Well, the clue was the fact that there were entries for NVIDIA in the Registry's enumeration section that I could not delete. Even though the device had been uninstalled those entries remained.
I have run things like CCleaner and Driver Sweeper/Fusion on this problem but only after uninstalling the device through Device Manager and, as I said, it would reinstall each time. This time I didn't uninstall the device, I disabled it before running the cleaner. When I rebooted it did not install anything but the NVIDIA entry was still there. I then re-enabled it and uninstalled it and re-booted the PC. When it came back up it did detect new hardware and installed a driver but it was the default Windows high definition audio driver, not the NVIDIA one.
It looks like the secret was disabling the device before running the cleaners.
Thanks so much for your help with this.