The title is somewhat misleading. This event happens maybe once every 20 plug-in attemps and only when I'm slightly up or down from a straight push in.
The device doesn't matter.
The port in question is one of four on the front panel of the case. The rebooting only happens with one port. Never had a problem with the other three.
I haven't pulled the front of the case because it happens so infrequently.
However, I would like some suggestions on the possible cause.