I believe Brink has already covered the questions in this thread, but I will add my personal observations anyway.
I have a USB flash drive with a red light on it. The light is on when there is some write/read activity going on, otherwise the light is off. Naturally I always remove the drive when the light is off. Usually I click the "safely remove" button, wait a sec for an OK, then remove it. Sometimes I forget to do so

. Nothing bad has ever happened, with one exception - but that was on a Mac.
I observe this behavior on XP, 7, and Vista. The only time I actually had a problem was when I used this drive on a Mac and forgot about safely remove button. The whole drive became corrupt and I had to use special software to retrieve what data I could from it.
I also have an external hard drive - Seagate FreeAgent or something. This drive has a cool looking ambient light thing, which goes on as soon as the drive is connected to a PC (it uses a separate power supply as well). It also has an on/off touch button. If the drive is powered up - i.e. plugged in - while the PC is operational, or the PC is booted up, while the drive is plugged in, then the light goes on. If I power the PC down, the light goes off. In these situations the on/off button is not used. Now, if I want to stop the drive without shutting down the PC, I click the "safely remove" button, wait for an OK (
at this point the light remains on!), then press the on/off button. At this point the light goes off and at the same time the drive stops spinning, after that it's safe to unplug it (I always unplug devices that are not in use).
I have never turned it off without using the safely remove procedure. The drive is connected to the Windows 7 machine, I have never used it in XP. I do not use any special software. Oh, and I have never unplugged the USB cable, I just turn it off and unplug from the electric grid.
Hope that helps.