Ah, it works for the Start Menu but not the Taskbar, my bad, should have just written the exact steps I took instead of trying to simplify it. Allow me to correct the steps written above (but still simplified):
1. Go to Control Panel
2. In the top right corner, change the view to show large/small icons
3. Find Device Manager and drag it to the desktop (or if you can't drag to desktop, then any other folder in Windows Explorer will do)
4. A shortcut is created for you for Device Manager. Drag the shortcut to the Start Button. The Start Menu wouldn't open, but the hover over message says "Pin to Start Menu". Release the mouse and it's done.
5. Cheers!
6. If it doesn't work for you (which it didn't for another PC I have), go into the Start Menu, right-click on any folder and choose "Open". The folder will open in a new window in Windows Explorer. Go up one level to the Programs folder.
7. Drag Device Manager from Control Panel to Programs folder window. A shortcut will be placed in it.
8. Head over to the Start Menu and drag the shortcut over to the "Back" menu entry at the bottom and hold the mouse there. The Start Menu switches from All Programs to the pinned programs list.
9. Drop the shortcut to the pin list.
P.S. Just an alternative way of doing it ("Look Mum, no keyboard!"), but doesn't work for pinning to the taskbar. Thanks for the tutorial btw!