Yep, did that. Also toggled the ability to sleep the device off, made sure it was on wireless N only on the router and enabled adhoc wireless-n on the adapter.
One of the last 2 things got me to get a ~1.7 down and ~4.0 up, which still baffles me, as I was under the impression that download speeds should at least match, and usually exceed, upload.