I think most people feel you can keep you battery in your laptop while you're using electric power. I've even heard experienced users say that there aren't any negative side effects to leaving a battery in the system after it is charged, and there can’t possibly be any, unless your charging circuit does not work, and continues to dump unregulated voltage into the battery.
But most importantly, you should use your battery and let it run all the way down to about 10% power (and never more than 5%) on a regular basis, about once a week or so.
By the way, heat is very bad for batteries, but leaving it plugged in will not cause the battery to overheat.