Windows 7: Connect 2 routers together?

Ahoy guys,
I have a network question...

I have 2 Linksys routers (WRT or something). Router A is in Room A along with the modem. Router B(collecting dust) is in Room B (upstairs) and Room B has most of my computers. The only "functional" computer in Room B is my laptop, but that's because it's Wi-Fi enabled. I would like to connect Router A to Router B, which would give computers around Router B an internet connection. I have some cat-5 cables laying around, which might help.
Any Help?
Thanks.

hi Metamencs, and welcome to sevenforums

this tutorial may help

Thanks for the help.. but I'd rather not go that way, as I feel I might botch something up..

Hmmm, this isn't what I had in mind earlier, but I guess it might do... is there some sort of hub full of Ethernet ports, that I can connect to with a cat-5 cable? (Modem>Router(downstairs)>Hub(upstairs)>Computers&WirelessPrinters&XBox360, is what I had in mind)

You should just be able to hook them up with the cat5 and use the second router as a switch/hub for the other computers.

You may need to log into the 2nd router and disable the DHCP server if enabled.

disable DHCP on the 2nd router... connect cat5 to either the last or 2nd port ( i forget which ) and that should do it

Thanks for the help guys. I have one last question...
I connected my cat-5 cable to the ports called "1" (there's 3 other ports, "2,3 and 4") instead of "Internet". However, whenever I connect the cat-5 cable to "Internet" my computers connected to Router B will not be able to connect to the internet. So my question is, how would you connect the cat-5 cable to "Internet" and have your computers' internet work? My reason is that the cat-5 cable takes up 1 LAN port or whatever, and I need to be able to connect my 4 computers(2)/XBox/printer. Sorry about this, but I don't know how to disable DHCP on a SECOND router. (Router A is 192.168.1.2).

Thanks for your help.

To change the router settings, you will need to log in to the router from a locally connected computer. What is the make and model number of this router?

Yeah, that's the part I don't understand....
Linksys WRT54G router.

Ugh, I seem to come across more problems than expected...
Because I can't get my Router B to work,(I musta botched something up ) I connected my cat-5 cable to computer A in the meantime. It says something
about "Dial Up Connection, PPPOE Broadband, Opening Port, Redialing, WAN Miniport, then finally Unable to Connect". Any help here?

Meh.

EDIT: (I don't know if this is my internet or what) Whenever I try to plug my cat-5 to Computer A, it gives me this Diagnostic of: "A problem with your network router... If you are connected to a home network.... turn off modem.... make sure modem is plugged into..." However, when I plug the cat-5 to Computer B(XP), it works dandy. My laptop works fine wirelessly. Computer A is: Gateway DX4710 desktop.