Router's firewall is always in action no matter what you do, which is a good thing. NAT stuff. "Network Address Translation" Look that up on Google if you're curious about it. Unrequested incoming packets are blocked, simply because they don't know where to go once they hit your router. That's why you are much more protected behind one.
Windows Firewall is activated by default which is also a good thing. Generally speaking, neither the router nor Windows Firewall needs attention by the average end user unless they want a server. Then ports need to be forwarded and permissions applied.