Just resize the Explorer window, make it as big or small you want to, and close it. Explorer remembers the last window size, and now all opened Explorer windows open in that size.
For example, open Libraries window. Resize it and close. Open now the Computer window and you'll notice it opens as big as the Libraries window was when you closed it.
Kari