Loud buzz on startup before windows loads

Occasionally when I start up my computer, before the bios beep plays and before the computer is displaying anything on the monitor, I hear a loud, solid buzzing noise that occurs for around 2 seconds, pauses for a short period and then another 2 second buzz. It is always twice, but does not always occur on startup. My computer logs no disk errors, all the fans seem to be running correctly, and this problem does not occur at any point when the computer is running. Is this a problem that is worth further looking into or is it something that I can safely ignore?

MORE INFORMATION
The buzzing does not appear to be coming from the front of my case, where my disk drive, CD drive, and GPU are located. It only happens the first time I boot up my computer for the day (I boot several times a day, at least twice), so it may have to do with time between boots. My computer has been idling at about 38 Celsius, and I have only noticed this problem as the weather has gotten colder in my location.