Random PC Freezing - no pattern

[dsperber]

I'm actually not of the opinion that it might be the PSU (on both ASUS and M910t machines???) which is suspect.

The ASUS machine has a 600W NoFan PSU but in fact the machine itself uses around 220W total even with both monitors powered on (everything is running through an APC UPS, so I know how much is being covered by battery backup). The new M910t only has the stock 250W PSU from Lenovo.

The GTX 1050ti is rated <75W TDP, but in fact I don't "game" and it typically runs at under 40W max.

I honestly don't think I have either inadequate power (on the M910t) nor flaky power (on either machine).

But this afternoon I did download five legacy NVidia driver versions going back to early 2018 when I know I had none of this instability. There's really nothing that's been added in newer driver versions which impacts the older GTX 1050ti card as I use it, and again I'm not a gamer so am non-impacted by any game-related fixes in newer driver version.

If the latest Aida64 tweaks don't seem to help I next plan to downgrade my installed NVidia driver, in order to see if my suspicion might be valid about a newer NVidia driver version perhaps introducing some issue. I definitely seem be "much more stable" (but not yet back to 100%) having turned off screen saver and power-save mode.

Anyway, current clock (since last freeze/re-boot) on the ASUS machine is at 1 day 6 hours, and the M910t shows almost 22 hours. Been able to watch/record TV, work, remotely connect with RealVNC and Team Viewer, etc., all apparently without mishap. We'll see how long the current streak remains in effect. Naively for now it does seem that turning off all of those "lockup might occur in rare cases" hardware monitoring settings in Aida64 has brought about an improvement.

 

[wither 2]

Yeah, I thought it would be a leap to have problems with PSUs on more than one system but just thought I'd throw it out there as a possibility.

If you read the release notes for driver updates at NVIDIA, most of them are to accomodate gamers. They are the ones driving the desktop business these days.

Greybear, in that forum, seems to be very knowledgeable about these kinds of problems. I read some other threads in which he was trying to help and he knew where to look for the cause and was able to provide details about what to try. Very technical. Perhaps you might want to ask about it there.

 

[dsperber]

Well, I'm setting new records for recent stability duration thanks to disabling those lower-level I/O sensor readouts from Aida64: M910t has now been up for 1 day 10 hours, and the ASUS has now been up for 1 day 17 hours. Both continue to be fully operational including still being connectable through RealVNC and Team Viewer.

I've now downloaded older versions of Aida64 to try again (with those I/O sensor settings turned back on), since I know this issue is only about 3-4 months old for me.

Current "stable" Aida64 - 5.99.4900 dated 11/10/2018 is the one I'm currently running on both machines (and that I've disabled those low-level items to avoid "may cause lockout in rare situations").

I now also now have retrieved older Aida64 5.98 from 09/20/2018, and Aida64 5.97 from 03/28/2018, and Aida64 5.95 from 11/27/2017. I know I must have been running with all of these over the past year with the low-level options active but also without the instability I'm currently observing, with the same ASUS machines (including the one which is now dead and gone). These older legacy versions are also running on other Lenovo desktop machines so I'm confident I can just use them without concern on my M910t.

This is encouraging. In addition to newer NVidia drivers installed over the past few months, there were these 5.98 and 5.99 versions of Aida64 installed around 9/20 and 11/19 respectively. And based on what I now see using 5.99 but without its low-level readout options active, maybe I'm really on the right trail now.

We shall see.

 

[dsperber]

Well, I had a dentist appointment today. When I returned late afternoon the ASUS machine had frozen at 4:50PM. So it made it 2 days 2 hours before giving out. The M910t is up and still running fine, 1 day and 18 hours since its last re-boot.

Just to be proactive I decided to back-level the ASUS machine for both NVidia graphics driver as well as Aida64. It seems that turning off the low-level sensor readouts in Aida64 had terrific effect on both machines (e.g. the M910t is still ticking since I made that change), but since both of these software products seem relevant and since I never had instability until installing recent updates to both late in 2018, I decided just to roll back both of them to get me back to a stabler environment. There is zero new or fixed functionality that, for me, justified applying any updates to either software product over the past year. So reverting to what was running perfectly fine a year ago is of most concern to me.

So I've now reverted to Aida64 5.95.4500 which was dated 11/26/2017. This replaces 5.99.4900 released 11/20/2018. I also have re-enabled all but one (SMBus operations) of the items I had been running previously so I once again have all of the hardware monitoring values I had always been displaying. The SMBus item doesn't impact what I do monitor, so turning it off is of no consequence.

And I've now reverted to NVidia WHQL 391.35 which was dated 3/27/2018. This replaces 417.71 which was released 1/15/2019.

Let's see how things go. I will defer possibly rolling back anything on the M910t until/unless it next freezes.

One more note regarding Aida64, and my ASUS motherboard. Late last year (and this may have been tied to the software update, rather than to my motherboard) I decided to turn off monitoring the temperatures of my three internal drives as reported by SATA/SMART or the PCIe/NVMe. Every so often I would observe that the temperature items for the SATA drives would simply disappear from Aida64. At the same time Windows Explorer (and Free Commander XE) would struggle to display the drives. I was unable to determine or decide if I was seeing a problem with Intel RST (which is installed, presumably to improve SATA performance for drives running AHCI, not RAID) or with the SATA controllers on my ASUS motherboard.

To avoid possible contribution to the issue which might be flaky SATA controllers, I decided to stop monitoring drive temperatures in Aida64. I also un-checked the related "SMART operations" setting, which was one of those "stability" group of settings. So I've been running with that option disabled and no drive temperature reported for a good number of months. For the time being, even though I've gone back to Aida64 from 11/26/2017, I'm not going to re-enable SMART functions and monitoring drive temperatures. I just want to be able to see what happens with the rollback of both Aida64 and NVidia.

We shall see.

 

[wither 2]

Why fool with Aida64? Just uninstall it to see what happens. It's not really necessary.

I think Firefox may still be causing your random freezes. I suggest not running it for several days. Since you're not around when these freezes occur, you wouldn't see the graphics driver stopped and then recovered error. This event occurs and recovers within about 30 seconds to a minute. Happened again to me yesterday when I was looking at Videos at YouTube even though it had been running for a week without a problem.

 

[johnhoh]

this very setup has been working 100% reliably since 2010

You have very old hardware. Heat or oxidation kills all hardware eventually. But before death there is usually a frustrating period of intermittent failures that are assumed to be caused by software or setup when in fact they're caused by failing hardware. my two cents...

 

[dsperber]

You have very old hardware. Heat or oxidation kills all hardware eventually. But before death there is usually a frustrating period of intermittent failures that are assumed to be caused by software or setup when in fact they're caused by failing hardware. my two cents...

When I said I had been running with this setup since 2010, I didn't mean I haven't upgraded hardware in those past 9 years. I simply meant I have been running the very same notion of WMC HTPC and essesntially the identical software for this period.

But in actuality, I have upgraded my physical machines three times over the past 9 years. All are homebuilt, the first with a SuperMicro motherboard and the last two with ASUS motherboards. It's this most recent machine upgrade which was only built early in 2017 which is causing my headaches now. It's a Skylake Z170-Deluxe with i7-6700 CPU. Essentially it will be the final Win7-supported hardware. But essentially it's brand new hardware, though clearly exhibiting definite signs of hardware instability both in SATA and USB.

I've also upgrade through many graphics cards in the past 9 years (ending up with the current GTX 1050ti) along with several generations of TV tuner cards (both OTA/ATSC and cablecard-enabled), first from AMD and then from Ceton and Hauppauge.


But back to the case...

Had freezes on both machines again this morning (including the M910t after remaining up for 2 days and 7 hours), one occurring while I was on a conference call and looking at some emails on the screen in front of me. I went to select another email to view and I was frozen. Turns out it had occurred about 40 minutes earlier but I hadn't noticed.

Not surprisingly (anymore) I went upstairs to check the ASUS machine and sure enough it too was also frozen, sometime after 6AM this morning after being up only 8 hours. And this is the machine which was running with the back-level NVidia driver and back-level Aida64. So this is a real disappointment.

I then tried to re-boot the M910t but it froze at the Windows welcome screen! I went upstairs and sure enough the ASUS machine was frozen again, though I had rebooted it less than an hour earlier.

Clearly there is something "sympathetic" going on between the two Win7 desktop machines on my LAN. Seems to be totally immune to Win10, which is the laptop I'm using to type this post right now. Given that my M910t is literally brand new I have to believe the source of the problem is fundamentally with the ASUS Z170 machine. Interestingly both are Skylake machines with i7-6700 CPU.

I've now powered down the ASUS Z170, and also flipped the power switch on the PSU so that it should not be capable of spewing out anything over the LAN that can be seen by the M910t. Although the Z170 was my primary HTPC (with 6-tuner Ceton and 4-tuner Hauppauge) I have relegated my HTPC chores to the new M910t (with older 4-tuner Ceton and 2-tuner Hauppauge) for the past few weeks. So I don't actually need to use the Z170 in my office. I can use the Win10 laptop which shares external mouse, keyboard and Eizo monitors and have the exact same daily "work" capability I require. This Win10 laptop has remained up and available and on the same LAN throughout all of this nonsense with my two Win7 machines, and appears completely unfazed by what's going on in the Win7 network environment.

I now want to see if the brand new M910t HTPC can run all by itself isolated from the ASUS Z170 which is now truly offline indefinitely, and exhibit stability. Or is it, too, strangely subject to this instability from something coming from within itself, perhaps from intermittent TV tuner cards? This all seems so improbable.

Anyway, we'll see if I can establish normalcy with just the M910t running. I will also back-level its Nvidia drivers and Aida64, just to try and help out for this next test.

I obviously am still procrastinating replacing the ASUS Z170, either with the second M910t that I also bought or just with a replacement Z170-Deluxe motherboard. I really have to establish proof that it is a hardware issue on this machine which is at fault before I embark on either of these extreme remedy attempts.

 

[wither 2]

If you want to establish it is a hardware issue, then why run aida64? Still can't ignore the Firefox issue.

 

[dsperber]

If you want to establish it is a hardware issue, then why run aida64? Still can't ignore the Firefox issue.

As I've mentioned before, I remotely maintain about 23 computers for friends and family, almost all of which are machines that I ordered/configured and installed for them. And NONE of them exhibit any of this freeze symptom. Typically they can remain up for weeks or even a month or more between re-boots, which are from me doing a monthly Windows Update or other software/driver update that deserves a reboot. And every single one of those machines runs Aida64 without an issue, and typically the latest version... all without any instability.

So I can't bear to think I shouldn't be allowed to run software that I myself have been running for years, on my own machines as well as others'. But I understand why should I tempt fate at all and run Aida64 no matter how it is disabled, when by just not running it I could certainly point the finger at some other cause (like hardware) if I still see a freeze. I'm just frustrated, and can't bear to not see the Aida64 OSD that I'm used to, which also happens to show me the current time and also up-time, and its non-movement is evidence of the "freeze" even though Perfmon and Clockwise and windows clock all continue to move and right up to the bitter end I can still usually access mapped network drives from the other non-frozen machine.

Anyway, I should be able to run the new M910t "fully configured as my HTPC and with Aida64 running fully configured) in a perfect world. With the ASUS machine now truly taken down and offline completely, I'm hoping to see the M910t remain up for days and weeks, as it should.

While I wait for this proof (of either success or once again disappointing failure), I may just bite the bullet and use the opportunity to take the day or two it's going to require to replace the motherboard in the ASUS machine as long as I do have the replacement new motherboard waiting use. Worst case, if this doesn't actually solve the problem on the ASUS machine I still have the second extreme solution option of just replacing the machine completely with my second in-the-carton new M910t. But I really would like to keep the current ASUS case (and 5.25" BluRay burner optical drive which can't be transferred to the M910t which only has room for a slim optical drive).

 

[wither 2]

I understand. Only reason I mentioned Aida64 is because you were concerned about the updates to it and were reverting back to previous versions. I just thought it would be easier to completely uninstall it temporarily.

You've got a lot of money tied up in all that equipment. This system is 11 years old and my other is almost 9 years old. Don't have any spare parts.

 

[dsperber]

Sad news to report. After being up for 22 hours, my M910t froze again this morning at around 9:25AM. Nothing else was going on, the only other PC on the LAN was my Win10 P70 laptop (which continues to operate normally and continuously now for the 3rd day since I booted it, even as the M910t has frozen several times). The ASUS machine has been powered off and unplugged so it couldn't have been a contributing factor in any way.

Strangely, I had actually gotten up early this morning and was working at the computer, maybe 30 minutes before the 9:25AM freeze occurred. I'd gone back to bed at around 9AM and the machine was clearly working perfectly at that moment and I was naively confident.

I decided to power off my mouse and plug it into the charger. Obviously the receiver was still plugged into a USB port on the front of the PC. Furthermore, my Samsung Galaxy S7 phone was also plugged in to another of the front ports of the PC through its own USB cable, also charging that way. The mouse power-off is the only "changed" thing I did relating to USB or anything after finishing up at 9AM, instead of just leaving the mouse on the pad in its power-on state when I went back to sleep.

Yes, I'm running with the back-level NVidia and Aida64 software. And I had reinstated all of the low-level hardware monitoring capability of Aida64 as I had been running last year without issue on all of my machines, being of supreme naive confidence using the back-level product might solve the freeze mystery. Unfortunately apparently not.

Given what I already have learned about the apparent ability of one "failing" networked Win7 PC on the LAN (e.g. my upstairs ASUS machine) to impact the other networked Win7 PC (e.g. my downstairs M910t) and cause the second machine to also freeze, my explanation for this is that the networked machines "talk to each other" regularly to ascertain the state of the connection and also the state of shared mapped network drives. If one machine enters the freeze state where access to mapped network drives is still active and enabled and Windows on the frozen machine isn't really truly frozen solid yet, eventually this seems to impact the second machine as well when the normal regular inter-machine handshaking eventually is fatally disrupted. And then the second machine will also freeze.

So if the Win7 M910t froze again this morning even with the Win7 ASUS machine out of the picture, but with the Win10 P70 still up and running normally, it would seem to be yet something ELSE on my LAN which presumably is regularly handshaking with the M910t and which might have flaked out or hiccup'ed or sent out a stray signal over the LAN or who knows what.

I have four WMC extenders plugged into the Netgear R7800 router via ethernet cable through a network that includes six Netgear switches and two other Netgear routers running "access point mode" for WiFi. Somewhere out there must be the component which is still causing the instability for the M910t, since it now cannot be the ASUS machine. I am going to unplug three of the four extenders and two of the switches and all other non-critical ethernet-cabled network-connected devices that I don't really need or can live without during the experimental period.

My P70 also has a Win7 dual-boot option. Since it is clearly "immune" when running Win10 I think I will re-boot it to Win7 and let it operate that way. I'm curious if it can be impacted if/when the Win7 M910t next freezes again.

The chase continues. So far the solution eludes me. Very frustrating. But at least I've managed to procrastinate successfully regarding the motherboard replacement for my ASUS machine. Had the M910t remained up forever with the ASUS offline I might have actually begun this several-day surgery, which for now still seems genuinely unnecessary.

There has to be some explanation. Maybe Meltdown/Sceptre changes impacing WMC-related hardware/software which is clearly Win7-only? But then what about all you other people who see freezes, and are not running WMC setups?

Very very frustrating to me, the super-detective.

 

[dsperber]

Latest update on the Wn7 "freeze" story... (forgive me if the early statements sound repetitive)

First, continues to freeze occasionally, still with no real clue as to what is going on at that instant which would cause the symptom. I've stopped running both Aida64 and DUMeter for the moment even though I have run these products for probably two decades without an issue. So I don't blame them, but rather feel they are "victims" of the real culprit whatever it turns out to be.

Second, I honestly don't believe it's a "failing hardware" issue on either the brand new Lenovo M910T or the ASUS home-built machine, although both machines have the same Ceton and Hauppauge TV tuner cards installed and also have the same GTX 1050ti graphics, and both run WMC and serve as HTPC systems feeding WMC extenders around the house. I know I had been suspicious of the ASUS motherboard, and USB connections, etc., but being able to run for days between freezes simply belies the theory that there is truly failing hardware.

Next, I think I have a clearer picture of what the actual "freeze" consists of, since it definitely is not actually Windows itself totally freezing up. In fact it is NOT freezing up at all, as the onscreen system clock continues to move and progress, at least the system clock in the lower-right corner of the screen as well as the digital HH:MM:SS clock showing in the Clockwise (app) that I also run. Both of these clocks are progressing normally, showing that Windows itself and the Clockwise task seem to be running as usual.

However other onscreen "motion" has actually frozen and stopped, even while the two other clock values I mentioned are still moving normally. For example, the two time values displayed in Aida64 (HH:MM:SS clock time and also up-time since last Windows boot) are frozen. Also, I run PERFMON.MSC in a window, kind of like an EKG for the CPU, to show percentage of CPU horsepower being used (between 0 and 100%). This is displayed on the Y-axis with time-of-day displayed on the X-axis, with the graph updated at intervals of every 1-second, with a vertical red line moving to the right every 1-second marking progress and a time-of-day value showing at the bottom of the window at increments of 30-seconds. This too is frozen, as is Aida64's two time values. And yet the Windows and Clockwise clock values continue to move.

Also, I continue to be able to run "silent" apps on other network-connected machines (which have mapped network drives on the "frozen" machine) which simply access data folders/files on mapped network drives hosted by the "frozen" machine. These don't seem to go through regular Windows Explorer to initiate, even while the frozen machine is already frozen. So, for example, I can run Quicken and Agent (my email program) from other network machines, both of which are apps whose data files are hosted by the now frozen machine. The apps still start and run from the network machines seemingly without impact, at least early on in the freeze. In fact early on all of the mapped network drives on the frozen machine are still "visible" in Windows Explorer run on other network machines, as well as from Free Commander XE (which doesn't go through official Windows Explorer but is similar) run on other network machines. Also, the frozen machine is still responding through Windows Explorer and it still shows up in the "network" group of Windows Explorer run on other network machines.

However later on when the frozen machine has had some kind of a "collision" with its own Windows Explorer, the freeze now seems to become more serious. Now its mapped network partitions are no longer visible through Windows Explorer on other machines, and in fact the frozen machine itself no longer responds and provides information about itself or its partitions. It no longer shows in "network", and in fact fatally impacts presenting the whole category of "My computer" drives (both local and network mapped) on other network machines. Running Windows Explorer on those other network machines now only shows Favorites, Libraries, etc., but no drives and no "Network". And yet, Free Commander for some reason continues to be able to access the drives and their contents on the frozen machine unimpacted, apparently because it must have its own non-Explorer method of accessing them across the network.

Furthermore, I believe that whatever is "freezing" writing to the screen for "moving values" (like from Aida64 and PERFMON) also must be what is preventing RealVNC and Team Viewer from completing their connection dialog from a remote machine and enabling remote access. When these products connect they also pop up a small dialog message window advising the user of that remote machine that "a remote user has connected". So it is necessary to produce that popup message on the screen of the frozen machine, but apparently that cannot be done in the frozen state. Apparently the characteristics that define exactly what is "frozen" on the frozen machine prevents that popup message, and hence back at the client connecting machine it appears the connection doesn't actually successfully initiate... although I'm sure it's trying and has probably truly made contact up to that point where the popup message is being "queued" but not actually displayed.

At this moment my M910t has been in its latest frozen state for almost 4 hours. I noticed it was frozen because the system clock time was about an hour "newer" than what showed at the bottom of the PERFMON window, and also that the red line in the PERFMON window (which is supposed to be moving right at 1-second intervals with the CPU EKG graph moving along right behind it) had frozen. For a while, I could run Agent and Quicken from remote network machines and they would start and stop normally. And for a time running Windows Explorer on network machines (who had network drives mapped to the frozen machine) also opened and closed normally, including showing "Network".

However at some point I experimented with RealVNC which of course fails to connect, because as I mentioned above wants to display its own popup message announcing the remote connection but can't. I believe this is what now takes the freeze further, and actually impacts Windows Explorer and the "network visibility" nature of the frozen machine. After this I now begin to see the negative impact on Windows Explorer run on other network machines, where "network" has disappeared along with all My computer drives and the mapped network drives as well. And yet, Free Commander even now continues to be able to see (and access, through the supposedly now seriously frozen machine) those mapped network drives.

So, I now do not suspect newer versions of Aida64 or the Nvidia graphics drivers of themselves being responsible for the freeze symptom. I believe they play no role in causing the situation on the screen which must be more complex. System clock values continue to display and move normally, while other program-generated outputs are frozen and must not actually be being posted "complete" by the desktop screen manager. And this, in turn, causes the queueing up of other outputs to the screen (e.g. the popup messages from RealVNC and Team Viewer) and thus preventing the normal completion of those processes, even back on the initiating remote client machines. And once this "queue backup" symptom is underway, Windows Explorer now stops working... both on the frozen machine as well as on remote network machines that are trying to "talk to" Windows Explorer on all other network machines.

And eventually, it's possible that the inability of Windows Explorer to communicate normally between all network machines eventually "brings down" other network machines, i.e. they too "freeze" at least on their own program-generated moving outputs (such as Aida64 and PERFMON running on those other machines) even while their own system clock values (e.g. in Clockwise and in the lower-right corner of the screen system tray) also continue to move normally. This isn't always the case, but it's certainly possible.

For example right now I'm working from the ASUS machine which is still up and running normally for almost 2 days since its own last freeze, while the M910t has been frozen for more than 4 hours.

 

[dsperber]

I'm going to have to re-boot the M910t now. Its freeze has moved into the more significant "second stage" of impacting things on network machines. from the ASUS machine I can no longer run Agent or Quicken, I can no longer run Windows Explorer or start programs which require Explorer functions, and I cannot boot machines (e.g. my P70 or W530 laptop) which by default have mapped network drives to the M910t machine which is frozen. The boot process on those other machines will make it as far as starting the Windows desktop but when they then must establish the network connections to the mapped network drives they, themselves, freeze (actually they're running, of course) awaiting reply from their own Windows Explorer trying to communicate with the M910t Windows Explorer which itself is now truly stuck and not responding.

Although I don't think it's relevant, I'm going to revert to a wired USB Lenovo mouse on the M910t. I'm going to uninstall the Logitech Setupoint software and (at least temporarily) not use the wireless Logitech Performance MX mouse. Though this software hasn't changed in a long time, I'm just trying to prove that it has nothing to do with the story by eliminating it for now. If the freeze continues to occur it must clearly be from yet something else which I haven't stumbled upon yet.

In other words, it's not really that jiggling the wireless mouse isn't waking up the screen from screensaver or power-save mode that's at fault. i now have my screen on all the time and can see it's really that the program-generated output to the screen (e.g. from Aida64's time clock values, or PERFMON's time-of-day) have stopped being updated. Win7's output to the screen from programs has been cut off for some reason, even though system clock output (including through an application program like Clockwise) continues to be displayed normally. So the Logitech wireless USB mouse itself, plus the Setpoint software that supports it, should also be considered blameless.

Although I won't do it yet, I'm very tempted to just go back and reinstall the latest Aida64 and NVidia drivers, as I now feel they are blameless in causing this freeze. And although both of my Win7 machines are WMC HTPC systems, I also hesitate to blame that either.

Interestingly the M910t was recording the Super Bowl yesterday evening when a freeze occurred. I happened to notice this around 8 minutes after it had occurred (as told to me by the frozen time on PERFMON vs. the moving time on Clockwise). So, not wanting to lose any recording more than necessary, I pushed-held POWER to shut down the M910t and ran upstairs to see if the ASUS machine was also frozen, which it was not. So I quickly started WMC and began recording the Super Bowl on that machine, allowing me to now more casually go back to the M910t to get it going again. And by the way the ASUS machine continue to record properly for hours, and in fact it is still up and running normally right this moment as I type, now un-frozen for almost two days.

Well, when I got the M910t re-booted I double-checked in WMC to see if the recording-in-progress when the freeze occurred had been "saved" (in its abbreviated length, due to my pushing the POWER button). This is actually what I generally observe, that magically Windows (which is actually not frozen but is actually still operating) is triggered into its forced-shutdown activities when I push-hold the POWER button, which includes closing out any recordings-in-progress from WMC. This is to guarantee not losing any partial recordings, and MS should get credit for making sure this is a feature of WMC and Win7, which it is.

Anyway, when I looked at the recording start-stop time of this partial recording-in-progress that had been manually interrupted by me when I pushed the POWER button, remarkably it actually went right up to the moment I pushed-held the POWER button, which was of course 8 minutes after the freeze itself had taken over the screen. This is consistent with my other observations about the fact that Windows itself is really still running inside, and even supporting (for the moment anyway, until Windows Explorer get impacted) mapped network drive access from remote LAN machines. Apparently WMC continues to operate, and record if that's what's going on at the time.

So in fact I didn't lose those 8 minutes of WMC Super Bowl recording time on the M910t at all, even though the screen itself had been frozen for 8 minutes. And so the start time of the "continuation" recording on my ASUS machine which I started maybe only a minute or two after shutting down the M910t and running upstairs, well it was just a minute or two later, so that's all the actual "recording" that I truly lost.


Which means I really still have not gotten any closer to determining exactly what it is about Win7 and how I use it and what common software/hardware I have installed on both of my WMC desktop machines must clearly be responsible for Win7 for some reason suddenly not able to complete a program-generated display of output to the screen... resulting in the first external symptom of the "freeze" symptom.

From that point on, it is the queued backup of subsequent output intended for display on the screen (so that nothing appears to be happening if we jiggle the mouse) which is what contributes to why we believe the system is "frozen"... which internally it actually is not. It's only output to the screen which has become inhibited.

It's not the NVidia drivers which are at fault here to prevent output to the screen, as the system clock and Clockwise time values are displayed normally and progress normally. It's only other program-generated outputs which are being suppressed. There must be some other internal Windows handling of screen output mechanism (inside of Win7, but seemingly not Win10) which has for some still unknown reason stopped working. This is the "culprit", eventually impacting Windows Explorer and related functionality both on the frozen machine as well as on remote network and remotely connecting machines.

I don't know the Win7 internals, or how output to the screen is programmed or handled. But I believe the freeze symptom has nothing directly to do with mouse input, or NVidia graphics drivers, or screen saver or power-save mode, or Aida64 or its low-level sensor processing, or anything else I've tried to back out or turn off in an attempt to get to the true root of the cause.

And yet SOMETHING is every so often triggering a disruption in the normal Win7 display of certain non-system program-generated classes of output destined for the screen, which itself now prevents all future program-generated screen output from making it to the screen, which thus backs up everything. Eventually Windows Explorer is impacted, with its own resulting snowball effect... but it all started with the "dropped/lost or somehow inhibited" original output to the screen from some program-generated output. It had nothing to do directly with the mouse itself.

If there are any Win7 systems programmers out there who might have some ideas as to what might be at fault here, or how we might diagnose what's happening (e.g. running some system monitoring tool of some sort, specifically to track output sent to the screen and when that might stop/freeze), I'm willing to donate my two failing machines to the project of debugging here.

Note that the latest M910t freeze (now about 5 hours ago) occurred with NOTHING RUNNING in the foreground, other than (a) Clockwise and (b) PERFMON, along with the rest of the normal Windows desktop and taskbar. There was otherwise NO APPLICATION PROGRAM ACTIVE!! The clock value within the Clockwise window was moving normally, and matched the windows system clock value shown in the lower-right corner of the taskbar. But the inner contents of the PERFMON window was frozen, with its last drawn time-of-day value about an hour earlier than the current true time-of-day when I noticed this freeze. Nothing was running when the freeze occurred an hour earlier, same as nothing was running when I first noticed the freeze an hour after it had occurred.

So there actually was nothing new and different which needed to be presented on the screen other than the regular ongoing normal every 1-second update from PERFMON to its window data. This was the ONLY program-generated output to the screen which was happening, and which had now frozen. The only other "moving" output to the screen was to the Clockwise window (and it was still working correctly and moving every 1-second), and the system clock at the right end of the taskbar (and it, too, was still working correctly and moving every 1-minute).

In other words the system-generated and Clockwise-generated time-of-day values continued unimpacted, even while the only other non-stationary screen output was the moving every-1-second EKG output from PERFMON and the every-30-seconds output of the time-of-day.

Just to show what would normally be on the screen when this latest M910t freeze occurred, here is the lower-right corner of what would be the M910t screen:

(a) Clockwise window showing day, HH.MM.SS time and date
(b) Perfrom window showing CPU EKG with vertical red line moving to the right every 1-second
(c) system tray at right-end of task bar showing Windows HH:MM system time and date

When the freeze occurs, only the PERFMON window is impacted and frozen. The Clockwise window contents and Windows taskbar system time/date are unimpacted and continue moving normally.

 

[dsperber]

I have one more idea for something to try, just to see if I can eliminate one more possible "culprit" from blame, perhaps exonerating it.

Since it is PERFMON which seems directly impacted by the freeze (and perhaps actually CAUSING the freeze???) and is definitely always running on both of my failing Win7 systems, maybe I should try eliminating it from the equation.

I had already seen that Aida64 would also be impacted by the freeze (although the freeze continues to occur even when Aida64 is not running at all!), but then PERFMON was also running and was also impacted/caused the freeze.

So I will reinstate running Aida64 (in its plainest trimmed-down non-low-level-I/O-sensor mode) just so that I'll have something else onscreen which is program-generated and supposed to be moving but other than PERFMON so that I can tell if a freeze has occurred or not. And I will not run PERFMON.

That means I'll now have Clockwise date/time, Aida64 time/up-time, and Windows system time-of-day, all on the screen. And the PERFMON EKG window will no longer be present.

Let's see if I can induce a freeze with that setup or not, thus permitting the drawing of conclusion that PERFMON is either the "culprit" or is just another "victim".

 

[dsperber]

Ok. New test. Running without PERFMON on both M910t (now re-booted and up for over an hour) and ASUS (still up and running now just over two days). I've now reinstated Aida64 on both machines instead of PERFMON, as my way of visibly detecting if/when a freeze has occurred. Clockwise is still running on both machines.

I left back-level NVidia drivers, back-level Aida64, existing Logitech Setpoint software installed and existing wireless Performance MX mice in use (wireless receivers still plugged into front USB ports of towers), still no USB cables connecting any monitor so all those USB hubs are out of the picture. This is how it's been for a while now, but to no avail. Freeze still occurs. Anyway for now this is all just as it's been for a week or so. I haven't yet tried using the wired mice, at least for now.

A few days ago I had already disconnected ethernet cables to a number of unnecessary network devices just to see if their presence or absence would have any effect, and it didn't. Freeze still occurs even with them gone. But for the moment I have not reconnected anything.

As far as Aida64 configuration, I have the setup running on the M910t with all of its lower-level I/O sensor readouts disabled. There is therefore nothing running in the program that is detecting anything lower-level that pretty much superficial generic Windows information.

In contrast, on the ASUS machine I've left the fully-configured setup, utilizing all of the required relevant low-level I/O sensor options required to present what I want to show.

The hourglass has been turned over. Let's see how long the sand continues to fall.

 

[dsperber]

M910t: up now 14 hours, recorded a number of new programs yesterday, watched 10 hours of recorded TV through three different extenders around the house

ASUS: up now 2 days 13 hours, connected without issue from M910t numerous times via RealVNC to check access. Did normal work on it throughout the day.


Cautiously optimistic, but it's still early in the new non-PERFMON game on both machines. If I can now log another 24 hours of solid-uptime on both through tomorrow morning, that would be significant.

Fingers crossed. I don't want to jinx it.

 

[dsperber]

:mad2:

Naturally, I jinxed it. Not two hours later another freeze on the M910t. ASUS continues to run quietly, now 2 days 18 hours.

I've now delved into the system logs to see if I can find anything. And I did. Turns out the 6008 event ID ("dirty unexpected shutdown") which gets created at the next re-boot has been tracking every one of these system freezes, going back to 1/12/2019 which is when my M910t was freshly built-out and put into production and the very first freeze occurred.

So here is the current filtered output showing all of these 6008 events. Their log time is when the system was next re-booted but the details for the event cite the so-called "previous system shutdown at HH:MM:SS on MM/DD/YYYY was unexpected". And of course I recognize the times of at least the last few freezes as precisely matching the "previous system shutdown" as recorded in these 6008 events.

This should be helpful, I would imagine, in confirming that SOMETHING is actually happening internally at the moment of the "freeze" which is being treated as a SHUTDOWN (unexpected). Interesting, because as I've reported the system isn't actually 100% frozen (or actually shutdown and inoperative). In fact it is largely still operating, and at least early on after the "unexpected shutdown, a.k.a. freeze", it continues to allow its mapped network drives to be accessed from other non-frozen PCs on the LAN.

I'm hoping somebody is following this thread who might be able to interpret what the following information really is showing, and/or suggest something I might now do or turn on or activate that can help me gather even more forensic diagnostic data regarding exactly what this "unexpected shutdown" really is. I'm hoping the details of that earlier "unexpected shutdown" which are captured in the later 6008 restart log entry mean something to a Windows system programmer.

Here is the system log filtered to show 6008 events. For example the first (most recent) shows that I re-booted at 11:34AM, and the details show that the "unexpected shutdown" (a.k.a. freeze) occurred back at 11:08AM, which is confirmed precisely by what I saw frozen on the screen in the Aida64 "time" value as contrasted with the un-frozen moving time-of-day value in the Clockwise window and which tipped me off that a freeze had occurred back at 11:08AM.

And here are the "event properties / details" for the top three most recent freezes. They are slightly unique, but for the most part seem pretty much the same.

 

[dsperber]

Just a couple of hours later the M910t froze again:

Interestingly, the event details claim that the earlier unexpected shutdown was at 3:30PM (I noticed the freeze about 3:53PM and re-booted at that time). But I could swear that the frozen time value in the Aida64 window was earlier at 2:50PM and not at 3:30PM as the event details claimed. I'm sure of this because I actually looked to see if a 3PM scheduled TV program recording actually kicked off which as it turns out it hadn't, confirming I believe that my memory of the 2:50PM was likely corrrect. Anyway, next freeze I will write things down as I see them on the screen before re-booting, just to be sure all the expected times match or not.


I was suspicious of the APC UPS battery backup I use (BACK-UPS NS 1050) which is managed by Powerchute Personal 3.0.2 through a connected USB cable between PC and UPS. Powerchute has the ability to shut down the machine after an extended power outage, if the battery level drops to be able to support only 5 more minutes with the connected devices.

So I looked at the UPS status in Powerchute, and actually it hasn't even kicked in for three days (although it might not report on very short duration incidents). Plus it currently is at 100% charge, supplying 170 watts of battery-backed power, which is estimated to support 24 minutes of running on battery should that situation arise... which it didn't.


I wish I had more insight into the event log 6008 incidents, or had some kind of diagnostic capability to either log things in more detail or have Windows itself produce some kind of rotating analytical log which could hopefully be viewed after the freeze at the next re-boot, to show what was going on 20 minutes earlier that has been captured as this 6008 situation.

Something is causing this "unexpected shutdown". Yet to be determined.

 

[dsperber]

So I did some browsing around the interweb looking for any insights from others regarding the 6008 error and what might cause it and how one might attempt to resolve it. Lots of fairly vague advice, as honestly this is one devil of a vague problem.

But I did find this "solved" article on TomsHardware which did suggest that going with what starts as the typical Windows "high performance" power plan and then tweaking out pretty much every possible leftover power-saving option to OFF or DISABLED or NEVER has a decent chance of eliminating any of the possibly contributory power-saving settings which really relate more to battery-saving on laptops rather than maximizing performance for high-end desktops.

Can't hurt I figure. So I went ahead and changed from my M910t "ThinkCentre Default" delivered power plan from Lenovo which I had already adjusted (as I've described previously) to eliminate much of the built-in power saving stuff especially with USB, to the suggested MS "high performance" plan which I then further modified as recommended in the article (see below).

Too soon tell if this will make any difference or not, but there's almost nothing here I would have any disagreement with. It is, after all, a high-performance desktop machine. The only thing I might not have touched is about the spin-down of hard drives after 15 minutes, which other than the resulting expected small delay when the drive is needed again and must first be spun up I always have had set that way. I have never let the drive spin 24/7 as I prefer the resulting reduced heat and reduced noise and reduced wear/tear. But I only have one remaining internal HDD spinner (my 6TB WD black drive used for WMC recording, which is built for 24/7 use anyway).

So, just to share here what the TomsHardware recipe is for "tweaked high-performance power plan":

---------------------------------------------------------------------------------------------------

Change your power settings to this:

This causes you to have problems on some computers. You expect high performance and you are not getting it. OR You system freezes…
If you are on laptop battery power, these high performance settings cause the battery to run down faster. These settings are really for high performance desktop PC.

A. When these power saver features are enabled, it causes a bunch of problems.
B. Windows shuts down your system to sleep, hibernate, standby, etc..etc...
C. After shutting down system to "save power" the system malfunctions when you try to wake it up again...and locks up, freezes, etc...
It locks the mouse, it locks the keyboard, it shuts off the display, and locks out the hard drive, it shuts off USB devices, etc...etc...
D. This will cause you to pull your hair out, and go to the funny farm...(those nice young men in their pretty white coats)
E. Make it stop, please make it stop.
F. Shut off all these "features" and USE your computer for a change):

Click Start, Click Control Panel,
Look at the top of the window, in the path bar you see “control panel >”
Click on “>” (in the path bar) now click on “all control panel options.”
(This will open up all the hidden controls available)
Click Power Options
click on the arrow to “show other plans”
Check the Box that says "high performance"
Click (in high performance) "change plan settings"
Turn off display: set to NEVER
Put the computer to sleep: set to NEVER
Click: Change advanced plan settings
Scroll down the list: Click on the + signs to expand the choices for each item on the list.
Require a password on wake up: set to NO
Hard disk: turn off the hard disk: set to NEVER
Wireless adapter settings:
Sleep: set to NEVER
Allow Hybrid sleep: set to NEVER
Hibernate after: set to NEVER
Allow wake timers: set to disable
USB settings:
USB selective suspend setting: set to NEVER
Power Buttons and lid:
Power button action: Setting: set to shut down
Sleep Button Action: set to: do nothing
PCI Express:
Link State Power Management, Setting: OFF
Processor Power Management: Minimum state (set to) 7%

System Cooling Policy: setting: Active
Maximum State (set to) 100%
Display
Turn off display after: setting: NEVER (turning off display automatically can cause freezing also)
Turn off the monitor power manually, when you want it off. Don’t use the auto monitor turn off.
Multimedia Settings:
When Sharing Media: Setting: Prevent idling to sleep
When Playing Video: Setting: Optimize Video
Click APPLY
Click OK

Open the bios set up and make sure "cool and quiet" is OFF. (AMD)
If there is a power saver or a "quiet mode" in the bios, shut it off...
There may be a performance setting in the bios setup you have...make sure it's cranked up to max.
in the bios, see that the allocation for video, if available, is maxed.

Now open the hardware manager profiles...
click start
click computer
click system properties
click device manager
double click on mice and other pointing devices
right click on HID compliant mouse
left click on properties
click on the power management tab
UN-check the box that says: "allow the computer to turn off this device to save power." (there is now NO check mark in this box)
click OK

Now repeat this procedure for all mice, monitors, keyboards, and ALL USB ports on the device manager list.

You must open ALL the devices one at a time, as above, and turn off the power saver, for each device.

NOW turn all the security back ON. NOW open your security antivirus. Make sure the antivirus is set to "gaming mode." Or "multimedia mode."
This prevents the security updating from interrupting your game / multimedia priority.
IF the security does not have "gaming mode" or "multimedia mode" get different security.
IF you are using "free" security downloaded from the internet, get rid of it NOW.
Use ONLY professional all in one security. DO NOT load multiple mismatched security programs, which conflict with each other.
DO NOT load free tools into your system such as: "driver sweeper" or any of that "free" goofy stuff.

-------------------------------------------------------------------------------------

We shall see if this accomplishes anything. Note that the ASUS machine continues to "set new records", now having remained up and running for just over three days!!

Also note that I've gone back to running PERFMON and Aida64 (fully configured again) since experimental results suggests that neither of them is the "culprit" behind what we now can see as an unexplained event ID 6008 situation made even more ambiguous and mysterious by the mostly still-running Windows even while the screen output is locked up, no matter what the event log claims.

 

[wither 2]

If what you're trying now doesn't work, I suggest disabling your graphics card driver and using the onboard chipset for a while.