Thousands of WMI Event ID 10 Errors after Server 2012 R2 Upgrade

I recently upgraded my Hyper-V host from Server 2008 R2 to 2012 R2. It’s not domain-joined, and I did an in-place upgrade. Every 11 seconds, I was getting the following error in the Application event log:

Log Name:      Application
Source:        Microsoft-Windows-WMI
Event ID:      10
Level:         Error
User:          SYSTEM
Description:
Event filter with query “SELECT * FROM __InstanceOperationEvent WITHIN 10 WHERE (TargetInstance ISA ‘Msvm_ExternalEthernetPort’) OR (TargetInstance ISA ‘Msvm_VmLANEndpoint’) OR (TargetInstance ISA ‘Msvm_SyntheticEthernetPort’) OR (TargetInstance ISA ‘Msvm_ComputerSystem’) OR (TargetInstance ISA ‘Msvm_VLANEndpointSettingData’)” could not be reactivated in namespace “//./root/virtualization” because of error 0×80041010. Events cannot be delivered through this filter until the problem is corrected.

Each section below lists an attempted fix. Jump to the last section for “The Short Answer.”

Network Drivers

Microsoft Partner Support suggested that this might be caused by network drivers. This machine has both Broadcom and Intel NICs. For both, when I tried to update the driver, I got the message, “Windows has determined the driver software for your device is up to date.” Nonetheless, I decided to uninstall and reinstall the drivers to see if it helped.

First I uninstalled the Intel driver, version 6.3.9600.16384. After a reboot, the driver had re-installed itself, but with version 9.6.10.0. The WMI errors continued.

Next I uninstalled the Broadcom driver, version 7.4.23.2. After a reboot, the driver had re-installed itself, but with version 7.4.23.2. The WMI errors continued.

Note that uninstalling the driver causes Hyper-V to lose the NIC association; it reverts to being an Internal network. After the drivers re-installed, I had to go back to the Hyper-V Virtual Switch Manager and re-assign the existing switch to the External network.

WMI Rebuild

Another suggestion was to rebuild the WMI repository. I ran this script from TechNet:

http://blogs.technet.com/b/askperf/archive/2009/04/13/wmi-rebuilding-the-wmi-repository.aspx

After running the script, I ran winmgmt /verifyrepository. The repository is consistent. However, the errors continue every 11 seconds.

Process Monitor

While working on another issue, I ran Process Monitor from Sysinternals. Interesting:  there are a lot of references to IProsetMonitor.exe, including unsuccessful attempts to read a “virtualization” key in the registry, followed by writing to a special event log, Microsoft-Windows-WMI-Activity/Operational.

That log is small (1MB) so it only holds about 750 events. There are so many 5858 events being written to that log that it only holds about five seconds worth. It took some fast keyboarding to grab this sample:

Log Name:      Microsoft-Windows-WMI-Activity/Operational
Source:        Microsoft-Windows-WMI-Activity
Event ID:      5858
Level:         Error
User:          SYSTEM
Description:
Id = {4C4C4544-0031-0002-EAE0-B8E795DDCE01}; ClientMachine = MYHVSVR; User = NT AUTHORITY\SYSTEM; ClientProcessId = 1732; Component = Unknown; Operation = Start IWbemServices::ExecNotificationQuery – root\virtualization : SELECT * FROM __InstanceOperationEvent WITHIN 10 WHERE (TargetInstance ISA ‘Msvm_ExternalEthernetPort’) OR (TargetInstance ISA ‘Msvm_VmLANEndpoint’) OR (TargetInstance ISA ‘Msvm_SyntheticEthernetPort’) OR (TargetInstance ISA ‘Msvm_ComputerSystem’) OR (TargetInstance ISA ‘Msvm_VLANEndpointSettingData’); ResultCode = 0x80041010;

That “ClientProcessId = 1732” is interesting because it points back to IProsetMonitor.exe in Task Manager:

The Culprit:  Intel PROSet Monitoring Service

So finally we know where the error is coming from:  the Intel PROSet Monitoring Service. Sure enough, when I stopped that service, it stopped both the 5858 events in the Microsoft-Windows-WMI-Activity event log and the WMI 10 events in the Application event log. Phew! That’s a relief. I bet my system will run better without hundreds of events being written every few seconds.

So I need a version of PROSet that works on 2012 R2.

I tried re-installing the version that was working before (Dell calls is 14.0.0; Intel calls it 17.3.66), but I still get the WMI errors.

I tried installing the latest from Dell (14.5.1, Intel 18.2.63.0), but that will require deleting my virtual NIC:

Well if I’m going to do that, maybe I should see if Intel has a later version specifically for 2012 R2. Yup, here’s ProWin64.exe version 18.7 for Server 2012 R2. I tried installing that and it said my computer has no Intel adapters (shown here next to Device Manager listing the Intel adapters):

So I went back to the latest Dell version 14.5.1. that installed okay, but the WMI errors were back.

Eventually I found this Intel page about the level of 2012 R2 support for each of their NICs. It turns out that my PRO/1000 PT Server Adapter only has “In Box Support,” which means that PROSet for Windows Device Manager is not included. So I wound up doing a Modify installation on the latest Dell version, leaving only the drivers and the SNMP Agent installed:

In that configuration, PROSet is not active and the WMI errors are not happening.

Re-Activate the Virtual Adapter

The last step is to re-connect the Hyper-V virtual switch to the physical NIC. After opening the Network Connections (ncpa.cpl) to confirm which NIC is plugged in, I went in to the Hyper-V Virtual Switch Manager and set the old virtual switch to use the “new” external NIC. That led to this message:

Great. How can every little step in setting up a computer be a failure? And why not give a reason for the failure?

Back to Google, I found lots of suggestions. One was to uninstall and re-install the network drivers. I thought I just did that, but I went to Device Manager, told it to uninstall and remove drivers on both Intel NICs, then rebooted.

Before the reboot, the Microsoft NIC driver was at version 9.6.10.0; an attempt to update gave a message that the latest driver is already installed. After the reboot, the Microsoft NIC driver was at version 9.13.41.3. Go figure.

Anyway, the driver uninstall/reinstall didn’t help. I kept getting the “Failed while adding virtual Ethernet switch connections” message.

This article had some low-level details on the errors. I followed the first part of those suggestions, including removing the old virtual switch from all virtual machines, but kept getting the error.

Finally on a whim, I tried unchecking “Allow management operating system to share this network adapter.”

Voila! The virtual switch was created with no problem. Then I went back in and re-enabled management:

That also completed with no problem (other than losing the Remote Desktop session—be sure you have console access).

Finally, I went in to the new vEthernet switch and set the machines fixed IP address and DNS servers:

The Short Answer

If you’re getting WMI 10 errors every 11 seconds after a Server 2012 R2 upgrade, try these steps:

1. In Event Viewer, go to Applications and Services Logs > Microsoft > Windows > WMI-Activity event log and check for 5858 errors. If you see errors that match the WMI 10 errors in the Application log, use the Process ID from the 5858 event to find out from Task Manager what process is causing the error.
2. If IProsetMonitor.exe is causing the error, check this Intel page to see what level of support your NIC has under Server 2012 R2.
3. If Full Support is available, you may be able to update your NIC driver. You’ll want to use the full package that includes the PROSet tools. Mine didn’t have full support so this didn’t work for me.
4. If only Out of Box support is available, either uninstall the Intel tools entirely and just use the Microsoft driver, or Modify the install to remove PROSet but leave the driver.

If you need to remove and re-create your Hyper-V virtual switch in order to uninstall the driver, see notes in the previous section.