HyperV unstable vmms service crashes periodically

[guest]

I am getting an error, that is causing the hyperV to be unstable. I have 3 hyperV servers in my test env, runnning about 45 clients. All 3 are reporting this error at some point throught the day. This often means that I need to restart the service hyperV Virtual Machine manager, and obviouisly during that time the SCVMM server (which is a VM client within these 3) looses contact until the service is restarted.



Log Name: System
Source: Service Control Manager
Date: 8/13/2008 11:39:38 AM
Event ID: 7011
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: BMA-HYPERV-02.xxxx.xx

Description:
A timeout (30000 milliseconds) was reached while waiting for a transaction response from the vmms service.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Service Control Manager" Guid="{555908D1-A6D7-4695-8E1E-26931D2012F4}" EventSourceName="Service Control Manager" />
<EventID Qualifiers="49152">7011</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2008-08-13T14:39:38.000Z" />
<EventRecordID>13345</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>BMA-HYPERV-02.xxxxx.xxx</Computer>
<Security />
</System>
<EventData>
<Data Name="param1">30000</Data>
<Data Name="param2">vmms</Data>
</EventData>
</Event>





Log Name: Application
Source: Application Error
Date: 8/13/2008 11:42:31 AM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: BMA-HYPERV-02.xxxx.xxx

Description:
Faulting application vmms.exe, version 6.0.6001.18016, time stamp 0x484fc22e, faulting module ntdll.dll, version 6.0.6001.18000, time stamp 0x4791adec, exception code 0xc0000374, fault offset 0x00000000000a6e97, process id 0x156c, application start time 0x01c8fd528d23eb06.

[guest]

I have found that if I do not use the same NIC for the server communication as for the HyperV virtual NIC, that I do not get this error and instability of hyperV.



I have gone back and reconfigure all hyperV servers and they all no longer have this issue.



In summay I now have one NIC for server to domain communication. I then configure another NIC or NICs for HyperV virtual NICs. If one of these NICs is on the same network as the server nic, I simply do not register this nic on DNS. I also remove all protocols from these vitual NICs, simply IP.



I am not sure if this is the correct resolution but has helped me.

[guest]

Guys, I have a solution for us. I got in contact with the product group and provided this the logs i captuer above for Traces, and it turns out that the issue is a Heap corruption.

The problem is a known bug and is fixed in Windows 2008 SP2.

The problem is due to having a virutal machine configured with a SCSI adapter that does not have a drive attached to it. I had virtual machines with this configuration. since removing the un used SCSI adapter, my Hyper V service does not stop and restart.

So go thru each VM and remove any SCSI adapters that does not have a drive associated with it.

Hope this helps you guys.

[guest]

You sorta saved my bacon! My VM Management Service was crashing every two minutes then recovering, crashing, etc.. Quite harrowing.. The VMs stayed running but I could not figure out why it was happing.. I had one VM out of about 20 on this host that had a SCSI without a VHD attached. Shut it down, removed it, started it back up and BOOM! No more crashes... ahhhh... the sweet smell of victory. Thanks again Richard for posting this!

[guest]

I can confirm that this solution helped me.
VMMS was crashing just about every time I rebooted the host servers, but would eventually start on its own a random amount of time later. Went into all my VMs, removed SCSI adapters where they were not in use, and voila, VMMS hasn't crashed since.

[guest]

Thanks, this appears to have fixed one of the issues we have with a legacy SP1 2008 box. That's the problem with so much virtualisation and no live migration, very difficult to find patch time (particularly due to fear that it won't come back up again).

Incidentally just to add to this, we did not encounter the problem until using SCVMM 2008 R2 to connect to the box as a managing host. When connected, the management service was going up and down every couple of minutes. When the host was removed from the console, the crashing stopped.

Since removing the erroneous SCSI adapter on one of the machines we can now use SCVMM just fine.

Just adding that to the mix to assist with google searches on the problem :-)

[guest]

Me too. I Added SCVMM to my Hyper-V mix. I then added my first "Virtual Machine Host" in SCVMM "Hosts". It happened to be a Hyper-V 2008 R1 machine. Next thing I know VMMS.EXE (Virtual Machine Manager Service) was up and down every couple of minutes. It took forever to find this blog as the solution.

Removed one SCSI adapter with no disk on it in one VM that was off but got started by SCVMM... so far no problem ... knock-on-wood.

Below for Google/Bing Searches:

7034 The Hyper-V Virtual Machine Management service terminated unexpectedly. It has done this x time(s).

7031 The Hyper-V Virtual Machine Management service terminated unexpectedly. It has done this x time(s). The following corrective action will be taken in 240000 milliseconds: Restart the service.

1000 Faulting application vmms.exe,

SCVMM Error (12704) Unable to connect to the Hyper-V service on the server