Forum Discussion
Anyone experiencing session hosts becoming unavailable at random
Since the end of last week we have had three occasions where one of the session hosts randomly becomes unavailable. This happened in two separate AVD environments.
- Users get kicked out of their session and cannot reconnect.
- The user sessions are still marked as Active/Disconnected according to the Azure portal.
- We cannot RDP to the session host through the internal network.
After we shutdown and reboot the session host, everything will work fine again.
We noticed the following notable things:
- There are no event logs generated at all, starting 30-60 min prior to the 'crash'.
- Since the 28th of October Event Viewer is getting spammed by the following warning:
- Microsoft.RDInfra.RDAgent.Service.AgentUpdateStateImpl
- Unexpected last recorded state
- Microsoft.RDInfra.RDAgent.Service.AgentUpdateStateImpl
- The "Remote Desktop Services Infrastructure Agent" has been updated on the 25th of October, to version 1.0.5555.1008
- The "Remote Desktop Services SxS Network Stack" has been updated on the 31st of October, to version 1.0.2208.17300
- This is also the first day that we experienced the problem.
I have yet to find anything on this problem. Is anyone else experiencing this with their AVD environments?
- Is there already some more information about this case? I have the same issue with my AVD-environment.
- This issue has long been resolved so whatever issue you have I'd suggest creating a new topic. As it's probably a totally different issue.
KristofH Great! But i cannot find the solution in this topic...
- I think it's caused by certain redirected client hardware. I turned off USB Redirection and it's looks much better so far....
AndreasJ5325 we also have this issue running Citrix in multi user Win 10 in azure. Out of 50+ session hosts running per day we get 2 or 3 isssues per week. Session host becomes unavailable in Citrix, rdp won’t work. We can still access the machine via unc path to c$ and get to services remotely. The machine isn’t completely dead but all sessions freeze on it. Event logs usually show a terminal services failure svchost. I have been working with MS on a ticket for a few weeks with logs/memory dumps. the host needs to be rebooted to bring it back to a working state. It could be down to certain we redirect as we also usually see audiobuilder service issues around the same time but that could just be because termservice is dead. The issue is completely random though intermittent we went a whole month between dec and jan with no issues and then started up again mid January. We’ve had these issues since November.
- Hi Dmvinay85, We had similar issue yesterday, but I'm not sure same problem on this post as we had that previously and seems different.
Please see this post: https://techcommunity.microsoft.com/t5/azure-virtual-desktop/some-session-hosts-became-unavailable-this-morning/m-p/3722261
I also logged a ticket. Today so far so good, but really need to get a handle on the issue.
- Hi, same problem today again. 2 out of 5 Session Hosts went unvailable. I have already created a MS Ticket a month ago, on the first time of the issue. Microsoft said this is not a global problem. This should be the solution they said:
- Remove the session host from the host pool
- Generate a new registration key for the VM
- Reinstall the agent and boot loader
- Restart Session Hosts
I have done all this steps one month ago. But now same problem...
Anyone got a other idea?
Greets Mariomariolener , I am also seeing this event in my logs. I had to restart our VDI instance for a user who was seeing issues, but at the expense of the others who weren't (it's a shared instance). We're using Sophos endpoint instead of Defender, though I'm sure some of the process is still present, even if AV duty has been handed off.
We have not seen this issue arise again yet (AVD in Aus East, Meta data in East US)
note: our SH agent versions is now listed as: 1.0.5739.9800, as normal there in nothing in the whats new KB about it so no idea if this has any fixes/includes new issue What's new in the Azure Virtual Desktop Agent? - Azure | Microsoft Learn
Agent appears to have been updated sometime this week.
- hi guys, glad to see the end of these issues. Just a note that we have had this agent version since Monday morning: 1.0.5555.1010. In the connection logs for a user it has a little more detail, calling it "SessionHostAgentVersion 1.0.5555.1010_hotfixProdR1" which I find interesting.
Not sure if anyone else has this version. Either way it seems stable.
Nothing in the KB about it, but that seems normal to be out of date.
https://learn.microsoft.com/en-us/azure/virtual-desktop/whats-new-agent- Hi Paul, how strange... This is the one we are seeing through the AVD deep insights workbook, we don't see that suffix in the AVD session hosts overview: 1.0.5555.1008_hotfixProdR1
KristofH Yes I witnessed it being installed on Monday morning.
We are in Aus East and have no scheduled updated configured.
- However, the eventlog continues to be flooded with warnings about:
Microsoft.RDInfra.RDAgent.Service.AgentUpdateStateImpl
Unexpected last recorded state
Is this the same for you guys? - I have gotten a hopeful response from Microsoft.
They believe it's been caused by Windows Defender. There has now been an update released that is available to download through Windows Update.
(version 4.18.2210.6)- Indeed, had the same feedback. All our AVD hosts are updated, let's see how the day goes.
- i checked some hosts. here as well updated to 4.18.2210.6. Fingers crossed.
We are running 1.0.5555.1008 on all our AVDs, most are located in UK South and so far no issues.
- We also received first reply from Microsoft through our Distri.
Multiple cases running. We are providing tenant details etc.
@Everyone, please refer to this thread when communicating with distri or microsoft.
We also took different approach from now on, we update to "Known / Microsoft documented" versions.
RDAgent: 1.0.5555.1008
Bootloader: 1.0.3.0
Geneva Agent: 45.3.1
SxSStack: 1.0.2208.17300
After that we remove and rejoin host to pool.
Now running some like this for about 3 hours straight, with decent workload (users connected).
We have our fingers crossed, but this is not how we like to work.- I would also like to launch a request here, because we are thinking further then our tenants or what we can reach or see in Azure.
Can you guyes check what broker-urls your sessionhosts use?
We use West-Europe region and our broker url is always:
https://rdbroker-g-eu-r1.wvd.microsoft.com
(Viewable on session host in registry Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RDInfraAgent)
Tnx
- That manual downgrade is fine for a couple of hosts but not at scale, terrible! Latest news on our ticket: .. because we have cases right now with these symptoms.
So hopefully a fix sometime soon. tomdw Any updates to this from anyone? to be honest I have not seen a repeat since friday (it is now Tues afternoon here in NZ).
Agents still on 1.0.5555.1008 but our 10 session hosts have been rock solid over 4 days ** TOUCH WOOD **
- We still had repeats over the weekend and yesterday (7/11)
We have been reverting RD Agents to version 1.0.4739.1000 because they remain stable and we disabled the scheduled agent updates in the host pool settings.
This docs page was updated today:
https://learn.microsoft.com/en-us/azure/virtual-desktop/whats-new-agent
They now talk about v 1.0.5555.1008, but no word about v 1.0.5555.1200
...
We go for our fix for now.- Thanks mate, good info. perhaps we have just been lucky but a ticking timebomb.
can you please advise the high level method of how you force the older agent?
