Issues with Invoke-Webrequest and IE on servers

I've been doing some work with Release Management in Visual Studio Online recently. Overall it's been a pretty positive experience, but there was one face-palm inducing moment I came across which needs writing down so I don't fall into the same trap next time I have to do this. When you're working with local release agents, you mustn't forget the security settings that your agent's server is configured with...

The scenario

I'd created a release which did a variety of deployment tasks before it tried to start up the instance of Sitecore it had just updated. Initially I set this up with a PowerShell script that just used a web request:

Invoke-WebRequest https://$(Website.HostName)/sitecore

					

I've used that commandlet in lots of scripts on my laptop before, so didn't think anything of adding it to my deployment task.

However, when I ran the deployment, it didn't complete as I expected – the UI showed that it got to my "warm up the site" step and then sat there neither returning an error or moving to the next step.

Retrying the deployment failed at the first step, which tried to download the deployment artefacts to my server, with an error saying that it could not write to the agent's working folder.

I scratched my head over this for some time. I ended up looking at the tasks running on the server, and the directories and files they had locks on. My tool of choice for this is SysInternals Process Explorer. With that you can search for a fragment of a path, and it will tell you which processes have handles referring to those paths:

The result I got on my server was an instance of Internet Explorer running in the background holding a handle to the agent's working folder.

Ending that process allowed a new deployment to start successfully, but again it stopped at the "warm site" task...

The issue a hand

Having scratched my head and looked for issues being reported about the release management system, I tried running the script above on my server myself. And that gave a big clue about what the issue was:

Under the surface, Invoke-WebRequest makes use of Internet Explorer DLLs, and hence has to follow the same security model that IE uses. On a normal copy of Windows that's not much of an issue (hence me not thinking twice about using the commandlet), but most servers have IE's Enhanced Security mode enabled – which causes the warning dialog above.

When the script was run by the deployment agent, the same dialog was being generated – but because the deployment agent is a service running in the background, no UI was being displayed. So the deployment would hang waiting for someone to click "Close" on a dialog that couldn't be seen...

Fixing it...

There is always the option to just disable the Enhanced Security setup for IE. While that will certainly address the issue, I'd not recommend this for two reasons: Firstly lowering security to deal with an issue is never a great idea, and secondly going down that road requires you to remember to make the same change on every server which you choose to deploy to – and you're bound to forget at some point...

The right way to deal with the issue is to fix your script:

The Invoke-WebRequest commandlet has a switch -UseBasicParsing which is supposed to bypass the use of IE code during requests. While this worked for me on my desktop test, I still had some issues with it via release management. However I've been unable to pin down why. So while that switch should fix it, you may find you need to go back to basics and use the standard System.Net.WebClient:

$url = "https://$(Website.HostName)/sitecore"
$wc = New-Object System.Net.WebClient
$result = $wc.DownloadString($url)

					

(Steve Vandenbush has a useful blog post if you want to be able to have a timeout here)

One of those two should solve the problem if you encounter it...

Updated to add: As pointed out by John Rappel on Twitter – if all else fails, you can always make use of other technologies to achieve the same effect – curl being a good example here

Further updated to add: Following on from that, Kamruz Jaman points out that in PowerShell v5 "curl" is configured as an alias for Invoke-Webrequest – so you need to take care what it is you're actually running if you want to fall back to curl.

↑ Back to top