Downloading stuff from dev.sitecore.net

There's been a bit of a theme in my recent posts about scripting stuff, and that continues this week. I've been looking at some ideas for automating tasks for developers recently, and one of the things I was interested in was being able to get stuff downloaded from dev.sitecore.net without having to do it manually. So, here's some PowerShell that can help you with that...

Standing on the shoulders of giants... url copied!

Getting files downloaded from Sitecore is something that SIM already does, so I based my efforts on the code that SIM uses when you ask it to download a particular version of Sitecore.

There are three things that need doing to make this work:

1. Get the user's login credentials for Sitecore's websites
2. Authenticate with the website to get the cookies required for a download
3. Fetch the required file and save it

So:

Getting your download on url copied!

The `Get-Credential` commandlet can be used to prompt for the users details - but it's a bit tedious to have to type them in repeatedly if you've got various downloads to do. One way to deal with that is to cache the credential object to disk with the `Export-Clixml`. According to the docs, the credentials are encrypted using a key from the current user account, so this is reasonably secure. So a function to grab the cached credentials if they exist, or prompt for them otherwise might look like this:

function Fetch-WebsiteCredentials
{
	$file = "dev.creds.xml"

	if(Test-Path ".\\$file")
	{
		$cred = Import-Clixml ".\\$file"
	}
	else
	{
		$cred = Get-Credential -Message "Enter your SDN download credentials:"
		$cred | Export-Clixml ".\\$file"
	}

	return $cred
}

					

Once the script has the user's website credentials, it needs to turn them into the cookies necessary for a download operation to succeed. Based on the code from SIM, that needs two operations. First you need to post the user's credentials as json to an API endpoint to get one cookie, and then you need to make a get request for the dev site homepage to get a second cookie.

So this function can perform those two web requests, and return the two cookies we need:

function Fetch-DownloadAuthentication($cred)
{
    $authUrl = "https://dev.sitecore.net/api/authorization"

    $pwd = $cred.GetNetworkCredential().Password

    $postParams = "{ ""username"":""$($cred.UserName)"", ""password"":""$pwd"" }"

    $authResponse = Invoke-WebRequest -Uri $authUrl -Method Post -ContentType "application/json;charset=UTF-8" -Body $postParams -SessionVariable webSession
    $authCookies = $webSession.Cookies.GetCookies("https://sitecore.net")

    $marketPlaceCookie = $authCookies["marketplace_login"]

    if([String]::IsNullOrWhiteSpace($marketPlaceCookie))
    {
        throw "Credentials appear invalid"
    }

    $devUrl = "https://dev.sitecore.net"

    $devResponse = Invoke-WebRequest -Uri $devUrl -WebSession $webSession
    $devCookies = $webSession.Cookies.GetCookies("https://dev.sitecore.net")

    $sessionCookie = $devCookies["ASP.Net_SessionId"]

    return "$marketPlaceCookie; $sessionCookie"
}

					

So finally the script can use the cookies to download the file we want. In the scripts from the last couple of posts I'd been using the Start-BitsTransfer commandlet for this. But that doesn't appear to allow you to pass cookies for authentication. So I've fallen back to using System.Net.WebClient. While you can write some really simple code to do this, that implementation doesn't give you the nice progress bar that the Bits transfer allows. You don't need that, but I wanted it.

So I did a bit of googling to see how you can implement the progress bar behaviour, and I came across this gist by Dave Wyatt. That implements pretty much what I needed, by attaching the progress bar code to the progress events raised by the WebClient. But there is a minor bug in that gist, which it took me a bit of time to work out. When you run a download with the code as-is, a nice progress bar is displayed, but there's no text visible, despite the value set for the -Activity parameter:

The issue here is that the original code is trying to access the function's parameters inside the Register-ObjectEvent‘s -Action block. It's not immediately obvious, but because of PowerShell scope those variables aren't actually accessible there. What you have to do is use the -MessageData property to pass in a data structure containing the values you want to be able to access:

$data = New-Object psobject -Property @{Uri = $Uri; OutputFile = $OutputFile}

$changed = Register-ObjectEvent -InputObject $webClient -EventName DownloadProgressChanged -MessageData $data -Action {
    Write-Progress -Activity "Downloading $($event.MessageData.Uri)" -Status "To $($event.MessageData.OutputFile)" -PercentComplete $eventArgs.ProgressPercentage
}

					

The $event variable inside the -Action block contains the data you pass in via -MessageData.

There are a couple of other changes needed to the code from the gist. It needs to make use of the cookies that we captured before, and based on a bit of experimentation it needs to make sure it can be stopped gracefully by a Ctrl-C. That's easily achieved by wrapping the main part of the download in a try block, so that the finally part can be run after the download is aborted by the user. And with those changes, the completed download function is:

function Invoke-FileDownload
{
    [CmdletBinding()]
    param (
        [Parameter(Mandatory)]
        [string] $Uri,

        [Parameter(Mandatory)]
        [string] $OutputFile,

        [string] $cookies
    )

    $webClient = New-Object System.Net.WebClient

    if(!([String]::IsNullOrWhiteSpace($cookies)))
    {
        $webClient.Headers.Add([System.Net.HttpRequestHeader]::Cookie, $cookie)
    }

    $data = New-Object psobject -Property @{Uri = $Uri; OutputFile = $OutputFile}

    $changed = Register-ObjectEvent -InputObject $webClient -EventName DownloadProgressChanged -MessageData $data -Action {
        Write-Progress -Activity "Downloading $($event.MessageData.Uri)" -Status "To $($event.MessageData.OutputFile)" -PercentComplete $eventArgs.ProgressPercentage
    }

    try
    {
        $handle = $webClient.DownloadFileAsync($Uri, $PSCmdlet.GetUnresolvedProviderPathFromPSPath($OutputFile))

        while ($webClient.IsBusy)
        {
            Start-Sleep -Milliseconds 10
        }
    }
    finally
    {
        Write-Progress -Activity "Downloading $Uri" -Completed

        Remove-Job $changed -Force
        Get-EventSubscriber | Where SourceObject -eq $webClient | Unregister-Event -Force
    }    
}

					

Those three functions can then be wrapped up in a script to invoke them:

param(
	[string]$url,
	[string]$target
)

$cred = Fetch-WebsiteCredentials
$cookie = Fetch-DownloadAuthentication $cred

Invoke-FileDownload -Uri $url -OutputFile $target -Cookies $cookie

					

If that script is named download.ps1 then you can download a copy of Sitecore 8.2 by running:

.\download.ps1 "https://dev.sitecore.net/~/media/82216B3D1FE245CFADC8B2C758E510C5.ashx" "sitecore.zip"

					

And you get a working progress bar:

followed by a downloaded file...

As before, the full code for this is available as a gist if you want to make use of it in your work.

↑ Back to top