Another package install performance boost

I came across an interesting approach for improving the performance of automated package installs the other day. In the space of one day of internet reading I saw the Sitecore.Data.BulkUpdateContext class mentioned in a blog post and it came up in MVP forum discussions. In both cases, it was mentioned in the context of making item operations faster. Hence it seemed like an interesting thing to investigate, to see what sort of difference it would make to my previous work.

How do you use it?

The class is simple to use - it's an IDisposable which can be placed into a using block wrapped around the rest of the logic for installing a package. The Page_Load() code from my previous blog post‘s package install endpoint can be updated simply by wrapping this class around the existing set of using statements:

public void Page_Load(object sender, EventArgs e)
{
    var files = WebUtil.GetQueryString("modules").Split('|');
    if (files.Length == 0)
    {
        Response.Write("No Modules specified");
        return;
    }
    Sitecore.Context.SetActiveSite("shell");

    using(new BulkUpdateContext())
    {
        using (new SecurityDisabler())
        {
            using (new ProxyDisabler())
            {
                using (new SyncOperationContext())
                {
                    foreach (var file in files)
                    {
                        Install(Path.Combine(Sitecore.Shell.Applications.Install.ApplicationContext.PackagePath, file));
                        Response.Write("Installed Package: " + file + "<br>");
                    }
                }
            }
        }
    }
}

					

What difference does it make?

The simplest way to work out what difference this code change makes is to measure the time taken to install some packages with each approach.

In PowerShell we can time code with the .Net Stopwatch class:

$sw = [Diagnostics.Stopwatch]::StartNew()

# Put the code to time here...

$sw.Stop()
Write-Host "Elapsed time: $($sw.Elapsed)"

					

If we wrap that around the PowerShell code which is calling the ASP.Net code above, we can measure the elapsed time for installing a set of packages:

function Add-Packages() {
    $siteName = Get-ConfigParam "InstanceName"
    $sitecoreFolder = "C:\Inetpub\wwwroot\$($siteName)\Data\packages"   
    
    $sw = [Diagnostics.Stopwatch]::StartNew()

    foreach($module in Get-ConfigModules) {
        # make querystring
        $query = (Split-Path -Path $module -Leaf)
        
        # Copy the package
        Write-Host "Copying package $query to Sitecore package folder..."
        Copy-Item $module $sitecoreFolder -force
        
        Write-Host "Calling package upload tool for $query"
    
        # call tool
        $url = "http://$siteName/PackageDeploy.aspx?modules=$query"
        $result = Invoke-WebRequest -Uri $url -TimeoutSec 600 -OutFile ".\$siteName-PackageResponse-$query.log" -PassThru
        
        if($result.StatusCode -ne 200) {
            Write-Host "StatusCode: $($result.StatusCode)"
            throw "Package install failed for $($query)"
        }
        
        # Removing installed package file
        $file = Join-Path $sitecoreFolder $query
        Remove-Item $file -Force
    }

    $sw.Stop()
    Write-Host "Package install time: $($sw.Elapsed)"

    write-host "Package tool done..."
}

					

If I run this as part of a scripted install of a new instance of Sitecore 6.6 and adding SPEAK and Email Campaign Manager packages then, an average of 10 runs gives me:

• Without `BulkUpdateContext`: 1 minute 25 seconds
• With `BulkUpdateContext`: 52 seconds

Repeating just the package installs a further 10 times on top of the already-installed instance gives an average elapsed time of:

• Without `BulkUpdateContext`: 48 seconds
• With `BulkUpdateContext`: 29 seconds

So while that's not a statistically rigorous analysis, it does suggest that it's well worth adding that extra line of code to improve the performance when installing big packages, or multiple packages...

---

Edited to add: As pointed out by @kamsar on twitter, the underlying code for BulkUpdateContext disables the publishing queue while the install proceeds - so you need to remember to publish things yourself after you've finished with your package installs.

Another point I failed to add in my original write-up was that you'll want to update your search indexes after using this approach. (Thanks to make use of the post I wrote on index builds to address this.