Sitecore builds with Visual Studio Online – Part 1: Private NuGet feeds

While I've read plenty of articles in the past about configuring build servers for Sitecore projects, I've not seen much written about using Visual Studio Online. (Or "Team Foundation Services", or whatever it's called this week) Since the company I work for are in the process of transitioning all their development infrastructure to Azure, trying to get hosted builds working in VSO seemed like a userful area of research for me. This is the first part of what will probably turn into a few posts on the topic of how I got to the point of being able to run a hosted build for a Sitecore / TDS / FakeDB solution...

Packages, and where to put them...

To get a build working you're going to need NuGet packages. Most of these are easy, as they can come from the main NuGet.org feed. But some of them are more complicated – the TDS build package and Sitecore references can't be added to public feeds. Historically MyGet has been a good solution for hosting these packages but Visual Studio Online has recently acquired a development feature for hosting private feeds. I don't think this is officially released yet (and I don't think they offer support on it outside of the US) but it does appear to work well enough.

The package management feature that lets you host feeds isn't enabled by default. So once you've logged in to your VSO account you need to add it. Got to the "Package Management" extension's marketplace page and click the "Install" button. Once that completes, you'll have a new "Package" option in the VSO menu:

Clicking that menu gives you the feed management page. From there you can click the "New Feed" button, to create yourself a feed that's private to users of your Visual Studio Online account:

The feed name needs to be valid for to be used as part of a URL, and it doesn't allow spaces. Whether you want to give all of your VSO accounts access or not is up to you, but you do need to allow both team members and build services to add packages, as we're going to need to push some packages up manually.

Once it's created, you'll see the "empty package" information page:

Don't ignore this! I had all sorts of bother with some of the steps below, before realising that there are two separate sorts of credentials involved in accessing your new feed. The default nuget.exe that you (probably) already have doesn't appear to know how to talk to VSO's security. You need to click that "Bundle: NuGet.exe + VSTS Credential Provider" button to get the right version of NuGet if you're going to be pushing packages or managing your feed from the command line.

(Edited to add: Turns out you can get this download at any time from the url https://your-account.pkgs.visualstudio.com/_apis/public/nuget/client/CredentialProviderBundle.zip if you need to)

Ensure you've clicked "unblock" in the file properties for the download and then unzip the resulting file somewhere sensible. Keep the contents for later...

With your feed created, you can now upload packages to your new feed in one of two ways:

You can use Hosted Build to do it...

The first option is that you can create a Visual Studio solution which contains the stuff to package and the appropriate .nuspec files. (Perhaps something like this but you no longer need to configure running NuGet to build the packages in your project file) If you run a cloud build of a solution like that, you can use the special NuGet packaging build tasks to construct packages from any .nuspec files in the solution, and to push them into your feed. In the build definition you need something like:

You'll need to know the "package source URL" for your feed. You can always get that from the feed page's "Connect to feed" button:

You paste that into the source URL for your package publishing build task.

Note that the VS Project you're building here might compile code to make a package, or it could just contain files and DLLs from other sources which you want to put in a package. Yes, sticking binaries into source control isn't great – but it's an approach that does work for some scenarios. It's most useful where you have 3rd party code that you want a build to reference but you don't have its source.

Or you can upload stuff yourself from the command line...

Your second option is to take packages you've obtained from any source (your own build internal build, a third party or whatever) and push them up to the feed manually. You can do this via the command line using the special NuGet.exe you downloaded above.

Again, you need that feed URL from above, and the command to run is:

nuget.exe push -Source https://your-account.pkgs.visualstudio.com/_packaging/your-feed-name/nuget/v3/index.json -ApiKey VSO your-package-name.nupkg

					

Replace the url here with your package source, and the .nupkg file with the name of the file you want to push.

You may get prompted for credentials. You should see a login-form dialog like this:

If you get prompted with text in the command window, then you're running the version of NuGet that doesn't have the VSTS Credential Extension. See above about the importance of downloading it.

Once you're authenticated, NuGet will validate your package and push it up to the cloud if it's acceptable.

Making use of your new feed

If you're building locally in Visual Studio to install and fetch packages from a private feed is pretty easy – there's a whole dialog box for it:

You just add the feed URL you created above as a new source here...

But we're aiming to build in the cloud, where we won't have access to this dialog box. So what do we do? The answer is that you can create a nuget.config file in the root of your solution. This allows you to specify various configuration settings - but the one we're interested in is the "Package Sources" option. You can have whatever other options you need, but the important line is:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
    <add key="YourName Repository" value="https://YourAccount.pkgs.visualstudio.com/_packaging/YourName-Sitecore/nuget/v3/index.json" />
  </packageSources>
</configuration>

					

Where we add the custom repository URL.

With that created and committed to your source control, you can now add any packages from your custom feed or from the main public feed. Local or hosted builds will be able to restore them.

And that's the first part of this odyssey. In part 2 we'll look at getting a solution set up so that the hosted build can generate TDS packages and run FakeDB tests.