A drink from the Gulp firehose

Having spent a bit of time recently looking at some of the new stuff included in the tools and frameworks for ASP.Net Core 1.0 and Sitecore's Habitat solution, one of the things that caught my eye is the Gulp task runner. So after a few days of poking around, here's a basic introduction for anyone else considering it for their Sitecore work.

What is Gulp?

It's a JavaScript-based task framework that can be used to automate recurring actions in your build process. Sort of like an alternative take on MSBuild – but not tied to the .Net stack. You write JavaScript functions which perform tasks for you. The code you write can be run manually, or can be tied to build-time events like "on clean" or "after build completes". The framework has been available to use via Node.js and command line tools for some time, but with Visual Studio 2015 there's now built-in support for using it with your .Net projects. Hence you can make use of it in Sitecore development if you think it's useful.

Gulp isn't the only tool you can use to achieve things like this – we've been able to use MSBuild to trigger code when build events occur for ages. However I've never really like having to unload my project to edit the .csproj file to modify this. Having the Gulp tasks visible in a window inside Visual Studio seems like a nicer solution to me.

It's also worth pointing out that there are other JavaScript based approaches to this that you can use as well as or instead of Gulp. Visual Studio has also added support for Grunt – which is just similarly named enough to be confusing I find. The difference seems to be that Grunt is based around a model where you use large plugins which automate a complete task – say "compile my LESS files, minify them and add a license header" – and they do all of this in one operation. Gulp has a more pipeline-based model, where each plugin does a small task, and you compose them together to solve more complex problems. So you'd find plugins for "compile LESS", "minify CSS" and "prefix a file with text". To my mind, that makes Gulp the more flexible and useful tool – but you're free to choose the one which best suits your processes...

How do you set up Gulp in a project?

In Visual Studio solutions it's pretty simple:

The first thing to do is configure your project to depend on Gulp. To do that you need to add a configuration file for the Node.js Package Manager (NPM). This is sort of like "NuGet for Node" – it lets Visual Studio automatically download any Node-based code that your solution requires at build time. Helpfully there's a template for this package.json file included in VS2015:

You need to add one line to the default content of this file to get going - to tell NPM that it needs to ensure a recent version of Gulp is installed:

{
  "version": "1.0.0",
  "name": "ASP.NET",
  "private": true,
  "devDependencies": {
    "gulp": "^3.9.0"
  }
}

					

Note that these are described as "developer dependencies" – they're things related to your build process for your code, and not to the execution of it. In a real project you'd probably end up with other things specified here, as this is also the pattern you use for installing any plugins for Gulp which you want to make use of in your build tasks.

Once you've configured NPM you can save this file to make sure all these extras get downloaded. (Note you can also right-click the file in the Solution Explorer to manually trigger a download) Then you need to add an instance of gulpfile.js to your project. This is the file that Visual Studio is going to find your build task code in, and again there's a helpful template to use:

The default file contains two things:

var gulp = require('gulp');

gulp.task('default', function () {
    // place code for your default task here
});

					

The first line tells the Node to get a reference to the Gulp framework. Think of it like a using statement. The remaining lines are the outline of a Gulp Task. You can have as many of these as you want, and each one gets a name that will be displayed in the Visual Studio "Task Runner Explorer" window:

Here you can see the list of your tasks on the left. And you can right-click the name of a task to bind it to the events shown in the right hand pane.

Note: I've found that when you're first creating your Gulp file and adding tasks or dependencies, this window can be a bit slow to update to reflect new data. Sometimes this is down to a missing dependency, and can be solved by doing a build before refreshing the Task Runner window. Sometimes it's an error processing your JavaScript, which should show up in the right pane of the Task Runner window. But occasionally it gets confused enough that you need to re-load your project (or even re-start Visual Studio) to get it to recognise what you've done. Hopefully these issues will be resolved in future patches to Visual Studio.

How can Gulp help with your Sitecore projects?

I'm sure there are lots of ways. The most obvious is as a mechanism for doing things like compiling LESS or SASS files to CSS, minifying and bundling your CSS or JavaScript files and similar web-development related tasks. This sort of application for Gulp is common to all web projects, and hence is well documented elsewhere, so I'll skip over that area and let you Google those features and plugins.

But looking at a few of the simple issues relating to Sitecore development:

Copy files from Sitecore to your solution

One example is the need to grab Sitecore binaries from a development instance to be the reference DLLs for your build. You shouldn't commit these to source control, so having a way for developers to fetch them when they first clone a solution is helpful. Copying the files out of your development Sitecore instance is one of many approaches to solving this problem. Copying the files is simple, but it raises the question of "where are the files going to and from?". And that means you need somewhere to put configuration.

Gulp lets you specify config data via JavaScript as well. Just add another .js file to your project named something sensible, and put your configuration data into a "module". For example if we need configuration settings for the website's root folder and it's binaries folder, we could use something like:

module.exports = function () {
    var config = {
        websiteRoot: "C:\\inetpub\\wwwroot\\DevTest\\Website",
        sitecoreLibraries: "C:\\inetpub\\wwwroot\\DevTest\\Website\\bin"
    }
    return config;
}

					

And then you can import this into your Gulp file by adding the following after the require statement for Gulp itself:

var gulp = require('gulp');
var config = require("./gulp-config.js")();

					

Then you can use config.websiteRoot (or whatever other config declarations you make) in your tasks.

You might find your developer spider-sense tingling when you look at that configuration though: We've got basically the same path in two places. That makes me want to try and factor out the common bits.

When you declare a set of name-value pairs for config like that, you can't use a reference to another key in the same dictionary. However you can declare more than one dictionary. So we could refactor that to something like:

module.exports = function () {
    var paths = {
        website: "C:\\inetpub\\wwwroot\\DevTest\\Website"
    }

    var config = {
        websiteRoot: paths.website,
        sitecoreLibraries: paths.website + "\\bin",
    }
    return config;
}

					

Now the root path is defined in a single place, no matter how many configuration values you need that include it.

Most operations in Gulp follow a similar pattern: First you call gulp.src() to specify the file (or files) you want to be the input to your operation. You then call the .pipe() method as many times as you need to perform any transformations required on the data. The final .pipe() can then use gulp.dest() to write out the results to new files.

So to grab a copy of the Sitecore.Kernel.dll from your website and store it in your project's lib folder, you can use the following:

gulp.task('Copy-Sitecore-References', function () {
    gulp.src(config.sitecoreLibraries + '\\Sitecore.Kernel.dll')
        .pipe(gulp.dest('.\\lib'));
});

					

We've given the task a meaningful name, and we've told it the source and piped it straight to the destination to create a copy. And once we've refreshed the Task Runner window this action will be available to right-click and run whenever we need to.

Copying files from your project to Sitecore

Taking this a bit further, you might also want to use it to copy things in the other direction. For example you might want to ensure the binaries required for Unicorn that are provided by NuGet can be deployed to your website when required.

You can just write multiple copy operations when you have more than one thing to transfer, but that's probably not the best pattern. Copying multiple files can use wildcards (for both files and folders), and you can also pass a set of paths. Since there's no common patern for the naming of the binaries in Unicorn, we can give Gulp a list. Each item in the list will get processed in turn. We can add a list of the required files to our configuration data:

module.exports = function () {
    var config = {
        // other configuration properties
        unicornFiles: [
            '.\\bin\\Kamsar.WebConsole.*',
            '.\\bin\\MicroCHAP.*',
            '.\\bin\\Rainbow.*',
            '.\\bin\\Rainbow.Storage.Sc.*',
            '.\\bin\\Rainbow.Storage.Yaml.*',
            '.\\bin\\Unicorn.*'
        ]
    }
    return config;
}

					

(Yes, you could probably reduce this list a bit by using more wildcards – you're free to optimise if you feel like it) And that allows for another single-line copy operation to transfer all the relevant files:

gulp.task('Copy-Unicorn-Binaries', function () {
    gulp.src(config.unicornFiles)
        .pipe(gulp.dest(config.websiteRoot + '\\bin'));
});

					

Note that you can't use the gulp.dest() call to rename the file. The path you specify there is always treated as a folder. If you need to rename things as you copy them, you need to include a .pipe() call to a renaming plugin - which might look something like this:

var rename = require('gulp-rename');

gulp.task('rename-file', function () {
    gulp.src('.\\old-file-name.xml')
        .pipe(rename('.\\new-file-name.xml'))
        .pipe(gulp.dest(config.websiteRoot));
});

					

(which of course requires you to add the plugin to your package.json developer dependencies list)

Running tasks on build events

Taking this concept a bit further, what about the need to copy files automatically after each build? A good example for this is deploying all your config patch files from the web project to your Sitecore instance. Creating a copy operation for this is just as simple as above:

gulp.task('Copy-Config-Patches', function () {
    gulp.src('.\\App_Config\\include\\**\\*.config')
        .pipe(gulp.dest(config.websiteRoot + '\\App_Config\\include'));
});

					

All that's different here is the use of wildcards. The ** wildcard specifies "any directory tree" so the source for this copy is any .config file that lives anywhere under our project's App_Config/include folder.

But to make this happen at build time, we need to bind this task to the "after build" event. That's done by right-clicking the task in the Task Runner window and choosing the "Bindings / After Build" option.

Doing this will add an extra comment to the top of your Gulp file:

/// <binding AfterBuild='Copy-Config-Patches' />

					

That fragment of XML is the binding required for Visual Studio to run your task automatically. Note that the binding uses the task name, so if you rename tasks you will need to re-do the binding. You're free to bind any set of tasks to events, but you can also specify dependencies in the definition of your tasks if you need to ensure that certain tasks run in a specific order. To do that you just include an array of names of tasks which must have run first in your definition:

gulp.task('Task-With-Dependencies', ['Task-1', 'Task-2'], function () {
    // code here runs after Task-1 and Task-2
});

					

Modifying files as they're transferred

None of the examples so far show the power of piping data between Gulp plugins, but a simple example of how that can work should be modifying XML documents as they're deployed. It's not uncommon to want to modify the odd setting in a config file at build time. You can do it using XDT in Visual Studio with the Slow Cheetah plugin, but it's another place where Gulp provides an alternative.

When I sat down to try and create a quick example of this I found two different XML-modifying plugins for Gulp. gulp-edit-xml provides a "map the XML tree to JavaScript objects" approach which seems frankly too complicated for most Sitecore scenarios. gulp-xml-editor has a much more useful xpath-based syntax, but (so far) I cannot get this plugin to run. It relies on another package called node-gyp which appears to be used to compile native code for the Node.js framework. In theory this involves installing Python and making various configuration settings to ensure Node can find appropriate compilers - which, frankly, is way too much effort for a simple task like this.

If it didn't require all the messing about and other installs, it should be possible to write simple replacements in a fairly sensible manner:

gulp.task('xml-transform', function () {
    gulp.src('.\\example.xml')
        .pipe(xeditor([
            { path: '/settings/setting[@name="changeMe"]', text: 'New text value' }
        ]))
        .pipe(gulp.dest(config.websiteRoot));
});

					

As with all the Gulp plugins, the source data gets piped in, and you define a set of name/value pairs for the XPath to find the element(s) to change and the new text to write into whatever the XPath selects.

I'll keep an eye out to see if anyone creates a more "windows friendly" plugin for patching XML. But given the complexity of making this work, I think I will be sticking to XDT for this requirement in the meantime...

↑ Back to top