Sitecore Habitat Deployment

For a recent client, starting a new Sitecore project, we made the decision to base the implementation on Sitecore Habitat.
For the uninitiated, Habitat is a fully functioning “real” Sitecore project that shows how to implement Helix principles.
I would highly recommend anyone that is involved in a Sitecore implementation, that they spend the time to read the Helix documentation, it is a font of knowledge about Sitecore implementations, pitfalls and recommendations for a successful project.

Once the decision was made to emulate the Habitat project for setup, we went upon creating our Visual Studio solution and projects.
Most of the Habitat foundation modules were brought across and re-namespaced to align with the organisation.
The fantastic thing about Habitat is that it provides many useful scripts and tricks for Sitecore development, the gulp files are a great place to look for any Sitecore project as they provide speed boosting features for any development team.

While building the solution however, we ran into the issue of how do we deploy this new solution to any environment that isn’t our local development environment.
Simple, I thought, we can just use the gulpfile-CI.js and run the CI-Do-magic task and our deployments will be taken care of for us!
This however, as you can probably guess, was completely wrong.
If you run this task, all you get is a nicely filled out package.xml file with a list of all the project related items and all the unicorn items listed as items in the script.
It turns out looking at Habitat Github issue 152 Habitat uses some internal tooling to take this file and turn this into something that can be deployed/ installed.
This isn’t really helpful for us though, so let the hacking and slicing begin!

Part 1: Creating the Website output

Normally, I would be using the traditional build script generated from a Visual Studio Online or msbuild with arguments, but the gulpfile-CI already gives us a great starting place for getting our build output.
I started with running the CI-Do-magic task, this does everything we want for creating the build artefacts, except that it removes all the artefacts from the folder after the build has completed. This has to be stopped for the good of the build.

I changed the CI-Do-magic to:

gulp.task("CI-Do-magic", function (callback) {
    runSequence(
        "CI-Clean",
        "CI-Publish",
        "CI-Prepare-Package-Files",
        "CI-Enumerate-Files",
        "CI-Enumerate-Items",
        "CI-Enumerate-Users",
        "CI-Enumerate-Roles",
        callback);
});

Great! Now we have our build output being generated for us to deploy, however I don’t like the idea of it being built into a folder named “temp”.
So let’s fix that, update the CI-Publish task to use a new websiteRoot, In this example I’m using the folder named “website” to correspond with the website folder we use when deploying Sitecore.

The CI-Publish task:

gulp.task("CI-Publish", function (callback) {
    config.websiteRoot = path.resolve("./Website");
    config.buildConfiguration = "Release";
    fs.mkdirSync(config.websiteRoot);
    runSequence(
      "Build-Solution",
      "Publish-Foundation-Projects",
      "Publish-Feature-Projects",
      "Publish-Project-Projects", callback);
});

But now we have a new folder for deploying, so let’s also update the CI-Clean:

gulp.task("CI-Clean", function (callback) {
    rimrafDir.sync(path.resolve("./Website"));
    callback();
});

Awesome, we now have a gulp task creating a release build for us, so all we need to do is have our build tool run:

    Nuget Restore
    NPM Install
    CI-do-Magic

That means we’re done right?
Not yet unfortunately, if this was a simple MVC application, we would be.
However Sitecore has one more important task that needs to be done and that’s migrating content.

Part 2: But how do we Unicorn?

Now for the hard(er) part.
We need a way to migrate new content between environments as part of the deployment.
Time to move the items, I decided following from earlier of having the website code build into a website folder, I’m going to move unicorn files to data\unicorn.
So I updated the CI-Enumerate-Items to simply copy the .yml files to the right location:

gulp.task("CI-Copy-Items", function() {
    return gulp.src("./src/**/serialization/**/*.yml")
        .pipe(gulp.dest('./Data/unicorn/'));
});

I did the same for CI-Enumerate-Users, CI-Enumerate-Roles and removed CI-Enumerate-Items.
To see the complete example look at the gist.

Next I updated the CI-Do-magic to:

gulp.task("CI-Do-magic", function (callback) {
    runSequence(
        "CI-Clean",
        "CI-Publish",
        "CI-Prepare-Package-Files",
        "CI-Copy-Items",
        "CI-Copy-Users",
        "CI-Copy-Roles",
        callback);
});

Now we are deploying the unicorn files to /data/unicorn and our build server needs to take both website and data folder to package as the artefacts.
This still doesn’t get us across the line as just deploying the unicorn files onto the server isn’t enough.
With the current CI-Prepare-Package-Files we are removing the unicorn serialization configs.
So we comment out the line removing the configs:

gulp.task("CI-Prepare-Package-Files", function (callback) {
    var excludeList = [
      config.websiteRoot + "\\bin\\{Sitecore,Lucene,Newtonsoft,System,Microsoft.Web.Infrastructure}*dll",
      config.websiteRoot + "\\compilerconfig.json.defaults",
      config.websiteRoot + "\\packages.config",
      config.websiteRoot + "\\App_Config\\Include\\{Feature,Foundation,Project}\\z.*DevSettings.config",
      "!" + config.websiteRoot + "\\bin\\Sitecore.Support*dll",
      "!" + config.websiteRoot + "\\bin\\Sitecore.{Feature,Foundation,Habitat,Demo,Common}*dll"
    ];
    console.log(excludeList);

    return gulp.src(excludeList, { read: false }).pipe(rimraf({ force: true }));
});

Note, you also need to deploy a config patch with:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <sc.variable name="sourceFolder" value="$(dataFolder)\unicorn" />
  </sitecore>
</configuration>

Now we have the unicorn files getting deployed with Sitecore and we can sync these files in manually.

Lastly, in our deploy pipeline, we took the PowerShell being called by the Sync-Unicorn task in the solution gulpfile.js and piped in the environment being used to call sync.

With this all hooked up we now have a deployment strategy for both our code and any modified Sitecore items.

I have uploaded the completed Gulpfile as a gist.
Hopefully this helps speed up your development!