Adventures in TrueNAS Scale- fix docker containers not connecting to the internet

As part of my migration from my Synology to my custom built server/ NAS hybrid I needed to move all my docker containers, thankfully I had them in docker-compose files and wasn’t creating them through the inbuilt wizard on my Synology. TrueNAS scale runs Kubernetes for it’s applications and I didn’t want to invest time learning Kubernetes… yet.

So I managed to move my docker-compose files and their data across to TrueNAS easily enough, I update the paths in my secrets file and run docker-compose and my containers are up and running, “great, job done” I think to myself. But when are migrations ever that simple?

I try to connect to my containers and notice that they can’t connect to any external systems. After a bit of playing around to see what I had done wrong, I find that iptables are disabled by default in TrueNAS as they run Kubernetes and that that is a completely different networking beast.

So how do we fix it? Simple. SSH into your TrueNAS box with your shell of choice and run “nano /etc/docker/daemon.json” you simply need to change “iptables”: false to “iptables”: true

If you’re using nano hit control + x and Y to save the file. Now we just need to restart docker for this to take effect “systemctl restart docker”.
Now your containers should be able to talk to the internet again.
Note: this file does get overwritten when upgrading so you will need to apply this fix again if you upgrade.

Hiding components from users when they are not logged in. AKA Personalisation and you!

Hiding components from users when they are not logged in. AKA Personalisation and you!
When you first install Sitecore 7.2 this is the page that greets you:
Sitecore 7.2 Home Screen

However for this example, let’s make it so that users that aren’t logged in can’t see the title and body, until they are logged in they shouldn’t be welcomed to our system.
While this is a silly example, the same mechanism can be used to achieve many other requirements, such as show specific content based on the users location, goals that have been set, or perhaps hiding welcome content after they have already used the website.
To make this even better, the rules engine is easily customised such that you can create custom rules based on applicable business requirements such as if the user has a certain attribute in their profile.

  1. Firstly, we need to make sure that analytics are enabled, personalisation is driven through the analytics engine and requires it to be turned on
  2. Log into Sitecore, go to the content editor and select the Home item.
  3. In the menu select Presentation and Detailssc72-presentation-details
  4. Under Sample Layout select editsc72-layout-edit
  5. In the Device Editor select Controls, then Sample Rendering and click Personalize.sc72-device-editor-personalise
  6. In the Personalize the Component window click New Condition
  7. Name the condition “Hide for Anonymous”
  8. Under the “Hide for Anonymous” condition click Editsc72-personalize-the-component-hide-edit
  9. In the rule editor, find the Security rules and click on “where the current user is anonymous” and click ok.
  10. Back on the Personalize the Component window to the right of where we clicked edit, check the “Hide Component” tick box.sc72-personalize-the-component-hide-with-rule
  11. Publish the home item.

If you open an incognito window or clear your cookies and go to the home screen you will get this cold unwelcoming page:
sc72-no-welcome

NOTE: To quote the great Stan Lee “With great power there must also come- great responsibility”, whilst the rules engine allows you to tailor content you must be careful to not over personalise and create content that is never seen because you have too many rules chained together or have catch all rules.

Now go forth, create custom rules and amazing websites.

Sitecore 7.2 Personalisation Buttons Not Showing

Sitecore has the fantastic ability to personalise content for users, however this is not enabled by default.
I recently had to jump into a Sitecore 7.2 solution and for the life of me couldn’t figure out why personalisation options weren’t showing.

sc72 Component No Personalise Button Showing

I finally remembered that for 7.2 personalistion requires analytics (DMS).
If you get to the point where you’re looking at the screenshot above and wondering where Personalise is, just follow these steps:

  1. Download the corresponding DMS version for your version of Sitecore from the SDN.
  2. Extract the DMS download (in my case DMS 7.2 rev.150408)
  3. In SQL Server Management Studio attach the Sitecore.Analytics database.
  4. Copy the 3 .config files into YOURSITECOREWEBSITE\App_Config\Include
  5. Update the connectionstring.config for the analytics database.
  6. <add name="analytics" connectionString="Data Source=localhost;Initial Catalog=Sitecore_analytics;Integrated Security=False;User ID=USERNAME;Password=PASSWORD" />
    

Now if you go to edit a component you will get:
sc72 Component Personalise Button

If you are dealing with a client where they don’t want an Analytics Database to be installed or configured, you can simply add the 3 config files and in Sitecore.Analytics.config change:

<!--  ANALYTICS DISABLE DATABASE
      Disables the database so that no reading or writing to the database occurs.
      Default: false
-->
<setting name="Analytics.DisableDatabase" value="false" />

to:

<!--  ANALYTICS DISABLE DATABASE
      Disables the database so that no reading or writing to the database occurs.
      Default: false
-->
<setting name="Analytics.DisableDatabase" value="true" />

Doing this also means you do not need to add the connection string for the analytics database, but this will mean you will lose the ability to personalise on any historical events such as previous visits or goals and is therefore not recommended.

Sitecore Habitat Deployment

For a recent client, starting a new Sitecore project, we made the decision to base the implementation on Sitecore Habitat.
For the uninitiated, Habitat is a fully functioning “real” Sitecore project that shows how to implement Helix principles.
I would highly recommend anyone that is involved in a Sitecore implementation, that they spend the time to read the Helix documentation, it is a font of knowledge about Sitecore implementations, pitfalls and recommendations for a successful project.

Once the decision was made to emulate the Habitat project for setup, we went upon creating our Visual Studio solution and projects.
Most of the Habitat foundation modules were brought across and re-namespaced to align with the organisation.
The fantastic thing about Habitat is that it provides many useful scripts and tricks for Sitecore development, the gulp files are a great place to look for any Sitecore project as they provide speed boosting features for any development team.

While building the solution however, we ran into the issue of how do we deploy this new solution to any environment that isn’t our local development environment.
Simple, I thought, we can just use the gulpfile-CI.js and run the CI-Do-magic task and our deployments will be taken care of for us!
This however, as you can probably guess, was completely wrong.
If you run this task, all you get is a nicely filled out package.xml file with a list of all the project related items and all the unicorn items listed as items in the script.
It turns out looking at Habitat Github issue 152 Habitat uses some internal tooling to take this file and turn this into something that can be deployed/ installed.
This isn’t really helpful for us though, so let the hacking and slicing begin!

Part 1: Creating the Website output

Normally, I would be using the traditional build script generated from a Visual Studio Online or msbuild with arguments, but the gulpfile-CI already gives us a great starting place for getting our build output.
I started with running the CI-Do-magic task, this does everything we want for creating the build artefacts, except that it removes all the artefacts from the folder after the build has completed. This has to be stopped for the good of the build.

I changed the CI-Do-magic to:

gulp.task("CI-Do-magic", function (callback) {
    runSequence(
        "CI-Clean",
        "CI-Publish",
        "CI-Prepare-Package-Files",
        "CI-Enumerate-Files",
        "CI-Enumerate-Items",
        "CI-Enumerate-Users",
        "CI-Enumerate-Roles",
        callback);
});

Great! Now we have our build output being generated for us to deploy, however I don’t like the idea of it being built into a folder named “temp”.
So let’s fix that, update the CI-Publish task to use a new websiteRoot, In this example I’m using the folder named “website” to correspond with the website folder we use when deploying Sitecore.

The CI-Publish task:

gulp.task("CI-Publish", function (callback) {
    config.websiteRoot = path.resolve("./Website");
    config.buildConfiguration = "Release";
    fs.mkdirSync(config.websiteRoot);
    runSequence(
      "Build-Solution",
      "Publish-Foundation-Projects",
      "Publish-Feature-Projects",
      "Publish-Project-Projects", callback);
});

But now we have a new folder for deploying, so let’s also update the CI-Clean:

gulp.task("CI-Clean", function (callback) {
    rimrafDir.sync(path.resolve("./Website"));
    callback();
});

Awesome, we now have a gulp task creating a release build for us, so all we need to do is have our build tool run:

    Nuget Restore
    NPM Install
    CI-do-Magic

That means we’re done right?
Not yet unfortunately, if this was a simple MVC application, we would be.
However Sitecore has one more important task that needs to be done and that’s migrating content.

Part 2: But how do we Unicorn?

Now for the hard(er) part.
We need a way to migrate new content between environments as part of the deployment.
Time to move the items, I decided following from earlier of having the website code build into a website folder, I’m going to move unicorn files to data\unicorn.
So I updated the CI-Enumerate-Items to simply copy the .yml files to the right location:

gulp.task("CI-Copy-Items", function() {
    return gulp.src("./src/**/serialization/**/*.yml")
        .pipe(gulp.dest('./Data/unicorn/'));
});

I did the same for CI-Enumerate-Users, CI-Enumerate-Roles and removed CI-Enumerate-Items.
To see the complete example look at the gist.

Next I updated the CI-Do-magic to:

gulp.task("CI-Do-magic", function (callback) {
    runSequence(
        "CI-Clean",
        "CI-Publish",
        "CI-Prepare-Package-Files",
        "CI-Copy-Items",
        "CI-Copy-Users",
        "CI-Copy-Roles",
        callback);
});

Now we are deploying the unicorn files to /data/unicorn and our build server needs to take both website and data folder to package as the artefacts.
This still doesn’t get us across the line as just deploying the unicorn files onto the server isn’t enough.
With the current CI-Prepare-Package-Files we are removing the unicorn serialization configs.
So we comment out the line removing the configs:

gulp.task("CI-Prepare-Package-Files", function (callback) {
    var excludeList = [
      config.websiteRoot + "\\bin\\{Sitecore,Lucene,Newtonsoft,System,Microsoft.Web.Infrastructure}*dll",
      config.websiteRoot + "\\compilerconfig.json.defaults",
      config.websiteRoot + "\\packages.config",
      config.websiteRoot + "\\App_Config\\Include\\{Feature,Foundation,Project}\\z.*DevSettings.config",
      "!" + config.websiteRoot + "\\bin\\Sitecore.Support*dll",
      "!" + config.websiteRoot + "\\bin\\Sitecore.{Feature,Foundation,Habitat,Demo,Common}*dll"
    ];
    console.log(excludeList);

    return gulp.src(excludeList, { read: false }).pipe(rimraf({ force: true }));
});

Note, you also need to deploy a config patch with:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <sc.variable name="sourceFolder" value="$(dataFolder)\unicorn" />
  </sitecore>
</configuration>

Now we have the unicorn files getting deployed with Sitecore and we can sync these files in manually.

Lastly, in our deploy pipeline, we took the PowerShell being called by the Sync-Unicorn task in the solution gulpfile.js and piped in the environment being used to call sync.

With this all hooked up we now have a deployment strategy for both our code and any modified Sitecore items.

I have uploaded the completed Gulpfile as a gist.
Hopefully this helps speed up your development!

Sitecore 8.0 EXM 3.1.1 Can’t create new message

For a client, I’ve been having to replicate some issues they are facing with EXM.
When going through the content editor, they can create new messages fine and they turn up in the Email Experience Manager dashboard, however when they use the Create button in Email Experience Manager, nothing happens.

When clicking, creating any of the message types, you would get the templates showing up fine, select any template and click create.
The create button disables, but no further action happens, no page refresh, nothing.

One-Time Message Create Greyed Out

After a long amount of time debugging this issue, it turns out that there is a switch statement in MessageCreationDialogBase.js that is case sensitive and if you have linkManager set with lowercaseUrls=”true” then the case statement fails as the creation type string won’t hit.

I believe this is fixed in a later version of EXM, but for me, the simple fix was to go into the webconfig and change:

   <linkManager defaultProvider="sitecore">
      <providers>
        <clear />
        <add name="sitecore" type="Sitecore.Links.LinkProvider, Sitecore.Kernel" addAspxExtension="false" alwaysIncludeServerUrl="false" encodeNames="true" languageEmbedding="never" languageLocation="filePath" lowercaseUrls="true" shortenUrls="true" useDisplayName="false" />
      </providers>
    </linkManager>

To:

    <linkManager defaultProvider="sitecore">
      <providers>
        <clear />
        <add name="sitecore" type="Sitecore.Links.LinkProvider, Sitecore.Kernel" addAspxExtension="false" alwaysIncludeServerUrl="false" encodeNames="true" languageEmbedding="never" languageLocation="filePath" lowercaseUrls="false" shortenUrls="true" useDisplayName="false" />
      </providers>
    </linkManager>

And voila! Once I had restarted Sitecore, the Email Experience Manager create page would take me to the next screen.

Download A Few Files from Azure Blob Storage

Azure Blob Storage is a great and cheap place to upload a lot of files.

One of the websites I am currently work on, stores all the PDFs that users can download, in an Azure Blob Storage container.

We needed to pull down a large amount of those PDFs to confirm the content in them.

So I wrote the following PowerShell:

$LocalTargetDirectory = "D:\Temp\FilesDownloaded\"
$StorageAccountName = "YourStorageAccountName"
$StorageAccountKey = "YourStorageAccountKey"
$ContainerName = "YourContainerName"
$Blobs = @("file1","file2")

New-Item -Force -ItemType directory -Path $LocalTargetDirectory
$Context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey

foreach($Blob in $Blobs)
{
Get-AzureStorageBlobContent -Blob $Blob -Container $ContainerName -Destination $LocalTargetDirectory -Context $Context
}

To get this to work for you, replace:
LocalTargetDirectory with where you want to download the files to.
StorageAccountName with the name of the Azure Storage Account Name.
StorageAccountKey with the name of the Azure Storage Account Key.
ContainerName with the container that you have the files saved in.
Blobs is an array of the files you want to download from the container.

AAA Testing Snippet

This is a snippet I use quite often and send around.

It simply a framework for getting people to organise there unit tests around the AAA principal of Arrange, Act and Assert.

What does it do? Well not much really, it’s just a useful construct I find that helps split up a unit test, it’s a snippet for MSTEST but can quite easly be turned into an xUnit or NUnit snippet by changing the [TextMethod] mark up.

<?xml version="1.0" encoding="utf-8" ?>
<CodeSnippets xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">
    <CodeSnippet Format="1.0.0">

<Header>
            <Title>AAA Test</Title>
            <Description>Arrange, act, assert test format</Description>
            <Author>James Hirka</Author>
            <Shortcut>aaatest</Shortcut>
            <SnippetTypes>
                <SnippetType>Expansion</SnippetType>
            </SnippetTypes>
        </Header>
        <Snippet>
            <Code Language="csharp"><![CDATA[[TestMethod] 
public void TestMethodName() 
{
 //-----------------------------------------------------------------------------------------------------------
 // Arrange
 //-----------------------------------------------------------------------------------------------------------

 //-----------------------------------------------------------------------------------------------------------
 // Act
 //-----------------------------------------------------------------------------------------------------------

 //-----------------------------------------------------------------------------------------------------------
 // Assert
 //----------------------------------------------------------------------------------------------------------- 

}]]>
            </Code>
        </Snippet>
    </CodeSnippet>
</CodeSnippets>

 

Check a list of urls in PowerShell to see what are valid

Recently, one of my colleagues was asked to check what of our products had images on our website and didn’t.

Being a typical developer, I didn’t like the idea of someone wasting energy on monotonous tasks like that when a machine could do it so much more accurately.

The outline of the code I used was:

$urls = @("http://www.google.com","http://www.google.com.au","http://this.doesnt.exist")

foreach($url in $urls)
{
$HTTP_Request = [System.Net.WebRequest]::Create($url)

Try
{
 $HTTP_Response = $HTTP_Request.GetResponse()
 $HTTP_Status = [int]$HTTP_Response.StatusCode
}
catch [System.Net.WebException]
{
 Write-Host "$url is invalid."
}

$HTTP_Response.Close()
}

The output for the above is: http://this.doesnt.exist is invalid.

This was a great start.
To complete this for the task at hand, I enhanced the code to populate the variable $urls with a list of all our products attached to the base image path.
This reduced the amount of time from hours to minutes to find all the missing images.

NuGet- Failed to initialize the PowerShell host issues

I recently had issues updating some NuGet packages in my solution at work, it was giving me the following error:

“Failed to initialize the PowerShell host. If your PowerShell execution policy setting is set to AllSigned, open the Package Manager Console to initialize the host first.”

nuget fail

There is a really simple fix for this.

1.Click start and type in powershell.
2.Right click “Windows PowerShell (x86)” then click run as administrator
3.Type in: “Set-ExecutionPolicy Unrestricted” (without the quotes) and hit enter.

Finally restart Visual Studio and try adding that NuGet package again.

Easily see all members of an Active Directory Group

Recently I had an issue with SQL Server permissions and someone not being able to log into one of our development machines.

I knew that the instance had read access for everyone in a specific Active Directory Group. After trying to log in as the user and seeing the issue, I decided I better check if she was added to the group or not.

First thing I did was crack open the ever handy AD Explorer and found that searching for the group, (which took a long time in our environment) did not show me who was members.

I then remembered that there was a command line way to check the users.
So I clicked, Start, Run, typed in cmd.exe and in the command prompt  typed:
NET GROUP “group name here” /DOMAIN
net group
The output will show you the group name, the comment about the group and the members of it.