Shipping custom logs from your v10 containers

My work on a container-based v10.0 project keeps raising interesting challenges – things that don’t work quite the same way in Docker or Kubernetes, compared to the old world of "bare metal" installs of Sitecore. Custom log files are an example here...

I realised recently that we had some custom code in our solution, whose log data was not appearing in the Docker streamed logs. (And hence wasn’t visible in the Kubernetes container log output either) After a bit of digging I released this was because the developers had quite sensibly moved these log entries to a separate log file. Turns out that the log streaming does not pick up all logs – you have to be specific:

How does it work?

Inside your containers, a tool called LogMonitor.exe is running. It uses disk monitoring (and other techniques, potentially) to monitor log data, and when it sees changes it streams these to “standard output".

Your container orchestrator is picking up that streamed data, and sending it to whatever console or log aggregation framework your using. That might be AppInsights or Prometheus from Kubernetes in production, and it might be the Visual Studio containers window or a console under Docker.

By default, this is set up to stream IIS logs and the main Sitecore log. So what do we do if we want other log files?

Extending this

If you look inside your Sitecore CM or CD containers, LogMonitor sits in a folder under the root:

By default it’s started by the entrypoint script for the container, and it reads is configuration settings from that json file sitting next to it.

It's default state is:

{
    "LogConfig": {
      "sources": [
        {
          "type": "EventLog",
          "startAtOldestRecord": false,
          "eventFormatMultiLine": false,
          "channels": [
            {
              "name": "system",
              "level": "Error"
            }
          ]
        },
        {
          "type": "File",
          "directory": "c:\\inetpub\\logs",
          "filter": "*.log",
          "includeSubdirectories": true
        },
        {
          "type": "File",
          "directory": "c:\\inetpub\\wwwroot\\App_data\\logs",
          "filter": "log.*.txt",
          "includeSubdirectories": false
        }
      ]
    }
  }
}

					

So to add extra log files we can modify this. There’s some documentation available on GitHub because it’s not just disk files you can process here. But I’m only interested in other Sitecore logs for the moment...

So we could change the existing pattern there to "*.txt" to bring in everything from the logs folder. But I wanted to be a bit more subtle. I tried adding an extra source which would cover just the files I care about:

{
    "type": "File",
    "directory": "c:\\inetpub\\wwwroot\\App_data\\logs",
    "filter": "MyCustomFile.log.*.txt",
    "includeSubdirectories": false
}

					

That block gets appended after the file source for the main Sitecore logs, as part of the array for "sources".

But to make this work, we need to get this into our container images. How do we manage that? Extending the base image build...

The example container setup for Sitecore includes a DockerFile for extending the base CM container. (You may want to do this on other roles too, of course – similar patterns apply)

We can extend that file to overwrite the default config with out version. Use the VS container browser shown above to find the file and right-click "open" it to see the contents. You can then modify this, and save a copy of your changes in your docker image build folder.

You can then add a simple "copy" operation into your Dockerfile:

# Update Log Monitor config
COPY CustomLogMonitorConfig.json C:\LogMonitor\LogMonitorConfig.json

					

That will ensure your modified file will end up in the container, ready to run on startup. Note that you will need to rebuild your container images for that to take effect.

But once that’s done, the extra data should appear in your streamed logs:

↑ Back to top