Adding Encryption Support to the SmartInspect Libraries

Update: SmartInspect 3.0 now includes built-in support for log file encryption.

Introduction
Implementation
Usage
Limitations
Closing Notes

Introduction

One of the great features of the SmartInspect libraries is their flexibility. It is very simple to customize and extend the built-in capabilities like shown in several articles on this site. This article shows how to implement a custom protocol on top of the existing file protocol which adds support for encrypting log files. If you haven't already, please see the Using Custom Protocols with the SmartInspect Libraries article for general information on implementing custom protocols.

Implementation

As already mentioned above, we add encryption support by implementing and registering a custom protocol. This is done by deriving from the existing file protocol and adding a bit of custom code. Please note that the examples throughout this article use the C# language. The concept is the same with the Delphi and Java libraries but the implementation differs a bit. Also, the example implementation uses the Rijndael (also known as AES) encryption algorithm with a fixed key length of 128 bit. If other algorithms or key lengths are preferred, the implementation needs to be changed a bit.

At first we need to find a name for our new protocol class. For lack of a better name, let's choose CryptoProtocol:

using Gurock.SmartInspect;

...

public class CryptoProtocol: FileProtocol
{
	protected override string Name
	{
		get { return "crypto"; }
	}
}

By deriving our class from FileProtocol instead of directly from Protocol, all the methods for opening and closing a file and writing log packets are already implemented. The only thing we need to do is to provide a wrapper stream for the underlying file stream which adds the desired encryption functionality. The FileProtocol class provides a special virtual method exactly for this purpose. It's called GetStream and is intended to return such a wrapper stream. We implement this method by wrapping the passed Stream object into a new CryptoStream instance:

using System.IO;
using System.Security.Cryptography;

...

protected override Stream GetStream(Stream stream)
{
	Rijndael rijndael = Rijndael.Create();
	
	return new CryptoStream(
		stream,
		rijndael.CreateEncryptor(),
		CryptoStreamMode.Write
	);
}

What's missing in our custom protocol is the encryption key and initialization vector handling. The above implementation uses a random key and initialization vector for encrypting the log data. That's obviously not very practical since we cannot decrypt the log files afterwards without knowing these two values. We therefore add support for two options to our custom protocol which allow specifying both the encryption key and initialization vector:

protected override bool IsValidOption(string name)
{
	return
		name.Equals("key") ||
		name.Equals("iv") ||
		base.IsValidOption(name);
}

protected override void LoadOptions()
{
	base.LoadOptions();
	this.fKeyOption = GetStringOption("key", null);
	this.fKey = GetHexadecimalBytes(this.fKeyOption);
	this.fIVOption = GetStringOption("iv", null);
	this.fIV = GetHexadecimalBytes(this.fIVOption);
}

protected override void BuildOptions(ConnectionsBuilder builder)
{
	base.BuildOptions(builder);
	builder.AddOption("key", this.fKeyOption);
	builder.AddOption("iv", this.fIVOption);
}

We pass the encryption key and initialization vector as hexadecimal strings to the protocol. The GetHexadecimalBytes method is responsible for converting such a string into a byte array which is the expected data type for the key and initialization vector of the encryption algorithm. Its implementation is omitted here but can be found in the CryptoProtocol.cs source file which can be downloaded at the end of this article. After adding support for the protocol options, the GetStream method is now also changed to use these two options instead of the random values:

protected override Stream GetStream(Stream stream)
{
	if (this.fKey == null || this.fIV == null)
	{
		ThrowException("Invalid key or initialization vector");
	}

	Rijndael rijndael = Rijndael.Create();

	return new CryptoStream(
		stream,
		rijndael.CreateEncryptor(this.fKey, this.fIV), 
		CryptoStreamMode.Write
	);
}

Usage

As outlined in the Using Custom Protocols with the SmartInspect Libraries article, we need to register the new custom protocol first before we can specify it in the connections string. This requires a single line of code only:

/* Register the new crypto protocol class */
ProtocolFactory.RegisterProtocol("crypto", typeof(CryptoProtocol));

Then, after registering the protocol class, we can use the new crypto protocol like any built-in protocol of the SmartInspect libraries. We can specify it in the connections string and optionally even make use of the protocol options which are common to all protocols (like the backlog functionality or log level, for example). A basic example of using the new crypto protocol could thus look like:

/* Build connections string */
ConnectionsBuilder builder = new ConnectionsBuilder();
builder.BeginProtocol("crypto");
builder.AddOption("filename", "log.esil"); /* .esil = encrypted */
builder.AddOption("key", /* Encryption key */ );
builder.AddOption("iv", /* Initialization vector */ );
builder.EndProtocol();

/* Assign connections and log some messages */
SiAuto.Si.Connections = builder.Connections;
SiAuto.Si.Enabled = true;
SiAuto.Main.LogMessage("This message is encrypted!");
SiAuto.Main.LogMessage("This message is encrypted!");
SiAuto.Si.Enabled = false;

This writes two encrypted log messages to a file named "log.esil". The normal file extension for a SmartInspect log file is ".sil" and the change to ".esil" indicates that the generated log file is not a normal SmartInspect log file which can be opened by the SmartInspect Console. Since the SmartInspect Console has no built-in support for decrypting log files, we need to decrypt the log file first before the Console is able to open them. A simple method for decrypting a log file could look like:

using System;
using System.IO;
using System.Security.Cryptography;

...

public void Decrypt(string inputFile, string outputFile,
	byte[] key, byte[] iv)
{
	Stream input = File.OpenRead(inputFile);
	try
	{
		Rijndael rijndael = Rijndael.Create();

		input = new CryptoStream(
			input,
			rijndael.CreateDecryptor(key, iv),
			CryptoStreamMode.Read
		);

		using (Stream output = File.Create(outputFile))
		{
			int n = 0;
			byte[] b = new byte[0x2000];

			while ((n = input.Read(b, 0, b.Length)) > 0)
			{
				output.Write(b, 0, n);
			}
		}
	}
	finally
	{
		input.Close();
	}
}

In practice, one would build a small console or WinForms applications around this method to specify the input and output files as well as the encryption key and initialization vector (for example by passing these values as command line arguments).

Limitations

There are a few limitations that need to be considered when using encryption support like shown in this article. The first limitation is that is no longer safe to use the 'append' file protocol option. The 'append' option normally specifies that the file protocol should append the logging data to an existing file rather than creating a new log file. This is unfortunately not supported when encrypting log files for various reasons.

Like all block cipher algorithms, Rijndael always operates on a block of bytes. These block cipher algorithms can make use of several different so-called cipher modes. Most cipher modes do not encrypt one block independently from the others, but operate in so-called feedback modes, i.e. the encryption of a certain block depends on the previous block. This greatly improves security but obviously also introduces several problems when trying to append logging data to an existing file since it is longer possible to correctly decrypt the generated log files in all cases. Another problem is related to padding. Since a block cipher algorithm can only operate on a block of bytes with a fixed length it can happen that the final block of a log file is padded with additional bits. If an existing log file is opened in append mode, it can happen that these padding bits result in a corrupt SmartInspect log file after decryption. It is therefore advised not to use the 'append' option in combination with encryption.

Another limitation is that opening logs in "live mode" (i.e. opening a log file while it is still in use by an application) can lead to unexpected results. Due to the block nature of the block cipher encryption algorithms it can happen that not all previously logged messages are completely written to a log file. Since with the block ciphers only full blocks of bytes can be written it can happen that the most recent messages of a log file are not yet stored in the underlying file.

Closing Notes

As we have seen, although the SmartInspect libraries do not have built-in support for encrypting logs, adding this functionality ourselves isn't that hard. Encryption support can be very useful or even required if you log confidential data that needs to be present in log files but must not be read by third-parties. If you have any questions about this article, please let us know at support@gurock.com.

CryptoProtocol.cs (2 KB)

Try SmartInspect

Get started in minutes and try SmartInspect free for 30 days.

Try SmartInspect