Web Rarely

Background

I created a rudimentary mod kit for the game Battle Brothers that allows editing the encrypted Squirrel scripts (.cnut files) within the game. Since almost the entire game is implemented in script, this allows tremendous flexibility for modding. It started when I wanted to know which character backgrounds were best to hire in a "value for money" sense. There was a spreadsheet that the community had painstakingly created over the years based on hundreds of manual tests showing the typical stat ranges of various backgrounds, but it was invalidated by the Beasts & Exploration DLC which rebalanced various parts of the game. I wished I could just take a look at how characters are generated, but like many people trying to peek under the hood I soon saw that the scripts are shipped only in a compiled form, and not only that but they appear to be corrupted or encrypted.

Accessing the files

Before we can start messing with the game data files, we first have to know where they are. If your Battle Brothers installation directory is bbros, then the data files are stored within bbros/data/. Within that directory you'll see files named like data_001.dat. These are actually .zip files, and if you rename it to data_001.zip you'll be able to open it. Images, sound effects, etc., are not encrypted, so you can examine them. (Don't be tempted to modify them inside the .zip file, though. See below for how to make changes.) The scripts are the .cnut files, but if you open them they just look like garbage.

Encryption and decryption

The "corruption" in the compiled scripts is actually an unusual encryption algorithm that encrypts only bits and pieces of the files, making them look like normal compiled scripts at a glance, but preventing them from actually working if you try to run or examine them. After painstakingly reverse engineering parts of the game, I was able to decipher the encryption algorithm. I spent quite a long time discovering how it worked and then a while longer translating the cryptic instructions into C code that I could run. Only after I was done did I realize that one of the constants in the algorithm is well-known, and that the core cipher is XXTEA.

But like I said, only parts of the file are encrypted. The encryption algorithm is unusual in that the format of a Battle Brothers encrypted .cnut file cannot be described in a stand-alone way since it's intimately tied to the internal implementation details of the Squirrel script engine. Specifically, the Squirrel API calls to parse a compiled script take a reader callback function, which gets called with a buffer and a number of requested bytes, and the function is supposed to read those bytes from the application's storage. Basically, it's a byte-oriented stream interface. Now, where it gets tricky is that the game only decrypts runs of 8 or more bytes requested through that stream interface, and it doesn't keep any state between invocations. If the Squirrel engine changed to break one read into two, or to combine two reads into one, or even just changed the size of any encrypted read, the file format would effectively change. So, to be safe, you should use the same version of Squirrel as the game. At the time of writing, this is 3.0.4, but I've also tested with 3.0.7. Also, only multiples of 4 bytes are encrypted, so a 15-byte read has the first 12 bytes encrypted and the last 3 bytes unencrypted.

So, without further ado, here's how you'd go about decrypting a Battle Brothers .cnut file:

// the standard XXTEA block cipher algorithm
#define DELTA 0x9e3779b9
#define MX(p) (((z>>5^y<<2) + (y>>3^z<<4)) ^ ((sum^y) + (key[((p)&3)^e] ^ z)))

void decrypt(uint32_t *values, uint32_t count, const uint32_t key[4])
{
    uint32_t rounds = 6 + 52/count, sum = rounds*DELTA, y = values[0], z;
    do
    {
        uint32_t e = (sum >> 2) & 3;
        for (uint32_t p = count-1; p; p--)
        {
            z = values[p-1];
            y = values[p] -= MX(p);
        }
        z = values[count-1];
        y = values[0] -= MX(0);
        sum -= DELTA;
    } while (--rounds);
}

// the key used by Battle Brothers
static uint32_t bbkey[4] = { 238473842, 20047425, 14005, 978629342 };

// the Squirrel reader function
SQInteger read_encrypted(SQUserPointer file, SQUserPointer buffer, SQInteger length)
{
  length = file_read(file, buffer, length); // read the data from the file
  if (length >= 8) decrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey);
  return length;
}

...
sq_readclosure(vm, read_encrypted, file);

To encrypt a file, you can't simply do the analogous approach of using a writer callback function to encrypt all blocks of 8+ bytes, since the Squirrel writer doesn't invoke the callback for the same offsets and lengths as the reader, which is needed for the two to match. (And even if they did match, there's no guarantee that it would stay that way.) So, I implemented encryption using a reader function as well, so you read the file that you want to encrypt, and as a byproduct of reading it the file gets encrypted.

void encrypt(uint32_t *values, uint32_t count, const uint32_t key[4])
{
    uint32_t rounds = 6 + 52/count, sum = 0, y, z = values[count-1];
    do
    {
        sum += DELTA;
        uint32_t e = (sum >> 2) & 3, p;
        for (p = 0; p < count-1; p++)
        {
            y = values[p+1];
            z = values[p] += MX(p);
        }
        y = values[0];
        z = values[count-1] += MX(p);
    } while (--rounds);
}

SQInteger read_and_encrypt(SQUserPointer file, SQUserPointer buffer, SQInteger length)
{
  length = file_read(file, buffer, length); // read the data from the file
  if (length >= 8)
  {
    // encrypt the file in place. you could also write the changes to a separate file
    encrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey); // encrypt the data
    file_seek(file, -length, SEEK_CUR); // seek back to the start of the block
    file_write(buffer, length); // write the encrypted bytes back to the file
    file_flush(file); // some C file APIs require a flush when switching from write to read

    // decrypt the data again so we don't return garbage to Squirrel
    decrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey);
  }

  return length;
}

Decompiling a script

Now we can decrypt a script file, but it's still in compiled form, so we can't easily edit it. To decompile it, I recommend using NutCracker. I actually don't like that decompiler, because the output is incorrect. It has numerous bugs, some of which I've fixed, and is more verbose than it needs to be. It also doesn't preserve all the information from the .cnut file, so if you decompile a file with NutCracker and then recompile it, the result will not be the same as the original. It still seems to work when loaded into the game, though.

Because of those problems, I wanted to write my own decompiler. I'm sure I can create a better one than NutCracker, but my free time is limited so I probably won't do so unless there's a real demand. The problems above are not fatal, since Battle Brothers is capable of loading plain-text scripts (for the most part) and most scripts decompile correctly. A few scripts don't, though, so be careful.

Anyway, here's an example of how to use NutCracker to decompile a script. By default it prints the script to the console, so you simply redirect the output to a file. (The NutCracker author doesn't distribute binaries for the decompiler, but a binary is included in my mod kit.) From a command-line:

nutcracker decrypted.cnut >decrypted.nut

Getting a modified script loaded by the game

Lets say you decompile scripts/items/weapons/knife.cnut, which starts like this:

this.knife <- this.inherit("scripts/items/weapons/weapon", {
	m = {},
	function create()
	{
		this.weapon.create();
		this.m.ID = "weapon.knife";
		this.m.Name = "Knife";
		this.m.Description = "A short knife, not made for combat.";
		this.m.Categories = "Dagger, One-Handed";
    ...

Just for a test, you change "not" to "totally" within the description. Now how to get the modification back into the game? The most convenient way is to just use the plain-text script. For many scripts (including this one), the game will prioritize a plain-text script named "foo.nut" over a compiled script named "foo.cnut" in the same directory, so you could drop the plain-text script into the directory with the .cnut file. But to be sure you can recompile the .nut file using the standard Squirrel compiler (sq.exe) into a .cnut file and then reencrypt it using the mod kit.

You could simply insert the file into the data_001.dat archive, but that's a bad idea, because your modification may cause file corruption if the game is patched, and any game update would undo your changes. We want the game to load the base data and your modified version. Thankfully, the game uses the PhysFS filesystem abstraction library, and configures it in such a way that the game will load files from subdirectories of the data directory as well as files from the .dat archives. So you could actually place your modified knife.nut (or knife.cnut) file in bbros/data/scripts/items/weapons/ and it will take precedence over knife.cnut from the data_001.dat archive. This is good for testing, but is not a good solution for mods in general because it makes it hard to combine multiple mods. (Their files would be all mixed together.)

A better way is to simply package your mod into its own .dat file. Since the .dat files are really .zip files, just create a .zip file with the scripts/items/weapons/knife.nut file inside and place it in the bbros/data/ directory along with data_001.dat. (It's not necessary to use the .dat extension.) The important thing here is that mods get loaded after the base data, and I believe this happens based on the file name. Since 'm' sorts after 'd', naming the archive "mod_knife.zip" hopefully ensures that it gets loaded after the file named "data_001.dat". If order among mods is important (to resolve some conflicts), you could tweak the names to get the right ordering.

Coding guidelines for better mods

Now we can mod the game, but mods created as above will not play well together either with other mods or with future official updates. The reason is that you are overwriting entire files rather than just tweaking the bits you want to change. If an official update changes the file, your copy won't have the update unless you recreate your mod based on the latest official version. And, two mods that update the same file will overwrite each other's changes even if the changes don't conflict. So, we need a way to express the changes relevant to us without duplicating the rest of the file. Consider the following script.

local super = this.knife.create;
this.knife.create = function()
{
  super();
  this.m.Description = "A short knife, totally made for combat.";
}

Something like this would allow us to change just the description while keeping the rest of the knife code the same, and we could put this script into its own file rather than overwriting the base knife file. The difficult part is getting this code loaded at the right time. By placing your file in a folder named !mod_mymod you can make a script load before most others, and by placing it in a folder named ~mod_mymod you can make it load after most others, but neither of those work in this case. That said, I did create a small mod called "modding script hooks" that allows something like this:

::mods_hookNewObject("items/weapons/knife", function(obj)
{
  obj.m.Description = "A short knife, totally made for combat.";
});

On a side note, any sizeable mod will have to deal with official updates. You wouldn't want to redo your mod from scratch after each official update, nor would you want to continue to use outdated script files in your mod. The solution here is to set up version control. Extract the game data, decompile the scripts, and put it into a Mercurial or Git repository. Create a branch for the mod and edit your mod in the new branch. After each official update, switch back to the main branch, extract and decompile the game data again, and check in any changes. Then merge the changes into your mod branch. This way you keep your mod up to date with official changes without having to redo it from scratch. Also, the repository should of course just be on your local disk. Don't go putting all the game files up on GitHub. This also helps if you need to merge multiple mods together due to conflicts.

The mod kit

I packaged together a few tools into a very basic mod kit to get people started modifying the scripts for Battle Brothers. See the README.txt file within the package for details.

Download: bbros.zip (874kb) Version 9 - released February 12, 2019

Changelog

• Version 1, January 20, 2019 - Initial release
• Version 2, January 21, 2019 - Attempted to fix a serious bug in NutCracker. I'm not sure if the fix is correct. Hopefully the author will make an official fix.
• Version 3, January 22, 2019 - Attempted to fix another bug in NutCracker.
• Version 4, January 23, 2019 - Attempted to fix another serious bug in NutCracker.
• Version 5, January 26, 2019 - Added a program (bbrusher) that can unpack and repack the game's sprite sheets
• Version 6, January 27, 2019 - Fixed two more serious bugs in NutCracker
• Version 7, January 28, 2019 - Expanded the types of sprites that can be added with bbrusher. (Should be all types now.) Made Nutcracker's floating point output prettier. Added a masscompile.bat tool.
• Version 8, February 3, 2019 - Fixed four more serious bugs in NutCracker
• Version 9, February 12, 2019 - Improved Nutcracker's handling of non-ANSI strings by using UTF-8

Postscript

So in the end, I got my updated character backgrounds spreadsheet. :-) I wrote a program to scan the script files and make a spreadsheet out of it. I guess that's mission accomplished!