Web Rarely

I like the game Capitalism 2, but it has some annoying bugs. I like to run companies that are vertically integrated, where I own the production all the way down the supply line to the raw resource extraction. If I recall correctly, one bug has to do with cotton, or perhaps textiles, which are made from cotton. It's difficult to profitably produce your own textiles because cotton is just too expensive. There are even worse problems with citric acid and wheat germ oil, which are very expensive to produce. Today I was motivated to reverse engineer the game enough to fix these problems. In the process, I created a mod kit that allows editing many parts of the game.

Table of Contents

• Mod kit
• Reverse engineering
  • The game database
  • The ICN/COL files in the "image" directory
  • The I_foo.RES and P_foo.RES files in the "Resource" directory
  • Other RES files in the "Resource" directory
  • IDA Pro 6.1

The mod kit

Since the most popular part of this post is now the mod kit, I'll place it first. The most recent release is version 3 (released September 16, 2013). It requires the .NET Framework version 3.5 or later, which you probably already have, and consists of four programs:

• dbmod allows editing the cap2 database (i.e. gameset\1STD.SET) file, which allows you to edit many types of properties within the game, and to add new products and production rules. It requires an installed copy of Microsoft Excel 2007 or later.
• icnedit allow editing the ICN/COL files within the game's "image" directory, which allows you to edit some of the user interface graphics.
• iresedit allows editing the I_foo.RES/P_foo.RES pairs in the game's "Resource" directory, which allows you to edit product and CEO images, but not to add new ones or delete existing ones. (However, see prodedit.)
• prodedit allows more advanced editing of the I_ITEM.RES/P_ITEM.RES pair in the game's "Resource" directory. In particular, it allows you to add new product images and delete existing product images.

The mod kit can be download here. See the README.TXT file for more detailed usage instructions. I've used the mod kit to create a mod that rebalances the whole game.

The game database

After some searching, I found that the game data relevant to my problem seemed to be stored in the gameset\1std.set file. I was able to largely decipher the database format. It begins with the following header. All integers are in little-endian form.

// represents a .set file -- a set of game files
struct databaseHeader
{
  uint16 fileCount; // the number of files in the set
  fileDescriptor files[fileCount+1]; // a number of file descriptors equal to 'fileCount' + 1
}

// describes a single file
struct fileDescriptor // size = 13 bytes
{
  char name[9]; // nul-terminated file name (8-bit chars)
  uint32 offset; // the position of the game file within the .set file
}

The last file descriptor has an empty name and an offset that points to the end of the .set file (i.e. that equals the length of the file). The file descriptors must be in order by offset; the length of a file equals the distance to the next offset. Each file I've seen has been a database table, and each database table has the following header:

// describes a database table
struct tableHeader // size = 32 bytes
{
  byte unk1; // unknown. always 3 so far. maybe file type? (3 = database table?)
  byte unk2[3]; // unknown. varies. first byte is usually 1. others are often 09 1B. probably unused. (i disassembled
                // the executable and couldn't see any usages. i changed them all to FF and the game ran fine)
  uint32 rowCount; // 32-bit size assumed, but at least 16 bits.
  uint16 dataOffset; // the position of the first row within the table file
  uint32 rowLength; // the size of a row in bytes
  byte unk3[18]; // unknown. usually all zero. two tables had unk3[15] equal to 2. the game doesn't seem to use it, though
}

This is followed by a number of column headers. The number of columns is not stored explicitly, but each column header is 32 bytes long, and the number of column headers equals (dataOffset - sizeof(tableHeader)) / sizeof(columnHeader), or (dataOffset-32) / 32. Each column header has this form:

// describes a table column
struct columnHeader // size = 32 bytes
{
  char name[11]; // nul-terminated column name
  char type; // 'C' = string, 'N' = number, 'L' = boolean (stored as "T" or "F")
  uint32 offset; // the position of the column within a row. 32-bit size assumed.
  union
  {
    struct ifNum // this is used if type == 'N'
    {
      byte length; // the length of the column data, in bytes
      byte decimals; // the number of decimal places
    }
    uint16 ifStr; // the length of the column data, in bytes. this is used if type != 'N'
  }
  byte padding[14]; // unknown. always zero so far, so probably just padding
}

The column headers are followed by a single extra byte, of value 0D (which marks the end of the headers), and then the row data starts. The row data is simply stored as 'rowCount' blocks, each 'rowLength' bytes in size. Within each row, the values appear to be stored as ASCII text. Portions of a row not allocated to any column have the value 20h (space), but you should preserve the values of unallocated bytes just in case. After the final row, there is a single extra byte with a value of 1A.

Using this information, I was able to write a program that extracts all of the game data from the database. (I'll mention that many of the .res files in the Resource directory are database tables too.) In the ITEM table, I found this (excerpt):

CODE    	SPENDING
ACID    	1
...
FLAX    	1
RUBBER_P	1
STRAWBEY	1
SUGARCAN	1
BARLEY  	1
COCOA   	1
COCO_NUT	1
CORN    	1
COTTON  	10
FLAXFIBR	1
GRAPE   	2
LEMON   	1
RUBBER  	1
SUGAR   	1
TOBACCO 	1.4
WHEAT   	1

Cotton costs about 10 times as much as the other crops. Hmm! That might have something to do with the problem I was seeing. I also found that it takes 10 units of lemon to produce one unit of citric acid. So it costs me about $10 to produce a unit of citric acid, but its expected price is only $1 (which is about what it costs to import and roughly what customers expect to pay). It turns out that citric acid, coconut oil, corn syrup, dyestuff, flour, linen, polyester, textiles, and wheat germ oil all have the same problem, where the expected profit is negative. Textiles is negative due to cotton's super high price, so that's easy to fix, but the way to fix the others without unbalancing the game is not so obvious. But honestly, the whole game needs rebalancing because a lot of products have ridiculous profit margins (ten thousand or even a million percent).

If you'd like to see the game database, I've collected all the tables into HTML files linked below. The mod kit, linked from the top of this article, contains a copy of the database in the form of a spreadsheet, including helpful calculated columns. (If you can view Excel spreadsheets, it's much more convenient to look at the spreadsheet than the HTML version.) The mod kit also allows you to edit the game database easily, using Excel. I've used the mod kit to create a mod that rebalances the whole game.

If you want to edit the database without using the mod kit, it's not too hard since everything is fixed-width, so a hex editor works fairly well, but adding new rows requires updating the offsets to all the subsequent database tables. If you find out the meanings of the database columns that aren't obvious, I'd be interested to know. Anyway, here's the game database in HTML. Click a link to view the table.

CAR | FARMCROP | FARMLIVE | FARMPROD | FBUILD | FIRM | FIRMJOB | FITEM | GROUP | HEADER | ITEM | ITEMCLAS | JOB | METHOD | PERSON | RAW | RICH | SHIP | TOWN | TREE | TUT | WALKER

Here are a few miscellaneous facts I've picked up while reverse engineering the game. They may be of help if you use the mod kit to create new mods.

• When a row references another row by ID, the ID is simply the one-based ordinal of the row within the table. If you add a new row in the middle of a table, be aware that it will shift the subsequent rows down, changing their IDs. You may then need to edit other tables to update the IDs there. This can be automated by replacing the literal ID references with an Excel formula that uses VLOOKUP to look up the correct ID based on the text code, which is usually stored in the same row as the ID reference.
• The freight cost calculation (in dollars per kilometer) is distance * spending * freightFactor * avgEconomyFactor / 200, where freightFactor = freight/100.0, avgEconomyFactor is the average of something to do with the economies of the source and destination city involved, and distance is in kilometers. (The distance is the distance between the buildings if they're in the same city or the distance between the cities plus 200km, representing movement to/from ports, if they're in different cities.)
• The "spending" column is not just about price. It also interacts with production speed, which also interacts with output quantity. Presumably more expensive items are manufactured more slowly and in smaller batches, so if you want to change the price without changing manufacturing speed, you should adjust the "prod_speed" column as well.
• In the methods table, input and output IDs can be negative. Negative IDs refer to item classes rather than items, and this allows you to use any item of that class as an input or output. The stock game database doesn't make use of this feature, though, so I can't say if it really works properly.
• Firm buildings must be in order by class, and new classes cannot be added.
• The game knows the position and length of all columns, so those cannot be changed.
• The game knows about all of the item classes, class IDs, and class codes, so item classes cannot be added, removed, or reordered.
• The game knows about building codes and firm IDs.
• Items must be grouped and ordered by class ID, so if you insert new items you must do it in the right place.
• The game knows about some item codes, so you should be careful about changing item codes. Also note that like item IDs, item codes are referenced in several tables and the references will need to be kept in sync.
• The game assumes that an image file exists for each item in the database (and vice versa), so to add a new item you also have to add the product image for it. This can be done using the prodedit program in the mod kit.

The ICN/COL files in the "image" directory

Although I had no need to edit files in the "image" directory, they were easy to decipher so I might as well describe the format. Each image is composed of two files: a COL file which contains the color palette and an ICN file that contains the image data using indexes into the color palette. The palette files have the following format:

struct colFile // size = 776 bytes
{
  uint size; // always equal to 776, or 08 03 00 00 in hex, which is the size of the palette file
  uint unknown; // always equal to 23 B1 00 00 as far as I've seen. seems to be ignored by the game
  struct color colors[256];
}

struct color // size = 3 bytes
{
  byte R, G, B; // standard RGB color where each channel value ranges from 0 to 255
}

The ICN files have this format:

struct icnFile
{
  ushort width, height; // the dimensions of the image in pixels
  byte data[width * height]; // each byte is an index into the palette
}

You can use the icnedit program in the mod kit to edit these files.

The I_foo.RES and P_foo.RES files in the "Resource" directory

The I_*.RES/P_*.RES pairs in the "Resource" directory contain sets of images. Similar to the ICN/COL pairs in the "image" directory, the I_*.RES files contain the image data and the P_*.RES files contain the palettes. All of them are databases, in that they begin with a databaseHeader as described above and have the same basic structure, except that instead of each embedded file being a database table (as with 1STD.SET), each embedded file is either a COL file or an ICN file as described in the previous section. To correlate palettes with image data, the names of the embedded files must be matched up. For instance, the embedded file named FOO in P_bar.RES is the palette for the embedded image data named FOO in I_bar.RES. If you want to edit the images stored within these files, don't change the size or else bad things will probably happen. You can use the iresedit and prodedit programs in the mod kit to edit these files.

Other RES files in the "Resource" directory

Most other RES files in the "Resource" directory contain images as well. There are two main formats. One is a database (e.g. starts with databaseHeader, etc.) containing image data with no obvious palette files. In this case, the game assumes the image data has a particular well-known palette, usually the pal_std.res palette, which is simply a COL file as described above. Examples of this format are I_BUTTON.RES and I_ICON.RES. The iresedit program in the mod kit contains experimental support for editing files in this format, but it's commented out. If you want, you can recompile the program and add the feature back in. Another RES format is more complex, since it contains a set of animations with the palettes embedded within the file and the ability for palettes to be used by multiple animations and possibly some support for animation loops. (Some "animations" contain only a single frame, making them effectively just image files.) An example of this format is FIRMICON.RES. I've largely deciphered this format as well, but don't have the results in an easily publishable form. If you're interested in having the description of this format, let me know and I'll try to write it up.

IDA Pro 6.1

You can download my IDA Pro 6.1 database for Capitalism 2 version 1.3 here. I've also posted a short video series about decompiling the game here.

As usual, feel free to contact me if you have any trouble.

ERROR: OverflowException: Arithmetic operation resulted in an overflow.
ERROR: IndexOutOfRangeException: Index was outside the bounds of the array.

ERROR: IndexOutOfRangeException: Index was outside the bounds of the array.