Web Rarely

Office Automation: The use of computers to improve efficiency in the office by removing anyone you would want to talk with over coffee.

Reverse Engineering Capitalism 2

2013-02-27

I like the game Capitalism 2, but it has some annoying bugs. I like to run companies that are vertically integrated, where I own the production all the way down the supply line to the raw resource extraction. If I recall correctly, one bug has to do with cotton, or perhaps textiles, which are made from cotton. It's difficult to profitably produce your own textiles because cotton is just too expensive. There are even worse problems with citric acid and wheat germ oil, which are very expensive to produce. Today I was motivated to reverse engineer the game enough to fix these problems. In the process, I created a mod kit that allows editing many parts of the game.

Table of Contents

The mod kit

Since the most popular part of this post is now the mod kit, I'll place it first. The most recent release is version 3 (released September 16, 2013). It requires the .NET Framework version 3.5 or later, which you probably already have, and consists of four programs:

  • dbmod allows editing the cap2 database (i.e. gameset\1STD.SET) file, which allows you to edit many types of properties within the game, and to add new products and production rules. It requires an installed copy of Microsoft Excel 2007 or later.
  • icnedit allow editing the ICN/COL files within the game's "image" directory, which allows you to edit some of the user interface graphics.
  • iresedit allows editing the I_foo.RES/P_foo.RES pairs in the game's "Resource" directory, which allows you to edit product and CEO images, but not to add new ones or delete existing ones. (However, see prodedit.)
  • prodedit allows more advanced editing of the I_ITEM.RES/P_ITEM.RES pair in the game's "Resource" directory. In particular, it allows you to add new product images and delete existing product images.

The mod kit can be download here. See the README.TXT file for more detailed usage instructions. I've used the mod kit to create a mod that rebalances the whole game.

The game database

After some searching, I found that the game data relevant to my problem seemed to be stored in the gameset\1std.set file. I was able to largely decipher the database format. It begins with the following header. All integers are in little-endian form.

// represents a .set file -- a set of game files
struct databaseHeader
{
  uint16 fileCount; // the number of files in the set
  fileDescriptor files[fileCount+1]; // a number of file descriptors equal to 'fileCount' + 1
}

// describes a single file
struct fileDescriptor // size = 13 bytes
{
  char name[9]; // nul-terminated file name (8-bit chars)
  uint32 offset; // the position of the game file within the .set file
}
The last file descriptor has an empty name and an offset that points to the end of the .set file (i.e. that equals the length of the file). The file descriptors must be in order by offset; the length of a file equals the distance to the next offset. Each file I've seen has been a database table, and each database table has the following header:
// describes a database table
struct tableHeader // size = 32 bytes
{
  byte unk1; // unknown. always 3 so far. maybe file type? (3 = database table?)
  byte unk2[3]; // unknown. varies. first byte is usually 1. others are often 09 1B. probably unused. (i disassembled
                // the executable and couldn't see any usages. i changed them all to FF and the game ran fine)
  uint32 rowCount; // 32-bit size assumed, but at least 16 bits.
  uint16 dataOffset; // the position of the first row within the table file
  uint32 rowLength; // the size of a row in bytes
  byte unk3[18]; // unknown. usually all zero. two tables had unk3[15] equal to 2. the game doesn't seem to use it, though
}
This is followed by a number of column headers. The number of columns is not stored explicitly, but each column header is 32 bytes long, and the number of column headers equals (dataOffset - sizeof(tableHeader)) / sizeof(columnHeader), or (dataOffset-32) / 32. Each column header has this form:
// describes a table column
struct columnHeader // size = 32 bytes
{
  char name[11]; // nul-terminated column name
  char type; // 'C' = string, 'N' = number, 'L' = boolean (stored as "T" or "F")
  uint32 offset; // the position of the column within a row. 32-bit size assumed.
  union
  {
    struct ifNum // this is used if type == 'N'
    {
      byte length; // the length of the column data, in bytes
      byte decimals; // the number of decimal places
    }
    uint16 ifStr; // the length of the column data, in bytes. this is used if type != 'N'
  }
  byte padding[14]; // unknown. always zero so far, so probably just padding
}
The column headers are followed by a single extra byte, of value 0D (which marks the end of the headers), and then the row data starts. The row data is simply stored as 'rowCount' blocks, each 'rowLength' bytes in size. Within each row, the values appear to be stored as ASCII text. Portions of a row not allocated to any column have the value 20h (space), but you should preserve the values of unallocated bytes just in case. After the final row, there is a single extra byte with a value of 1A.

Using this information, I was able to write a program that extracts all of the game data from the database. (I'll mention that many of the .res files in the Resource directory are database tables too.) In the ITEM table, I found this (excerpt):

CODE    	SPENDING
ACID    	1
...
FLAX    	1
RUBBER_P	1
STRAWBEY	1
SUGARCAN	1
BARLEY  	1
COCOA   	1
COCO_NUT	1
CORN    	1
COTTON  	10
FLAXFIBR	1
GRAPE   	2
LEMON   	1
RUBBER  	1
SUGAR   	1
TOBACCO 	1.4
WHEAT   	1
Cotton costs about 10 times as much as the other crops. Hmm! That might have something to do with the problem I was seeing. I also found that it takes 10 units of lemon to produce one unit of citric acid. So it costs me about $10 to produce a unit of citric acid, but its expected price is only $1 (which is about what it costs to import and roughly what customers expect to pay). It turns out that citric acid, coconut oil, corn syrup, dyestuff, flour, linen, polyester, textiles, and wheat germ oil all have the same problem, where the expected profit is negative. Textiles is negative due to cotton's super high price, so that's easy to fix, but the way to fix the others without unbalancing the game is not so obvious. But honestly, the whole game needs rebalancing because a lot of products have ridiculous profit margins (ten thousand or even a million percent).

If you'd like to see the game database, I've collected all the tables into HTML files linked below. The mod kit, linked from the top of this article, contains a copy of the database in the form of a spreadsheet, including helpful calculated columns. (If you can view Excel spreadsheets, it's much more convenient to look at the spreadsheet than the HTML version.) The mod kit also allows you to edit the game database easily, using Excel. I've used the mod kit to create a mod that rebalances the whole game.

If you want to edit the database without using the mod kit, it's not too hard since everything is fixed-width, so a hex editor works fairly well, but adding new rows requires updating the offsets to all the subsequent database tables. If you find out the meanings of the database columns that aren't obvious, I'd be interested to know. Anyway, here's the game database in HTML. Click a link to view the table.

CAR | FARMCROP | FARMLIVE | FARMPROD | FBUILD | FIRM | FIRMJOB | FITEM | GROUP | HEADER | ITEM | ITEMCLAS | JOB | METHOD | PERSON | RAW | RICH | SHIP | TOWN | TREE | TUT | WALKER

Here are a few miscellaneous facts I've picked up while reverse engineering the game. They may be of help if you use the mod kit to create new mods.

  • When a row references another row by ID, the ID is simply the one-based ordinal of the row within the table. If you add a new row in the middle of a table, be aware that it will shift the subsequent rows down, changing their IDs. You may then need to edit other tables to update the IDs there. This can be automated by replacing the literal ID references with an Excel formula that uses VLOOKUP to look up the correct ID based on the text code, which is usually stored in the same row as the ID reference.
  • The freight cost calculation (in dollars per kilometer) is distance * spending * freightFactor * avgEconomyFactor / 200, where freightFactor = freight/100.0, avgEconomyFactor is the average of something to do with the economies of the source and destination city involved, and distance is in kilometers. (The distance is the distance between the buildings if they're in the same city or the distance between the cities plus 200km, representing movement to/from ports, if they're in different cities.)
  • The "spending" column is not just about price. It also interacts with production speed, which also interacts with output quantity. Presumably more expensive items are manufactured more slowly and in smaller batches, so if you want to change the price without changing manufacturing speed, you should adjust the "prod_speed" column as well.
  • In the methods table, input and output IDs can be negative. Negative IDs refer to item classes rather than items, and this allows you to use any item of that class as an input or output. The stock game database doesn't make use of this feature, though, so I can't say if it really works properly.
  • Firm buildings must be in order by class, and new classes cannot be added.
  • The game knows the position and length of all columns, so those cannot be changed.
  • The game knows about all of the item classes, class IDs, and class codes, so item classes cannot be added, removed, or reordered.
  • The game knows about building codes and firm IDs.
  • Items must be grouped and ordered by class ID, so if you insert new items you must do it in the right place.
  • The game knows about some item codes, so you should be careful about changing item codes. Also note that like item IDs, item codes are referenced in several tables and the references will need to be kept in sync.
  • The game assumes that an image file exists for each item in the database (and vice versa), so to add a new item you also have to add the product image for it. This can be done using the prodedit program in the mod kit.

The ICN/COL files in the "image" directory

Although I had no need to edit files in the "image" directory, they were easy to decipher so I might as well describe the format. Each image is composed of two files: a COL file which contains the color palette and an ICN file that contains the image data using indexes into the color palette. The palette files have the following format:
struct colFile // size = 776 bytes
{
  uint size; // always equal to 776, or 08 03 00 00 in hex, which is the size of the palette file
  uint unknown; // always equal to 23 B1 00 00 as far as I've seen. seems to be ignored by the game
  struct color colors[256];
}

struct color // size = 3 bytes
{
  byte R, G, B; // standard RGB color where each channel value ranges from 0 to 255
}
The ICN files have this format:
struct icnFile
{
  ushort width, height; // the dimensions of the image in pixels
  byte data[width * height]; // each byte is an index into the palette
}

You can use the icnedit program in the mod kit to edit these files.

The I_foo.RES and P_foo.RES files in the "Resource" directory

The I_*.RES/P_*.RES pairs in the "Resource" directory contain sets of images. Similar to the ICN/COL pairs in the "image" directory, the I_*.RES files contain the image data and the P_*.RES files contain the palettes. All of them are databases, in that they begin with a databaseHeader as described above and have the same basic structure, except that instead of each embedded file being a database table (as with 1STD.SET), each embedded file is either a COL file or an ICN file as described in the previous section. To correlate palettes with image data, the names of the embedded files must be matched up. For instance, the embedded file named FOO in P_bar.RES is the palette for the embedded image data named FOO in I_bar.RES. If you want to edit the images stored within these files, don't change the size or else bad things will probably happen. You can use the iresedit and prodedit programs in the mod kit to edit these files.

Other RES files in the "Resource" directory

Most other RES files in the "Resource" directory contain images as well. There are two main formats. One is a database (e.g. starts with databaseHeader, etc.) containing image data with no obvious palette files. In this case, the game assumes the image data has a particular well-known palette, usually the pal_std.res palette, which is simply a COL file as described above. Examples of this format are I_BUTTON.RES and I_ICON.RES. The iresedit program in the mod kit contains experimental support for editing files in this format, but it's commented out. If you want, you can recompile the program and add the feature back in. Another RES format is more complex, since it contains a set of animations with the palettes embedded within the file and the ability for palettes to be used by multiple animations and possibly some support for animation loops. (Some "animations" contain only a single frame, making them effectively just image files.) An example of this format is FIRMICON.RES. I've largely deciphered this format as well, but don't have the results in an easily publishable form. If you're interested in having the description of this format, let me know and I'll try to write it up.

IDA Pro 6.1

You can download my IDA Pro 6.1 database for Capitalism 2 version 1.3 here. I've also posted a short video series about decompiling the game here.

As usual, feel free to contact me if you have any trouble.

Comments

Error when run 2013-04-14 06:11AM
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Office.Interop.Excel, Version=14.0.0.0, Culture=neutral, Public
KeyToken=71e9bce111e9429c' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.Office.Interop.Excel, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'
at Cap2Mod.Program.Main(String[] args)
an anonymous MK Uy
RE: Error when run 2013-04-17 10:59PM
Do you have Microsoft Excel installed? You need Excel 2010. Later versions may also work, but I'm not sure. .NET programming support needs to be installed. It's installed by default. If you'd like, I can help you with it. Email me.
RE: Error when run 2013-05-15 09:28PM
I see, I only have office 2007. Is there any way to make it work with that?
an anonymous MK Uy
2013-05-16 01:39PM
Yeah I got the same problem trying to get this to work with Office 2007.
Is there any other way? I'm not buying Office 2010.
Cheers
an anonymous BOB RIVERS
RE: Error when run 2013-05-16 07:04PM
[UPDATE: Somebody was kind enough to help compile the mod kit for Excel 2007, so it now works!]

Yes, it should be possible to create a version that works with Office 2007. However, I don't have a copy of Office 2007 so it would be difficult for me to create such a version, but the source code is included, so it would be possible for you to compile the program on your own machine, and link it against your Office 2007 DLLs. You should already have the necessary tools to do this, since they're included in the .NET Framework, which is included with Windows. The tricky part is knowing how to do it. If you want, you can send me an email -- see the Contact link at the top of the page -- and I can help with that. (I would need to gather a few pieces of information about your computer first.) Once a version has been compiled for Excel 2007, I can put it here for other people to use if you'd like, or I can just put the steps that we had to go through so other people can do it.

Another possibility is to send me the spreadsheet, and I can send back the modified game database, but that doesn't lend itself to rapid iteration.

A third possibility is for me to change the program so it doesn't require Excel at all. I could probably write my own code to edit .xlsx files, but that's a large undertaking which I wouldn't like to do unless there's significant demand.

A fourth possibility is to use a hex editor to edit the game database as described on this page, but that's much more difficult than editing a spreadsheet. There are several decent, free hex editors. wxHexEditor isn't bad.
Hex 2013-05-28 07:59AM
Chose the Hex editor route.
Thank you for doing this.
an anonymous Bob Rivers again
RE: Hex 2013-06-15 04:58AM
I'm glad to help. :-)
images format 2014-07-05 04:17AM
Thank you very much for your mod kit 。I want to make a MOD for children's 。so i need Replace the Products, buildings。But I don't know programming。
Come on,“have the results in an easily publishable form.”
You are the best。
https://github.com/capitalism
There is a group of enthusiasts from China.
an anonymous bill_bob
RE: images format 2014-07-10 12:14PM
Hi, thanks for the interest. To be honest, I don't really want to dig into the animation file format again at the moment given it's complicated and you will require some programming (or at least rather technical) knowledge to usefully exploit the information. However, the existing mod kit can edit the products and product images as well as a number of other things. If you actually create your mod and replace the products, and are just left with the task of replacing the building animations, then let me know and I'll help out. Good luck!
res viewer 2015-01-20 11:38AM
I couldn't view/edit other .res file using your tools (it missing the palette file)
I want to edit company logos. and if possible to add another car in the store. Is there any way to do it? Thanks for your info.
an anonymous parlin
RE: res viewer 2015-01-21 06:07PM
For your first question, I'll quote myself above: "Most other RES files in the "Resource" directory contain images as well... the game assumes the image data has a particular well-known palette, usually the pal_std.res palette, which is simply a COL file as described above." So although it's missing the palette, you can recover it from the pal_std.res file. Furthermore, as I mentioned in the same paragraph, there is experimental support for editing these paletteless images in the source code, commented out. If you like, you can uncomment it and recompile the tool for an easy way to edit the files. If you don't know how to do that, feel free to send me an email and I can do it for you. (Since you didn't leave a valid email address, there's no way for me to contact you.)

For your second question, it's possible to add new items to some item classes, and cars might be one of the classes that you can edit. (The game has hardcoded knowledge of farm products and semi-finished products, but I forget at the moment whether the game hardcodes their IDs or their codes. If it's the former you can't add new cars, but if it's the latter you can...) You'll need to add new rows to the ITEM and METHOD tables, update the IDs and counts in the other tables, and add a new product image (using prodedit) with the same name as the product code in the database.
2015-01-22 06:44AM
Hi Adam,
Thanks for your reply.

Oh i see.. so we still can use the pal_std.res file to export the other .RES file
As for the compiling, I don't have the proper knowledge to do it. So I'll be glad if you can send me the recompiled tool.

Currently I'm editing the database file. Yup, I'm aware we have to update/sync all related item ID in the corresponding tables.
an anonymous parlin
RE: res viewer 2015-01-22 03:20PM
It's not just the related item IDs but, annoyingly, all the IDs for all the items that come after the items you're adding, since they get shifted down and the ID is simply the row number. If you're going to change any IDs, I'd recommend switching the ID reference columns to use formulae that use VLOOKUP to look up the IDs based on the item codes, and also to update the item TOTALTYPE column in the ITEMCLAS table to use a formula that counts the number of items of the particular class rather than having to maintain the counts by hand.
res viewer 2015-01-23 11:31AM
Thanks for the file. I've downloaded & give it a try but no luck.

I'm using command "iresedit export firmicon.res logo" and got error message:
"ERROR: Missing palette file: pal_std.res"
same thing happen to the other .res files although the actual "pal_std.res" exists there.

Anyway, I'm still editing the database & indeed it's an annoying job especially when you adding dozens of items :p
an anonymous parlin
RE: res viewer 2015-01-23 05:58PM
Ahh, whoops. I uncommented the code to edit paletteless images, but didn't comment out the code to display the error message for paletteless images. Try the updated file at the same URL I gave you before.

If you're going to add dozens of items, I definitely recommend investing in making all the reference columns use VLOOKUP formulas so that you can just insert rows without having to update the references and counts. Also, you probably shouldn't add farm products or semi-finished products. They may appear to work, but they are likely to cause obscure bugs, since the game has a lot of hard-coded logic about those (especially farm products).
res viewer 2015-01-24 03:23AM
Ok now the error changed when i try to export the .res file (firmicon.res & logo.res).
ERROR: OverflowException: Arithmetic operation resulted in an overflow.
ERROR: IndexOutOfRangeException: Index was outside the bounds of the array.
a folder was created but there's no image inside. I also have problem with prodedit. I couldn't import my added images to the I_ITEM.RES file.
ERROR: IndexOutOfRangeException: Index was outside the bounds of the array.
Did my computer missing some system files to be able operating this program properly?
an anonymous parlin
RE: res viewer 2015-01-24 04:05AM
Yeah, alright. I guess that old, experimental code is too old, and uncommenting it isn't enough to make it work anymore. The problem is that I no longer have the game installed, which is why I haven't been able to just test it out and make sure it works. Anyway, if you send me the files you're trying to edit I'll be able to actually run the program.
RE: res viewer 2015-01-24 05:19AM
Oh, those aren't image files. Those are animation files. As I wrote above:

"Another RES format is more complex, since it contains a set of animations with the palettes embedded within the file and the ability for palettes to be used by multiple animations and possibly some support for animation loops... An example of this format is FIRMICON.RES."

The image tool can't be used to edit the animation files, unfortunately. But the cause is not obvious from the error message, so I've improved the error message.
Number of Cities 2015-06-02 08:35PM
Hi Adam,

Is there anyway to increase the number of cities you can choose in custom mode. It be cool to be able to choose more than 4.
an anonymous Clos
RE: Number of Cities 2015-06-07 07:54PM
Hi Clos. Sorry for the delay, but I have to dig out my old laptop to answer questions like this.

You can increase the number of cities, although I'm not sure how stable the game will be. It might crash. The following should work: 1) Start the game, set up the custom game options as you like, and start a custom game. 2) Quit the game. 3) Open CONFIG.DAT with a hex editor. 4) Change the byte at offset 8 (i.e. the 9th byte in the file) to the number of cities you want. 5) Start the game again. 6) Choose to start a new custom game and don't change any options but just click "Start".

There may be two copies of the game's CONFIG.DAT file on your computer, but only one is used by the game (depending on the game version). First look in My Documents\My Games\Capitalism II (or similar) and then look in the game's install directory.
Great Job Adam any new updates? 2016-06-05 03:38PM
this game has been my favorite for a very long time, I have just added ver 1.07 from 1.01...yeah i don't get to play very often lol

I have always wanted to edit one product in this game and it seem the world may be making it legit. Is there anyway to add Medical Weed ?? and yes Growing threw manufacturing (paper ) to the sale and price it right??? This would make my year man, I lost a kid a few years ago and lost a pretty large block of time
an anonymous CJ White
RE: Great Job Adam any new updates? 2016-06-05 11:21PM
Hi CJ,

The easiest way to add marijuana would be to replace either cigarettes or cigars, which are very similar. You'd have to change the ITEM table (name, spending, freight, and unit, at least) as well as the METHOD table. You'd probably want to rename the Tobacco farm product as well. Just change the Name column. Don't mess with the Code columns.

If you have Capitalism 2 and Excel, you can simply download the mod kit (linked above), edit the spreadsheet, and import it into your game. With Capitalism 2, the iresedit program in the mod kit can let you replace the tobacco, cigarette, and/or cigar images with different graphics.

If you don't have Capitalism 2 or Excel, you may have to use a hex editor to edit the database file. The instructions are on this page, but it's more complicated.

Good luck!
-- Adam
New Products 2016-08-23 02:04AM
Plain and simple, congratulations! But how to add new products, I did not understand?
an anonymous Hawk
data.xlsx 2016-09-06 01:35AM
Understood)) But why is not considered surplus data.xlsx
an anonymous Hawk
RE: New products & data.xlsx 2016-09-15 12:08AM
Hi Hawk,

Adding new products is quite complicated. I recommend editing existing products instead. But if you want to add a new one, you have to follow some rules, like: 1) you must keep products grouped by class, 2) if you insert a new row, you must update all of the references to the rows below (formulas can help here), 3) you might not be able to usefully add new farm products or semifinished products, 4) each new product must have an image. I know that's not a simple, step-by-step guide, but it's quite complicated so I don't recommend it unless you have a good understanding of databases, etc. First try modifying products, and if you get good at that, you might try adding one.

I'm not sure what you mean about data.xlsx being surplus. If you want to export or import the game database from 1STD.SET using the mod kit, you'll need that file (or another spreadsheet with the same structure). If you just want to edit the files directly, you don't need it of course, but it's intended for use with the mod kit.

I hope that helps at least a little. Take care,
-- Adam
2016-09-21 07:53AM
Thanks for the answer. With bases understood data. As with the addition of new products. But when you add new products to put his limit ... Over 148 of image files "resource" is not loaded and the game crashes. Do you know anything about this?
an anonymous Hawk
Farm 2016-10-24 04:21AM
Small and large farms have the same production rate ... Change it does not work ... What to do?
an anonymous Hawk
RE: Farm 2016-10-25 11:36PM
What do you mean by "production rate"? I think small and large farms only differ in size, which means it takes the same amount of time to grow crops, but the larger farm should produce more crops by the end of the season. Anyway, if changing the building size has no effect, you may need to start a new game. (Some things are only read from the database when you start a new game.)

As for the game crashing when you add more than 148 image files, I'm not aware of any particular limit. All the file-loading code I've seen so far has not had any hardcoded item limit. Are you sure you added the resources correctly? If a resource doesn't have the right name or format, the game will crash... That said, I can't say for sure that there are no limits.
Capitalism 2 Lab 2019-01-22 04:41PM
Hey Adam,

I'm late to the party, but did you know that there's a great expansion made by some guys who reverse engineered the game like you? They're here: https://www.capitalismlab.com/

It could be what you were looking for, but without all the hacking work ;)
an anonymous SJ

Add a comment

Note: The information you enter (including your name and email address) will be displayed publicly.




Line breaks are converted to <br>'s, and all HTML tags except b, u, i, tt, and pre are filtered out.