Spreadsheets and Game Decryption

[Ishad Nha]

This is generally useful for any game system not just Spiderweb games. If you were decrypting games with walls that are one square in thickness, you might want to go with BoE instead. (As I did for looking at towns in SSI's Ravenloft: Stone Prophet.) Here I will be using G5 and AEFTP as examples because they are familiar to most people in this board.

 

One good way to help decrypt a game's town record structure is to make a mock-up of a town record: translate the data into BoA terms and paste it into a scenario.

(A BoA scenario can't handle a few features of Geneforge like Objects though.)

Break up the game file into town records. Remove the header, a file splitter can resolve the remaining data into town records, because they are all the same size.*

Take a town record, copy it as Assembler data or C data and paste into a spreadsheet.

 

Using a G4 or G5 converter as an example, the data will commence in cell D6 and end in cell S3261. (A G4/G5 record has 52,088 Bytes. Data will be posted in rows of 16 Bytes, that is the way that my hex editor does things. 52,088 = 3,255*16, with 8 left over. Hence cells L3261 through S3261 will be vacant.) Remove all extraneous stuff, "db ", "h".

Now create an array of equal size, a couple of columns to the right. Just copy and paste the D6:S3261 array so that the second array lies in the range W6:AL3261. In cell W6, write in the formula "=D6", omit quotes. Fill to the right, then fill down. All cells in the second array will have this formula. Delete the formulas in cells AE3261 through AL3261, they are not wanted.

We want things so that each line has the correct number of Bytes, this number can be anywhere from 2 through to 80. Here I am going to write a program that does exactly that.

First step is to copy the formulas ( =D6 through =K3261) and paste them into a page of source code. If I just do a straight copy and paste, the formulas will be pasted as numerical results not formulas, not what I want. In the second array, use Replace All to change all equality signs to something else, say asterisks. This will cause them to be pasted as text:

*D6 *E6 *F6 *G6 *H6 *I6 *J6 *K6 *L6 *M6 *N6 *O6 *P6 *Q6 *R6 *S6

*D7 *E7 *F7 *G7 *H7 *I7 *J7 *K7 *L7 *M7 *N7 *O7 *P7 *Q7 *R7 *S7

(a few thousand lines are omitted here)

*D3260 *E3260 *F3260 *G3260 *H3260 *I3260 *J3260 *K3260 *L3260 *M3260 *N3260 *O3260 *P3260 *Q3260 *R3260 *S3260

*D3261 *E3261 *F3261 *G3261 *H3261 *I3261 *J3261 *K3261

 

Save all the data into a file, then open that file in a hex editor. Here "\t" equals a tab stop. In each line, between each formula we have \t* and we actually want ","=

In Windows a tab stop usually has a hex value of 09 and an asterisk is 2A. While ","= has a value of 22 2C 22 3D. Simply use Replace All to change all instances of 092A with 222C223D. This still leaves problems at the beginning and ends of lines.

Representing a paragraph mark as "\n" we have \n* and we want ",\n"=

In Windows a paragraph mark usually has a hex value of 0D0A and an asterisk is 2A. While ",\n"= has a value of 22 2C 0D 0A 22 3D. Simply use Replace All to change all instances of 0D0A2A with 222C0D0A223D.

Beginning and end of file will need to be manually altered. I will assign a name of "buffer" to the formulas, result should be as below:

char *buffer[52088] = {

"=D6","=E6","=F6","=G6","=H6","=I6","=J6","=K6","=L6","=M6","=N6","=O6","=P6","=Q6","=R6","=S6",

"=D7","=E7","=F7","=G7","=H7","=I7","=J7","=K7","=L7","=M7","=N7","=O7","=P7","=Q7","=R7","=S7",

"=D8","=E8","=F8","=G8","=H8","=I8","=J8","=K8","=L8","=M8","=N8","=O8","=P8","=Q8","=R8","=S8",

(a few thousand lines are omitted here)

"=D3259","=E3259","=F3259","=G3259","=H3259","=I3259","=J3259","=K3259","=L3259","=M3259","=N3259","=O3259","=P3259","=Q3259","=R3259","=S3259",

"=D3260","=E3260","=F3260","=G3260","=H3260","=I3260","=J3260","=K3260","=L3260","=M3260","=N3260","=O3260","=P3260","=Q3260","=R3260","=S3260",

"=D3261","=E3261","=F3261","=G3261","=H3261","=I3261","=J3261","=K3261",

};

 

Next, write a program using clauses like this:

for (i = 51648; i < 52048; i++) { // Automap Notes

if (i % 40 == 8)

printf("%d,%d,,",i,(i - 51648)/40);

printf("%s,",buffer);

if (i % 40 == 7)

printf("\n");

}

 

When it is finished, run the program to print out the formulas so they can be entered into a spreadsheet.

 

* In AEFTP you would create one data file consisting only of outdoor records and another file consisting only of town records.

In AEFTP this also works for the outdoor records too.

 

(to be continued)

Edited December 22, 2013 by Ishad Nha

[Ishad Nha]

Now the idea is to create a blank data structure for a large BoA town:

 

Data type Name Starts Length

char town_name[20]; 0 20

short respawn_monsts[4][6]; 20 48

location respawn_locs[6]; 68 12

macRECT special_rects[60]; 80 480

unsigned char spec_id[60]; 560 60

short lighting; 620 2

location sign_locs[15]; 622 30

char sign_text[15][256]; 652 3,840

location start_locs[4]; 4,492 8

location exit_locs[4]; 4,500 8

macRECT in_town_rect; 4,508 8

item_type preset_items[144]; 4,516 1,152

preset_field_type preset_fields[60]; 5,668 240

short wall_1_sheet, wall_1_height; 5,908 4

short wall_2_sheet, wall_2_height 5,912 4

short cliff_sheet; 5,916 2

short beam_type; 5,918 2

short environmental_sound; 5,920 2

short is_on_surface; 5,922 2

short town_kill_day; 5,924 2

short town_linked_event; 5,926 2

short external_floor_type; 5,928 2

short monster_respawn_chance; 5,930 2

char town_script[14]; 5,932 14

in_town_on_ter_script_type ter_scripts[100]; 5,946 3,800

macRECT room_rect[16]; 9,746 128

char info_rect_text[16][30]; 9,874 480

creature_start_type creatures[80]; 10,354 5,920

short extra[20]; 16,274 40

location waypoints[10]; 16,314 20

short exit_specs[4]; 16,334 8

short spec_on_entry; 16,342 2

short spec_on_entry_if_dead; 16,344 2

short terrain[64][64]; 16,346 8,192

unsigned char floor[64][64]; 24,538 4,096

unsigned char height[64][64]; 28,634 4,096

unsigned char lighting[64][64]; 32,730 4,096

EOF: 36,826

 

We want a blank town because at least some of the features of a BoA town won't occur in the game we seek to decrypt. Then again, maybe they do, but it might take a while to correctly identify where they occur.

Now a program can be written to print out the data in the correct number of columns.

Once you find out where the AEFTP equivalent of BoA data occurs, you can import it into a BoA town data structure by a simple use of Fill Down.

 

Once the spreadsheet is complete it can be easily used. All data is found in a 1,274x74 rectangle. If data is pasted into Hexplorer as Hex numbers empty cells in the spreadsheet will be ignored, hence you will have exactly the right number of Bytes: 36,826.

[Little Fyora]

I'm reminded of the time when a group of fellows got together and decrypted the entire mission data for an FPS PC game. It did take a while, but they had brilliant results; they made a function-oriented language and a compiler to create new scenarios, like what it would have been if the whole Blades of Avernum was created by fans alone.

[Ishad Nha]

It is possible to edit an entire town record using a spreadsheet and one of my printout programs. I have finished an editor for AEFTP and am now working on the readme.