Save Game File Decrypting

[Ishad Nha]

This is the first installment of decrypting the BoE save game file. It is based upon the source code file "global.h". Simply copy into the word processor of your choice and change all the tildes into tab stops. (Fine print, the save file uses an idiosyncratic system for recording values of letters and numbers. I have not fully decrypted it, but 0 thru 10 is usually:

k,j,i,h,o,n,m,l,c,b,a.)

 

PARTY RECORDS

Data type~name~length~starts at

(short)~(scenario flags)~6~0

long ~age;~4~6

short~gold, food;~4~10

unsigned char~stuff_done[310][10]~3,100~14

unsigned char~item_taken[200][8];~1,600~3114

short~light_level;~2~4714

location ~outdoor_corner, i_w_c, p_loc, loc_in_sec;~8~4716

boat_record_type ~boats[30];~300~4724

horse_record_type ~horses[30];~300~5024

creature_list_type ~creature_save[4];~33856~5324

short~in_boat, in_horse;~4~39180

outdoor_creature_type ~out_c[10];~290~39184

item_record_type ~magic_store_items[5][10];~3300~39474

short~imprisoned_monst[4];~8~42774

char~m_seen[256];~256~42782

char~journal_str[50];~50~43038

short~journal_day[50];~100~43088

short~special_notes_str[140][2];~560~43188

talk_save_type ~talk_save[120];~720~43748

short~direction, at_which_save_slot;~4~44468

char~alchemy[20];~20~44472

Boolean ~can_find_town[200];~200~44492

short~key_times[100];~200~44692

short~party_event_timers[30];~60~44892

short~global_or_town[30];~60~44952

short~node_to_call[30];~60~45012

char~spec_items[50]~50~45072

char~help_received[120];~120~45122

short~m_killed[200];~400~45242

long~total_m_killed, total_dam_done~8~45642

long~total_xp_gained, total_dam_taken;~8~45650

char~scen_name[256];~256~45658

 

"Age" means time, day of scenario&.

 

Lengths of data types in bytes:

1~Boolean

1~char

1~unsigned char

2~location

2~short

4~long

6~talk_save_type

10~boat_record_type

10~horse_record_type

29~outdoor_creature_type

66~item_record_type

8,464~creature_list_type

 

PC RECORDS

short ~main_status;~2~0

char ~name[20];~20~2

short ~skills[30];~60~22

short ~max_health~2~82

short ~cur_health~2~84

short ~max_sp~2~86

short ~cur_sp~2~88

short ~experience~2~90

short ~skill_pts~2~92

short ~level;~2~94

short ~status[15];~30~96

item_record_type ~items[24];~1584~126

Boolean ~equip[24];~24~1710

Boolean ~priest_spells[62]~62~1734

Boolean ~mage_spells[62];~62~1796

short ~which_graphic~2~1858

short ~weap_poisoned;~2~1860

Boolean ~advan[15]~15~1862

Boolean ~traits[15];~15~1877

short ~race~2~1892

short ~exp_adj~2~1894

short ~direction;~2~1896

 

A lot of the Debug functions don't work. "Actions.cpp" lines 1,586 onwards, Debug keystrokes are listed. Only the case > and case W work. Apart from minor message keys: &,*,(,).

[Celtic Minstrel]

Is the structure of the Mac save file the same? Anyone know? I know it has an endian difference, but it's quite possible that's the only difference...

[Ishad Nha]

I imagine that it is the same. But I have yet to check the two different source files. If you have the official Spiderweb source code you will have both the Mac and the Windows versions of "global.h". In Windows the above tables come from the data structures "scenario_data_type" and "party_record_type".

 

A difference of endianness won't by itself affect the above tables.

 

Edit: I just checked the two source files and they both gave the same result, thus the above tables are valid for both Mac and Windows.

[Ishad Nha]

Change of topic: hex editing.

 

The notes above are scary to people who are not programmers. But if you open an exs file in a hex editor they start to make sense. Perhaps I should include some notes on how to hex - edit, when I find the time.

 

How to Hex edit

Download a free hex editor. I personally use Hexplorer, found at http://hexplorer.sourceforge.net

Then install it and start it up.

Open the file you want, a good choice to start with is an exs file. You can compare what you see in the hex editor with what the Scenario Editor displays. (Also it lacks the encryption found in the save game files.)

 

The place that you are looking for is found in the left part of the status bar. Say you want to edit the Special Items in a password - protected Windows scenario. You are looking for place 3,028 which equals hexadecimal address BD4.

 

Edit: to hex-edit is easy enough, if you can understand Blades of Avernum you are clever enough to hex-edit.

 

From the BoE Compiled Suggestion List, what hex-editing enables you to do:

 

Character Editor

(Ishad Nha)Allow the player to add items from the scenario list rather than the default list.

 

Scenario Editor

(CM)Remove editing restrictions on the first 91 terrain types.

(CM)Allow the designer to resize the outdoors

(CM,Hrothgar)Allow the designer to import an outdoor section

[Ishad Nha]

A new alphabet:

Code:

*)(/.-,#"! '&%$;:98?>=<321

(These are the capital letters used to record the names of items in the PC records part of a save game file.)

 

Lower case letters are recorded by ASCII symbols with numerical values between 0 and 31. Now h,i,j,k have symbols with values of 3,2,1,0. In the numerical areas of item records 0,1,2,3 are represented by k,j,i,h.

[Celtic Minstrel]

Note: The whole point of the suggestion list is to make those things possible without hex-editing.

[Ishad Nha]

If you really want a Mac Editor that works you could try passing around the hat and hiring someone who actually knows how to program. (The Spiderweb fans may not be so wealthy being students and so on.)

 

You could learn programming and do it yourself.

 

But apart from those two options, there is only hex - editing. If you have not tried it, it is quite easy once you know what you are doing. You can consult my notes, which you will find in this forum. It is also relatively fast.

[Celtic Minstrel]

I've looked at the Mac source code, and I honestly can't figure out why it doesn't compile. I'll keep trying every so often, though.

[Ishad Nha]

Whenever I try to compile any Spiderweb source code I get 10 or 11 pages of errors and warnings. This is on Dev-C++. Borland is even worse.

 

I think Jeff uses Code Warrior, he has praised it in the Credits part of a help file somewhere. He may use some professional compiler for all I know.

[Prince of Kitties]

Hmmm. Is it possible to edit things like e.g. PCs' intrinsic speed?

 

(I just had an amusing thought involving an entirely naked god party...)

 

On a more serious note, I've wished for a while to introduce stat boosting/draining potions to BoE, and wholeheartedly approve of anything that might further that goal.

[Ishad Nha]

Increase a PC's inherent speed? No, the monsters use the data structure "monster_record_type;", this is from the source file "global.h", there is mention of both speed and "ap", presumably the latter is action points. Meanwhile the PCs use

struct pc_record_type

{

short main_status;

char name[20];

short skills[30];

short max_health,cur_health,max_sp,cur_sp,experience,skill_pts,level;

short status[15];

item_record_type items[24];

Boolean equip[24];

Boolean priest_spells[62],mage_spells[62];

short which_graphic,weap_poisoned;

Boolean advan[15],traits[15];

short race,exp_adj,direction;

 

/* functions */

bool isAlive() { return (main_status == MAIN_STATUS_ALIVE); }

};

 

This does not mention speed or action points.

[Ishad Nha]

Carrying on from my post of April 29, Ormus said that the funny numbering system is in fact XOR encryption. He wrote a simple program to remove this.

 

In Hexplorer, the XOR encrytion can be added and removed at will: Edit > Operation > XOR Selection

 

The encryptions used are:

places 6 through 45,913: XOR = 92

places 62,298 through 73,685: XOR = 107.

The first encrypted part of the save file represents the party records, while the second part is the collection of records for the individual PCs. No other parts of the file are encrypted.

 

In decrypting the first part remember that the character for 92 is \, when the XOR dialog box comes up just type this in the Text box. Ditto in the second part, 107 corresponds to "k" in ASCII so just type k in the Text box. Mercifully the same process is used to both encrypt and decrypt a selection of characters.

 

Hexplorer also has Shift + Click selection,

[Ishad Nha]

One use for save game decrypting is figuring out just where you put that boat or horse. In BoE save game files this information is found early in the file:

Start Length Data Type

4,724 300 boat_record_type boats[30];

5,024 300 horse_record_type horses[30];

 

In each record there are ten bytes;

two bytes for coordinates

four bytes about outdoor location

two bytes for town number

one byte to say that the boat or horse exists

one byte to say whether it is anyone else's property.

 

This will tell you about all boats and horses, including the ones you don't currently know about.

[Ishad Nha]

Write Up: in town save game files.

 

Record~Start~Length~Start: Hex

party_record_type~0~45,914~0

unsigned char setup[4][64][64];~45,914~16,384~B35A

pc_record_type[6]~62,298~11,388~F35A

out_info_type;~73,686~9,216~1,1FD6

current_town_type~82,902~16,014~1,43D6

big_tr_type~98,916~6,056~1,8264

stored item lists, current town~104,972~7,590~1,9A0C

stored item lists, 3 saved item rectangles~112,562~22,770~1,B7B2

stored_town_maps_type town_maps[100][8][64];~135,332~51,200~2,10A4

stored_town_maps_type town_maps[100][8][64];~186,532~51,200~2,D8A4

stored_outdoor_maps_type outdoor_maps[100][6][48];~237,732~28,800~3,A0A4

sfx~266,532~4,096~4,1124

misc_i~270,628~4,096~4,2124

 

If saving maps, grand sum is 274,724 bytes.

 

Town location, when in town mode and only when in town mode, town number seems to be found at: 98,909 and x,y coordinates at 98,914 and 98,915. Town number also occurs at place 82,902

 

unsigned char item_taken[200][8];

Question arises, if there are 64 possible items in a town and there are only 8 entries in the items taken for each town, what is going on? It seems that there must be bitwise manipulation going on, 256 = 2^8.

 

struct current_town_type

~82,902~2~short ~town_num

~82,904~2~short ~difficulty;

~82,906~3,506~town_record_type~town;

~86,412~4,096~char ~explored[64][64];

~90,508~1~Boolean~hostile;

~90,509~8,400~creature_data_type ~dudes[60];

~98,909~2~short ~which_town;

~98,911~2~short ~friendly;

~98,913~1~Boolean~in_boat;

~98,914~2~location ~p_loc;

~total size:~98,916

 

Here big_tr_type is just the usual for the variable part of any big town record: terrain, room rectangles, creature start type and lighting. For medium and small towns, their records will always fit into a big town record.

[Tonweya]

You guys left me so far behind that after the dust settled I felt like I'm standing in a desert. ;^{

I do seriously thank you for the response.

What I was hoping for was a way to know which Flags may have been tripped inadvertently, or if a Node wasn't being called, and why.

"scentext" and "specdata" have been a God-send to someone with my limited abilities. I can "Find" an item with these and was hoping I could "Find" the actual "Status" of the Flags and Nodes.

When I couldn't cast "Ritual Sanctification" on an Evil Altar, I had no idea if the Flag had been inadvertently tripped, or if the Node was non-functional.

N-E-ways,

Again, thanks,

me

[Ishad Nha]

In BoA it is very simple to see if a certain state is being called or not, just insert messages before it is meant to happen and afterwards too. This would work in BoE too, though you would need to use one node per message.

 

Stuff Done Flags start at offset 14, there are 3,100 of them. They are of the form "unsigned char", that means that each SDF occupies one space/place in the file. Now the last 100 SDFs are used by the game for internal purposes, only the first 3,000 are actually used by scenarios. So you are looking at addresses 14 thru 3,013. 300x * 10y = 3,000.

I imagine that they are sorted by "x" then by "y", but I have never checked this. Now in the open source versions of BoE you can activate Debug Mode with Shift + D, then you can set a SDF with the Shift + S keys. Then you could save the game and open the save file in Hexplorer and see where the relevant numbers were. Currently there is no easy way to print out BoE current SDF values.

(If you have Caps Lock on you can just press D and S.)

 

Edit:

Using "A" and "B" rather than "x" and "y". The set SDF function seems to crash if the A part is 0. Actual formula seems to be: offset = 10A + B + 14. Remember that:

many hex editors display the value of a byte in hexadecimal not decimal

this part of the save file is encrypted with use of the XOR function.

 

In Hexplorer the XOR dialog box fills up with junk regularly, (coming from the buffer?) Before each use make sure that the Text box contains only one character, a backslash: \.

 

[Chokboyz]

Originally Posted By: Ishad Nha

Currently there is no easy way to print out BoE current SDF values.

While in Classic BoE game, enter Debug Mode, press Shift-S, enter the SDF flag to check (e.g 150 and 5 for SDF[150][5]) and enter -1 for the value. It should print out the SDF value (BoA style if i'm not mistaken).

Originally Posted By: Ishad Nha

The set SDF function seems to crash if the A part is 0.

It doesn't crash : 0 being the result returned for non integer characters, the SDF[0][x] are not checked.

Concerning nodes checking, it's bit more complicated because nodes positions are loaded into the misc_i array (which are the last bytes of the savefile, if i remember well) each time a town is entered. Nonetheless, the SDF checks should suffice, since if a one-time node is triggered the corresponding SDF is set ...

Hope it helps,
Cbokboyz

[Celtic Minstrel]

Originally Posted By: Chokboyz

Originally Posted By: Ishad Nha

The set SDF function seems to crash if the A part is 0.

It doesn't crash : 0 being the result returned for non integer characters, the SDF[0][x] are not checked.

Well, that's no good. We need to fix that somehow.

[Chokboyz]

Originally Posted By: Celtic Minstrel

Well, that's no good. We need to fix that somehow.

Fixed.
If the first character entered is a non-integer, the function will now behave has if 0 had been entered ...

Chokboyz

[Celtic Minstrel]

Not quite what I meant, but close enough.