Savefile Compatibility

[Niemand]

I'm still working on trying to ensure exact binary compatibility between old style savefiles across versions and platforms. The problem that I have currently run into is that the current Mac development version of the game cannot correctly read old or current Windows files in my testing.

 

These findings are based on revision 120 of the source code, and the windows savefiles CHEESLOC.SAV(An original Windows BoE savefile,VoDT, Sweetgrove, party location(9,45), decrypted copy:decCHEES.sav) and WINTEST.SAV(A Win32_CBoE_1.0 savefile, VoDT, Fort Talrus, party location(7,6), decrypted copy:decWINTEST.sav).

 

Using the aforementioned program version and files, loading fails, and in addition I have identified what appear to be critical incompatibility problems that occur before the program aborts loading: In the Windows savefiles the scenario name is found at offset 45658. Given that this is within the party_record_type structure, which is preceded only by the 6 byte header (3 16 bit flags), it is implied that the scen_name member of the party_record_type structure should be at offset 45652 wthin that structure. However, in the current Mac version, that member is at offset 46142 within that structure, a difference of 490 bytes. This causes reading of the scenario name and all subsequent data to fail, and implies that an unknown amount of data earlier within the party_record_type structure is also read incorrectly. For reference, the structure in question is:

Code:

	struct party_record_type {		int32_t age;		int16_t gold,food;		unsigned char stuff_done[310][10],item_taken[200][8];		int16_t light_level;		location outdoor_corner,i_w_c,p_loc,loc_in_sec;		boat_record_type boats[30];		horse_record_type horses[30];		creature_list_type creature_save[4];		int16_t in_boat,in_horse;		outdoor_creature_type out_c[10];		item_record_type magic_store_items[5][10];		int16_t imprisoned_monst[4];		char m_seen[256];		char journal_str[50];		int16_t journal_day[50];		int16_t special_notes_str[140][2];		talk_save_type talk_save[120];		int16_t direction,at_which_save_slot;		char alchemy[20];		Boolean can_find_town[200];		int16_t key_times[100];		int16_t party_event_timers[30];		int16_t global_or_town[30];		int16_t node_to_call[30];		char spec_items[50],help_received[120];		int16_t m_killed[200];		int32_t total_m_killed,total_dam_done,total_xp_gained,total_dam_taken;		char scen_name[256];	};

The first three members definitely match, so the incompatibility exists in one or more of the members between food and scen_name. As though this weren't enough, the problems seem to worsen as reading the file progresses; by the time the program reaches the point of reading the field I was originally investigating (the party's position in the current town), it is reading form approximately offset 99526 rather than the correct offset of 98914 for a difference of 612 bytes.

 

I'll continue trying to figure this out, but I wanted to ask: 1) Can anyone else (Celtic Minstrel) get the development Mac version to read these files (that is, I want to make sure that this isn't something I've somehow broken in my copy), and 2) Does anyone have ideas about how these incompatibilities came about? I'm stumbling around in the dark at the moment, so any clues would be helpful.

[Chokboyz]

I noticed that too when implementing PPC Mac saves compatibility : the two structures are not read the same way (even the final number of byte read differs).

 

I have no clue of what is causing that, but found that, in Windows reading PPC Mac, when reading the int32_t total_m_killed,total_dam_done,total_xp_gained,total_dam_taken two bytes were skipped, messing with the rest of the loading sequence ...

I manually correct those bytes (and maybe others things) in the load_file() function.

 

If the problem is Intel Mac no being able to read Windows format file, i'm afraid i can't help

(note that, Windows loading procedure don't read structures as whole but field by field ... that shouldn't make a difference, but i had to mention it)

 

Hope it helps,

Chokboyz

[Celtic Minstrel]

I don't think I ever got the development version to be able to read Windows saves properly.

[alexcurylo]

You're almost certainly running into structure padding differences. Even when you define the padding as closely as possible given compiler directives, platform/compiler incompatibilities will still creep in.

 

For starters, I see a "Boolean" in your definition above. That can range from one to four bytes. I also see some char[] fields that are not long word multiples, which is another likely place for a trying to be helpful compiler to insert some pad bytes for you behind your back.

 

Having debugged more of this kind of thing in my game-porting life than I really care to think about, you might as well just print out a long list of printf(sizeof(StructureXXX)) statements at the beginning of the file code reading, and see where the differences show up on each platform. Trying to analyze where the differences will be from reading source code, although you can often make some good guesses (any compiler-defined type like "Boolean" is a big waving red flag, for starters), can very well actually be completely impossible.

[Niemand]

We have that already, you can do better than just sizeof and also use offsetof to examine structure layouts, which I did (See this post for details). The numbers of bytes by which we're off are large enough that I'm not sure I'd blame padding alone, although we've been trying to get that under control from the start. The job isn't impossible, as in theory we could always add enough padding bytes to both platform's definitions to make them match, but it would likely just be better, given that we can distinguish the two platform's files, to just have separate reading routines to handle them, if we can determine what the important differences are. In part this has stalled a bit since I hadn't gotten around to trying to do the same layout examination of the structs from the Windows side, so so far we've only seen half of the picture.