File formats

[Celtic Minstrel]

This is here for reference and discussion; when finalized, I'll add it to the repository in a text file.

 

Basically, I'm sticking to my plan of the scenario and the save game both being a gzipped tarball with a custom header. The graphics sheet, however, will be replaced with a folder containing all the graphics, possibly on more than one sheet. (Some day it may be possible to edit these from within the scenario editor. That day however is quite distant.)

 

The files in the tarball will come in three basic flavours: data files, string files, and grid files. All will be plain text, but I'll give them unique extensions to help identify the differences.

 

The data files will be easily extensible: each line contains a keyword followed by arguments. One line will be read at a time and the relevant property set according to the arguments. So it will be easy to add additional keywords without breaking the format.

 

The outlines below are a draft of what it might look like. The headers are not included in the description (yet).

 

Click to reveal.. (Structure of a Scenario)

(Note: x or y indicates an unspecified digit)

Code:

scenname.exs (gzipped tarball)  _scenname.exs/    scenname.dat    scenname.str    items/      xxxxx.dat    monsters/      xxxxx.dat    terrain/      xxxxx.dat    towns/      xxx.dat      xxx.str      xxxdlg.dat      xxxdlg.str      xxx.ter    outdoors/      xx_yy.dat      xx_yy.str      xx_yy.terscenname.exr/  scenname.png  sheetxxx.png  ??????.png

 

(I think that's everything needed.)

 

Click to reveal.. (Structure of a Saved Game)

Code:

save.exg (gzipped tarball)  _save.exg/    party.dat    magic_store/      0.dat      1.dat      2.dat      3.dat      4.dat    notes/      journal.dat      journal.str      talk.dat      talk.str      encounter.dat      encounter.str    saveitem0.dat    saveitem1.dat    saveitem2.dat    summons.dat    pcs/      pcx.dat      pcxitems.dat

(Can't think of anything else at the moment.)

[Niemand]

What are the 0, 1, . . . , 4.dat files in the save archive, and what is the difference among the multiple saveitemN.dat files? Also, are you putting every monster, terrain, or item definition in its own file, from what I read of the scenario outline? That seems a bit excessive.

 

Lastly, I think this may have been discussed before, but will your header actually be compatible with the standard gzip file format, such that a normal gzip decompressor could unpack the file?

[Celtic Minstrel]

Oops, I forgot to use code tags to preserve indentation. That has been fixed.

Originally Posted By: Niemand

What are the 0, 1, . . . , 4.dat files in the save archive,

Those are the contents of the random item shops - one file per shop.

 

Originally Posted By: Niemand

what is the difference among the multiple saveitemN.dat files?

Again, one file per saved item rectangle.

 

Originally Posted By: Niemand

Also, are you putting every monster, terrain, or item definition in its own file, from what I read of the scenario outline? That seems a bit excessive.

I was considering it, yes. Each monster/item/terrain has many properties, so storing them in separate files is slightly simpler. However, it could just as easily be one file for all monsters, one for all items, and one for all terrain, if that is preferred. I'd probably just add an extra parameter to all keywords (preceding all others) which specifies the number of the object. Either that or add a keyword to specify that all subsequent keywords apply to a specific object, until the keyword appears again.

 

Originally Posted By: Niemand

Lastly, I think this may have been discussed before, but will your header actually be compatible with the standard gzip file format, such that a normal gzip decompressor could unpack the file?

I don't believe so. I plan to use the "compress" functions in zlib, rather than the gz* functions, because I need a custom header. I'm not quite sure how these work though – it may be possible to uncompress it with a regular decompressor if you delete the custom header.

 

(To be fair, the custom header is only really needed for the scenario format, so that if the scenario is read by an older version of the game it will be recognized as a scenario created by a later version which it cannot read. Since the save file had no version info, there's no way to get this behaviour anyway, and so the custom header could be compressed along with the rest of the file.)

[Niemand]

Okay, so your mention of gzip in the first post was just a red herring. I would suggest then that there may be no reason to adhere to the tar format internally, just use something similar (blocks which know their name and length) but simpler (leaving out permissions data, owner name data, and so on) so that your block headers could be much smaller than the tar format's half kilobyte headers.

[Celtic Minstrel]

My mention of gzip was an indication that I intended to use zlib for DEFLATE compression, more than anything else.

 

Using a custom tarball-like format does sound like a good idea.

 

For the scenarios, at least, I intend to extract them to a "temporary" location (possibly within the application package? Either that or in the Application Support folder). This gives a few benefits. For one thing, it eliminates the need to save before doing certain things such as adding or deleting a town (just save automatically in the temporary location; when the user explicitly saves, the temporary copy will be compressed into the actual file). It also means an auto-recover feature can be implemented – if a file already exists in the temporary location when starting the program it's an indication that the program did not terminate properly (ie it crashed), because the temporary copy is deleted on exit. Therefore, it can load up the temporary to recover a good chunk of data that the person was working on before the crash (it might not save everything, but it would at least save any changes towns or outdoors apart from the currently opened one).

 

I'm not sure whether it's worth doing the same with save files. It might be, though. Perhaps it could be a way of implementing an auto-save feature without overwriting the actual file unless you save explicitly.