Blades Save File Format

[Celtic Minstrel]

At some point (possibly fairly soon) I'll be writing code to save a game in a new format, and I'd like to get people's opinions on what (if anything) should be saved in the savegame file that isn't already?

 

The only thing I've determined so far is that the first two bytes will be 0x0B0E to distinguish it from the old format.

 

Click to reveal.. (List of things stored in the current file format)

Flags:

1. In town or outdoors?
2. In scenario?
3. Maps saved?

(The new format will have the 0x0B0E identification flag preceding these, and the version number following them.)

 

Party info:

• Age (which is, I believe, the number of moves since entering the scenario; it's used to calculate the day)
• Gold
• Food
• Stuff done flags (310 high by 10 wide)
• Item taken (200 by 8; this may indicate whether the party has taken the preset items in a town.)
• Light level
• Outdoor location (4 points!)
• Boats (not sure if this include boats not owned by the party; it likely does though)
• Horses (ditto)
• Monster lists for the last 4 towns visited
• short in_boat, in_horse (uncertain why this is a short)
• Monster list for the outdoors
• The items in the 5 random item shops.
• Monsters in the Soul Crystal
• char m_seen[256] (I think this may be for the monster roster menu?)
• Journal entries (which string and the day occurred – not currently used)
• Special encounter notes (currently references the scenario, town, or outdoor strings by number)
• Conversation notes (similarly, references personality and node by number, so would be lost when leaving scenario)
• Facing direction
• short at_which_save_slot (no clue what this does)
• Known alchemical recipes
• Whether each town is hidden
• Whether special events have occurred
• Active event timers (including whether it's a town timer and the node to call when it expires)
• Special items
• Whether each instant help message has been received
• Number of monsters killed in each town
• Total of monsters killed (not sure whether this is reset when entering a scenario)
• Total damage done
• Total experience gained
• Total damage taken
• Name of current scenario

(I've made a few minor changes to the above so far: gold and food can no longer become negative, and the final four statistics have an increased limit of 2^64-1 rather than 2^32-1. I have no idea whether that would be truly necessary, but it seemed like a good idea.)

 

Setup save:

4 arrays of 64x64 (I suspect this contains the terrain for the last 4 towns visited.)

 

Character info: (x6)

• Main status (ie dead, petrified, dust, etc)
• Name
• Skill levels/
• Maximum and current health
• Maximum and current spell points
• Experience, skill points, level
• Status effects (eg curse, web, magic resistance, etc)
• Inventory list, including whether each is equipped
• Known spells
• Which graphic
• Which weapon is poisoned
• Traits (oddly, there are two arrays for this – one is advan[15] and is never checked or written, the other is traits[15] which is checked and written)
• Race
• Experience adjust (which doesn't appear to be used – experience adjust is calculated on the fly)
• Facing direction (I think)

 

(If not in a scenario, stop here – I think)

 

Outdoor info:

An array of 96x96 (I'm pretty sure this contains the outdoor terrain for 4 outdoor sections)

 

Current town info (if in town):

• Town number
• Town difficulty
• Town record (the entire town as saved in the scenario; the extended size-dependent town record is also saved)
• Exploration (I think this is whether the party has seen each space; it's a 64x64 array)
• Whether the town is hostile
• The list of monsters in the town
• bool in_boat (I'm not sure why this is here as well as in the party record)
• Town location (ie where they are in the town)
• List of items in the town

 

Stored items:

3 lists of items (these are the items that you placed in a saved item rectangle)

 

Saved maps (if enabled):

Town maps

Outdoor maps

 

SFX and MISC_I:

Two 64x64 arrays of something. They are included even when the party is not in town, yet from their size I think they are actually part of the town info (for example, they may include quickfire or barriers placed in the town).

It appears that some stuff is in fact redundant...

 

Anyway...

 

Some things that aren't currently included that I have thought of maybe adding:

• Custom graphics (so that items with custom graphics can be taken out of the scenario). This would most likely be a separate companion file, though. It could theoretically also include custom PC graphics. I can't think of any other kind of custom graphic that may need to be saved in the savegame file.
• Instead of references to the journal, special encounter, and conversation strings, it might be nice to include the actual string. This would mean that they can be carried between scenarios if you wish (and if you don't, they're easily deleted).
• Statistics such as "number of scenarios won" would be nice.
• More stuff done flags. I don't know if anyone has ever run out of stuff done flags, but adding a few more can't hurt.
• For the outdoors monster list, include each monster's sector of origin.

[Celtic Minstrel]

Another idea: if the party has any items that summon monsters, we could include the descriptions of those monsters in the save file when the party leaves the scenario, and if they have custom graphics we could copy them to the party graphics sheet.

 

 

Does anyone else have any ideas?

[Ishad Nha]

Custom graphics problem should be addressed one way or another. More SDFs won't hurt, BoA has three times as many.

As for the strings and text generally, they would only be relevant in the original scenario.

[Celtic Minstrel]

Originally Posted By: Ishad Nha

As for the strings and text generally, they would only be relevant in the original scenario.

Not entirely true. If someone wanted to keep a record of past scenarios they've visited with the party, for example.

But it should be the player's choice, at least.

[Celtic Minstrel]

Okay, I'm making the following changes to the party record class:

1. The name of the custom graphic file. It should be the same as the save file's name except with a different extension; if it isn't, a new copy that satisfies this will be created the next time the game is saved.
2. An array to keep track of whether each custom graphic in the sheet is used. I think it should be an array of bool[250], meaning that the party can't have more than 25 rows of custom graphics. If every PC had a different custom graphic, and every one of their items had a different custom graphic, and all 6 PCs have a full inventory, this still leaves room for 20 of those items to summon a monster from another scenario. That's definitely plenty (maybe even too much).
3. An array of monster types to support taking items that summon creatures out of the scenario.
4. Expand the width of the SDF array from 10 to 50.
5. A "number of scenarios won" statistic, and a "number of scenarios played" statistic. Perhaps there could also be a "number of scenarios lost" statistic? Only, that would require a notion of "losing" a scenario which, as far as I know, doesn't exist (apart from death, of course). I highly doubt there's any reason to let these stats get higher than 65535, so I'll make them shorts.
6. Outdoor wandering's sector of origin, so the right special node(s) can be called. Perhaps for compatibility we should also add a setting to outdoor encounters to keep the old behaviour (which could even be quite useful in certain circumstances). If the outdoor wandering encounter should call specials from the current sector rather than its sector of origin, it will have a sector of origin of (100,100). This is guaranteed to be invalid, since there can be no more than 100 sectors and 10000 would be required for (100,100) to be valid.

 

 

Any other suggestions are still welcome. In particular, I'd like to hear other opinions about including the recorded strings in the save file. Advantages include: easier to find the strings (from a programming standpoint), strings can be preserved when leaving the scenario. Disadvantages include larger save files.

[The Almighty Doer of Stuff]

Perhaps have a switch in the preferences for the saved text, like the Don't Save Maps switch? While we're at it, maybe the same for the custom graphics.

 

I'm curious as to the necessity of including the PC graphics in the save file, though. The scenarios themselves don't actually affect the PC graphics, so perhaps it would make more sense just to add the custom PC graphics to the collection of PC graphics of the program itself.

[Celtic Minstrel]

The reason for potentially including PC graphics is that I'd like to create a special node that can add players to your party (if there's free space, of course), in which case the scenarios suddenly will affect the PC graphics – to be precise, they'd specify the graphic for the added character, which could easily be a custom monster graphic.

 

 

The switches are a good suggestion, but they won't affect the in-memory format of the party record – only the file format. It'd mean the list of header flags needs to be expanded even more, to 7, in order for the game to recognize whether each thing is saved.

 

Also, custom graphics won't be saved in the party until they leave the scenario, because until then it's not necessary. The same could easily be applied to the strings. I'll just go ahead and add fields for the strings – it can't hurt.

[keira]

Maybe it could be possible to actually have the custom graphics embedded in the save file rather than a separate file? Becasue some moron (ie. me) will forget to copy/move/whatever the graphics file, and then spend forever trying to figure out why Graphic X won't appear.

[Niemand]

The tricky aspect of that would be that your file format would then need to be able to contain arbitrarily sized chunks of data in another format (such as bmp data or PNG data). Such a thing certainly isn't impossible, but it might be a bit more cumbersome to program.

 

Implementation thought: It seems to be that the best way to do such a thing would be to embed a block of image data with a small header to tell yourself things like what type of data it is, the image number and most importantly, the length of the image data. after reading such a header, you would just read in a block of binary data of the stated size and pass it off to your graphics library to be decoded as an image.

 

On the one hand this would have the nice feature that it makes your file format really simple for users to handle; they don't need to know or worry about the data you've embedded in it. On the other hand, it adds significant complication to parsing the file format.

[Celtic Minstrel]

Originally Posted By: w-dueck

Maybe it could be possible to actually have the custom graphics embedded in the save file rather than a separate file? Becasue some moron (ie. me) will forget to copy/move/whatever the graphics file, and then spend forever trying to figure out why Graphic X won't appear.

Yes, that would be very nice. However, as Niemand said it would greatly complicate matters.

It wouldn't be hard for the game to give an error message if the graphics file is not found, though. Then at least you know why Graphic X is not appearing.

[*i]

I am indeed in favor of putting in more information in the save files. Hard disk space is cheap these days and internet transfer speeds are fast enough. If you have the time, I would even go so far as to say this data can be ASCII as opposed to binary to get rid of the cross-platform portability issues.

 

Also, start giving the formats version numbers and place it near the head of the file. Version 0 would be the default format, 1 would be your new one, and future versions would have further increments. This allows future designers to change the format specifications with greater ease.

 

What I would also do is make a lot of these things dynamic. The header information can contain info on the size of the arrays and stuff like that. One of the most frustrating things with the old scenario file format were these hard coded values. If you grow the SDF array to 310x30, put these values in the save file. If someone later decides to change the dimensions to 310x50, it will be easy to reconcile the two versions.

 

I'm seriously considering getting back into rewriting the scenario format to be more extensible. Sadly, graduate school has been a huge drain on time until the last couple months.

[*i]

One more thing that just crossed my mind is campaign flags so data can be ported between scenarios. Sending these in the save file would be a very good thing. It may be worthwhile to start a discussion in another thread about the details of access/implementation.

[Aran]

Quote:

I would even go so far as to say this data can be ASCII as opposed to binary to get rid of the cross-platform portability issues

This is an interesting point. If you do go with a plain-text readable format, however, I suggest it be gzipped (like Battle for Wesnoth does). You could save substantial space at no cost.

Edit: The proposal to make the SDF matrix dynamically sized opens a different can of worms, namely, "why use a matrix at all"? The one benefit of using a matrix is that you can hardcode its position and size in a binary file. The cost involves having to use coordinates instead of variable names, keeping notes to avoid collisions, and either running out of space or leaving much space unused. If you decide to go with an ASCII-based format, you could as well use variable names like "town42.rentar.isDead", "town1.motrax.isHostile" etc.

Of course, that is a drastic digression from the existing save file format, and it would be difficult to convert old save files to new ones (if that is what you wanted).

[*i]

I totally agree with gzipping. The main reason one would preserve the notion of the SDF array is for legacy reasons. The SDF array is fairly rigid in BoE that changing to a more flexible variable system would create headaches elsewhere in the code.

[Celtic Minstrel]

Originally Posted By: *i

Also, start giving the formats version numbers and place it near the head of the file. Version 0 would be the default format, 1 would be your new one, and future versions would have further increments. This allows future designers to change the format specifications with greater ease.

The lack of a version number will indicate the old format, because the old format doesn't have a version number. The presence of 0x0B0E at the start of the header will indicate that it's a new format save, with a version number. I'm pretty sure I mentioned that plan in the first post.

Originally Posted By: Stareye

What I would also do is make a lot of these things dynamic. The header information can contain info on the size of the arrays and stuff like that. One of the most frustrating things with the old scenario file format were these hard coded values. If you grow the SDF array to 310x30, put these values in the save file. If someone later decides to change the dimensions to 310x50, it will be easy to reconcile the two versions.

I'm planning to make as many things dynamic as possible, but I was going to store their length just before the data. So, for example, if there's an array of 72 preset items in the town, the value 72 appears followed by each item in sequence. And strings would be preceded by two bytes that specify their length. (Probably all sizes would be two bytes.)

The scenario format currently stores a list containing the size in bytes of each of the towns and outdoor sections. I was debating on the workability of inserting the word OUT just before each outdoor section followed by two bytes containing the coordinates of the section, and the word TOWN just before each town section followed by a byte containing the town number. Then the program would have to seek until it finds the keyword, though, and I'm not sure if that would work well. It could be applied to save files too, for example inserting the word PC followed by the number before each pc record, but since the save file is read all at once it wouldn't really be very useful.

Of course, that's another option – if the editor loaded all 100 (max) outdoor sections and all 200 (max) outdoor towns into memory at once, such marker points would be completely unnecessary.

Originally Posted By: *i

I'm seriously considering getting back into rewriting the scenario format to be more extensible. Sadly, graduate school has been a huge drain on time until the last couple months.

It would be nice to have some help. If you do decide to work on it, email me and I'll give you repository access.

Originally Posted By: *i

One more thing that just crossed my mind is campaign flags so data can be ported between scenarios. Sending these in the save file would be a very good thing. It may be worthwhile to start a discussion in another thread about the details of access/implementation.

What sort of format would these flags be?

Originally Posted By: *i

I totally agree with gzipping. The main reason one would preserve the notion of the SDF array is for legacy reasons. The SDF array is fairly rigid in BoE that changing to a more flexible variable system would create headaches elsewhere in the code.

I really wish I could create a Mac package format, but portability dictates that I can't. Zipping is the next best choice. Using a tarball type format would, in my opinion, make it easier to locate things such as a specific outdoor section – in this case, each outdoor section would be a separate file within the tarball. I can also imagine that creating the potential for an auto-recovery system in the scenario editor.

If we did that, though, I'm pretty sure that the current ways of identifying a file (0x0B0E / the two in_town flags for savegames and 0x0A141E28 / 0x14283C50 for scenarios) would no longer work. Unless you can prefix a few bytes onto the file, but then ordinary unzipping and rezipping would invalidate it (if it even worked at all). And if those methods of checking the file don't work, the original version can't give a "created for a later version" message.

Originally Posted By: Arancaytrus

Of course, that is a drastic digression from the existing save file format, and it would be difficult to convert old save files to new ones (if that is what you wanted).

I don't think it would be hard to convert. As long as there's a function to read data in the old format, and a function to write it in the new, conversion will be as simple as reading it in and writing it back out. And if conversion required using text variables instead of stuff done flags, the conversion could just give the variables names in the form "sdf.x.y". I don't think that using variables would be plausible at this time, though. And I don't plan to make the SDF array dynamic either.

[Aran]

You still need a version number. Even if it's just a third byte numbered 0x00 (or, preferably, 0x01 since the 0th version is the old BoE one, even though it doesn't call itself that) byte after the 0x0B0E (which admittedly is a nice pun).

 

The reason is that there will doubtlessly be further versions of this format, and having to add a new distinguishing feature every time quickly gets tedious. Better to code the thing robustly.

[Niemand]

Quote:

If we did that (gzipping the data), though, I'm pretty sure that the current ways of identifying a file (0x0B0E / the two in_town flags for savegames and 0x0A141E28 / 0x14283C50 for scenarios) would no longer work. Unless you can prefix a few bytes onto the file, but then ordinary unzipping and rezipping would invalidate it (if it even worked at all). And if those methods of checking the file don't work, the original version can't give a "created for a later version" message.

I would argue that prefixing the gzipped dat with the file type identification data (file header) is exactly what should be done. That way, when you go to load a file you just read the header, examine it's contents, and decide what to do next. If the header begins with what looks like the start of a version 0 file, start over and read the entire file as version 0. If the header is a valid version 1 header, take all the data after the header, un-gzip it, and read it as version 1. If theheader is valid, but contains a version number higher than your program knows how to handle, display a message that the format is too new and the program cannot read it. If the header is totally unrecognizable, say that.

A detail about the header: Assuming that version 0 has n bytes of header data at the beginning, design you new header to be m bytes long where m>=n. That way, you can open a file and just grab the first m bytes to examine. If the first n of those are identifiably version 0, it won't matter that you may have also grabbed some of the data along with the header, since you can always jump back to the beginning of the file and start over.

Lastly, CM, I think your complaint about using gzipped data with a header is that it wouldn't really be a valid gzip file. That, however, is a good thing: the goal is to create a valid BoE file, not a gzip file, so gzip shouldn't be used, or be able to be used on it. The point is that gzip really has nothing to do with it, you just want to compress the data inside your file. The PNG format also happens to use DEFLATE compression, but it's not as hough you try to use gunzip on your image files and then get upset when it doesn't work.

[Aran]

If your format cannot be decompressed with a normal archiving tool, then there is no purpose whatsoever to using a compressed ASCII format rather than a native binary one.

 

The file can only be opened with your software, meaning your software may as well handle all its manipulation rather than exporting it to a form humans could read if they could figure out how to decompress it.

[Niemand]

I don't agree with you entirely Aran, although I agree that it would decrease the usefulness somewhat.

 

That being said, compression is still a very good idea for a binary format. Many portions of the data, even when represented in binary are very sparse, like the SDF array. In some quick tests with BoA savefiles, I get compression factors ranging from 6 to better than 65(!), and for BoA scenarios factors which are consistently over 10. That's good enough to be worth pursuing.

 

Also, I just realized, there's no reason not to put your new style header inside the compressed data, I think. That way, you'll be able to have your cake and eat it too. I'll expand on this later, as I have to run to a meeting.

[Celtic Minstrel]

Originally Posted By: Arancaytrus

You still need a version number. Even if it's just a third byte numbered 0x00 (or, preferably, 0x01 since the 0th version is the old BoE one, even though it doesn't call itself that) byte after the 0x0B0E (which admittedly is a nice pun).

The reason is that there will doubtlessly be further versions of this format, and having to add a new distinguishing feature every time quickly gets tedious. Better to code the thing robustly.

Yes, I know I need a version number, but the fact is that the old savefile format simply doesn't have one. It has a six byte header which specifies certain details about the format: the first two byte specifies whether the party is in town, the next two specify whether the party is in a scenario, and the final two specify whether the savefile contains saved maps.

So, the first two bytes are the first step of identification. If they match the old-format flag for "in town" or "not in town", then it's an old format save. Otherwise, if the first two bytes are 0x0B0E, it's a new format save, whose header will include a version number.

This problem does not occur with the scenario format, since it already has a version field.

Originally Posted By: Niemand

I would argue that prefixing the gzipped dat with the file type identification data (file header) is exactly what should be done. That way, when you go to load a file you just read the header, examine it's contents, and decide what to do next. If the header begins with what looks like the start of a version 0 file, start over and read the entire file as version 0. If the header is a valid version 1 header, take all the data after the header, un-gzip it, and read it as version 1. If theheader is valid, but contains a version number higher than your program knows how to handle, display a message that the format is too new and the program cannot read it. If the header is totally unrecognizable, say that.

Hmm, okay then. I guess I was seeing a problem where it didn't exist then.

Originally Posted By: Niemand

A detail about the header: Assuming that version 0 has n bytes of header data at the beginning, design you new header to be m bytes long where m>=n. That way, you can open a file and just grab the first m bytes to examine. If the first n of those are identifiably version 0, it won't matter that you may have also grabbed some of the data along with the header, since you can always jump back to the beginning of the file and start over.

Yes, I'm planning something like that. The old savefile header is 6 bytes long, and the new savefile header will be longer. The new scenario header will probably just be the same length, but I may change the meaning of some fields other than the version one.

Originally Posted By: Niemand

Lastly, CM, I think your complaint about using gzipped data with a header is that it wouldn't really be a valid gzip file. That, however, is a good thing: the goal is to create a valid BoE file, not a gzip file, so gzip shouldn't be used, or be able to be used on it. The point is that gzip really has nothing to do with it, you just want to compress the data inside your file. The PNG format also happens to use DEFLATE compression, but it's not as hough you try to use gunzip on your image files and then get upset when it doesn't work.

Hm, I guess that makes sense. I see your point here.

Originally Posted By: Arancaytrus

If your format cannot be decompressed with a normal archiving tool, then there is no purpose whatsoever to using a compressed ASCII format rather than a native binary one.

The file can only be opened with your software, meaning your software may as well handle all its manipulation rather than exporting it to a form humans could read if they could figure out how to decompress it.

Well, in fact the point of compression isn't simply to reduce filesize, as I see it. It's to simplify manipulation and avoid having to calculate offsets. I don't want to simply gzip it – I want a tarball archive containing multiple files. Then when loading a scenario into memory, it will untar it to a temporary location and read in one of those files at a time as appropriate.

For the savegame file there isn't really any need for a tarball, since the whole file needs to be read in all at once. However, for the scenario format, we could then keep each town in its own file, and each outdoor section in its own file. For the scenario editor, this could even help in some ways – it could give a way to recover data after a crash, or support an option to revert a town to the last saved version. It would remove the need to force the user to save before changing towns or outdoor sections (if the user doesn't save, it's automatically saved to the temporary location). And, if an interface for adding custom graphics was put into the editor, it could even remove the need for a separate custom graphics file.

And even for the savegame file, storing the values as ASCII rather than binary would remove the need to handle endianness. I know that with the Intel switch, the endians are now the same; but we also need so support Mac OS PPC, so if a binary format is used we'd still have to consider endianness. An ASCII format, whether compressed or not, would remove that requirement since there are no multibyte values to swap in an ASCII file.

[*i]

I guess I was not clear exactly what you meant. I'm just advocating for some common sense system that we all decide on now so that versioning is standardized.

 

The way I planned things with the scenario format was a routine that updated things from format n to format n+1 until the most recent format was reached. This way the Scenario Editor would auto-convert old scenario files without much effort.

 

CM touches on the endianness advantage of going to an ASCII format. Even if we make the assumption of an Intel chip set (which is not advisable at this point), there is no guarantee that the endian consistency will continue forever and an ASCII format saves future developers from any headaches. In addition, there are definite advantages to an ASCII format in that it allows a user/designer to more easily modify things about their party and allows for a more free format specification for greater flexibility.

 

As for storing the SDF array, it is probably going to be a bunch of zeroes. A good way to help with the ASCII format is to simply store an ordered triplet of each non-zero element:

 

SDF 1 6 2

 

This would say SDF(1,6) = 2. Upon loading the scenario, the SDF array will need to be set to zero. Each read would check for the SDF prefix, and modify the appropriate element.

 

Campaign flags would depend on how one handles scenario records and retrieval.

[Celtic Minstrel]

A nice thing about an ASCII format is that we wouldn't really need to store the length of each string – a simple readline would grab the entire string (since they can't contain newlines).

 

If we're using an ASCII format, I'm definitely going to use the iostream library instead of the File Manager to read anything, and I'll use Quicktime for the graphics and store them as PNG. (Unless someone has a good reason not to use Quicktime?) Then I'll only need the Resource Manager to access program data (such as dialogs).

 

(Speaking of which, would anyone happen to know if there is a way to get a dialog from a nib file, retrieve a list of its elements, and then process and draw them with the game's current dialog code? Because if I could do that, then I'd at least be able to edit the dialogs with a graphical interface.)

 

 

 

 

As for campaign flags, one thought I came up with is making them a dictionary with the key being either the scenario's name or a key chosen by the scenario. In the former case, when retrieving the flag a scenario would need to specify the key; in the latter case it would not. And the "value" would in fact be an array, most likely (though I suppose it could be another dictionary if we want named variables...).

[Niemand]

I would suggest that you go ahead with Quicktime. I'm sure Windows has native libraries to load and display PNG images as well, so there doesn't seem to be any reason to do something really low level like building in libpng.

 

I don't think that it would work very well to try to attack the old code to UI elements stored in a nib file. I'm not even sure if you can get the controls stored that way into an antiquated enough form since Jeff used the Control Manager, rather than the more modern HIView library. I looked at that, and decided that nothing short of a full conversion to nibs and custom controls seemed sensible. I've done a little work in that direction, but it won't be useful for some time to come.

 

I agree one the campaign flags, my first thought was to also store them by scenario name. I think that this has a couple of advantages: Firstly, the designer doesn't have to worry about choosing a key. Secondly, scenario names are already essentially unique, so one doesn't need to worry too much about collisions. Having the stored data also be a dictionary would also be nice, although designers could probably deal with having to index their data numerically.

[Celtic Minstrel]

Quicktime it is then. I know there's Quicktime for Windows, but I didn't think relying on that would be a good idea since it's not exactly a standard component.

 

I pretty much copy-pasted the load/save image routines from your Graphic Adjuster code and changed BMP to PNG, so that's actually already dealt with.

 

Originally Posted By: Niemand

I don't think that it would work very well to try to attack the old code to UI elements stored in a nib file. I'm not even sure if you can get the controls stored that way into an antiquated enough form since Jeff used the Control Manager, rather than the more modern HIView library. I looked at that, and decided that nothing short of a full conversion to nibs and custom controls seemed sensible. I've done a little work in that direction, but it won't be useful for some time to come.

I couldn't find any equivalent of CountDITL that would work on nibs. I know you were working on the dialog engine more for BoA, right? Would it work just as well for BoE? The current dialog code is fine for now, but using Rezilla to alter a dialog (or add one!) is going to be tedious (I'll need to guess at each item's rect and then test to see if it's right). I do have the demo of Resourcerer, but I think it doesn't export clipboard data (though if it did that would be a decent temporary measure...)

 

Originally Posted By: Niemand

I agree one the campaign flags, my first thought was to also store them by scenario name. I think that this has a couple of advantages: Firstly, the designer doesn't have to worry about choosing a key. Secondly, scenario names are already essentially unique, so one doesn't need to worry too much about collisions. Having the stored data also be a dictionary would also be nice, although designers could probably deal with having to index their data numerically.

Yes, the designer won't have to worry about choosing a key, but they'll need to specify the key to load the information from another scenario. That's probably not too hard to manage, though – even if the special node format is still confined to only numbers, the key could be stored in a scenario string (and there'll be no more limitations on the number of strings).

[Niemand]

Oh, right. I'd forgotten that the whole point of campaign flags is to transfer data between scenarios (Hits self in head). Anyway, I think your work around for the fact that the nodes can't directly work with strings sounds good. It'll be annoying, but no more so than any other aspect of that system.

 

I worked a little on the existing dialog engine for the 3D Editor to fix some minor bugs (at which I succeeded), and to try and fix some major annoyances (at which I failed miserably). I've also started trying to put together a Cocoa replacement, but as far as I've gotten is a custom NSWindow sublcass with the appropriate wood textures. I got stuck trying to mimic Quickdraw's rather unique font rendering, and haven't even started on the many types of buttons that will be needed. Also I decided that I would do it in Cocoa, with an eye toward a glorious day when the Mac 3D Editor can be rewritten in Objective-C++, but that day is far away, and I've got nothing that would be useful to BoE.

[*i]

The issue, as people said, is retrieval. However, there is an easy way to do handle this. Recall that the scenario text area has strings 10-59 that are essentially unused. One could craft a node to accept a string ID as an argument along with a campaign flag (much like an SDF). BoE can then do a string compare and copy the value of the flag. I would imagine if the scenario or relevant flag is not found, the value returned is a zero. One could do more elaborate checks, of course.

 

So here is my proposal for the node:

 

Set Campaign Flag:

SDF A: x-coordinate of Campaign Flag

SDF B: y-coordinate of Campaign Flag

Extra 1a: Value to set flag

 

Get Campaign Flag:

SDF A: x-coordinate of SDF to set

SDF B: y-coordinate of SDF to set

Message 1: Scenario Name String

Extra 1a: x-coordinate of Campaign Flag

Extra 1b: y-coordinate of Campaign Flag

Extra 2b: Special to call if scenario or flag not found

 

Note that this would need to be stored in some array for campaign flags that would get transferred to the save file upon saving or leaving the scenario.

[Celtic Minstrel]

Originally Posted By: Niemand

It'll be annoying, but no more so than any other aspect of that system.

Less so when you realize that a button can be included to edit the string right in the special node dialog.

Originally Posted By: Niemand

=Also I decided that I would do it in Cocoa, with an eye toward a glorious day when the Mac 3D Editor can be rewritten in Objective-C++, but that day is far away, and I've got nothing that would be useful to BoE.

Hm, okay then. I guess the current code will be around for quite a while then...

Originally Posted By: *i

Recall that the scenario text area has strings 10-59 that are essentially unused.

In fact, I intend to return those strings to their originally intended use. It doesn't really matter, though, since I also intend to remove the limit on the number of strings available.

Originally Posted By: *i

Set Campaign Flag:
SDF A: x-coordinate of Campaign Flag
SDF B: y-coordinate of Campaign Flag
Extra 1a: Value to set flag

Get Campaign Flag:
SDF A: x-coordinate of SDF to set
SDF B: y-coordinate of SDF to set
Message 1: Scenario Name String
Extra 1a: x-coordinate of Campaign Flag
Extra 1b: y-coordinate of Campaign Flag
Extra 2b: Special to call if scenario or flag not found

Hmm, interesting, you're treating it as a private SDF array. I was thinking simply a one-dimensional array, but either way would work.

town->max_dim()Note that this would need to be stored in some array for campaign flags that would get transferred to the save file upon saving or leaving the scenario.Yes, I'd store it in a map. This fact would also mean that using a one-dimensional array is actually easier, since the STL has no multidimensional containers. I'd just need to code something like "campaign_flags[scen_name][num] = value" – after pushing back enough 0's for the requested flag to exist.

Saving it in the save file, I'd probably begin a line with the scenario name, then follow that by a list of all the flags sorted by number and separated by spaces. (Since they're in order, the flag's number is given implicitly by its position.) Or, if the scenario name can contain spaces (as would be the case if the full scenario name is used) then I could use two lines: "CAMPAIGN scen_name" followed by the aforementioned list of flags. Or "CAMPAIGN scen_name num_flags" if it helps to know the number of flags in advance.

[*i]

Mainly I used the 2D notion because they act like an SDF flag this way. It is not essential and could easily be 1D. I think giving people campaign flags 0 to 255 would be reasonable.

 

Somewhere in BoE, you would require this new array whose non-zero contents would be stored to the save file. This brings up an interesting thought about saving after leaving. What happens if the player forgets to save and moves on to another scenario? How will this data be remembered? Something we shoudl consider.

 

A proposed format:

 

CAMPAIGN "Scenario Name" flag value

 

Note that I put Scenario Name in quotes. This solves the problem of white space. The variables flag and value are the campaign flag index and the value it is set to respectively.

 

Each campaign flag assigned could have a separate entry eliminating the need for a list of that type. Upon entering the scenario, all previous campaign flags would need to be loaded, but this should be fairly simple. Somehow the data structure for it would need to have a notion of Scenario Name. Again, not too difficult.

[*i]

As a side note, I said this before a while back, but I think it should be said again. Remember, when Jeff wrote BoE, it was 1997 and he wanted the game to run on machines from the early 90s. He did a lot of things that, back then, were necessary.

 

These days, our requirements are far less stringent and we should not be afraid to use the space if it makes things clearer for the programmer, designer, or player. I am not saying we should be wasteful, but we don't have to be a bunch of misers when it comes to memory usage.

[Niemand]

Quote:

Somewhere in BoE, you would require this new array whose non-zero contents would be stored to the save file. This brings up an interesting thought about saving after leaving. What happens if the player forgets to save and moves on to another scenario? How will this data be remembered?

Well, it's in memory before you write it to disk, so why wouldn't it remain in memory? I would suggest that this would work most naturally if no special distinction is made (in the data structures) between the current scenario's flags and other scenarios'; just shove it into the map with the rest. You can still make it easy for designer to access the scenario's own flags without having to supply the scenario's own name by making the default value be zero, or an empty string or whatever, and then substituting the current scenario's name when the node runs with the name input left at the default.

Something that I would be somewhat more concerned about: What stops these campaign flags from just building up in the savefile? Assumedly when the party enters a scenario that has been played before you would erase all the flags for that scenario to allow them to be set again, but it doesn't appear that anything would ever remove them permanently. While I agree that there's no need to be miserly, the idea of allowing the savefile to slowly grow without bounds makes me nervous.

[*i]

I'm not sure I understand exactly what you are saying in your first paragraph. I get the point about storing everything, but robust access via the nodes is quite tricky because, unlike BoA, the fields are fixed and we lack a lot of flexibility. In other words, I do not see how one would rewrite every node (and maintain backward compatability!) to differentiate between flags in this scenario and the another.

 

So yes, campaign flags are not elegant, but neither is the BoE interface. It's elegance comes from its simplicity on the user's side. Of course, we lose a lot of power, but BoE seemed to do pretty darn well.

 

I agree letting save files grow without bounds is a bad thing. Strictly speaking, the number would be bounded by the number of campaign flags used in scenarios, which would tend to be small. This is part of the reason why I said to only copy very specific data, at the discretion of the designer, to alleviate this issue. Of course, the data will be there "forever", but that is the point.

 

I also feel that all campaign flags for a scenario should be cleared if you reenter the scenario. If you load a party that completed a scenario, enter the scenario, and save, all the campaign flags from this scenario should be gone.

[Niemand]

Suppose we have a node: "Get campaign flag A of scenario B". A is either a number, or the index of a string which stores the name of the flag to set, depending on which dictionary format is implemented. B is the index of a string which stores the name of the scenario for which we want to access a flag. Then, if B is 0, or -1, or whatever the default value is, assume that this refers to the current scenario. Alternately you could not bother with this convenience feature, and the designer would just have to insert a string which is the scenario's own name and use the number of that string anytime he wants to check the value of a flag from this scenario, just as he would for flags from any other scenario. It would be nice not to make him do this, but not necessary.

 

An analogous issue doesn't even exist for setting flags if we assume that a scenario may only set its own flags, the the node need only be "Set flag A to C" where A is a string index again and C is the value to assign. The game will implicitly know that the name under which to store this flag is that of the current scenario.

 

Finally, I'm not sure what you mean by 'rewrite every node'. Campaign flags don't yet exist right? So there are only a couple of new node types to be added, no alterations to existing node types.

[Lilith]

Originally Posted By: *i

I also feel that all campaign flags for a scenario should be cleared if you reenter the scenario. If you load a party that completed a scenario, enter the scenario, and save, all the campaign flags from this scenario should be gone.

This should certainly be the default behaviour, but I wouldn't mind having the option to override it to allow a New Game+ mode.

[The Almighty Doer of Stuff]

Returning to an old point, Windows (XP at least) can natively read PNG graphics. I have a question though: Will I need to install Quicktime to use BoE? I'm somewhat confused about what you guys were talking about. I hate Quicktime and don't have it installed, nor will I ever have it installed. It makes my computer lag and screws with my file associations in ways that I can never figure out how to permanently fix without uninstalling the program.

[Celtic Minstrel]

ADoS, that would depend entirely upon how the Windows version reads the images. I'll be using Quicktime in the Mac version, but the Windows version can something else (and probably should, since Quicktime's not a standard component of Windows).

 

Originally Posted By: Thuryl

This should certainly be the default behaviour, but I wouldn't mind having the option to override it to allow a New Game+ mode.

Uh, what?

 

Originally Posted By: *i

Mainly I used the 2D notion because they act like an SDF flag this way. It is not essential and could easily be 1D. I think giving people campaign flags 0 to 255 would be reasonable.

...You're still talking about a limited number of flags.

 

...Which, come to think of it, actually may be a good idea, to avoid allowing the savefile to grow arbitrarily large.

 

Originally Posted By: *i

Somewhere in BoE, you would require this new array whose non-zero contents would be stored to the save file. This brings up an interesting thought about saving after leaving. What happens if the player forgets to save and moves on to another scenario? How will this data be remembered? Something we shoudl consider.

It will be saved the next time he saves – it won't matter that he's in a different scenario, because it'll be stored as a map with the scenario name as a key.

 

 

Originally Posted By: *i

A proposed format:

 

CAMPAIGN "Scenario Name" flag value

 

Note that I put Scenario Name in quotes. This solves the problem of white space. The variables flag and value are the campaign flag index and the value it is set to respectively.

True, that would be a good format if we have a fixed number of flags for each scenario.

 

Originally Posted By: *i

Each campaign flag assigned could have a separate entry eliminating the need for a list of that type. Upon entering the scenario, all previous campaign flags would need to be loaded, but this should be fairly simple.

Yeah, no problem there.

 

Originally Posted By: *i

Somehow the data structure for it would need to have a notion of Scenario Name. Again, not too difficult.

Sorry, I'm not quite sure what you mean by this?

 

Originally Posted By: Niemand

Well, it's in memory before you write it to disk, so why wouldn't it remain in memory? I would suggest that this would work most naturally if no special distinction is made (in the data structures) between the current scenario's flags and other scenarios'; just shove it into the map with the rest.

That's what I was talking about all along.

 

Originally Posted By: Niemand

You can still make it easy for designer to access the scenario's own flags without having to supply the scenario's own name by making the default value be zero, or an empty string or whatever, and then substituting the current scenario's name when the node runs with the name input left at the default.

Sure, that wouldn't be too hard to do.

 

Originally Posted By: Niemand

Something that I would be somewhat more concerned about: What stops these campaign flags from just building up in the savefile? Assumedly when the party enters a scenario that has been played before you would erase all the flags for that scenario to allow them to be set again, but it doesn't appear that anything would ever remove them permanently. While I agree that there's no need to be miserly, the idea of allowing the savefile to slowly grow without bounds makes me nervous.

I was proposing having an unlimited number of flags available for each scenario, but if we limited the number of flags as Stareye is suggesting, it would help with this. Also, we could add something to the Character Editor to clear campaign flags for a given scenario or for the current one. If we implement a concept of "winning" a scenario, as BoA has, we could clear the campaign flags when the user leaves the scenario without winning.

 

Originally Posted By: *i

I'm not sure I understand exactly what you are saying in your first paragraph. I get the point about storing everything, but robust access via the nodes is quite tricky because, unlike BoA, the fields are fixed and we lack a lot of flexibility. In other words, I do not see how one would rewrite every node (and maintain backward compatability!) to differentiate between flags in this scenario and the another.

I don't think backwards compatibility is quite that much of an issue. Yes, we need to preserve it, but we don't need to retain anything in the format to do this. Backwards compatibility is in the conversion process. (And now that I think about it, I don't think the idea of upgrading a file one version at a time would work so well – it seems it would require that we retain a version of the data structure for each now-unused version, which I don't think is a good idea. On the other hand, it would mean we don't need to upgrade all the conversion code when making a new version...)

 

Originally Posted By: Niemand

I agree letting save files grow without bounds is a bad thing. Strictly speaking, the number would be bounded by the number of campaign flags used in scenarios, which would tend to be small. This is part of the reason why I said to only copy very specific data, at the discretion of the designer, to alleviate this issue. Of course, the data will be there "forever", but that is the point.

Again, limiting the number of campaign flags would help with this.

 

Originally Posted By: Niemand

I also feel that all campaign flags for a scenario should be cleared if you reenter the scenario. If you load a party that completed a scenario, enter the scenario, and save, all the campaign flags from this scenario should be gone.

Sure, it would make sense to clear flags belonging to a specific scenario when entering that scenario, so that it can reset them as you progress through it.

 

Originally Posted By: Niemand

Suppose we have a node: "Get campaign flag A of scenario B". A is either a number, or the index of a string which stores the name of the flag to set, depending on which dictionary format is implemented. B is the index of a string which stores the name of the scenario for which we want to access a flag. Then, if B is 0, or -1, or whatever the default value is, assume that this refers to the current scenario.

This is a good idea.

 

Originally Posted By: Niemand

An analogous issue doesn't even exist for setting flags if we assume that a scenario may only set its own flags, the the node need only be "Set flag A to C" where A is a string index again and C is the value to assign. The game will implicitly know that the name under which to store this flag is that of the current scenario.

Yes, and not allowing the scenario to set flags for a different scenario is a Very Good Thing.

 

Originally Posted By: Niemand

Finally, I'm not sure what you mean by 'rewrite every node'. Campaign flags don't yet exist right? So there are only a couple of new node types to be added, no alterations to existing node types.

Correct.

[Niemand]

Quote:

Returning to an old point, Windows (XP at least) can natively read PNG graphics. I have a question though: Will I need to install Quicktime to use BoE? I'm somewhat confused about what you guys were talking about. I hate Quicktime and don't have it installed, nor will I ever have it installed. It makes my computer lag and screws with my file associations in ways that I can never figure out how to permanently fix without uninstalling the program.

Yes, the idea is to use native libraries on each platform so that the Windows version would not be dependent on Quicktime. The Mac version of the program would use it because the ancient graphics library it's currently saddled with using doesn't support much of anything, and Quicktime gives a convenient way to load in modern data.

(As an aside, I find it unfortunate how many Windows users dislike Quicktime, as it's by far the best media player I've used on any platform. However, it's your computer, so I won't try tell you how to run it, and I agree that requiring it just to load a common image type would be ridiculous.)

[*i]

You are correct that I suggested a fixed number to constrain the amount of data we would need to keep. Now I do think we should set this number to be high.

 

Quote:

True, that would be a good format if we have a fixed number of flags for each scenario.

 

Just to clarify, what I am proposing for the token would not care if there are a fixed number of these or not.

 

An advantage to the ASCII format means that there does not need to be a prescribed order to anything aside from a few header things. Otherwise, you can just read the line, parse the arguments, and by way of a select case you just modify the default data. This parser type implementation makes changing the file a lot easier in the future.

[Celtic Minstrel]

I've completely forgotten why I said that in that way. Of course it has no bearing on whether the number of flags is fixed.

 

The lack of a prescribed order is certainly true, but there will probably be some sort of order imposed anyway simply because the whole file will be overwritten when saving, which imposes an order on it. Still, if people want to hack their save files, it's more forgiving. And it does make future changes simpler.

 

Note that a switch statement would not work unless we have a function to convert the line prefix into an enum or something. You can't switch on strings – I think that applies to STL strings too.

 

By the way, you missed a question in my previous post which was hard to see due to a formatting error on my part.

[Niemand]

The usage of an actual switch statement or some more complex, composite construct is fairly irrelevant. However, it can be just as easy to handle a binary format if it's well designed.

 

The real pain with parsing ASCII text that you allow a human anywhere near is thinking of all the ways it may go wrong. Also, it can be tricky to make a format hold complex data that is easy for both a computer and a human to understand. For instance, storing every record on a single line will be easy for the computer, since it doesn't care how long the line is or get confused about the ordering of properties, while humans do.

[Slawbug]

How exactly do carriage returns make things any harder for the computer?

[Niemand]

Carriage returns! Heathen! The only true way to end a line is with a line feed!

 

They matter only if your format says they do. If the format is defined so that each data record is delimited by your line ending character of choice, then they are crucial. If your format says that fields are delimited by whitespace, in any amount and consisting of any combination, then they don't matter at all. My point was that *i was suggesting a line based format, but somethings don't fit well all onto one line, like say a creature or item definition. If you keep the format line based but allow certain definition types to be split into multiple lines of different subtypes, each of those lines now has to indicate to which record it belongs.

 

Maybe I was reading what *i wrote too literally, but I interpreted it as a suggestion for a parser which has no state from one line to the next, merely judging each line solely on its contents and not at all on context.

[Celtic Minstrel]

If it's going to be compressed, and a custom header added, I don't think we need to worry about readability. Using one line per record should be fine, I think, even for a monster or item definition.

 

The goal is to make it fairly simple for the computer to parse. Human readability, if it exists, is just a bonus.

 

If we really need line breaks within a given record, se can just split that record into a different file within the archive. For example, we could keep each pc record in a separate file if we wanted to.

[*i]

Yes, binaries would the easiest to read. If one is clever, one can make a pretty generalized format too. I mean, at the end of the day, everything is just 1s and 0s.

 

Mainly I am suggesting the ASCII format not so much to be easy to read, but to be easy to modify in the event something is wrong. If I want to go in and change a flag because something got messed up because of a bug, this is very easy to do in the ASCII format. Granted, it might take an "expert" user to do it, but it still is possible to do.

[Celtic Minstrel]

Though, that's kind of what the character editor is there for.

 

 

At least partly.

[*i]

Well yes, the Character Editor is nice for simple things. Checking the state of flags, doing other diagnostics, and doing in depth modification is not one of them. Granted we can edit the source code for that too, but I do think there is some virtue in keeping things simple for players.

[keira]

Couldn't y'all just have it ignore any and all whitespace (except when in a string, of course) and use semicolons or commas or something at the end of every piece of data? I don't know much about this crap, but doesn't seem to had to do. Is it?

[Niemand]

That's also a perfectly valid approach. It wouldn't really be significantly harder to do, or significantly easier than being whitespace sensitive. Whitespace is a fairly natural choice for delimiting data, since this is what it does in written human languages (generally speaking; I'm not a language expert), and so it's common for this to be a default behavior for input libraries, like iostream in C++. However, there are still choices about exactly how to delimit with whitespace, such as treating it all alike or treating newlines specially. Ultimately, I'd say that none of these methods is fundamentally better than the others; they all have their details to deal with, and one just needs to be chosen.

[*i]

It's not an issue of carriage returns versus any other delimiter. There are plenty of STL C++ functions that make reading in string data rather simple. The issue is deciding whether to use an ASCII (readable text) or a binary (machine code) format and how to best organize/parse it.

[Lilith]

Originally Posted By: Celtic Minstrel

Originally Posted By: Thuryl

This should certainly be the default behaviour, but I wouldn't mind having the option to override it to allow a New Game+ mode.

Uh, what?

http://tvtropes.org/pmwiki/pmwiki.php/Main/NewGamePlus

What I'm saying is that it should be possible for a scenario to remember that you've played it before, if the designer so chooses. So campaign flags should be deleted by default, but there should be an option to override that behaviour.

[Slawbug]

The TV Tropes page claims the concept "dates back to the Zelda series" but I don't think that's true. It's a clear reference to the "second quest" in the original Legend of Zelda, which is indeed accessed in the same way New Game+ is typically accessed. However, it's not a New Game+, it's essentially a New Game in a different scenario.