Jump to content


Some ideas on scenario editing

forge

  • Please log in to reply
22 replies to this topic
Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 25 September 2013 - 11:12 PM #1 Some ideas on scenario editing

This all started with thoughts about the Blades Forge site (still want to bring it back some day), and potential features for it. For example, it could retrieve information from scenarios - maps, statistics, that sort of thing - for display. Or it could try to lend a hand with actual development work and version control, rather than just being a place for hosting zip files.

All that, of course, required me to finally take an actual look inside the .bas file format. Considering all that's been done in the past nine years or so, I'm really late to the party.

Well, the first thing I found out is that nobody bothered to document the structure of the .bas format; probably because all the work so far has been in C++ where you can just reuse the classes from the editor, and don't need to pay attention to which record is stored where.
So I took a look at the editor source and made a start at writing this stuff down. It's mostly a list of byte offsets from adding up the variable lengths, plus some notes on value ranges and usage. Still has a lot of errors, probably.

After getting acquainted with the format, I wanted to play around with it a bit, and wrote some Python scripts. Managed to print out a text-based map (hex codes) for VoDT.
The next, somewhat more ambitious idea was a colored pixel map (it's tiny of course), where the pixel colors are derived from averaging the floor graphics.
I'm not sure what to do next. If I set out to create an entire new scenario editor on my own, it'll join the scrapyard of unfinished projects pretty quickly.

But I rather like the idea of having a Python toolset that can read and write scenario files, even without a GUI. So here's the small amount of stuff I made so far:

http://github.com/cburschka/forge

Ishad Nha Ishad Nha

Hand of Avadon

  • Member
    • Member ID: 4,773
  • 2,366 posts

Posted 26 September 2013 - 02:02 AM #2 Some ideas on scenario editing

I don't think the game pays any attention to these:
boat_record_type scen_boats[30]; 6,436 300
horse_record_type scen_horses[30]; 6,736 300
At some point I can experiment with this.

I did a Bas decrypt some time ago, it attracted little attention:
http://www.freewebs....BasDecrypts.zip

Town Records
class town_record_type
Data type Name Starts Length
char town_name[20]; 0 20
short respawn_monsts[4][6]; 20 48
location respawn_locs[6]; 68 12
macRECT special_rects[60]; 80 480
unsigned char spec_id[60]; 560 60
short lighting; 620 2
location sign_locs[15]; 622 30
char sign_text[15][256]; 652 3,840
location start_locs[4]; 4,492 8
location exit_locs[4]; 4,500 8
macRECT in_town_rect; 4,508 8
item_type preset_items[144]; 4,516 1,152
preset_field_type preset_fields[60]; 5,668 240
short wall_1_sheet, wall_1_height; 5,908 4
short wall_2_sheet, wall_2_height 5,912 4
short cliff_sheet; 5,916 2
short beam_type; 5,918 2
short environmental_sound; 5,920 2
short is_on_surface; 5,922 2
short town_kill_day; 5,924 2
short town_linked_event; 5,926 2
short external_floor_type; 5,928 2
short monster_respawn_chance; 5,930 2
char town_script[14]; 5,932 14
in_town_on_ter_script_type ter_scripts[100]; 5,946 3,800
macRECT room_rect[16]; 9,746 128
char info_rect_text[16][30]; 9,874 480
creature_start_type creatures[80]; 10,354 5,920
short extra[20]; 16,274 40
location waypoints[10]; 16,314 20
short exit_specs[4]; 16,334 8
short spec_on_entry; 16,342 2
short spec_on_entry_if_dead; 16,344 2

&&&&&&&

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 26 September 2013 - 03:17 AM #3 Some ideas on scenario editing

Quote

I don't think the game pays any attention to these:
boat_record_type scen_boats[30]; 6,436 300
horse_record_type scen_horses[30]; 6,736 300

Yep, this information is only accessed by the scenario editor to generate script code. You could presumably leave this part blank (or put it to other creative uses) and only break inter-editor compatibility rather than game compatibility. I'll move it into the "editor information" heading.

(The format appears to have several quirks like this, like the platform identifier that is always set to Mac.)

What I haven't figured out is this:

1194..119b location exit_locs[4];
119c..11a3 macRECT in_town_rect;

Do these just store the same information; once as corners and once as a rectangle?

Ishad Nha Ishad Nha

Hand of Avadon

  • Member
    • Member ID: 4,773
  • 2,366 posts

Posted 26 September 2013 - 04:46 AM #4 Some ideas on scenario editing

A macRECT is eight bytes long! If you look at the length of the in_town_rect section, you can see there is only one such rect. (It is the town boundary.)
location exit_locs[4]; 4,500 8
macRECT in_town_rect; 4,508 8
Exit locations are the outdoor exit locs, where you end up when you leave town by a given direction.
If you have the editor source code you can decrypt these mysteries.

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 26 September 2013 - 06:37 AM #5 Some ideas on scenario editing

Yeah, that's how I got the docs I made so far. ;) Right, forgot there has to be a reference to the outdoor location... and I guess that unless the section is changed with a script, you always exit in the currently loaded outdoor section.

Anyway, mapping code updated a bit. It can now generate a 1:1 isometric view of the outdoor map, but it only uses floor type so far (not height map or terrain).

Unfortunately, my code for writing such large bitmaps is slow, and takes 10 seconds per outdoor section. It might be a good idea to look at tiles instead of a single file, like the Minecraft Overviewer does it.

http://stuff.ermaria...tar/valley2.png

And with terrain:

http://stuff.ermaria...lleyterrain.png

Note that the wall-drawing code in BoA is quirky, and determines wall graphics by environment rather than defining a new terrain for each. The wall graphic is also shifted around a bit, and according to the script comments some of it is hardcoded into the engine. Haven't tried to touch that yet.

Kelandon Kelandon

!!!

  • Global Moderator
    • Member ID: 2,682
  • 9,815 posts

Posted 26 September 2013 - 08:47 PM #6 Some ideas on scenario editing

View PostArancaytar, on 26 September 2013 - 06:37 AM, said:

Wow, that looks pretty nice. Maybe when you get the code settled, a view like that could be glommed onto the most recent version of the 3D Editor, even if it's not editable from that view. It could be nice to be able to zoom out like that and see everything all at once.
VCH: I believe we settled this way back when: Kelandon was the most attractive.
SoT: You'd be happy, too, if you were such a clever spider.
Ephesos: In conclusion, yarr.

Kelandon's Pink and Pretty Page!!

Ishad Nha Ishad Nha

Hand of Avadon

  • Member
    • Member ID: 4,773
  • 2,366 posts

Posted 27 September 2013 - 02:14 AM #7 Some ideas on scenario editing

We would need to alter the Editor so it could load all outdoor zones into memory when it was started. A multi-zone display would need to be devised too.

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 27 September 2013 - 03:31 AM #8 Some ideas on scenario editing

It would probably be feasible to make a new editing GUI with the pygame library. If I did try to make a new editor, that'd be where I'd start looking - but as mentioned I don't have the stamina or time to stick with that kind of project for long.

(I'm doubtful that any of this - especially the Python code - could be integrated in the existing editor, because it's an ancient codebase and apparently isn't cross-platform.)

(Not to bash the existing editor, but this whole concept of loading only one zone and one town seems like an obsolete legacy from back when memory was scarcer.)

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 27 September 2013 - 05:09 AM #9 Some ideas on scenario editing

Quote

// This bug affects not only terrain 6, but terrain 7, 42 and 43.
// And because of this bug, NW corner wall display in realistic mode becomes odd.
// It's better to fix this bug on corescendata2.txt itself, but it may cause unpredictable errors on the BoA Game.
// For this reason, correct them here.
void patch_corescendata2( void )
{
	// equivalant of "te_blocks_view_e = 0;"
	scen_data.scen_ter_types[6].blocks_view[3] = 0;
	scen_data.scen_ter_types[7].blocks_view[3] = 0;
	scen_data.scen_ter_types[42].blocks_view[3] = 0;
	scen_data.scen_ter_types[43].blocks_view[3] = 0;
}


AAAAAAAUUUGHH

I found some other bug in the scenario scripts, by the way.

begindefineterrain 332; 
	import = 122;

#122 is never defined anywhere. I'm going to treat that as a "clear" command in my parser for now.

Edit:

New AvernumScript parser, and some better mapping code (see github) that works with walls, offsets and multi-icon terrains. New VoDT map:

http://stuff.ermaria...tar/valley4.png

(Wall corners aren't drawn yet; that takes more special code.)

Edit: Apparently my bitmap-drawing code is crazy-slow. I replaced it with pygame and now the VoDT map is done in 2 seconds.


Edit2: Scrollable Map GUI is done. Kind of primitive (no zoom, resize and stuff), but done.
Scripts are at https://github.com/cburschka/forge
Now includes a setup.py installer for configuring the location of the BoA data (Windows version required; it can't handle resource forks).
I'm kind of surprised that I got this much working in three days.

Edit3: In fact, there's an experimental GTK+ based program now that can open scenario files to scroll around the world and stuff.

Also, I'm noticing that my current mapping method (make one big picture) is only useful on roughly square outdoor maps. With crazy dimensions like 1x15 (not that that would be something anyone would ever do), the isometric map is as big as a stupendously massive 7x8 map, rendering a whole lot of blackness just for a single diagonal strip. For ZKR, pygame just gives up. A better idea would be to render each sections individually, to avoid wasting space.

nikki. nikki.

Humany-Wumany

  • Global Moderator
    • Member ID: 2,313
  • 7,449 posts

Posted 29 September 2013 - 04:15 PM #10 Some ideas on scenario editing

View Postξ, on 26 September 2013 - 08:47 PM, said:

Wow, that looks pretty nice. Maybe when you get the code settled, a view like that could be glommed onto the most recent version of the 3D Editor, even if it's not editable from that view. It could be nice to be able to zoom out like that and see everything all at once.

Yeah, that'd be amazing - whenever I work on Blades I tend to create huge maps by pasting zoomed-out sections of the outdoors into MSPaint. Having an option to do this in the editor, in 3D, would be great.

sylae sylae

Wisebottom

  • Global Moderator
    • Member ID: 5,482
  • 3,336 posts
  • LocationPortland, Oregon, United States

Posted 29 September 2013 - 05:44 PM #11 Some ideas on scenario editing

it's gonna be awesome being able to use this stuff on a webserver :D

(also yay i'm inside nikki's signature)

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 29 September 2013 - 06:21 PM #12 Some ideas on scenario editing

Definitely!

As far as the python GUI goes, I'm having some difficulty with different toolkits and libraries that don't work well together.

pygame is essential for the image manipulation and drawing - it can "blit()" (no relation) bitmaps into other bitmaps (such as tons of sprite images into a map) incredibly fast. But there are no GUI toolkits that have been maintained for any length of time - one of the most popular died on Google Code about two years back. GTK+ is an awesome toolkit for that, but connecting it to pygame has some pitfalls I haven't worked out yet.

Kelandon Kelandon

!!!

  • Global Moderator
    • Member ID: 2,682
  • 9,815 posts

Posted 29 September 2013 - 06:35 PM #13 Some ideas on scenario editing

View PostS▒y░░e▒, on 29 September 2013 - 05:44 PM, said:

(also yay i'm inside nikki)
This is how I initially read that. o_O

Loading everything all at once would make a heck of a lot more sense. If you're starting to move in that direction, that'd be great. But trying to design a new editor from scratch is the quagmire to end all quagmires. Abandon all hope, ye who enter here!
VCH: I believe we settled this way back when: Kelandon was the most attractive.
SoT: You'd be happy, too, if you were such a clever spider.
Ephesos: In conclusion, yarr.

Kelandon's Pink and Pretty Page!!

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 30 September 2013 - 04:05 AM #14 Some ideas on scenario editing

Pretty much, yeah. I don't think I can come up with any kind of long-term effort in that direction.

Best I can do while playing around is to have my source well-documented and public, so it may at least be useful if someone else wants to take over. Choosing Python over some flavor of C like everyone else probably doesn't help, but at least this should never get stuck in a porting nightmare.

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 30 September 2013 - 02:41 PM #15 Some ideas on scenario editing

Ugh. I have learned more about Gtk than I wanted today. Viewer basically works now, though, and you can open scenarios and stuff.

Not sure if this even works on any other system. If any of you feel up to installing Python 3, PyGObject, GTK+ and the Windows version of BoA - regardless of the platform - some feedback would be awesome.

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 02 October 2013 - 03:30 PM #16 Some ideas on scenario editing

Now with scroll-wheel zooming and stuff.

(Note: github URL is changed to http://github.com/cburschka/forge )

Edit: The next couple of updates will mostly clean up the code to change it from a dirty prototype into something easier to extend and maintain. Nothing interesting, in other words.

Celtic Minstrel Celtic Minstrel

Eye of Avadon

  • Global Moderator
    • Member ID: 5,296
  • 3,891 posts

Posted 09 October 2013 - 06:38 AM #17 Some ideas on scenario editing

This looks interesting, but I kinda wish the same investigation was happening with BoE...
Exile 2 Book of Items (I didn't create it, I just converted it to HTML.)

"Man, I know how you feel. I once spent an hour playing WordPad before realising that it was a text editor." – Thuryl

"Dikiyoba just hopes no one ever blows up Saturn. Getting those rings back into their proper position would take hours."

"—Alorael, who spells phoenetically. No matter how much his orthography is a wreck, intelligibility rises from the ashes."

"I've never played a Spiderweb game. I didn't even know SW made games until I saw this thread." —Nikki

Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 10 October 2013 - 09:05 AM #18 Some ideas on scenario editing

Shouldn't be too tricky; it basically involves finding the part of scenario.cpp where the various structs are defined, and start counting byte-lengths.

It's not an investigation I can undertake, though. In fact the whole Forge thing is going on the backburner for the next month or two; I have classes starting next week, and another TA job.

Ishad Nha Ishad Nha

Hand of Avadon

  • Member
    • Member ID: 4,773
  • 2,366 posts

Posted 11 October 2013 - 01:17 AM #19 Some ideas on scenario editing

I decrypted the Exs files years ago with my Exs Decrypt file:
http://www.freewebs....EXSDecrypts.zip
(It also contains stuff from Of Good and Evil&)

Edited by Ishad Nha, 11 October 2013 - 01:58 AM.


Arancaytar Arancaytar

Archivist

  • Global Moderator
    • Member ID: 2,001
  • 10,474 posts
  • LocationThe Empire of Ermarian

Posted 11 October 2013 - 06:01 AM #20 Some ideas on scenario editing

You keep using that word. The file format isn't even obfuscated, let alone encrypted. :p

Celtic Minstrel Celtic Minstrel

Eye of Avadon

  • Global Moderator
    • Member ID: 5,296
  • 3,891 posts

Posted 11 October 2013 - 06:23 AM #21 Some ideas on scenario editing

Hm, not sure why I didn't download that... it looks helpful...
Exile 2 Book of Items (I didn't create it, I just converted it to HTML.)

"Man, I know how you feel. I once spent an hour playing WordPad before realising that it was a text editor." – Thuryl

"Dikiyoba just hopes no one ever blows up Saturn. Getting those rings back into their proper position would take hours."

"—Alorael, who spells phoenetically. No matter how much his orthography is a wreck, intelligibility rises from the ashes."

"I've never played a Spiderweb game. I didn't even know SW made games until I saw this thread." —Nikki

Ishad Nha Ishad Nha

Hand of Avadon

  • Member
    • Member ID: 4,773
  • 2,366 posts

Posted 11 October 2013 - 03:44 PM #22 Some ideas on scenario editing

I work with great big chunks of hex data, where normally you can't tell what is what, hence the word "Decrypt" I suppose.

Celtic Minstrel Celtic Minstrel

Eye of Avadon

  • Global Moderator
    • Member ID: 5,296
  • 3,891 posts

Posted 12 October 2013 - 06:01 AM #23 Some ideas on scenario editing

I think "decode" might be a little more accurate.
Exile 2 Book of Items (I didn't create it, I just converted it to HTML.)

"Man, I know how you feel. I once spent an hour playing WordPad before realising that it was a text editor." – Thuryl

"Dikiyoba just hopes no one ever blows up Saturn. Getting those rings back into their proper position would take hours."

"—Alorael, who spells phoenetically. No matter how much his orthography is a wreck, intelligibility rises from the ashes."

"I've never played a Spiderweb game. I didn't even know SW made games until I saw this thread." —Nikki




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users