Magnificent Ornk Aran Posted September 26, 2013 Share Posted September 26, 2013 This all started with thoughts about the Blades Forge site (still want to bring it back some day), and potential features for it. For example, it could retrieve information from scenarios - maps, statistics, that sort of thing - for display. Or it could try to lend a hand with actual development work and version control, rather than just being a place for hosting zip files. All that, of course, required me to finally take an actual look inside the .bas file format. Considering all that's been done in the past nine years or so, I'm really late to the party. Well, the first thing I found out is that nobody bothered to document the structure of the .bas format; probably because all the work so far has been in C++ where you can just reuse the classes from the editor, and don't need to pay attention to which record is stored where. So I took a look at the editor source and made a start at writing this stuff down. It's mostly a list of byte offsets from adding up the variable lengths, plus some notes on value ranges and usage. Still has a lot of errors, probably. After getting acquainted with the format, I wanted to play around with it a bit, and wrote some Python scripts. Managed to print out a text-based map (hex codes) for VoDT. The next, somewhat more ambitious idea was a colored pixel map (it's tiny of course), where the pixel colors are derived from averaging the floor graphics. I'm not sure what to do next. If I set out to create an entire new scenario editor on my own, it'll join the scrapyard of unfinished projects pretty quickly. But I rather like the idea of having a Python toolset that can read and write scenario files, even without a GUI. So here's the small amount of stuff I made so far: http://github.com/cburschka/forge keira 1 Quote Link to comment Share on other sites More sharing options...
Well-Actually War Trall Ishad Nha Posted September 26, 2013 Share Posted September 26, 2013 I don't think the game pays any attention to these: boat_record_type scen_boats[30]; 6,436 300 horse_record_type scen_horses[30]; 6,736 300 At some point I can experiment with this. I did a Bas decrypt some time ago, it attracted little attention: http://www.freewebs.com/ishadnha/BasDecrypts.zip Town Records class town_record_type Data type Name Starts Length char town_name[20]; 0 20 short respawn_monsts[4][6]; 20 48 location respawn_locs[6]; 68 12 macRECT special_rects[60]; 80 480 unsigned char spec_id[60]; 560 60 short lighting; 620 2 location sign_locs[15]; 622 30 char sign_text[15][256]; 652 3,840 location start_locs[4]; 4,492 8 location exit_locs[4]; 4,500 8 macRECT in_town_rect; 4,508 8 item_type preset_items[144]; 4,516 1,152 preset_field_type preset_fields[60]; 5,668 240 short wall_1_sheet, wall_1_height; 5,908 4 short wall_2_sheet, wall_2_height 5,912 4 short cliff_sheet; 5,916 2 short beam_type; 5,918 2 short environmental_sound; 5,920 2 short is_on_surface; 5,922 2 short town_kill_day; 5,924 2 short town_linked_event; 5,926 2 short external_floor_type; 5,928 2 short monster_respawn_chance; 5,930 2 char town_script[14]; 5,932 14 in_town_on_ter_script_type ter_scripts[100]; 5,946 3,800 macRECT room_rect[16]; 9,746 128 char info_rect_text[16][30]; 9,874 480 creature_start_type creatures[80]; 10,354 5,920 short extra[20]; 16,274 40 location waypoints[10]; 16,314 20 short exit_specs[4]; 16,334 8 short spec_on_entry; 16,342 2 short spec_on_entry_if_dead; 16,344 2 &&&&&&& Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 26, 2013 Author Share Posted September 26, 2013 I don't think the game pays any attention to these:boat_record_type scen_boats[30]; 6,436 300 horse_record_type scen_horses[30]; 6,736 300 Yep, this information is only accessed by the scenario editor to generate script code. You could presumably leave this part blank (or put it to other creative uses) and only break inter-editor compatibility rather than game compatibility. I'll move it into the "editor information" heading. (The format appears to have several quirks like this, like the platform identifier that is always set to Mac.) What I haven't figured out is this: 1194..119b location exit_locs[4]; 119c..11a3 macRECT in_town_rect; Do these just store the same information; once as corners and once as a rectangle? keira 1 Quote Link to comment Share on other sites More sharing options...
Well-Actually War Trall Ishad Nha Posted September 26, 2013 Share Posted September 26, 2013 A macRECT is eight bytes long! If you look at the length of the in_town_rect section, you can see there is only one such rect. (It is the town boundary.) location exit_locs[4]; 4,500 8 macRECT in_town_rect; 4,508 8 Exit locations are the outdoor exit locs, where you end up when you leave town by a given direction. If you have the editor source code you can decrypt these mysteries. Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 26, 2013 Author Share Posted September 26, 2013 Yeah, that's how I got the docs I made so far. Right, forgot there has to be a reference to the outdoor location... and I guess that unless the section is changed with a script, you always exit in the currently loaded outdoor section. Anyway, mapping code updated a bit. It can now generate a 1:1 isometric view of the outdoor map, but it only uses floor type so far (not height map or terrain). Unfortunately, my code for writing such large bitmaps is slow, and takes 10 seconds per outdoor section. It might be a good idea to look at tiles instead of a single file, like the Minecraft Overviewer does it. http://stuff.ermarian.net/arancaytar/valley2.png And with terrain: http://stuff.ermarian.net/arancaytar/valleyterrain.png Note that the wall-drawing code in BoA is quirky, and determines wall graphics by environment rather than defining a new terrain for each. The wall graphic is also shifted around a bit, and according to the script comments some of it is hardcoded into the engine. Haven't tried to touch that yet. keira and nikki. 2 Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Kelandon Posted September 27, 2013 Share Posted September 27, 2013 http://stuff.ermarian.net/arancaytar/valley2.png And with terrain: http://stuff.ermarian.net/arancaytar/valleyterrain.png Wow, that looks pretty nice. Maybe when you get the code settled, a view like that could be glommed onto the most recent version of the 3D Editor, even if it's not editable from that view. It could be nice to be able to zoom out like that and see everything all at once. nikki. 1 Quote Link to comment Share on other sites More sharing options...
Well-Actually War Trall Ishad Nha Posted September 27, 2013 Share Posted September 27, 2013 We would need to alter the Editor so it could load all outdoor zones into memory when it was started. A multi-zone display would need to be devised too. Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 27, 2013 Author Share Posted September 27, 2013 It would probably be feasible to make a new editing GUI with the pygame library. If I did try to make a new editor, that'd be where I'd start looking - but as mentioned I don't have the stamina or time to stick with that kind of project for long. (I'm doubtful that any of this - especially the Python code - could be integrated in the existing editor, because it's an ancient codebase and apparently isn't cross-platform.) (Not to bash the existing editor, but this whole concept of loading only one zone and one town seems like an obsolete legacy from back when memory was scarcer.) keira 1 Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 27, 2013 Author Share Posted September 27, 2013 // This bug affects not only terrain 6, but terrain 7, 42 and 43. // And because of this bug, NW corner wall display in realistic mode becomes odd. // It's better to fix this bug on corescendata2.txt itself, but it may cause unpredictable errors on the BoA Game. // For this reason, correct them here. void patch_corescendata2( void ) { // equivalant of "te_blocks_view_e = 0;" scen_data.scen_ter_types[6].blocks_view[3] = 0; scen_data.scen_ter_types[7].blocks_view[3] = 0; scen_data.scen_ter_types[42].blocks_view[3] = 0; scen_data.scen_ter_types[43].blocks_view[3] = 0; } AAAAAAAUUUGHH I found some other bug in the scenario scripts, by the way. begindefineterrain 332; import = 122; #122 is never defined anywhere. I'm going to treat that as a "clear" command in my parser for now. Edit: New AvernumScript parser, and some better mapping code (see github) that works with walls, offsets and multi-icon terrains. New VoDT map: http://stuff.ermarian.net/arancaytar/valley4.png (Wall corners aren't drawn yet; that takes more special code.) Edit: Apparently my bitmap-drawing code is crazy-slow. I replaced it with pygame and now the VoDT map is done in 2 seconds. Edit2: Scrollable Map GUI is done. Kind of primitive (no zoom, resize and stuff), but done. Scripts are at https://github.com/cburschka/forge Now includes a setup.py installer for configuring the location of the BoA data (Windows version required; it can't handle resource forks). I'm kind of surprised that I got this much working in three days. Edit3: In fact, there's an experimental GTK+ based program now that can open scenario files to scroll around the world and stuff. Also, I'm noticing that my current mapping method (make one big picture) is only useful on roughly square outdoor maps. With crazy dimensions like 1x15 (not that that would be something anyone would ever do), the isometric map is as big as a stupendously massive 7x8 map, rendering a whole lot of blackness just for a single diagonal strip. For ZKR, pygame just gives up. A better idea would be to render each sections individually, to avoid wasting space. keira 1 Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk nikki. Posted September 30, 2013 Share Posted September 30, 2013 Wow, that looks pretty nice. Maybe when you get the code settled, a view like that could be glommed onto the most recent version of the 3D Editor, even if it's not editable from that view. It could be nice to be able to zoom out like that and see everything all at once. Yeah, that'd be amazing - whenever I work on Blades I tend to create huge maps by pasting zoomed-out sections of the outdoors into MSPaint. Having an option to do this in the editor, in 3D, would be great. Quote Link to comment Share on other sites More sharing options...
Easygoing Eyebeast keira Posted September 30, 2013 Share Posted September 30, 2013 it's gonna be awesome being able to use this stuff on a webserver (also yay i'm inside nikki's signature) Aran 1 Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 30, 2013 Author Share Posted September 30, 2013 Definitely! As far as the python GUI goes, I'm having some difficulty with different toolkits and libraries that don't work well together. pygame is essential for the image manipulation and drawing - it can "blit()" (no relation) bitmaps into other bitmaps (such as tons of sprite images into a map) incredibly fast. But there are no GUI toolkits that have been maintained for any length of time - one of the most popular died on Google Code about two years back. GTK+ is an awesome toolkit for that, but connecting it to pygame has some pitfalls I haven't worked out yet. Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Kelandon Posted September 30, 2013 Share Posted September 30, 2013 (also yay i'm inside nikki) This is how I initially read that. o_O Loading everything all at once would make a heck of a lot more sense. If you're starting to move in that direction, that'd be great. But trying to design a new editor from scratch is the quagmire to end all quagmires. Abandon all hope, ye who enter here! Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 30, 2013 Author Share Posted September 30, 2013 Pretty much, yeah. I don't think I can come up with any kind of long-term effort in that direction. Best I can do while playing around is to have my source well-documented and public, so it may at least be useful if someone else wants to take over. Choosing Python over some flavor of C like everyone else probably doesn't help, but at least this should never get stuck in a porting nightmare. Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted September 30, 2013 Author Share Posted September 30, 2013 Ugh. I have learned more about Gtk than I wanted today. Viewer basically works now, though, and you can open scenarios and stuff. Not sure if this even works on any other system. If any of you feel up to installing Python 3, PyGObject, GTK+ and the Windows version of BoA - regardless of the platform - some feedback would be awesome. Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted October 2, 2013 Author Share Posted October 2, 2013 Now with scroll-wheel zooming and stuff. (Note: github URL is changed to http://github.com/cburschka/forge ) Edit: The next couple of updates will mostly clean up the code to change it from a dirty prototype into something easier to extend and maintain. Nothing interesting, in other words. Quote Link to comment Share on other sites More sharing options...
Understated Ur-Drakon Celtic Minstrel Posted October 9, 2013 Share Posted October 9, 2013 This looks interesting, but I kinda wish the same investigation was happening with BoE... Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted October 10, 2013 Author Share Posted October 10, 2013 Shouldn't be too tricky; it basically involves finding the part of scenario.cpp where the various structs are defined, and start counting byte-lengths. It's not an investigation I can undertake, though. In fact the whole Forge thing is going on the backburner for the next month or two; I have classes starting next week, and another TA job. Quote Link to comment Share on other sites More sharing options...
Well-Actually War Trall Ishad Nha Posted October 11, 2013 Share Posted October 11, 2013 (edited) I decrypted the Exs files years ago with my Exs Decrypt file: http://www.freewebs.com/ishadnha/EXSDecrypts.zip (It also contains stuff from Of Good and Evil&) Edited October 11, 2013 by Ishad Nha Quote Link to comment Share on other sites More sharing options...
Magnificent Ornk Aran Posted October 11, 2013 Author Share Posted October 11, 2013 You keep using that word. The file format isn't even obfuscated, let alone encrypted. keira 1 Quote Link to comment Share on other sites More sharing options...
Understated Ur-Drakon Celtic Minstrel Posted October 11, 2013 Share Posted October 11, 2013 Hm, not sure why I didn't download that... it looks helpful... Quote Link to comment Share on other sites More sharing options...
Well-Actually War Trall Ishad Nha Posted October 11, 2013 Share Posted October 11, 2013 I work with great big chunks of hex data, where normally you can't tell what is what, hence the word "Decrypt" I suppose. Quote Link to comment Share on other sites More sharing options...
Understated Ur-Drakon Celtic Minstrel Posted October 12, 2013 Share Posted October 12, 2013 I think "decode" might be a little more accurate. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.