Jump to content
  • Announcements

    • The hills are bridges

      Logging in   05/10/2017

      If you had an account and can't log in as of 5/9/17, this may be because of a change in logins with new forum software. You can log in using your publicly displayed name (not your username) or your email address and the password you used before.   If you have problems with this, please ask any of the mods or admins. 
Sign in to follow this  
Ishad Nha

Blades of Geneforge - How it could be done

Recommended Posts

Placed specials are not hard to understand, you step on a square and something from your townscript happens. In Geneforge games, unlike BoA, a placed special can also call a state from a town dialog script.

Sign text is a special state = 100 + Townscript state

States 1,3,4,9: check the begintalknode # in the town dialog script.

 

Edit:

Now I am starting on altering the source code for the BoA 3D Scenario Editor.

I will equate GF5 zones to BoA towns, hence all traces of the BoA outdoor zones will be removed from the code. The 100x20 outdoor map data structure will be placed in the Scenario menu. Horses, Boats, variable towns will all need to be removed too.

This alteration will easily take days or weeks.

 

A GF1 zone record has 38,796 = 12*53*61 Bytes. Thus if you display it in a hex editor with line width set to 53 or 61, the towns will always start in the same column.

GF2: 3,529,440 = 41,040 * 86 = 5*16*27*19

GF3: 4,421,144 = 48,584 * 91 = 8*6,073. 6,073 is a prime number, no use to us here.

For GF4 and GF5, 52,088 = 8*17*383 =68*766

Edited by Ishad Nha

Share this post


Link to post
Share on other sites

I can write a program that turns a hexadecimal town record into a decimal printout. Also, the printout has the correct number of bytes for each line. If the creature records each have 80 bytes then the printout can have one record per line. It should be useful in decrypting the records.

The program can be re-written to make it applicable to any Spiderweb game.

Share this post


Link to post
Share on other sites

I read all the records byte-by-byte and discarded the unwanted bytes in the sequence, only because I couldn't come up with a structure which exactly matches the one used by Jeff. Even with byte-padding put in, I don't see the need for so many empty spaces between the the values in the file.

 

From a programming interest, I'm interested to know what methods will you be using (since I didn't use structures I just used the raw read() and write() methods for individual bytes) ?

Share this post


Link to post
Share on other sites

Jeff does use 2 Bytes for a value when 1 Byte would do.

As for empty space, we would need to read all of the town records before we could be sure that empty space was never used.

 

"From a programming interest, I'm interested to know what methods will you be using"

Do you mean with the G5 printout program? I have not gotten around to read/write, I am not much of a programmer. I will be using fopen if and when I get around to writing a proper program. Said program is only a means to decryption, not an end in itself. I have a 52,088 Byte table compiled into the program. When I want to printout a new town I just copy and paste a town record from the G5Scendata.dat file right into the program.

 

I am making some progress with the creature records, in BoA the first number for town creatures is 6 in G5 it is 8.

 

The creature data structure seems to be like this:

1-2: Creature type number

5: Adjustment to X coordinate of position

6: X coordinate of position

7-8: Empty

9: Adjustment to Y coordinate of position

10: Y coordinate of position

11-12: Empty

13-24: Creature script

25-44: Memory Cells[10]?

45-52: looks like items carried and chances of them being carried

53-54: nothing?

55-56: attitude

 

65-66: facing

Here I am numbering from 1 to 80.

 

Where are the navpoints? Alter the scripts to send the creatures marching towards navpoints.

Edited by Ishad Nha

Share this post


Link to post
Share on other sites

Jeff does use 2 Bytes for a value when 1 Byte would do.

Jeff does use 4 Bytes for a value when 1 Byte would do.

 

I found the navigation points at offset 48,840:

In town 1, Minallah, a servile called Gevik had the "mobleadpc" script, which involves the use of way points, navigation points. He seemed to stop at a square x = 5, y = 20. I found a possibility in the town record at offset 48,840. It was prudent to check that this was really the waypoint. Hence I altered it to read x = 49, y = 8. Now Gevik leads the PC to the town 0 entrance in the NE corner of town.

 

 

????[250][28] 41,840 7,000. This may be a set of paths followed by creatures with certain creature scripts, I will need to research this further.

Share this post


Link to post
Share on other sites

I feared that if Jeff did not hand over the source code to the Editor this whole process would be slow. It sure has been.

Can't identify the places where the following are recorded:

spawn point, floor types, wall types, town lighting levels.

Offsets 48.920:48.927, this is the one area where something might be stored.

 

I created a zone printout program for G4, this may yield some answers.

Share this post


Link to post
Share on other sites

Floor types, wall types, town lighting levels may all be hard-coded for all I know.

It may be possible to track down and decrypt the tables concerned, it seems they are found in the program.

(Mercifully, Tridash found the place where the wall/floor types are set. Hence the rest of this post, on how to handle hard-coding of floor/wall types, is not needed.)

Share this post


Link to post
Share on other sites
tridash   

floor types etc. aren't hard-coded, they're about where you said they might be (i think walls are at +48908). when i'm a bit more certain of things i'll put details of how they work up.

Share this post


Link to post
Share on other sites

I thought walls and floors were set at offsets 48.920:48.927. But actually altering these bytes seems to affect nothing, even when I start with a new game.

In GF4, the values are:

Rebel Base Docks 0 0 0 0 0 0 0 0

Southforge Citadel 0 0 0 4 255 255 255 255

Upper Southforge 0 1 255 244 255 244 255 244

Citadel Gates 0 0 0 0 0 0 0 0

I don't see any pattern, towns 0 and 3 don't have identical floors/walls.

 

By contrast, the last two "nav points" look more credible:

0 0 0 0 0 1 0 3 0 0 0 0 0 0 0 0

0 0 0 2 0 1 0 2 0 0 0 0 0 0 0 0

0 0 0 2 0 5 0 2 0 0 0 0 0 0 0 0

0 15 0 0 0 4 0 3 0 0 0 0 0 0 0 0

It would be nice if floors and walls are found at 48,904 through 48,919.

I can start altering them and see if it has any effect.

Edit:

It worked for the first two towns in GF4, now I will try it for GF5. Just as well Tridash was on the ball. I will then go through one Byte (or Short) at a time and see what changes.

Share this post


Link to post
Share on other sites

Comparing towns 0 and 2:

0 0 9 0 4 0 5 0 0 0 0 0 0 0 0 0

0 0 3 0 7 0 2 0 0 0 16 1 1 0 0 0

 

Changes and their effects:

9 to 3, Wall 1 type changed.

4 to 7, Wall 2 type changed.

5 to 2, outdoor floor type changed.

Share this post


Link to post
Share on other sites
tridash   

I wonder if something like a wiki (or maybe just a shared google doc / piratepad) would be a sensible place to keep what's been worked out about the format, otherwise it's getting a bit scattered through the thread.

Share this post


Link to post
Share on other sites

Yes, that would be a good idea. As understanding of data structures improves we will need to re-write our notes...

I still have some stuff to upload about the G5 program itself. Program contains Special Item names, Special Item descriptions, Quest names, Quest descriptions, Intro text and Item Descriptions.

Share this post


Link to post
Share on other sites

Geneforge 5.exe

 

Generic Item descriptions: 2DD3A0 = 3,003,296

 

Lists of folders and files written into the program.

These can hopefully be altered to enable the program to directly read a scenario's files.

Altering the name "Scripts" would have the program look for scripts in a folder with the name you specify. (Said name should be no longer than seven letters.)

 

2D,FFE8 = 3,014,632: Scripts

2E,1290 = 3,019,408: gf5objsmisc.txt gf5itemschars.txt gf5floorster.txt

2E,3524 = 3,028,260: Geneforge 5 Files\Graphics B\G%d Geneforge 5 Files\Graphics A\G%d Geneforge 5 Files\Graphics Core\G%d .png .jpg .gif .tga .bmp

 

Unlike BoA some things are written right into the program: Special Item names, Special Item descriptions, Quest names, Quest descriptions, Intro text and Item Descriptions.

Geneforge 5.exe has 256 Byte strings of text, they start no later than offset 3,462,208. There are 1,152 Bytes of blank space before this, it could start as early as 34,D040 = 3,461,184.

I copied all the text into a spreadsheet.

 

34,D440 = 3,462,208: Creation descriptions. Ends at 3,470,143. Blank space till 3,474,752.

35,0A40 = 3,476,032: start of descriptions of statistics.

 

0 3,559,232 3,584,832

(100) 3,584,832 3,610,432

 

Special Item names start: 3,559,232

Special Item descriptions start: 3,584,832

 

There are 100 possible slots but only about 60 can be used for special items, other 40 are reserved for various G5 game dialogs. Figures above are for the 100 possible slots.

Highest-occurring number for a Special Item is 54: Twisted Key

 

Quests

Names Descriptions

0 3,710,272 3,735,872 only 256 Bytes available not 512

1 3,710,528 3,736,128 Escape the Fields.

2 3,710,784 3,736,640 Enter Minallah.

3 3,711,040 3,737,152 See Rawal.

4 3,711,296 3,737,664 The Presence.

 

99 3,735,616 3,786,304

(100) 3,735,872

Not clear if Quest 0 can be used.

 

Intro texts: 3A0740 = 3,802,944

Graphics are G170 through G173, found in Graphics Core, don't know how to alter this in the program. It could be altered in the graphics folder, by renaming photos.

It ends at 3A193F = 3,807,551.

 

Item Descriptions: it_extra_description

 

0 3,815,744 Not clear if this can be used.

1 3,816,000

 

32 3,823,936

33 3,824,192 Empty

 

48 3,828,032

49 3,828,288

50 3,828,544 Firebolt, this would seem to be the end of item-descriptions

Share this post


Link to post
Share on other sites
sylae   

I wonder if something like a wiki (or maybe just a shared google doc / piratepad) would be a sensible place to keep what's been worked out about the format, otherwise it's getting a bit scattered through the thread.

If y'all want, I would be willing to host a Mediawiki wiki for this project.

Upon Mars. likes this

Share this post


Link to post
Share on other sites

Yes, that would be desirable, just like it is for BoA, only more so.

What form it takes is not a big issue, but we need something that is big and that can be updated. What is being learned needs to be collated, all in the one place. It is getting a bit big for one post on this forum.

 

Sections on the site could include:

How to do things

Workarounds

Calls used by the game

Differences between the BoA and Geneforge games, an awful lot of differences exist.

Structure of the ScenData.dat file

Text and folder parts of the G5 program

How we enable a scenario structure.

Share this post


Link to post
Share on other sites
sylae   

Okay, here it is: http://bgwiki.calref.net/Main_Page

 

I haven't done any content editing, just the set-up. I'll prolly import some big templates like {{infobox}} at some point, but for the most part i'll probably just stick to keeping it running and formatting any minor issues i see with other people's edits :p

 

Also, I don't think it's really needed but I do have a CI server as well, and since it spends most of its time idle y'all are more than welcome to run jobs on it if need be.

Edited by S▒y░░e▒
If the appropriate persons would inform me when they make an account and give me their username, I will toss out some sysop permissions

Share this post


Link to post
Share on other sites

Excellent! We have so much to upload.

 

TGA files are a type of graphic, they can be opened with IrfanView or the GIMP for starters. I could not find the floor types occurring in the town 0 outdoors, then I did a batch conversion of the Graphics A TGA files...

For the latest version of IrfanView:

http://www.irfanview.com/

(Now I can't find the original TGA files?! Next time do a Batch Conversion using copies of the original files.) No problem, I changed the file association, now they are listed as IrfanView files not TGA.

Next, I need to figure out how the floor graphics relate to the floor types listed in the town record.

 

Edit:

I just uploaded some Scendata.dat stuff to the Wiki. How do I enter a table?

Edited by Ishad Nha

Share this post


Link to post
Share on other sites
sylae   

You can find answers to any formatting questions here (I'll copy this over to bgwiki at some point).

 

For tables, this code is a decent template:

{|
|  Orange    ||   Apple   ||   more
|-
|   Bread    ||   Pie     ||   more
|-
|   Butter   || Ice cream ||  and more
|}

 

It will produce this:

fc3e8dd7.png

 

I would definitely recommend reading the help page on tables to see all the formatting options. Really, the help pages as a whole are quite useful. MediaWiki can do some very useful things, and I've got several useful extensions (ParserFunctions, SMW, etc) that make it even more so

Share this post


Link to post
Share on other sites
tridash   

i did a bit of reorganisation, putting the scenario data stuff into a separate page, and added some stuff that may or may not be correct.

Share this post


Link to post
Share on other sites

Template stuff can be checked versus known graphics in the G5 folders.

Six of the templates use the TGA format, that will need to be converted before it can be viewed as thumbnails.

I uploaded a quick list of the templates and their names.

 

Edit:

Tridash seems to be correct about the Template numbering. I checked the numbers for templates and graphics, there were no exceptions.

I have attempted a set of names for the templates. This is not always easy, some templates are shapeless grab-bags without rhyme or reason.

Edited by Ishad Nha

Share this post


Link to post
Share on other sites

A text editor and a hex editor can make this quite easy. Now I have a list like this:

 

0 1,000 Floorset: Grass; Water

1 1,010 Missiles1

2 1,020 Missiles2

11 1,110 Fyora

(a whole lot of lines have been deleted here...)

209 3,090 Extra Big Floors

240 3,400 Rain

241 3,410 Snow

242 3,420 More Projectiles

 

I would like to get it into a format suitable for the BG Wiki:

{|

|0||1,000||Floorset: Grass, Water

|-

|1||1,010||Missiles1

|-

|2||1,020||Missiles2

|-

|11||1,110||Fyora

|-

 

(a whole lot of lines have been deleted here...)

 

|-

|209||3,090||Extra Big Floors

|-

|240||3,400||Rain

|-

|241||3,410||Snow

|-

|242||3,420||More Projectiles

|}

 

First, save the list into a file.

Second, open the file in a hex editor. Comparing the text file with its hex translation we see that tab stops equate to "09" when the file is viewed in the hex editor.

Third, change all tab stops to ||.

 

Now we want odd lines to start with a | and even lines to start with a |-

Again, comparing the text with the hex, we see that paragraph marks equate to "0D0A". Here I will show all such marks as "\n":

 

We have:

0 || 1,000 || Floorset: Grass; Water\n

1 || 1,010 || Missiles1\n

2 || 1,020 || Missiles2\n

 

We want:

|0||1,000||Floorset: Grass, Water\n

|-\n

|1||1,010||Missiles1\n

|-\n

|2||1,020||Missiles2\n

 

Solution:

Change an existing "\n" to "\n|-\n|".

 

Save it in the hex editor. File should automatically reload in the text editor. Then, in the latter editor, add the {| and|}.

Edited by Ishad Nha

Share this post


Link to post
Share on other sites

Interesting Titbit : When tinkering around with the Geneforge RAM I found that adding 12661220 (Dec) to the file address gives us the corresponding RAM address.

Share this post


Link to post
Share on other sites

One thing I don't know is where the intro graphics, G170.bmp through G173.bmp, are being called from. Titbit might help.

 

Edit:

This is the latest guess about how the templates relate to the wallsets listed in the town records:

0 20 1,200 Wallset: Stone

1 24 1,240 Wallset: Light Smooth

2 22 1,220 Wallset: Stone

3 23 1,230 Wallset: Gray Stone

4 21 1,210 Wallset: Wood

5 44 1,440 Wallset: Worn Walls

6 52 1,520 Wallset: Rough Stone

7 62 1,620 Wallset: Big Stone

8 63 1,630 Wallset: Big Old Stone

9 64 1,640 Wallset: Ice Stone

10 65 1,650 Wallset: Rubble Stone

(11 Crasho, floor type madness!)

I tried one value at a time and thus could see what wallsets are produced by what numbers.

Why the numbers 0 thru 10 are assigned escapes me. In BoA you just use the straight graphics sheet numbers. Then it is not currently clear to me why one sheet is "0" and not "2".

Edited by Ishad Nha

Share this post


Link to post
Share on other sites

Today I had a stab at decrypting the number assigned to each wallset:

0 1,000 Floorset: Grass; Water 0 TGA

36 1,360 Floorset: Grass; Water 1 TGA

38 1,380 Floorset: Cave 2 TGA

42 1,420 Floorset: Mountains 3 TGA

53 1,530 Floorset: Gully 4 TGA

55 1,550 Floorset: Outdoor Snow 5 TGA

25 1,250 Floorset: Grass; Stone 6

37 1,370 Floorset: Orange Wood 7

39 1,390 Floorset: Cave 8

43 1,430 Floorset: Mountains 9

54 1,540 Floorset: Gully 10

56 1,560 Floorset: Outdoor Snow 11

200 3,000 Floorset: Big 1 12

201 3,010 Floorset: Big 2 13

202 3,020 Floorset: Big 3 14

203 3,030 Floorset: Big 4 15

204 3,040 Floorset: Big 5 16

205 3,050 Floorset: Big 6 17

12:17 are unverified, may be indoors only

Columns show: the template number, the graphics sheet number, the unofficial name, the number that represents this floor set in the town record and whether it is a TGA graphic.

I don't know how many floor sets each town can have.

Strange, some terrain types like trees varied with the floor type. Changing the floorset also changed certain terrain types.

Share this post


Link to post
Share on other sites

Fantastic work guys! If you want I can even make images for the prologue and endings like Andrew Hunter did once, that is when you've finished cracking this hard nut and start thinking storywise! Keep it up, you're going to make it through!

Share this post


Link to post
Share on other sites

I can't access the Wiki with Javascript enabled, with it disabled I can't save a page!?

 

Offers of art are welcome. It will probably be some time before the ScenData.dat file is properly understood. Then we will need to find someone who can program better than I can.

Share this post


Link to post
Share on other sites
sylae   

try it now ishad, the database got a little pissy and locked a table, preventing proper wiki function. in other news i need to get innodb back again :p

 

 

Share this post


Link to post
Share on other sites

In the Wiki I posted a table listing the 24 Byte wall/floor record for each of the 82 towns: offsets 48,904 through 48,927.

(Assuming it is all walls and floors, which may not be the case at all for all I know.) It is listed on the main page as "Today's Mystery".

I have not played the game through so I don't know a lot about the various towns.

Share this post


Link to post
Share on other sites
tridash   

Floors correspond to the numbers in scendata like this (I think, going from the executable)

 

Floor number / Templates

0 / 0, 25, 100

1 / 36, 37, 101

2 / 38, 39, 102

3 / 42, 43, 103

4 / 53, 54, 104

5 / 55, 56, 105

Upon Mars. likes this

Share this post


Link to post
Share on other sites

We will need a proper programmer at some point fairly soon.

I am currently attempting a Blades of Nethergate Editor, which is the simplest possible Blades of Spiderweb project. Even on the simplest project my programming skills are not up to the job when it comes to the file initialization.

At this forum we do have proper programmers, they are always trying to do a cross-platform version of BoE or the BoA 3D Editor, (and never finishing it).

Share this post


Link to post
Share on other sites
sylae   
they are always trying to do a cross-platform version of BoE or the BoA 3D Editor, (and never finishing it).

thats a sure way to get their help ishad

 

anyway i would recommend looking at existing source code, iirc nethergate is based off BoE so the CBoE code might be a good starting point?

Share this post


Link to post
Share on other sites
simplest possible Blades of Spiderweb project

 

About that.

 

Let's assume that your end goal is a reasonably usable Total Conversion maker from scratch: Something with a learning curve like Blades, but that you can use without knowing the word "hexadecimal". Not necessarily WYSIWYG but still with a graphical interface. And you want the source to be in a state where anyone can join in or take over, otherwise there's no point.

 

That is a huge project regardless of what game you want to do it to. Examining the binary game data is the smallest aspect, while the vast majority is simple generic software design: Data structures, modules, design patterns, UI, documentation, testing. The parts you notice the least in the end product are some of the hardest.

Share this post


Link to post
Share on other sites

"They are always trying to do a cross-platform version of BoE or the BoA 3D Editor, (and never finishing it).

thats a sure way to get their help ishad"

(But it sure is the naked truth!)

 

"anyway i would recommend looking at existing source code, iirc nethergate is based off BoE so the CBoE code might be a good starting point?"

That sounds like a simpler code to copy, I intend that the Editor would be in the same folder as the game program.

 

I will be staying with the BoA code, it far exceeds what I could do by myself.

Share this post


Link to post
Share on other sites
tridash   

I'm not sure if it's worth spending the time on a proper BoG editor if nobody is going to use it, though.

Share this post


Link to post
Share on other sites

Once it is finished it may create demand for G5 scenarios. I am only wasting my own time not Jeff's... It also gives me feelings of programming grandeur.

Share this post


Link to post
Share on other sites

Playing scenarios which are based on the original story or even storyline-extensions would be fun. The ability of the characters to shape is of course a major aspect which shouldn't be ignored.

Share this post


Link to post
Share on other sites

Just got back to where I was a week ago with the Neth Editor.

There is a lot of work involved in just removing extraneous BoA-only stuff from the source code. N:R is much closer to BoA than ever G5 is.

I compiled a list of the BoA calls and a list of the G5, the lists only agree only about half of the time. Mercifully, to get an Editor going I only need to grasp the cr,fl,tr,it,ob calls not all the others.

Upon Mars. likes this

Share this post


Link to post
Share on other sites
sylae   

(Sorry for the resurrection)

 

The Blades of Geneforge wiki (linked in this topic (and elsewhere)) will no longer exist by February of this year.

 

An XML dump for people wishing to start up a successor wiki is available here: https://c312441.ssl.cf1.rackcdn.com/files.calref/16/ec277aa6_Blades+of+Geneforge-20160106210122.xml (you can also generate this file yourself using Special:Export if you so desire, the site will most likely remain up for a few more days).

Edited by sylae
if you have questions about the implimentation of a new wiki, i would be more than happy to share further coniguration details.

Share this post


Link to post
Share on other sites
BMA   

I've been wanting to make a zone-editor using the floor/terrain/character bytes info on the BoG Wiki for a while now (using C#/WPF and SDL 2) but have been putting it off as usual.

 

The number of zones are hard-coded at the beginning of the scenData file. So if we were to create a completely new zone (say a basement, since that would probably be easiest), and add the new bytes to the file, what else then would have to be changed, what info should be added to the header (if at all, like Ishad Nha was saying) or would it simply suffice to create an entry point (ladder or steps) from inside an existing zone which points to the zone-number of the custom zone ? I'll be fiddling around with the bytes to find out how, and I'll post here if I do.

Share this post


Link to post
Share on other sites

Zone editor, Little Fyora made one a while back.

http://spiderwebforums.ipbhost.com/index.php?/topic/20452-blades-of-geneforge-zone-simulator/

I came to the conclusion that the GF5 floor/terrain format was more complex than in BoA, a tree could vary according to the nearby floor types...

ScenData file was decoded by me, see the Wiki for details.

Edited by Ishad Nha

Share this post


Link to post
Share on other sites
BMA   

Yeah I know, I had been following the byte-deciphering here from the start. I've saved copies of the BoG wiki's web pages jic, but it would be nice if it can be hosted online. I've wanted to make a zone-maker, kind of like LF's Zone Simulator but which can create new zones from scratch (or say from a template) instead of just editing them.

 

It should be offset 106, but I have never attempted to add additional towns myself...

Then you might want to give a new town its own record in the header of the scenario data file.

 

@Ishad If we were to create a new zone, do you think it would be enough to change offset 106 (number of towns) from 82 to 83 and then add 52088 bytes of new zone data after the last zone, and then add it's own 'town location record' (20 bytes) into the header ? Oh yeah and there's the entrance into the new zone from an existing one. Whew, that does sound a lot. But if we could manually create a zone by moving and adding bytes around on a hex-editor, then making a program out of it shouldn't be too hard.

Share this post


Link to post
Share on other sites

I simply have no idea if it will be enough, go ahead and try it, see what happens! It will probably work or crash the game, so back up your ScenData file.

BMA likes this

Share this post


Link to post
Share on other sites

I'd like to see a BOG but the game would need to sell well or it could cause financial troubles :(

I would love to have a moddable Geneforge game.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×