Picky details

[Niemand]

So I'm working more on my Avernumscript documentation, and I've run into a couple of things that I wanted to run by other people to make sure that I get them right:

 

As we all know there is no get_max_energy() call. Am I correct in claiming that this code will produce the desired effect of that call?

Code:

 *(2 + get_stat_levels_bought([which_char,2) + 	get_stat_levels_bought(which_char,11) + 	get_stat_levels_bought(which_char,12)) 

(This is 3*(INT + MAG + PRI) omitting points in those skills coming from items the character may have equipped, at least I think, with an extra 2 points for base INT of a level 1 charcter that get_stat_levels_bought doesn't seem to count. )

 

The editor docs list no creature scpecial abilites numbered 25 or 31. Are there just no abilities with these numbers, or are they just not listed correctly?

[Lazarus.]

Seems alright, though I never use get_stat_levels_bought, so I'm just going to have to assume that it works the way the docs describe.

 

Those two abilities seem to do nothing. No default creatures use them, and if you assign them to a creature they just behave like a normal ability-less creature.

[Ishad Nha]

What about get_stat(36)? A quick test shows that it does actually work in practice. Is this code of yours meant for PCs or monsters or both?

 

If you have problems with the get_stat(36) including the effects of adjustments from items, such items aren't all that common. For instance the editor documentation states: "Note that you can’t change Health or Spell Energy with an item, and changes to Intelligence and Endurance won’t change Health or Spell Energy".

[Celtic Minstrel]

While we're on the topic of documentation, there's this:

Quote:

From the Scenario Workshop Support page:

The call print_big_str_color is actually called print_big_str_num. It works exactly like print_big_str_color, as described in the editor documentation.

When I tried out AScript, I noticed that the error checker considered print_big_str_color to be valid while print_big_str_num was not.

[Ishad Nha]

The Ascript error checker may be based upon Alint, and the latter has a few errors itself. Thus the error checker itself has errors. One example is the "fl_shimmers" property, to Alint the correct answer is "floor shimmers", which is wrong.

[Niemand]

Yes, AScript uses Alint for its checking engine. I've fixed several of these bugs in Alint, but the fixes may not be in the beta releases, let me check. . . yes, both of those are fixed in my development version. When I release the full 1.0 version, I will also be releasing the updated Alint code along with it.

[Ishad Nha]

As I learned the hard way, the variables in the various calls are not checked for magnitude. Thus for put_item_on_spot Alint accepted an item with the number 3,840.

 

Then it may reflect errors in the documentation. For example, if you have about 255 states in your scenario script, Alint will spam you with

"Error line ###: Invalid state"

 

Then it found an error in the sanctitem.txt script.

[Celtic Minstrel]

When will version 1.0 be released? Are you planning to finish the documentation first?

[Niemand]

I'm hoping to have v1.0 within a couple more weeks. This will mean that the avernumscript documentation will not be complete, but I do have another large installment of it done, which I also plan to post on my website later today.

 

In fact, development toward v1.0 is going well: I've fixed a stupid mistake in the syntax coloring code that was causing the entire document to be re-colored after every key press, I've added a live counter that shows how many characters are in the string literal that the cursor is currently in, and I've added an export feature that exports the colored script as HTML code.

[Celtic Minstrel]

It's good to have the documentation online, of course, but I recommend distributing the program with an offline copy, presumably accessible through the Help menu - for the benefit of people who have dial-up or non-constant internet access (eg. only at school / work / whatever).

 

I'm not one of those people, but I would still like to have offline reference. Sometimes my internet connection randomly breaks.

[Niemand]

Way ahead of you, Minstrel. The documentation's embedded inside AScript's Application bundle and pressing a key combination looks up the currently selected symbol in a reference panel. (It's been in development since June; one day to write the code, 6 months to write a fifth of the content for it to display. )

 

So a new chunk of the documentation is done, here as usual. I've now finished the boats and horses calls and the character calls, and added a a set of important lists, of stuff like numbers of sounds, creature special abilities, and so forth.

 

Nice thought on using get_stat( ,36), Ishad Nha, I found it to work perfectly no both PCs and NPCs, so it's the method I'm recommending.

[Ishad Nha]

As for error - checking: I am not sure how many states a script can use. I have had no trouble using 255 states in a scenario script, (the port of Shadow of the Stranger.) Also I have used all sorts of numbers without crashing the script. Possibly you could include an option to disable the "Invalid state" check, at least until we all know just what the BoA program will and won't accept. The "State already defined" check would be retained.

[Celtic Minstrel]

The automatic porting feature of the editor is proof that the game accepts more states than the documentation claims. I suggest raising the limit on state number to 255 - that is, after all, the logical limit if the number is stored in a char variable (ie one byte).

[Niemand]

So I did a bit of testing:

-The editor has definite problems placing special rectangles for states higher than 254 .(As it turns out it should, see end.)

-When I placed a special rectangle for state 254, the game did nothing when I stepped in it. (This I'm still confused by.)

-When I placed a special rectangle for state 200, the game gave "Error: Accessed a town special state out of range."

-A special rectangle for state 199 worked fine.

-Using a set_state_continue call from state 199, I had no trouble with states up to 1000, that is, not just a single state numbered 1000 either, but in a script containing a thousand states.

 

My conclusions: You can probably have 32000 or so states. (Jeff almost certainly used a short for storing state numbers in his code, it's the variable type he tends to default to. He used shorts in many of the game's data structures, even when it's far larger than necessary. About the only place he stuck to chars was for the SDF matrix, unfortunately. )

However, despite having 32000 states to work with, only up to 199 will work for certain from special rectangles, and even if the pointless limit at 200 were removed, the absolute limit is 255; the state number for a special rectangle to call is stored in an unsigned char.

 

This leaves us with the problem of how to check scripts. Do we tell people, "No you can't have states higher than 254!" and lie to them about the possible existence of the other ~32500 allowed states, or do we tell them, "Sure you can have 32000 states! Oh, yeah, but if you want to use most of them, you have to remember to call them from another state in this one special way, unless you want to just use them for. . . " etc?

[Ishad Nha]

Interesting. In my experience, when a state does nothing: the state has an error in it. Unless, the scenario script has an error in it. If it contains one error anywhere, it may crash or it may simply not run at all.

 

Edit: I took the five minutes to test this, and it is strictly a function of the number of the town script state. Change the number to 200 and you get the error message, then the party can't enter the space occupied by the placed special. Change the number to 250 and you get no message and no halting of the party movement at all, odd.

 

Another limitation on a valid script, which the program itself showed me. Currently my observation is that a scenario script cannot exceed 4,257 lines in length. This puts a definite limit on the number of possible states.

[Niemand]

Ah, yes, I'd forgotten about that limit; the limit is not actually the number of lines (I got a script with 8500 short lines to run) but rather the number of tokens, as counted by BoA, which must be less than 32768. Assuming the game's script parser is still identical to the editor's, tokens are counted as the number of times that the script text switches from any of four modes, whitespace, digits, alphabetic or underscore, and other, to any other mode, with the distinction that in other mode, every character is counted as a separate token. 20 is then arbitrarily added to the total count.

 

So, 'print_num(5);' has 5 tokens, the print_num, the open parenthesis, the five, the close parenthesis, and the semicolon. Likewise, 'i++;' has 4 tokens and 'a = b + c;' has 6 tokens.

 

The annoying thing is that the the token counting code stores the number of tokens in a long, but then returns a short, and it's the size of the short that determines how many tokens a script may have. Unfortunately, there's some sense i this, as each token is expanded into a 5 byte object, so currently a full sized script expands to be 160Kb when tokenized. Of, course, this could be improved if the token's type an sort were packed into a single short, cutting the size down to 4 bytes, or better still if each token didn't store what line number it came from. One could just have a special line number token type: when the script is loaded at the start of each line that contains other tokens first insert a line number token, then the real script tokens. Then when executing the script, just skip over line number tokens, unless an error occurs, in which case you back up to the last line number token to find out what line you were on. (How to tell line number and non-line number tokens apart? Sacrifice 1 bit from the non-line number tokens, leaving 10 for the sort and 5 for the type, and set the most significant bit of the non-line number tokens to be the opposite of the most significant bit of the line number they're on. You start out skipping tokens whose msb is 0 as line number tokens, but when you hit a token that is 2^15, 100...00, you switch and skip tokens whose msb is 1, since this can only be the line number token for line 32768.) By doing this, each token would be 2 bytes, and in the worst case there would be only one real token on each line, which along with a line number token would use 4 bytes, the same as with the line number being stored in every token, but in better cases with, say, 10 tokens per line, you would go from using 40 bytes per line to using 22. With both optimizations a script with 32767 tokens would use at most 128Kb, and if it had 10 tokens per line would use only 70Kb. The token limit could then reasonably at least be relaxed to the size of an unsigned short (the line number should really be treated as unsigned as well) with memory usage never being worse than the current implementation. In order to improve performance with large scripts, the locations of numbered states could be found while the token stream is being parsed and cached rather than the current linear search through all tokens.

 

My rant about how I would write the avernumscript interpreter aside, I will look into adding the token counter either to Alint or AScript (preferably the former) so that scripters can at least be warned when they run up against some of these dumb limitations.

 

EDIT: Adding the token counter code to Alint its looks rather nontrivial, since it is important to preserve the counting system that BoA uses, and I really have no clue how to work with flex and bison. So, for now it looks like I'll be adding that as a second stage to AScript's checking system, which will be transparent to the user, but will actually be distinct from Alint.

[Ishad Nha]

Then there is the 750 - strings limit to be considered too. So now we have to consider states, tokens and the number of strings. If the programming of all this gets a bit tough, one approach is to just inform the Ascript user of all these considerations, you already have a special help file.

 

How best to use the information about tokens?

 

As for the help file, there is no problem with just copying in the entire Wiki file now, then returning later to add your own corrections and comments. While you are at it, you could add the corrections listed in the forum http://www.ironycentral.com/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=18;t=001016

[Niemand]

Does Alint not count strings toward the 750 limit? Even if not, that change I think I could add directly to it without it having to be a seperate stage. (Although that is what I said about my last idea, and it didn't work. ) In any case, it's not that the prgramming for these extra checks it at all difficult, it's just threatening to yield rather inefficient solutions, which I don't like. Besides, any additions I can make to Alint can easily benefit Windows users as well.

 

And meaning no offense to the people who put effort into the wiki, but I find that thing unusable. I would really rather keep at writing up my own documentation, slow though it is. As to the topic you point to, I have been referring to that and several other sources often to try to make my documentation as complete as possible.

[Celtic Minstrel]

I think the wiki, in its current state, is little more than a dump of the documentation file. So starting with the wiki is no different than starting with the docfile.

[Ishad Nha]

As for Niemand's post #52650 - Nov 27, 2007 7:21 AM:

tokens are counted as the number of times that the script text switches from any of four modes,

Whitespace,

Digits,

Alphabetic or underscore, and

Other,

 

to any other mode, with the distinction that in Other mode, every character is counted as a separate token. (Thus in Other mode, every character is a mode in itself.)

 

He said 'a = b + c;' has 6 tokens. (How do you go about representing a white space in text posted in this forum? I will use a tilde, "~".) I would count it as 'a~=~b~+~c;' equals 10 tokens.

 

I have been trying to figure out how to use the information, this not always easy because there may be only one way of encoding some feature.

If you were using "who_used_custom_item()" a lot, it would be better to replace it with a single letter or word.

Each repetition takes 3 tokens: who_used_custom_item()

Setting up the replacement, as say “d”, takes up 8 tokens: d~=~who_used_custom_item();

 

[Niemand]

No, white spaces are not tokens. I think I left out a detail, which is that switches into the whitespace mode do not count towards the number of tokens. So "anton = bruno + caesar;" becomes the six tokens 'anton', '=', 'bruno', '+', 'caesar', and ';'.

 

If you want to see the code involved for yourself take a look at text_block_type::estimate_num_of_tokens() in EdParser.cpp of the editor code.

 

You've made a mistake with 'who_used_custom_item', which is neglecting that A-Z, a-z, and _ all belong to the same mode, so no switches occur and this function name counts as a single token.

[Celtic Minstrel]

Actually, "who_used_custom_item()" would be 3 tokens: "who_used_custom_item", "(", and ")". The name is a single token, yes, but there's also those parentheses.

[Niemand]

Ouch. Sorry Ishad Nha, I didn't pay enough attention to what you wrote.

[Ishad Nha]

Right, line 174 is where you find the start of this function:

if ((space == FALSE) && ((old_mode != current_mode) || (current_mode == 3)))

num_tokens++;

 

In plain language for the general reader of this topic:

(a) if a character is not a whitespace:

(b)if the character's mode does not equal the mode of the preceding character or if the character is mode 3

 

Then the number of tokens is increased by 1.

 

This means that "d = who_used_custom_item();" uses exactly twice the number of tokens used by "who_used_custom_item()". Hence the replacement is worth it if you use the latter call three or more times.

[Ishad Nha]

Reading the code below, it seems that not only final semi-colons will cause an increase in the number of tokens but also ordinary English language punctuation marks in blocks of text!

 

else if ((text_block >= '0') && (text_block <= '9'))

current_mode = 2;

else if ((text_block >= 'a') && (text_block <= 'z'))

current_mode = 1;

else if ((text_block >= 'A') && (text_block <= 'Z'))

current_mode = 1;

else if (text_block == '_')

current_mode = 1;

else current_mode = 3;

[Celtic Minstrel]

Huh? Isn't anything enclosed in quotes treated as a single token?

[Niemand]

By the actual parser, but not by the (exceedingly simpleminded) token counter. I think Ishad Nha is right, and text inside of string literals will inflate the game's estimate of your script's length.

 

Note however, that comments are handled correctly, text_block_type::preprocess_text() is used to remove them before the token count is estimated. This function also has the interesting side-effect (read that: annoying effect) that variable names, and indeed everything else, is not case sensitive. I just tested, and the game is completely unbothered by a town script which contains this:

 

Code:

beginstate StARt_StAtE;	PRINT_STR("Yo?");break;

 

The more importantly, this means that the behavior of variable names is somewhat screwed up. The game lets you declare two variables whose names differ only in capitalization (in fact, it seems not to mind if you explicitly declare two variables with the same name; they can even have different types!), but if you try to use them, you end up only accessing which ever was declared first:

 

Code:

variables;int a;int A;string a;body;beginstate START_STATE;	a=16;	A=5;	print_num(a);	print_num(A);break;

 

This will print:

Quote:

5

5

[Kelandon]

Ach, I could've sworn calls were case sensitive. Are they really not? PRINT_STR works?

[Niemand]

It does indeed. Weird, no?