Avernum 5/6 and a precipitous drop in some skills at 10

[Brocktree]

I was reading through the effects of some skills in Avernum 6 that Slarty had posted. For quick action, he had deduced the following:

 

QUICK ACTION (Cost:1 -- No Trainers)

+4% chance of double strike on all melee/pole attacks (10-cap)

 

 

The three most common explanations for the way skills work is either:

 

1. A flat, constant bonus for each point added. For example, adding 1 to melee always gives you +5% to hit, and +1 die damage.

 

2. Capped. No further bonus is obtained after investing more than the capped skill level.

 

3. Diminishing returns. For example, each point invested up to 10 grants you a +5% bonus to double strike. After 10, each *two* points grants you a +5% to double strike.

 

I decided to perform my own little test. I invested 5, 10, 15 and 20 points of quick action in the same character, and attacked an enemy 37 times on each skill level. Here were my results

 

[Edited in: QA 1 = Hit 6% of the time (6% per skill point)]

 

QA 5 = Hit 30% of the time (6% per skill point)

 

[Edited in: QA 9 = Hit 30% of the time (3.5% per skill point)]

 

QA 10 = Hit 22% of the time (2.2% per skill point)

 

QA 15 = Hit 38% of the time (2.5% per skill point)

 

QA 20 = Hit 44% of the time (2.2% per skill point)

 

Note the precipitous drop in your chance to double strike when you invest 10 skill points. My testing does not gel with any of the above three explanations.

 

1. A flat, constant bonus for each point added. For example, adding 1 to melee always gives you +5% to hit, and +1 die damage -

Clearly there is a decline in the benefits you receive from quick action after investing somewhere between 6 and 10 points.

 

2. Capped. No further bonus is obtained after investing more than the capped skill level -

Quick action is clearly *not* capped, as a 20 point investment grants you a greater chance to double strike than a 5, 10, and 15 point investment.

 

3. Diminishing returns. For example, each point invested up to 10 grants you a +5% bonus to double strike. After 10, each *two* points grants you a +5% to double strike.

 

This is where it gets interesting. Clearly quick action gives diminishing returns. However, the traditional explanation doesn't fly. Even if you received diminishing returns after investing 5 point, you should *still* have a greater chance to double strike at 10 points than at 5 points.

 

For example, let's assume quick action grants you 5% to hit for every point invested up to 5, and then every second point invested up to 10. 10 points in quick action invested should give you: (5%*5) + (2.5% * 5) = 37.5%

 

Now, I would usually brush off the drop at 10 points as a statistical aberration. Perhaps a sample size of 37 isn't enough , and I'll be damned if I'm going to repeat the test 100 times. However, there's one problem. I've noticed the exact same phenomenon in the past for magical efficiency and lethal blow! 5 points gives you a better cumulative benefit than 10 points, but 20 points grants you a better cumulative benefit than 10 points.

 

What am I trying to say? I'll put it simply:

 

Up until somewhere between 6 and 10 (let's call this point X) points of quick action, each point you have invested grants you roughly a 5% chance to double strike. However, once you hit X points of quick action, the benefit for each point you have invested drops to 2.5%. The benefit you receive from each point has effectively *reset* to a lower value upon reaching the X point threshold.

 

I know that my hypothesis is a bit 'out there', but it's the only one which really fits with my observations. The next question is: How would I confirm or refute this hypothesis?

 

 

 

 

 

 

 

[Brocktree]

A second trial of QA 10 gave a 48% chance to double strike. Good lord.

 

I'm not sure if I can pick a trend here. All I can conclude is that it's not worth investing over 5 points in quick action, as any benefit after this point is minimal.

[Aoslare]

Quote:

3. Diminishing returns. For example, each point invested up to 10 grants you a +5% bonus to double strike. After 10, each *two* points grants you a +5% to double strike.

This is what is referred to by "10-cap." It does not fit QA results perfectly, but it fits them better than other options and has been used very widely in Jeff's games. "10-cap" is not a perfect name for this but "diminishing returns" isn't great either since that name actually describes every single skill with a rising skill cost and flat additive effect.

Quote:

Clearly there is a decline in the benefits you receive from quick action after investing somewhere between 6 and 10 points.

In fact, this is not clear. First of all, if we are expecting to see a clear difference between 20% and 40% (for example), 37 tests are likely to show this difference but do not guarantee it. Considering that it differs from previous data it is hardly conclusive. Second, some empircal questions: did you attack the exact same entity each time? How did you deal with misses?

Quote:

I know that my hypothesis is a bit 'out there', but it's the only one which really fits with my observations. The next question is: How would I confirm or refute this hypothesis?

Lots and lots of tests.

To prove the point about testing numbers, I posted some QA tests for A6 a long time ago. They were similarly sized (30 successful attacks each) and I got results of 37% for 10 QA and 60% for 20 QA. These results are striking since they are not close to yours at all and they are both higher. That suggests that either (1) something else affects QA efficacy as well, (2) QA results include a large random factor, or both.

The point about observing the same thing for ME and LB is interesting. I'm not convinced, but it's interesting.

One possibility: in Exile, Jeff used lookup tables rather than formulas for most skills. Weapon skills, for example, would go up by 10-20% for the first few points, then 5% for a bit, then slowly decrease down to tiny percentage increases as you neared 20. While there are many skills where we KNOW he has given up lookup tables, I guess it is possible they were preserved for something like QA. Personally I find that hard to believe, but it's possible.

[Brocktree]

Originally Posted By: HOUSE of S

This is what is referred to by "10-cap." It does not fit QA results perfectly, but it fits them better than other options and has been used very widely in Jeff's games. "10-cap" is not a perfect name for this but "diminishing returns" isn't great either since that name actually describes every single skill with a rising skill cost and flat additive effect.

10-cap does appear to be the 'best' explanation at this time, but it is far from perfect. However, I think it is obvious that a sky high investment in quick action does not give you the benefit it ought to.

Quote:

Second, some empircal questions: did you attack the exact same entity each time?

No, I used minor summons. Do you think your chance to have a double strike differs depending on the creature type? I find that hard to believe, but it is one possible explanation for the huge and erratic variation.

Quote:

How did you deal with misses?

I didn't count outright misses, as I would assume that one would have had the same probability to double strike with those attacks if they had connected.

If I missed on the double strike, I still counted the attack as a double strike.

Quote:

Lots and lots of tests.

Yeah, I was hoping you wouldn't say that.

Quote:

To prove the point about testing numbers, I posted some QA tests for A6 a long time ago. They were similarly sized (30 successful attacks each) and I got results of 37% for 10 QA and 60% for 20 QA. These results are striking since they are not close to yours at all and they are both higher. That suggests that either (1) something else affects QA efficacy as well, (2) QA results include a large random factor, or both.

I think Explanation 2 is more likely. Perhaps each point does not give a flat % value to double strike, but a value randomly selected from between a range of values (eg. between 1-5%)

Quote:

One possibility: in Exile, Jeff used lookup tables rather than formulas for most skills. Weapon skills, for example, would go up by 10-20% for the first few points, then 5% for a bit, then slowly decrease down to tiny percentage increases as you neared 20. While there are many skills where we KNOW he has given up lookup tables, I guess it is possible they were preserved for something like QA. Personally I find that hard to believe, but it's possible.

In consideration of the data, I don't find it that hard to believe.

[Aoslare]

Originally Posted By: Brocktree

Quote:

Second, some empircal questions: did you attack the exact same entity each time?

No, I used minor summons. Do you think your chance to have a double strike differs depending on the creature type? I find that hard to believe, but it is one possible explanation for the huge and erratic variation.

I doubt it differs directly depending on the creature type. However, it's always possible that it differs based on hit chance or something like that, or that it has some unexpected interaction with level, dexterity, parry, gymnastics, luck, or the like.

Quote:

Lots and lots of tests.

Yeah, I was hoping you wouldn't say that.
Well, there is one other possibility, which is that we email Jeff and ask. He will sometimes (but not always) answer these types of game mechanics questions. However, he will not have a long conversation about it so the original question needs to be really thoughtful to be sure the answer you get gives you all the information you might want.

Quote:

I think Explanation 2 is more likely. Perhaps each point does not give a flat % value to double strike, but a value randomly selected from between a range of values (eg. between 1-5%)

The only time I have seen this much variation was testing armor effect in N:R. Thurilith suggested the involvement of a random factor as in the following scenario: for each point of damage, the game rolls a percentile to see whether or not it is blocked. That will give you a distribution of damage blocking that is the actual % on average, but is often higher or lower.

Quote:

One possibility: in Exile, Jeff used lookup tables rather than formulas for most skills. Weapon skills, for example, would go up by 10-20% for the first few points, then 5% for a bit, then slowly decrease down to tiny percentage increases as you neared 20. While there are many skills where we KNOW he has given up lookup tables, I guess it is possible they were preserved for something like QA. Personally I find that hard to believe, but it's possible.

In consideration of the data, I don't find it that hard to believe.
I think it's unlikely given that it would have to be hardcoded, and Jeff has moved pretty consistently towards modular definitions for almost everything.

[Thoukydides]

Brocktree, I doubt that any randomness you're seeing is because of a reduction in Quick Action's overall effectiveness. It’s probably just a result of your sample size. As an example, after 37 tests of QA 10, I had 19% double strikes, after 200, I had 37%. While several of my tests were accurate after 30-50 tests, sometimes you do get the extremes.

 

In light of Brocktree's theory, I decided to do some tests of my own against huge rats on Normal and with base stats, except for Endurance and Quick Action, at character level 1. Killing blows and misses were not counted unless the second strike performed the kill or the miss.

 

Since I wanted to test Brocktree’s hypothesis that QA, and possibly ME and LB, have diminishing returns after level 5, or a “5-cap”, I decided to add levels 3 and 7 since they would give us a better idea of how Quick Action acts. Each skill level was tested 200 times.

 

QA 1 = 13 double strikes out of 200 attacks, or 6.5% (6.5%/skill level).

Standard Deviation – 3.49

 

QA 3 = 15% double strikes, or 5%/skill level.

Std. Dev. – 5.05

 

QA 5 = 20% double strikes, or 4%/skill level.

Std. Dev. – 5.66

 

QA 7 = 28% double strikes, or 4%/level.

Std. Dev. – 6.35

 

QA 9 = 35% double strikes, or 3.9%/level.

Std. Dev. – 6.75

 

QA 10 = 37% double strikes, or 3.7%/level.

Std. Dev. – 6.83

 

QA 15 = 39% double strikes, 2.8%/level.

Std. Dev. – 6.9

 

QA 20 = 46% double strikes, 2.3%/level.

Std. Dev. – 7.05

 

QA 26 = 54% double strikes, or 2%/level

Std. Dev. – 7.05

 

QA 30 = 57% double strikes, or 1.9%/level

Std. Dev. – 7

 

QA 36 = 65% double strikes, or 1.8%/level

Std. Dev. – 6.75

 

QA 40 = 70% double strikes, 1.75%/level.

Std. Dev. – 6.48

 

My results do not suggest that Quick Action's effectiveness seems to decrease after level 5. I also never saw a significant drop in the % occurrence like Brocktree did in his first QA 10 test, so I'm more inclined to think it’s just a reduction in the effectiveness of each new point rather than a total drop in effectiveness. Brocktree is correct in at least one respect – Quick Action does not appear to work according to the traditional explanation of the 10 cap. While levels 1-10 do give ~4-5% per level, after level 10 the percent of expected double strikes does not fit the assumptions of the 10-cap.

 

For example, the following describes how Defense increases dodging -

At levels 1-10, Defense adds 3% dodging every level.

At levels 11-20, dodging increases by 3% at levels 12, 14, 16, 18, and 20.

At levels 21-30, dodging increases by 3% at levels 23, 26, and 29.

At levels 31-40, dodging increases by 3% at levels 32, 36, and 40.

 

If Quick Action followed this and increased the chance of a double strike by 4% per level, then QA should give 40% chance of a double strike at level 10, 48% at levels 14-15, 60% at levels 20-22, 72% at levels 29-31, and 84% at level 40. Sadly, after level 10, the 4% per level and 10-cap explanation does not even fit within 2 Standard Deviations of my observed values.

 

Alternatively, I considered the possibility that QA gives 3% per level with a 10-cap. Although this does not describe my results perfectly, it is more accurate than 4%. 3% per level of QA does not fit at levels 3, 7, and 9, but does fit within 2 Std. Devs. of every other observed value.

 

Since the 10-cap may not explain the behavior of Quick Action, one possible explanation could be, as Brocktree said, that beginning at level 6 the effect of adding more QA is halved, so instead of adding 4-5%, each new level adds 2-2.5%. However, my data suggests that it is unlikely that levels 11-40 give further reductions, and instead add a flat 2-2.5% every 2, 3, or 4 levels. Even though an initial 5% from levels 1-5, and 2.5% from 6-40 fit my data better than the previous explanations, this seems unlikely to me. Reducing the effectiveness of QA at level 5, but not at any subsequent levels makes little sense.

 

As Slarty mentioned it’s possible, though unlikely, that QA uses a lookup table. Since I did not test every level of QA I won’t venture a guess for how such a table might work and whether it fits with my findings.

I suppose this -

Quote:

I think Explanation 2 is more likely. Perhaps each point does not give a flat % value to double strike, but a value randomly selected from between a range of values (eg. between 1-5%)

 

is possible, and it might explain the discrepancy between my observed values and those we expect to see with the 10-cap. Unfortunately I’m not too sure about the actual implementation of this. Would 10 Quick Action give a 0-50% chance to double strike? One turn you would have a 10% chance and the next you might have 40%? Or would it be calculated like weapon skill damage?

[Brocktree]

Oh wow, awesome results Thoukydides. I admire your dedication.

 

I graphed your results in Excel (Quick Action value on X axis and % chance to double strike on Y axis), and it's obvious that *something* changes when you have 10 points of quick action. You get a roughly linear association up until 10, a plateau between 10 and 15, and then a new linear association with a smaller gradient.

 

Funnily enough, I noticed this association for magical efficiency in Avernum 5, although I'll admit that my sample sizes ranged from 30 to 50.

[Aoslare]

Here's my graph:

 

 

Up to 10, 4% per point is DEFINITELY closer than 3% per point. I was able to correct for the change in rate after 10 best by switching gains down to 1% per point, and that lined up better if I began the 1% gains at 10 rather than at 11, which is certainly a believable circumstance. The resulting line (the red one) may not be perfect, but it seems to be much more reasonable in describing QA's effects than what we had previously.

[Brocktree]

A linear association of "y = 3.5x + 3.4" fits the data almost perfectly from Quick action values 1 to 9. This means that each QA point would add 3.5% chance to hit up until a quick action of 10. A linear association of "y = 1.2x + 21.3" fits very well for QA values 15 to 40, meaning 1.2% to hit per point.

 

There is a definite plateau between values 10 and 15 until the new association kicks in. Could it be possible that adding points in Quick Action between these values doesn't actually provide a benefit?

 

[Lilith]

Originally Posted By: Brocktree

A linear association of "y = 3.5x + 3.4" fits the data almost perfectly from Quick action values 1 to 9. This means that each QA point would add 3.5% chance to hit up until a quick action of 10. A linear association of "y = 1.2x + 21.3" fits very well for QA values 15 to 40, meaning 1.2% to hit per point.

There is a definite plateau between values 10 and 15 until the new association kicks in. Could it be possible that adding points in Quick Action between these values doesn't actually provide a benefit?

Be wary of overfitting the data: we don't really have all that much of it, which means that a simple model that fits the data slightly less well is probably more plausible than a complex model that fits it better.

[Aoslare]

And also don't forget the common sense test. Jeff was not likely to say, "hey, for Quick Action, I'm going to use a set of two linear equations with decimal coefficients."

[Micawber]

Not unless he was just trying to mess with us

 

Actually the best way to do that would be with the lookup table; if no pattern exists, imagine all the false leads when you try to find one.

[Zummi]

Would the Almighty care enough for us lowly peons and grunts to clear our baffled thoughts on how stuff works in his world?

[Earth]

Jeff? keep dreaming.

[Brocktree]

Originally Posted By: HOUSE of S

And also don't forget the common sense test. Jeff was not likely to say, "hey, for Quick Action, I'm going to use a set of two linear equations with decimal coefficients."

He's unlikely to use a decimal coefficient. And I never said he used two linear equations. I'm using those equations to pick any trends.

I would conclude that:

1. From QA 1-9, each points increases the chance to double strike by approx. 3-4%.

2. From 9+, each point increases the chance to hit by approx. 1-2%. This would fit with the 10-cap hypothesis, *except* that the cap only occurs once at the first 10 points.

3. There is some base %to double strike once you invest in QA (probably 3-4%). This is why the first point gives a whopping 6% to hit.

4. It's not wise to invest in over 9 points in quick action. 9 QA gives 35%, wheres 20 QA only gives 46% (an additional 11% chance to double strike). What a waste of skill points.

[Thoukydides]

I don't believe the first point actually gives an increased chance of double strike. Using my test, with an average of 6.5% and assuming a normal distribution, there is a 95% chance that 1 Quick Action gives between 3% and 10% double strike. In light of this I don't think we should conclude that the first point acts differently than the second. It's definitely possible that I saw a couple (5 for 4% per point) more double strikes than I "should" have and the actual average is lower than 6%.