Hit chance DOES counter evasion. Also deadeye runes seem better than listed.

[VenomPhoenix]

Because there was no consensus on whether hit chance interacts with evasion, I decided to test this by taking 4 level 13 characters into the Ukatish memory fight and fire bows at the celebrants. Two characters were equipped with 2x deadeye runes for a +10% hit chance, and the others simply had empty slots on weapons and helm. I then recorded the hits and misses made by each group until the fight was over.

 

Bonus group: Shot 47 arrows, hit 46 times. Hit rate of 97.9%

No bonus group: Shot 63 arrows, hit 50 times. Hit rate of 79.4%

 

I could have collected more data but I spent an hour prepping and recording this, and this seems more than adequate to prove that hit chance DOES negate evasion. For the sake of argument, I ran a chi-squared test and got a value of 8.3007 (p < 0.05), proving the difference.

 

What I find interesting is that the deadeye runes seem to give much more than a 5% hit chance. Although it looks to be around 10%, it may actually have hit the ceiling (a lot of games max hit chance at 95%), meaning it could be even higher.

 

The moral of the story is, ALWAYS have a deadeye rune in your weapon and helm.

Edited August 22, 2021 by VenomPhoenix
Change statistical test to a more appropriate one.

[Mechalibur]

I think the lack of consensus wasn't so much "do deadeye runes help with evasion" but rather "is evasion rolled separately from hit chance?" (like how it is in Mutagen). For example, if you have 85% percent natural hit, and the enemy has 10% evasion, is your chance to hit 75%, or a 76.5% chance (.85 x .9). In both cases, a Deadeye rune will help with your overall hit rate, but there is still a distinction. Did you check the evasion of the monsters your were hitting? That could help us determine which case is true.

[VenomPhoenix]

Afaik there is no way to implicitly find the evasion of a particular mob. For reference, they were the "Ukatish Celebrants" found in the misty maze.

 

The problem here is that even if there were separate rolls, there is no way to account for how a 10% increase in nominal hit chance changes the hit rate from 80 to 95%. In fact, if there were separate rolls, it would make the final hit rate lower than the stated hit rate increase. Let's use some examples with an approximately 50% final base hit rate.

 

Separate rolls:

Base: 70% hit chance, 30% evasion, final hit rate 49%.

Improved: 80% hit rate, 30% evasion, final hit rate 56%. Improvement of 7%.

 

One roll:

Base: 50% hit rate.

Improved: 60% hit rate. Improvement of 10%.

 

There is no way to account for why +10% gave more than +15% to the final hit rate. The only possible explanation, and it is a huge stretch and very presumptuous, is that there are separate rolls, and Jeff knew that this would lower the effectiveness of any static hit% increase, so he made deadeye runes give MORE than +5% to make up for this. I'm open to other options though.

 

Anyway, this would all be virtually impossible to test further without knowing the true values for mechanics behind the scenes (e.g., enemy evasion statistics). What makes this difficult is that every single miss results in an "evade" message, so I can't see any way to delineate between a true "miss" and an "evade". For pragmatic purposes (which is why I tested it), deadeye runes are fantastic and provide superb value considering they are so incredibly cheap.

Edited August 23, 2021 by VenomPhoenix
typo

[VenomPhoenix]

Also, if there were separate rolls, how could I even begin to approach a 95% hit rate? The enemy would have to have a lower than 5% evasion rate. If they truly were separate rolls, then an enemy with 30% evasion would evade 30% of the time minimum.

[Mechalibur]

You can right click an enemy to see their evasion. If you were against an enemy with 0 evasion, the numbers you ended up with would be inconclusive for proving single rolls. That's why I'm asking. 

[VenomPhoenix]

Ok I loaded up the game and checked. Ukatish celebrants have a physical evasion rating of 15%.

 

Thank you for the right click tip; I had no idea that worked.

 

Anyway, that kind of proves that it's a single roll, because you can't go up to 95% hit rate against an enemy with 15% evasion unless the hit rate directly counters evasion.

[Slawbug]

4 hours ago, VenomPhoenix said:

There is no way to account for why +10% gave more than +15% to the final hit rate.

 

I think you've identified the right question to be asking here.  But there is at least one very plausible solution: natural variation in data.  50/63 (79.4%) is literally just a few arrows away from a number that would line up better with your speculation.

 

Also, a question... why are the two groups of results different sizes?  You had the tests all intermixed, on fixed (different) characters, rather than separate with the same characters, so it's also possible there's some other factor about the combat that affected things.  Since the two groups didn't even end up with roughly the same number of shots, as we would expect if all other things are equal, this seems plausible.

[VenomPhoenix]

The groups were 2 members each. Unfortunately, since the Ukatish memory boss casts slow every now and then, it just so happened that the two members with the hit% bonus ended up with less turns due to being slowed more often. I saw this happening but thought the results would be robust anyway, since chi-squares and t-tests do not require equal group sizes (nor even equal variances for that matter, but that's an aside). I continued until I ended up dying, and at that point felt I had more than enough data to merit retrying for more.

 

But yes, it is an interesting question as to why +10% is giving more than +10%. All the weapons and equipment were the same, saving a few exceptions that shouldn't matter (e.g. healing vestments vs the blessing vestments from the weaver), and jewellery (also which shouldn't matter). I wish I had an answer for that.

[Slawbug]

What I was saying in the second bit is it could be an unexpected result of how combat played out.  Just as the 2 members with the hit% bonus just happened to be the ones hit with slow, there could be some other thing that just happened to impact their attacks more (or less) than the other 2 members.  This could be as a result of affecting them, or affecting the enemies.

 

Definitely appreciate that you actually did statistical tests.  Still, the fact that 10% produces an inflated impact does create a feeling that there's something we're missing here.  This might not have anything to do with your tests, specifically, and could be a third factor that nobody expected to be relevant.  Nonetheless, it definitely makes me a little uncomfortable when you keep saying the data "proves" this particular conclusion... it supports it at least partly, but the conclusion doesn't completely explain the results you got, so "prove" seems like a stretch.

[Mechalibur]

11 hours ago, VenomPhoenix said:

Ok I loaded up the game and checked. Ukatish celebrants have a physical evasion rating of 15%.

 

Thank you for the right click tip; I had no idea that worked.

 

Anyway, that kind of proves that it's a single roll, because you can't go up to 95% hit rate against an enemy with 15% evasion unless the hit rate directly counters evasion.

 

Yeah, if they have 15% evasion then it's definitely likely that the rolls aren't done separately. Thanks for double checking.

 

The 15% vs 10% difference in hit rate is likely just data variance like Slarty said. It looks like the statistical significance test you did was measuring whether or not deadeye runes had any effect, not whether the 5% higher-than-expected hit rate is statistically significant or not. And even then, remember than p < .05 means there's less than a 5% chance the data difference are due to random chance - not that there's 0 chance of the data being anomalous.

[VenomPhoenix]

6 hours ago, Punctuation rains from the heavens said:

What I was saying in the second bit is it could be an unexpected result of how combat played out.  Just as the 2 members with the hit% bonus just happened to be the ones hit with slow, there could be some other thing that just happened to impact their attacks more (or less) than the other 2 members.  This could be as a result of affecting them, or affecting the enemies.

 

Definitely appreciate that you actually did statistical tests.  Still, the fact that 10% produces an inflated impact does create a feeling that there's something we're missing here.  This might not have anything to do with your tests, specifically, and could be a third factor that nobody expected to be relevant.  Nonetheless, it definitely makes me a little uncomfortable when you keep saying the data "proves" this particular conclusion... it supports it at least partly, but the conclusion doesn't completely explain the results you got, so "prove" seems like a stretch.

Apologies, I never meant to say I "proved" anything about deadeye runes. Only that there is statistically significant support for hit chance countering evasion. When I talked about deadeye runes I was careful to use the word "seems", but looking back I can see how someone could infer I was trying to say I had proven deadeye runes give more than 5% each.

 

4 hours ago, Mechalibur said:

 

Yeah, if they have 15% evasion then it's definitely likely that the rolls aren't done separately. Thanks for double checking.

 

The 15% vs 10% difference in hit rate is likely just data variance like Slarty said. It looks like the statistical significance test you did was measuring whether or not deadeye runes had any effect, not whether the 5% higher-than-expected hit rate is statistically significant or not. And even then, remember than p < .05 means there's less than a 5% chance the data difference are due to random chance - not that there's 0 chance of the data being anomalous.

No problem. Yes it is entirely possible it is random chance I happened upon more than a +10% rate in my data. I am very aware of this. In fact, I explicitly decided not to test this for the sake of time. The sample sizes required to estimate accurately within a small percent is extraordinary.

 

The fact that there are two different groups and we are measuring the difference between the outcomes between them means I'd have to do a probability distribution function for the both of them assuming certain base rates, and comparing them from there becomes extraordinarily complicated and time consuming. An easier method is to estimate true probabilities and look at the overlap found from 95% confidence intervals. However, I think I can prove my point by looking at the sample size required for this.

 

For the non-bonus group, assuming a probability of 80%, we can derive the standard error (Se) below:

 

Se = Sqrt ( p(1-p)/n ) = Sqrt ( 0.8(0.2)/63 ) = Sqrt ( 0.16/63 ) = 0.050395

 

The z-score of a 95% confidence interval is 1.9599, therefore we can estimate the confidence intervals by multiplying the z-score by the standard error to obtain the Maximum error (Me).

 

Me = 0.050395 * 1.9599 = 0.09877

 

Therefore the confidence intervals are:

 

(p-Me) : (p+Me) = (0.70123 : 0.89877)

 

Quite a wide range. The "true" hit rate is 95% likely to lie somewhere between 70 and 90%. This could be reduced with more data though. But how much? To obtain the appropriate sample size to get a minimal Me, we need to rearrange some formulas. Since:

 

Me = Z*Se = Z * Sqrt (0.16/n), then:

 

Me/Z = 0.4/Sqrt (n), then:

 

1/n = (2.5*Me/Z)^2 = 6.25*Me^2 / Z^2, therefore:

 

n = Z^2 / 6.25*(Me)^2

 

Assuming we want to be accurate to within a 1% hit rate, and we want to be 95% certain, then we sub in:

 

n = (1.9599)^2 / 6.25*(0.01)^2 = 3.841208 / (6.25*0.0001) = 6145.933

 

Yes, that's a sample size of 6,146 needed in order to accurately determine the true hit rate within 1%, assuming it's around 80%. This excludes the sample required for the bonus group, assuming 95% hit rate (a additional sample size of 1,825; forgive me for omitting the formulas for this).

 

Hence why I stopped where I did, the data was more than sufficient to prove my original point, but testing deadeye runes with any degree of certainty would take an extraordinary amount of time. Hence why I decided not to do it.