Statistics dialog "Inflicted" and "Taken" unclear

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderators: Forum Moderators, Developers

Forum rules
Before posting a new idea, you must read the following:
Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Yes, of course I did, I check statistics often. The luck was positive pretty much every time. Also After making a few loads during the campaign (not a lot) in the later scenarios i had I think +2% dealt -2% taken in campaign statistics while in earlier parts when I wasn't loading saves yet the overal luck was pretty much perfectly equal.

I am sure some other people think similarly but maybe they havent checked statistics/just didnt want to voice their opinion.

Developer
Posts: 339
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Sounds to me more like the Gambler's Fallacy than anything else.
I forked real life and now I'm getting merge conflicts.

Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

I was sure somebody will make a comment like this, that all this is just an illusion, but its fine.

Developer
Posts: 339
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Let's take a [supposedly] fair coin. Let's throw it 100 times. And let's repeat that test for 100 trials..

We think it's a fair coin, so we expect that, for each run of 100, we'll see about 50 heads and about 50 tails. And, out of 100 trials, we'll expect some to be exactly 50/50.

I've had (at other games) people come along and complain that, since all 100 trials were not exactly 50/50, there was something wrong with the RNG. My response: No, if they were ALL exactly 50/50, it would prove the coin was NOT fair and we would need a better RNG.

And I've had people come along and claim that, because one trial came up with a surprisingly large variation (say, 60/40 or 70/30), there was something wrong with the RNG. My response: No, if surprisingly large variations never occurred, it would prove the coin was NOT fair, and we would need a better RNG,

So, you say you've played a few campaigns without any save/reload and you've seen some were the inflicted/taken luck stats were dead-on, and you've played a few where there was some variation. My response: So far, so good. You're seeing what I'd expect. And I'd remind you to expect, eventually, maybe, to see some surprisingly large variations .. so large that it looks like you actually did save/reload. [btw, I **have** seen it.]
I forked real life and now I'm getting merge conflicts.

Inky
Moderator
Posts: 516
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

Oh, I thought you meant you kept getting positive luck on the turn you save loaded but it appears you're talking about the overall campaign stats? In that case of course the overall luck would be positive most of the time, that's the whole point of save loading. Only the turns you accept get factored into the statistics, so if you threw out the most negative results by reloading the turns with very bad luck of course the overall stats are going to end up positive. That's not because the RNG is biased, but the way in which you selected which results to include in the statistics calculation was biased.

Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

So what you said basically means that for you at least there is no "a lot of evidence" that can prove that the RNG is broken, you could apply the same argument to any possible luck rolls. So my point is impossible to prove, like I said earlier it is dismissing the problem but I expected that some may consider it impossible to prove and assume that RNG gods are always right no matter the amount of skewed rolls. Maybe lets finish at this.

@Inky

Developer
Posts: 339
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

There are a number of tests for randomness. The RNG used by Wesnoth is sufficient for the use it's put to. It is NOT sufficient for other uses, such as cryptography.

You're thinking of save-and-reload as "I don't like this one combat, so I'll reload until I get what I want" but your use, "things are not going my way so I'll reload and try something different," can also skew the results. If, however, by "reload" you mean "start the campaign over" and you do that a few thousand times, and can then show a significant bias, you might get some interest.
I forked real life and now I'm getting merge conflicts.

Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

I do not, I thought i made myself clear in the last post, why such assumption?

"If, however, by "reload" you mean "start the campaign over" and you do that a few thousand times, and can then show a significant bias, you might get some interest"

Are you serious? Ok, so I should not load under any circumstances, and the only acceptable form of "reloading" is starting the campaign over? Lets say the RNG does not work correctly when loading, so if it does not, then its all fine, no need to worry because it only matters when you start the campaign over and only if then it wasn't working correctly then that would mean something right? Also you said "do that a few thousand times, and can then show a significant bias, you might get some interest", obviously it is not feasible, so it is impossible to prove. So even ignoring that you implied that you think "starting the campaign over" is the only form of loading worth looking at and RNG rolls after classical loading mean nothing, you said that I would need to do that a few thousand times to get big enough sample size, which is just impossible, so yeah, seems that you do indeed think that proving that the RNG does not work right is impossible. But now, if you do indeed think that, then why do you keep up the discussion, if you consider it impossible to prove then there is nothing to talk about.

Inky
Moderator
Posts: 516
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

It's not the reloading itself that invalidates the statistics, it's the fact that you are choosing which turns to reload, which skews the results. Even if you reload only to change strategy and not to explicitly get better RNG, wouldn't you agree that it is more likely for you to reload a turn that had bad luck than good luck? (Because if you've been getting good luck things will be going well so why change strategy?)

If you are claiming that reloading causes positive luck in the turns after the reload, then in order to back up this claim you would need to look at the statistics on the later turns only, not the overall statistics which you've been using.

Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Likeliness plays no role here because i said that when i was reloading turns with normal luck it was the same, and even if i was reloading turns with positive luck, does that justify the fact that on every or almost every single load i was getting positive luck? The luck i got before loading does not matter AT ALL because it should not influence the luck I got after the load.

Developer
Posts: 339
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

To reset the running total 'luck' you need to load the scenario-start. Loading a saved game won't do it.

It should not be surprising to see surprisingly long strings of good or bad luck ... All other things being equal (which will rarely occur). Assuming only fresh scenario played to completion.

The assertion that, after a series of 'good luck', you should expect 'luck' to return to some neutral value is the very definition of the Gambler's Fallacy.
I forked real life and now I'm getting merge conflicts.

pauxlo
Posts: 1038
Joined: September 19th, 2006, 8:54 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Tad_Carlucci wrote:Let's take a [supposedly] fair coin. Let's throw it 100 times. And let's repeat that test for 100 trials..

We think it's a fair coin, so we expect that, for each run of 100, we'll see about 50 heads and about 50 tails. And, out of 100 trials, we'll expect some to be exactly 50/50.

I've had (at other games) people come along and complain that, since all 100 trials were not exactly 50/50, there was something wrong with the RNG. My response: No, if they were ALL exactly 50/50, it would prove the coin was NOT fair and we would need a better RNG.
Nitpick: Actually, an always-50/50-coin would be fair, it would just not be random (and likely not a coin at all, but some broken algorithm).

Elder2
Posts: 361
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

"To reset the running total 'luck' " what do you mean?

Inky
Moderator
Posts: 516
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

ElderofZion wrote:The luck i got before loading does not matter AT ALL because it should not influence the luck I got after the load.
Yes that's correct, but then you can't look at the overall campaign stats which also include stats from before and on the reloaded turn. You'd have to only look the stats from after the reload.
The overall displayed campaign statistics will only be unbiased if you restarted the whole campaign.

Pentarctagon
Posts: 3659
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Statistics dialog "Inflicted" and "Taken" unclear

To clarify what I mean by "a lot of evidence":

You'd need to gather a lot of data - let's arbitrarily say a thousand data points - and the result would need to be large enough to be statistically significant. So far all we really have are a claim with no data along with the usual back and forth that tends to happen whenever luck is brought up. If someone was able to provide a set of 1000 games with no save-loading that averaged ~+0.5% damage dealt, and then another set of 1000 games that did have save-loading which averaged to ~+5% damage dealt(only counting anything that happened after save-loading, and not counting the attack the save-load was done to change the result of), then that would be something worth looking into in more depth.

Additionally, at least since 1.13.0, Wesnoth uses the Mersenne Twister RNG.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code