Statistics dialog "Inflicted" and "Taken" unclear

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderators: Forum Moderators, Developers

Forum rules
Before posting a new idea, you must read the following:

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 11:03 am

Yes, of course I did, I check statistics often. The luck was positive pretty much every time. Also After making a few loads during the campaign (not a lot) in the later scenarios i had I think +2% dealt -2% taken in campaign statistics while in earlier parts when I wasn't loading saves yet the overal luck was pretty much perfectly equal.

I am sure some other people think similarly but maybe they havent checked statistics/just didnt want to voice their opinion.
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Tad_Carlucci » October 18th, 2017, 12:54 pm

Sounds to me more like the Gambler's Fallacy than anything else.
I forked real life and now I'm getting merge conflicts.
Tad_Carlucci
Developer
 
Posts: 203
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 2:37 pm

I was sure somebody will make a comment like this, that all this is just an illusion, but its fine.
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Tad_Carlucci » October 18th, 2017, 3:30 pm

Let's take a [supposedly] fair coin. Let's throw it 100 times. And let's repeat that test for 100 trials..

We think it's a fair coin, so we expect that, for each run of 100, we'll see about 50 heads and about 50 tails. And, out of 100 trials, we'll expect some to be exactly 50/50.

I've had (at other games) people come along and complain that, since all 100 trials were not exactly 50/50, there was something wrong with the RNG. My response: No, if they were ALL exactly 50/50, it would prove the coin was NOT fair and we would need a better RNG.

And I've had people come along and claim that, because one trial came up with a surprisingly large variation (say, 60/40 or 70/30), there was something wrong with the RNG. My response: No, if surprisingly large variations never occurred, it would prove the coin was NOT fair, and we would need a better RNG,

So, you say you've played a few campaigns without any save/reload and you've seen some were the inflicted/taken luck stats were dead-on, and you've played a few where there was some variation. My response: So far, so good. You're seeing what I'd expect. And I'd remind you to expect, eventually, maybe, to see some surprisingly large variations .. so large that it looks like you actually did save/reload. [btw, I **have** seen it.]
I forked real life and now I'm getting merge conflicts.
Tad_Carlucci
Developer
 
Posts: 203
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Inky » October 18th, 2017, 5:04 pm

Oh, I thought you meant you kept getting positive luck on the turn you save loaded but it appears you're talking about the overall campaign stats? In that case of course the overall luck would be positive most of the time, that's the whole point of save loading. Only the turns you accept get factored into the statistics, so if you threw out the most negative results by reloading the turns with very bad luck of course the overall stats are going to end up positive. That's not because the RNG is biased, but the way in which you selected which results to include in the statistics calculation was biased.
Commented playthroughs with screenshots:
Let's Play Dead Water, Let's Play Invasion from the Unknown and Let's Play After the Storm (in progress)
User avatar
Inky
Moderator
 
Posts: 451
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 5:20 pm

So what you said basically means that for you at least there is no "a lot of evidence" that can prove that the RNG is broken, you could apply the same argument to any possible luck rolls. So my point is impossible to prove, like I said earlier it is dismissing the problem but I expected that some may consider it impossible to prove and assume that RNG gods are always right no matter the amount of skewed rolls. Maybe lets finish at this.

@Inky
These werent the turns with very bad luck, when I load I never use the save and reload method, i just change my strategy. Obviously in the scenarios the luck also jumps, and usually on the turn immediately after loading you get the luck but, sometimes its not that simple, you can get the luck a few turns later, ultimately it almost always ends with having more luck after the loading than before it. If you get pretty balanced luck, you load, and then immediately your luck in the scenario jumps by a few percent, by just loading, does it looks like the RNG is working as intended? If you would say yes then that would be an argument for abandonning loading altogether.
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Tad_Carlucci » October 18th, 2017, 5:46 pm

There are a number of tests for randomness. The RNG used by Wesnoth is sufficient for the use it's put to. It is NOT sufficient for other uses, such as cryptography.

By your own admission, you reload. So, YOU are skewing the results, not the RNG.

You're thinking of save-and-reload as "I don't like this one combat, so I'll reload until I get what I want" but your use, "things are not going my way so I'll reload and try something different," can also skew the results. If, however, by "reload" you mean "start the campaign over" and you do that a few thousand times, and can then show a significant bias, you might get some interest.
I forked real life and now I'm getting merge conflicts.
Tad_Carlucci
Developer
 
Posts: 203
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 6:22 pm

I do not, I thought i made myself clear in the last post, why such assumption?

"If, however, by "reload" you mean "start the campaign over" and you do that a few thousand times, and can then show a significant bias, you might get some interest"

Are you serious? Ok, so I should not load under any circumstances, and the only acceptable form of "reloading" is starting the campaign over? Lets say the RNG does not work correctly when loading, so if it does not, then its all fine, no need to worry because it only matters when you start the campaign over and only if then it wasn't working correctly then that would mean something right? Also you said "do that a few thousand times, and can then show a significant bias, you might get some interest", obviously it is not feasible, so it is impossible to prove. So even ignoring that you implied that you think "starting the campaign over" is the only form of loading worth looking at and RNG rolls after classical loading mean nothing, you said that I would need to do that a few thousand times to get big enough sample size, which is just impossible, so yeah, seems that you do indeed think that proving that the RNG does not work right is impossible. But now, if you do indeed think that, then why do you keep up the discussion, if you consider it impossible to prove then there is nothing to talk about.
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Inky » October 18th, 2017, 6:40 pm

It's not the reloading itself that invalidates the statistics, it's the fact that you are choosing which turns to reload, which skews the results. Even if you reload only to change strategy and not to explicitly get better RNG, wouldn't you agree that it is more likely for you to reload a turn that had bad luck than good luck? (Because if you've been getting good luck things will be going well so why change strategy?)

If you are claiming that reloading causes positive luck in the turns after the reload, then in order to back up this claim you would need to look at the statistics on the later turns only, not the overall statistics which you've been using.
Commented playthroughs with screenshots:
Let's Play Dead Water, Let's Play Invasion from the Unknown and Let's Play After the Storm (in progress)
User avatar
Inky
Moderator
 
Posts: 451
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 8:18 pm

Likeliness plays no role here because i said that when i was reloading turns with normal luck it was the same, and even if i was reloading turns with positive luck, does that justify the fact that on every or almost every single load i was getting positive luck? The luck i got before loading does not matter AT ALL because it should not influence the luck I got after the load.
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Tad_Carlucci » October 18th, 2017, 8:43 pm

To reset the running total 'luck' you need to load the scenario-start. Loading a saved game won't do it.

It should not be surprising to see surprisingly long strings of good or bad luck ... All other things being equal (which will rarely occur). Assuming only fresh scenario played to completion.

The assertion that, after a series of 'good luck', you should expect 'luck' to return to some neutral value is the very definition of the Gambler's Fallacy.
I forked real life and now I'm getting merge conflicts.
Tad_Carlucci
Developer
 
Posts: 203
Joined: April 24th, 2016, 4:18 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby pauxlo » October 18th, 2017, 9:36 pm

Tad_Carlucci wrote:Let's take a [supposedly] fair coin. Let's throw it 100 times. And let's repeat that test for 100 trials..

We think it's a fair coin, so we expect that, for each run of 100, we'll see about 50 heads and about 50 tails. And, out of 100 trials, we'll expect some to be exactly 50/50.

I've had (at other games) people come along and complain that, since all 100 trials were not exactly 50/50, there was something wrong with the RNG. My response: No, if they were ALL exactly 50/50, it would prove the coin was NOT fair and we would need a better RNG.

Nitpick: Actually, an always-50/50-coin would be fair, it would just not be random (and likely not a coin at all, but some broken algorithm).
User avatar
pauxlo
 
Posts: 1011
Joined: September 19th, 2006, 8:54 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby ElderofZion » October 18th, 2017, 10:00 pm

"To reset the running total 'luck' " what do you mean?
@Elder but we have asked for the disclosure of all the cheese and we treat it seriously. Everyone on this site can get banned, everyone.
@Elder ale prosiliśmy o wyjawianie wszystkich serów i traktujemy to poważnie. Każdy na tej stronie może zostać zbanowany, każdy.

- Sagez
User avatar
ElderofZion
 
Posts: 214
Joined: July 11th, 2015, 2:13 pm

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Inky » October 18th, 2017, 10:36 pm

ElderofZion wrote:The luck i got before loading does not matter AT ALL because it should not influence the luck I got after the load.

Yes that's correct, but then you can't look at the overall campaign stats which also include stats from before and on the reloaded turn. You'd have to only look the stats from after the reload.
The overall displayed campaign statistics will only be unbiased if you restarted the whole campaign.
Commented playthroughs with screenshots:
Let's Play Dead Water, Let's Play Invasion from the Unknown and Let's Play After the Storm (in progress)
User avatar
Inky
Moderator
 
Posts: 451
Joined: September 22nd, 2014, 1:02 am
Location: USA

Re: Statistics dialog "Inflicted" and "Taken" unclear

Postby Pentarctagon » October 19th, 2017, 12:05 am

To clarify what I mean by "a lot of evidence":

You'd need to gather a lot of data - let's arbitrarily say a thousand data points - and the result would need to be large enough to be statistically significant. So far all we really have are a claim with no data along with the usual back and forth that tends to happen whenever luck is brought up. If someone was able to provide a set of 1000 games with no save-loading that averaged ~+0.5% damage dealt, and then another set of 1000 games that did have save-loading which averaged to ~+5% damage dealt(only counting anything that happened after save-loading, and not counting the attack the save-load was done to change the result of), then that would be something worth looking into in more depth.

Additionally, at least since 1.13.0, Wesnoth uses the Mersenne Twister RNG.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Pentarctagon
Forum Administrator
 
Posts: 2948
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

PreviousNext

Return to Ideas

Who is online

Users browsing this forum: Google [Bot] and 4 guests