Split from [is AI cheating?]

[Tad_Carlucci]

If you think the numbers being displayed are not correct, you could look at the code, it is all open source.

The most significant issue in this regard in wesnoth 1.12 is that we use a much older random number generator, the C "rand" function.
[...]
Whether it makes a noticeable difference for the user, I can't say.

1) If you think there's an error in the math, point it out and it will be corrected.

Saying "I tried 20 times and it always ..." won't do it. For example, if we assume the odds are 1-in-4 and there are 5,000 players today playing a total 100,000 games, (skipping a pile of mathematics) expect that at least one player (probably several dozen ... or even a few hundred ... I did skip the math) would be able to make that statement on a daily basis.

2) The discussion on the random number generators really only matters if we're talking about computer security such as encryption.

From the point of view of game design, the issue was that the output from the old "rand()" function is predictable. While I'm sure I could use the RAW results from a few calls to the old random number generator to PREDICT all future results, I would be amazed it anyone could take a few (or even a few hundred) 'cooked' samples from combat and do the same. Yes, it's possible; no, you probably won't live long enough to do it with certainty.

3) I would be amazed if a normal user could show any meaningful difference between the combat results obtained from the two generators.

4) While the comments about real and random number epison values are true, they are generally immaterial. Most game odds are rather large numbers (1-in-2, 1-in-6, 1-in-20, things like that). While the NET odds from a series of random events (such as combat) might be very small and might even approach epsilon, that is simply a theoretical value and has no effect.

[Inky]

On the subject of the "force chance to hit" macro, I never liked how the displayed percentages were wrong. I wish it would just display 0% for units which are impossible to hit, otherwise you only find out after you wasted several units' moves (and probably got lots of retaliation damage in the process). (I know it's usually clear that you're not supposed to attack these units, but part of the fun of wesnoth is doing things you aren't supposed to )

On the original topic, a good way to get a feel for your luck is to constantly check the statistics table (s hotkey). You're probably getting better luck than you think

[The_Gnat]

In response to Tad_Carlucci i ran a randomizer 300 times between 1 and 100, here is the result:

I have highlighted in red the times when the randomizer could be claimed to be cheating, overall it appears to be fairly reliable.

[GunChleoc]

I think people have been complaining about the RNG since Wesnoth was first published...

The thing is, you are less likely to notice your own luck than the luck of the AI. Also, the AI uses the same RNG as the player, so if the AI cheats, you cheat.

[MagnetoRN]

Well, considering that each combat would use 6-8 results in average (3 or 4 attacks of each opponent), I would say that pretty regularly the RNG delivers a quite huge almost perfect sequence of numbers above 50% (and sometimes even 60% and 70%). I really don't know how random is that, since I didn't find such long strings with values below 50%. These ones I highlighted can pretty much frustrate the player in combats where he should have a clear advantage, creating many opportunities for a character with 70% (like Delfador) to hit only once.

[Whiskeyjack]

I think people have been complaining about the RNG since Wesnoth was first published...

The thing is, you are less likely to notice your own luck than the luck of the AI. Also, the AI uses the same RNG as the player, so if the AI cheats, you cheat.

As someone that has played a lot of Wesnoth over the last 10 years I can say this, even without looking at the maths: This statement is true. If you pay self-critical attention you will realize that the odd one out results occur for the player on the same rate as for the AI.

However, I think there might actually be a problem with the probabilities of edge cases. If you calculate the probabilities of some events and then the probability of multiple such events happening in the matter of 10 actions, you sometimes get probabilities that are not like 1/10.000 but rather close to impossible. And this is something that occurs every so often. I´m not sure if the not so absolute randomness results in there being a string of numbers that lead to this phenomenon (as in certain conditions you will get said string of very improbable events), it might also just be me and my brother being the odd ones out among all the Wesnoth players in the world.
I guess I´d have to prove the math to you to convince anyone and since this is my subjective assessment, I also might just be wrong. It´s a mood point anyway, except for the occasional frustration every couple hundred games it´s not so much of a problem.

[skeptical_troll]

This qualitative and subjective approaches look really pointless to me. There are rigorous ways of testing randomness, looking at a string and searching for odd things certainly isn't one of them, especially if you haven't defined what is 'odd' in advance.

The probability of having exactly that string of 300 numbers was 1 in 100^300. That's ridiculously low, but it happened! I guess you can see the fault in this way of reasoning.
There are loads of 'improbable' events, each of which has a very low chance of happening. But since there are many, finding some is not so unlikely.

[Tad_Carlucci]

This qualitative and subjective approaches look really pointless to me. There are rigorous ways of testing randomness, looking at a string and searching for odd things certainly isn't one of them, especially if you haven't defined what is 'odd' in advance.

The probability of having exactly that string of 300 numbers was 1 in 100^300. That's ridiculously low, but it happened! I guess you can see the fault in this way of reasoning.
There are loads of 'improbable' events, each of which has a very low chance of happening. But since there are many, finding some is not so unlikely.

"Really pointless" vastly under-states things.

In actual point of fact, one of those "rigorous checks" is to specifically check for those "odd strings" and ensure they occur. Failure to find them is proof of failure in the random number generator.

I see conversations like this many, many times. I find it amusing that the people who begin by claiming they have found an issue almost always present proof that things are working properly as proof of their claims something is wrong.

[Whiskeyjack]

Spoiler:

This qualitative and subjective approaches look really pointless to me. There are rigorous ways of testing randomness, looking at a string and searching for odd things certainly isn't one of them, especially if you haven't defined what is 'odd' in advance.

The probability of having exactly that string of 300 numbers was 1 in 100^300. That's ridiculously low, but it happened! I guess you can see the fault in this way of reasoning.
There are loads of 'improbable' events, each of which has a very low chance of happening. But since there are many, finding some is not so unlikely.

While I do get your point of reasoning, I´m not fully convinced it applies here. Add all the almost impossible edge cases up and you will still have a terribly low probability (say, for you hitting all 8 strikes and the enemy missing all 8, you add the opposite probability of you missing all and him hitting every time. If you feel generous, also add all cases of at least one of them still hitting/missing at least one strike (as "edge cases" (which are not identical to your "'improbable' events") is a relative term). You can´t tell me those cases together have a high probability. Then take an event like that happening multiple times in a very short time. Then take all that happening multiple times over the course of a couple weeks, without playing so excessively that the amount of fighting would balance this out). I´m not combating the occurrence of such events, I´m saying that, from player experience, I can imagine there being a problem with the edge cases (the distribution not being perfect). I didn´t even want to say it is wrong, just that my experience says it might.

I am aware of the fact that I might be exaggerating in the example above (different kind of memory bias) and that this is my subjective experience. And as I´m not interested in checking the math at the moment, I´ll trustfully take iceiceice´s and Tad_Calrucci´s word for it that I´m wrong.

The misguided (and unnecessary, as I realized some time after posting) point I wanted to make was too much of a psychological nature in a tech discussion and I humbly apologize for chiming in at all and my choice of words, which, on reread, implies things I didn´t intend to say.

[iceiceice]

+1 skeptical_troll

These pictures of numbers, with various colored boxes, supposed to be "proving" that the RNG is biased... it reminds me of the "crazy walls" scene from the movie "A Beautiful Mind".

Here's a way that you could attempt to give evidence that the RNG is biased.

1. Think of a random variable that shows the supposed bias. Like, consecutive runs of large numbers.
2. Calculate what the expected value is. I.e. how many runs of a certain length there would be, for sequences of a certain length, if the dice were totally fair.
3. Calculate what the chance of a "large deviation" is. I.e. "I calculate that the chance to get twice as many bad runs as expected is less than .01%"
4. Make a hypothesis, the "null hypothesis", that the large deviation won't occur.
5. Then, take actual data, and see if the event actually happened or not. If it did happen, despite low probability, then it's considered "statistically significant", and suggests the fair dice assumption isn't correct.

The important thing is, you don't get to change the null hypothesis after you see the data.

There's a saying, "if you torture the data, it will confess".

If you just collect random data and then look for patterns, you will not fail to find patterns. The human brain is very good at that. They just won't have any meaning.

[The_Gnat]

I fully agree with iceiceice and the reason I ran the randomizer was infact not an attempt to claim unrandomness. I personally believe that out of those 300 times (run from the 1.12 generator) it was reliably random. I think this because only 3 times (as highlighted in red) were there strings of similar numbers. Even though magneto has a valid point with the yellow boxes it is not very significant because like iceiceice says if you exam the numbers close enough you can alwaya find something.