Statistical analysis of the RNG

[torangan]

I was thinking more of analysing the numbers produced by the rng rather then how much damage is done in game play.

I told you right at the start that it's the default random number generator provided by C. While it's obviously pseudorandom, you can very well expect it to provide reasonable results for this use case. It's part of an international standard for about 20 years by now...

[mrchadt]

Just because a finite set of numbers are random doesn't mean they are uniformally distributed or average. I am talking about the numbers for any specific game.
I had left the post as the people who would likely make use of my suggestion showed little or no interest, I returned to it as others had posted interest.
I did take notice of your original post and tried to explain myself better, I hope my idea is clearer now.

[torangan]

The random generator provides numbers close enough to random for usual purposes. So the question would be: "does Wesnoth use them in a bad way which makes the distribution worse?" or "what does the EV statistic dialog really tell me?".
BTW a finite set of random numbers which is uniformly distributed or average is a rare special case of sets of random numbers. Your usual set of random numbers will be biased towards some spikes.

[mrchadt]

I think you are missunderstanding my motive for proposing this idea. It is not because I think that the RNG is in anyway bad or biased. I am very happy with the wesnoth RNG and the game in general. I am not a complainer of bad luck nor do I think wesnoth is a game of luck like some have proposed. There is however an element of luck which I think can sometimes make the difference between winning and losing, but only between players with similar skill levels (a good player will never lose to the ai with equal settings on the standard maps, imo). I think that measuring the luck in the way I suggested would be better then the one currently employed. Although the current way is better then what I am willing to do, I just have the idea and as previously stated am unwilling/unable to program it.

[commanderkeen]

Not everybody understands Standard Deviation. I think the current way of seeing your luck is simpler and more user-friendly.

[mrchadt]

Sorry if I'm dragging this point out, please ignore this thread if I'm boring you.
My reason for having standard deviation in is that sometimes people say they have been really unlucky, or vice versa, it would be interesting to know how unlucky one has been, which is what SD would measure. I don't think this concept is much more complex then EV if at all. Maybe a small histogram representing all posibilities of damage done with the players actual damage done highlighted. To this lines showing EV and 1,2,3 SD's from the mean on each side.

ps, I haven't played much of late but think this would be a good addition to the game as so much talk of luck goes on and a good way of measuring it is needed if these discussions are to have any meaning. I really don't mind if it is implemented or not but it would be nice if others liked my idea.
Thanks for giving it consideration.

[zookeeper]

If there was some graph showing how lucky one has been, it could be a horizontal graph showing the amount of luck on every round, so you could see on which turns you got lucky and so on (instead of just seeing the average of the whole scenario).

[Clonkinator]

You already fought in the first turn? Must be a pretty small map...

[torangan]

Personally I think the best would be to take out any statistics showing your luck. They'll most likely only be used to feed complaints about bad luck. Really understanding the effects of random values on this scale isn't something you can do without some mathematical training.

[mrchadt]

If only discussions on luck could be solved so easily. Worth a try though, remove all statisical data viewing and see if the luck complaints and discussions end.

[taro]

Personally I think the best would be to take out any statistics showing your luck. They'll most likely only be used to feed complaints about bad luck.

It would be sad if a useful tool would be removed from the game only because a few people use it to justify their complaints.
The real versus expected damage statistics is a helpful instrument to analyze your own matches. A tactic, though successful, may have been wrong if it was successful only because of good luck. And vice versa ...

[torangan]

The current stats are most likely not even accurate and still - it's not really usefull even with correct values. Every valid strategy has to consider luck so the result of one or a few games don't tell you much. If you play with a strategy which considers luck and adapts then luck will simply not effect you much anymore. It may decide the game in a few cases but usually not.

[Wintermute]

Not everybody understands Standard Deviation. I think the current way of seeing your luck is simpler and more user-friendly.

Many of the developers do, I would imagine, and the game is by them, for them. Personally, I think it is an interesting idea. Many people don't understand/use the current damage statistics, but I imagine that any changes will be in the form of adding functionality rather than taking things away.

[Yogin]

Actually, I can guarantee the current EV #'s are incorrect. The only way to correctly do EV on a battle by battle basis is to calculate the EV based on attack_prediction.cpp output distribution. This takes into account strikes, probability of strikes, probability of death, probability of slow, drain, etc. I'm almost certain the current EV engine doesn't use this class.

If the EV calculator is going to be rewritten to use this class, a simple variance calculation can be done with it as well. For instance to calculate EV of damage expected for attacker, take the defender's hp distribution P(hp==x), and calculate:

Code: Select all

EV(hp) = Sum(P(hp==x)*x)    and
Var(hp) = Sum(P(hp==x)*(x-EV(hp))^2).  
Both sum's are across all x's in the distribution.

Then,  Damage stats are simply the reverse of the hp stats:
EV(dmg dealt by attacker) = orig. HP of defender-EV(hp)      and
Var(dmg dealt by attacker) = Var(hp)

Across multiple combats,
EVTotal = Sum(EV) for all combats 
VarTotal = Sum(Var) for all combats   {Note 1}

StdDevTotal = Sqrt(VarTotal)
-------------------------------
{Note 1}: Essentially this is what rrenaud pointed to in his second post.  

Also, Note that eventhough many people don't understand standard deviation, they _do_ understand +/-. So EV can be expressed in terms of
37.5 +/- 9.6, where 9.6 is the standard deviation.

Furthermore, if we assume the sum of the distributions are normal, then we can figure out how unlucky a person is by comparison to the normal. I'm not too comfortable with this assumption, though given that the Central Limit Theorem applies only to iid r.v.'s.


=================

P.S. OMG. I overlooked the forest for the trees. If we have the actual distribution of expected damage dealt and received for each combat, we can just convolve them and construct the exact, true distribution for damage dealt and received for all battles seen so far. With the true distribution, we can give the exact likelihood of unluckiness or luckiness.


=================

P.P.S. I wanted to note that this is simple a measure of luck for each battle. Obviously, if you're unlucky and lose a unit you shouldn't have, then that unit will be unavailable to press an advantage and to take part in future battles. But I believe others should have discussed this in other threads already, which is why an indicator of luck by turn is useful, as well as luck in combats which result in kills. Both are easily implementable with the above system.[/u]

[rrenaud]

Furthermore, if we assume the sum of the distributions are normal, then we can figure out how unlucky a person is by comparison to the normal. I'm not too comfortable with this assumption, though given that the Central Limit Theorem applies only to iid r.v.'s.

By the initial text at http://mathworld.wolfram.com/CentralLimitTheorem.html I believe that the samples do not need to be drawn from a single distribution to for the result to approach the normal distribution.

Certainly you must agree that by the "fuzzy" CLT (the sum of a bunch of independent events with small magnitudes) that the results will be normally distributed for wesnoth attacks. After you have attacked say, 100 times, the effect of no single attack will be that substantial.