How probable is extreme (un)luck? A mathematical study

[dwarftough]

After a game you often see chatlines like "I was soo unlucky, -20% inflicted" or "he was too lucky, +10% over the whole game" and so on, and so on. Luck in Wesnoth is a never-ending source of emotional comments. And at some point, the question arose in my head: How much luck/unluck is really too much?

In this study I'll consider the most obvious metric of luck in Wesnoth (with Standard RNG): percentages of inflicted/taken strikes in the "Statistics" menu (highlighted by red).

The statistics we discuss highlighted in red

hits.png (41.65 KiB) Viewed 1075 times

I will use a simple model to get reasonable estimates on the probabilities and luck ranges: each real game is a lot more complex but these numbers should at least give you some idea of how it can go.

This forum post is a summary of my results. The full article is presented in an attached pdf and will consist of several parts: basic model and results, discussion and some extra info, and detailed calculations that give the results.

Main Result: Probable luck ranges for units on 50% defense
That would be our simplified model: suppose we have only units fighting on 50% defense (for example, Orcish Grunts on hills). We want to estimate which percents in the stats window (for strikes) we'll see with the probability of, let say, 95%. We also suppose, for simplicity of calculation, that the number of strikes n is enough large. Here is the result:

Main result formula

200 and 95% are easy to remember numbers, but there isn't anything magical about 95%: a similar formula can be derived for other probabilities. For 98% the formula's numerator is 233, for 99% it's 257. Below is the table with some precalculated results for different probabilities and numbers of strikes n.

Table of precalculated probable ranges

Снимок экрана от 2025-09-27 04-48-43.png (16.4 KiB) Viewed 1075 times

Some conclusions

What are some brief conclusions we can derive from these results and what do I have to say about this?

1. The possible percentage range gets narrower when n rises, but at a slower pace

As you can see from the table, ±20% strikes luck range is achieved (with 95% probability) only at n = 100 strikes (which is a long enough game). And ±10% is reached only at n = 400 strikes (that's an excruciatingly long game)

Don't forget that it's the range that occurs with 95% probability, here goes the second conclusion (let's take n = 100 for example)

2. For n = 100 roughly 1 of 20 games will face more than 20% stats, roughly 1 of 50 games will face more than 23.3%, roughly 1 of 100 games will face more than 25.7%

As small as it may seem, 1% isn't that small probability and such games will appear once in a while. And this 25.7% range is calculated for long enough games (n = 100), if the game is shorter, the range may be bigger than that.

[the_kaygan]

[Vilebeggar]

Calculations aside i want to talk how bad rolls affect a competitive game. If you are generally unlucky in a game that will negatively affect your chances of winning since your units wont be able to hit/dodge. But what is worse in my opinion in unluck in KEY moments, like failing to kill a unit on an important chokehold or an important unit of yours dying despite being on favourable terrain.

It is known that a match between a superior player and an inferior one will most likely result in the superior players victory. However due to the rng and faction matchup factor anyone can beat any player at least once in a wesnoth game. Sometime extreme unluck is enough to settle a game. However a game between 2 even players in my opinion will most likely be decided by the rng or the faction matchup which has been a longstanding issue that frankly cannot be fixed since this game has 6 of them and multiple different maps.

Though some matchups like for example a loyalist vs dwarf or undead vs drake have always been problematic and on certain maps have been borderline unplayable despite the rng. Many attempts have been made by some parties (namely the ladder era guys) to fix the balance issues between factions for a spam of over 20 years that somehow are still considered a work "in progress". New patches of updates are still released frequently that only further mish mash the unit pool and widen the inbalance. To me they are only trying to hide the fact that they have failed.

So yes i would say wesnoth is an imperfect game where luck or the matchup type will get the best of you sometimes but you must play it with a grain of salt.

[newbieA]

Hmm maybe in even games yeah, critical rng will decide games..
How about make timing pushes and respect ToD. Surely the -25% whatever would boost survivability a lot... And +25% will increase chance to kill by a lot...

[Tonepoet]

Luck isn't all math, mind you. Some of it is timing. Good luck on a crucial turm is more important than bad luck throughout an entire game. Grig the Goblin basically turned around a game of Clash for me by successfully dodging Mage blows when I was just about to lose if I recall correctly. It was a glorious comeback, as he lead Goblins down the other side of the battlefield.

[dwarftough]

Luck isn't all math, mind you. Some of it is timing. Good luck on a crucial turm is more important than bad luck throughout an entire game. Grig the Goblin basically turned around a game of Clash for me by successfully dodging Mage blows when I was just about to lose if I recall correctly. It was a glorious comeback, as he lead Goblins down the other side of the battlefield.

That's true, luck in critical moments is more important than the overall luck. Note, however, that you may use this same method to look at a critical moment itself, and as the critical moment is by definition short, the range of possible luck or unluck in it is wide.

[dwarftough]

Hmm maybe in even games yeah, critical rng will decide games..
How about make timing pushes and respect ToD. Surely the -25% whatever would boost survivability a lot... And +25% will increase chance to kill by a lot...

There are a lot of things that decide games. Strictly speaking, the Wesnoth game isn't a row of indepedent trials because the results of previous fights open or close possibilities for future fights (in two ways, a player may decide to do/not to do something base on the outcome, and the outcome may eliminate any chance for something entirely, let say you either kill a guard and may try a leaderkill after that, or you don't and the leaderkill is impossible)

I'm still thinking about what models could be used to evaluate more things about the game, this is really the first step. And I don't think it's justified to declare Wesnoth the game of chance right from it, but the first attempt to evaluate actual numbers, what's lucky and what's unlucky, showed that the ranges of possible luck are wider than one might think

[mctom]

I think Wesnoth is a perfect platform to demonstrate the butterfly effect - pit two AIs against each other, save the game once they encounter each other in the battle, then reload it twice and observe the effects of different randomness seeds. Even though AIs are completely deterministic (I assume), I suspect the games will diverge completely after a few more turns.

An approach known from quantum physics could be used - most popularly known as Schrödinger's cat which is partially alive until the box is open. With this approach unit's HP would not be numbers but functions: probability density of their HP. Each unit would start with a 100% probability of having maximum HP, which over time would be modified by battling other units, with their own HP probability curves. If the model is well constructed, and the "duel function" precise enough, this framework would allow predicting the outcomes of the series of duels between many units, and in a greater perspective, the entire games.

Now to make it actually useful, in such model no units would ever die - even if the entire HP distribution goes below 0. However the model would assign no effect to these units, or perhaps even boost the HP of the other combatant, since a lack of unit to fight against is an advantage (no loss of movement for example).
Such model would benefit from being linear, thus the entire algebraic toolbox could be used to estimate more complex scenarios.

Chances may be modeled, but nobody doubts that skill is a major factor in winning a game to begin with. That's why math was never proven useful in balancing factions.
Managing randomness and uncertainty is a part of the strategy - a well known fact for those who have spent enough time in real life.

[PreposterousChicken]

So what you could theoretically do, I am putting it out there despite there are practical flaws unfortunately in the concept, is try and measure variance in impact of specific combat's result on game result. And compare that with correlation between basic "luck" as measure of hits/damages/kills and game result.

If the variance is small and correlation is high, that would suggest that tested game* is decided more by army size and luck than strategic choices. On the other side of the spectrum there'd be high variance in importance of instances of combat, basically we could detect (theoretically) key combats existence, and correlation between luck and result should be weakened, especially if we cut off luck-outliers.

Now, I marked word game with asterisk, because what we really could study is a combination of game and player(s). Which is very important if the players are automatons, because they are prone to rely more on large scale computation and less on conceptual strategy. This can skew results toward "shallow game".

Another problem is how do we actually study that. For each combat to check it's statistical impact on game result we'd need to play the game out from that point numerous times. Exploring the resultant tree of possibilities is madness, so we'd need to sample combats randomly and that introduces many methodological questions, mostly: how to ensure sample is representative of anything true about the game?