Playing skill levels in strategy games

[Spirit_of_Currents]

Let me explain that concept in the title: if two players play a strategy game against each other really many times, no one has a handicap, both play as well as they can and either one nearly always wins, then those players have a different playing skill level in that game.

Let's think about how many such levels there are in different strategy games. I would particularly like to know whether a nearly average Wesnoth player can have a playing skill level different to that of an average Wesnoth player, and how many levels there are in TBfW. I also ask a question: do you believe good strategy games have more possible that kind of levels than bad ones? My personal answer is now closer to yes than no.

I presume that it's best to have the same topic for both Wesnoth's and other games' playing skill levels.

[tekelili]

I also ask a question: do you believe good strategy games have more possible that kind of levels than bad ones? My personal answer is now closer to yes than no.

I am convinced that is exactly the meassure of a strategy game quality. You just have to examine the lowest possible range to confirm it; In tic-tac-toe there are only 2 possible skills levels: perfect and dumb.

[nuorc]

There's Elo, also in Wesnoth.

[GbDorn]

In the ELO rating system a 400 rating difference between players A and B means A has an expected score of 0.909 (that is the probability of winning and half the probability of drawing). I'll take this value for your "either one nearly always wins" requirement.

For example the FIDE ratings for chess start at 1000 and the current max is 2865 according to Wikipedia. Thus there are 5 "skill levels" among FIDE rated players with your definitions.
Edit: The Wesnoth MP ladder starts at 1300 and the current max is 2236. That's 3 "skill levels".
Now if you use a lower value than 400, you'll get more classes.

[iceiceice]

It's a nice general idea about strategy games, here's what I think about it:

• "Let me explain that concept in the title: if two players play a strategy game against each other really many times, no one has a handicap, both play as well as they can and either one nearly always wins, then those players have a different playing skill level in that game."
  
  Actually if I read it literally, what I would think should happen is that with enough repetitions both players should become of equal skill level, and neither one should win more on average. Unless one of them is mentally handicapped, I think that with sufficient repetitions, one of them and then the other will determine the optimal strategy for the game. (If it's a finite game anyways. If it's an infinite game then I'm not so sure.) It might take longer than their lives though, so maybe we should assume they are spirits for this argument :p
• If you look at the assumptions of ELO ranking systems (like used in wesnoth ladder) then its quite close to what you are saying. They think each player has a secret number which is their average quality of play. When two players play, their performance is some random variable distributed about their secret number, but it might be higher or lower in that game because of dumb mistakes. Whichever has a higher performance number is thought to be the winner.
  
  In this model you have `classes` in the sense like shown on wesnoth ladder: http://wesnoth.gamingladder.info/gadder.php
  
  I can't remember all details anymore -- for each player it is assumed that they have about the same "variance" from game to game, only their expected quality of play is different, and is supposed to be reflected in ELO score. If two players have same ELO it should be 50:50 for them to win. I think maybe 100 points of ELO is supposed to correspond to one standard deviation. So if a 1600 player beats a 1700 player, it's an upset of like 2:1 odds or 3:1 odds or something. And the same is true if 1500 beats 1600 etc. But if 1500 beats 1700, that's two standard deviations, not one, so its an upset of like 20:1 odds. When I was 1500 and I played Cackfiend with like 2400 something, that was like 9 standard deviations so it would have been like a 10^9:1 odds or something, depending on details of model.
  
  So players in a range of 100 or so are thought to be in the same competitive class.
  
  But the notion of class is continuous. There aren't discrete strategy classes in this model, so this `number of skill levels` doesn't seem to work, and its inherently linear, so the classes are just like `stronger` vs `weaker`.
• I don't know if there is a theoretical way to separate games based on "numbers of viable strategies", which depends only on the game and not on the assumption of human players. Maybe for games of imperfect information this makes sense somehow... Intuitively I'd like for "game quality" to be a mathematical property of the game which makes sense whether you play it with smarter people or less smart people, or with computers or chipmunks.
• If you instead want to define game quality with respect to some reference population, and take a machine learning / statistics point of view, I guess maybe what you will do is associate to each player a vector of their measured probabilities of winning against all the other players, then cluster the best players into groups whose vectors are near to eachother, and think of each cluster as corresponding to a strategy. Then you can count strategies this way. But its a bit fuzzy because its hard to know what the cluster size should be and there's some noise in the measurements.

Edit: I read your post again. I'm not sure right now if you care mostly about "number of stronger-weaker classes" and not "number of viable strategies".

"I also ask a question: do you believe good strategy games have more possible that kind of levels than bad ones?" If that means that there are multiple viable strategies, I think the answer is yes personally.

If you talk only about ELO-type "linear" skill levels, I think there are games that are going to have a very wide range of stronger-weaker classes among human or computer players, but strategically they will be very boring.

[GbDorn]

It seems to me that the larger the ELO spread for a game the more "cognitive abilities" players display. So you get a better view of the extent of (some sort of) human intelligence.

But I doubt one can compare games with this criteria because games use varying degrees of randomness. And in practice the ELO spread is dependent on the players population. For example if all chess players (or at least the best ones) in the world played on the Wesnoth ladder its ELO spread would probably change (assuming a chess player would have the same skill in Wesnoth MP). But it's not going to happen (any chess world champion on Wesnoth?) so we'll never know.

I don't know if there is a theoretical way to separate games based on "numbers of viable strategies", which depends only on the game and not on the assumption of human players. [...] Intuitively I'd like for "game quality" to be a mathematical property of the game which makes sense whether you play it with smarter people or less smart people, or with computers or chipmunks.
[...]
I think there are games that are going to have a very wide range of stronger-weaker classes among human or computer players, but strategically they will be very boring.

I would say it differently. IMO a more intellectually-demanding game does not necessarily imply a more fun game. I'm not sure where the number of viable strategies comes into play or how it correlates with the other two factors.

But I think you can add strategies and fun to a game without modifying the ELO spread. There are many chess variants that probably wouldn't change the ELO spread. Yet they add more strategies and probably some fun.

Does adding a new MP faction in Wesnoth increase the number of viable strategies? I don't know. But I think it does increase the fun and the game quality.

[Velensk]

I think it depends on how you define a good strategy game.

I'm inclined towards the definition that a good strategy game is one which can be picked up by anyone fairly quickly but which you do not stop improving on for a very long time. As progress of this time is incremental there will by nature be many many levels of player skill. So by (my) definition, the answer is yes.

A second definition which I could propose and also like (though somewhat less) for a good strategy game, is a game where the vast majority of questions a person could ask about what is the best course of action given a general situation is "It depends" where the factors it depends on are many, interesting, varied, and to some extent can be played around in advance of becoming directly relevant. That said, I don't think the purpose of the game should be to memorize all these factors and their meanings but rather to develop an intuition for them.