Main Factions Balance

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderators: Forum Moderators, Developers

Post Reply
User avatar
Xalzar
Posts: 290
Joined: April 4th, 2009, 10:03 pm
Location: New Saurgrath

Re: Main Factions Balance

Post by Xalzar » June 9th, 2019, 4:31 pm

I know this topic is dwindling down, I only wanted to chime in and link the page of a proposed project which fits what I summarily suggested during this discussion (I knew I heard something similar to what I have in mind in the past, finally I've found it!). Here for the curious, it's only a page with the proposal, the project sadly never took off.

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 9th, 2019, 5:38 pm

Xalzar wrote:
June 9th, 2019, 4:31 pm
I know this topic is dwindling down, I only wanted to chime in and link the page of a proposed project which fits what I summarily suggested during this discussion (I knew I heard something similar to what I have in mind in the past, finally I've found it!). Here for the curious, it's only a page with the proposal, the project sadly never took off.
Frankly at this point with so low high level player count statistics might be not as good as some time ago. Maybe downloading and counting replays from past could work (you could then also only get replays form good players). In short i don't think that they are key to solve this (if using only future replays). Well I was never a fan of them to begin with (especially in game with not very big community).

Oh and it wouldn't work for non-default eras.

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 9th, 2019, 5:52 pm

Xalzar wrote:
June 9th, 2019, 4:31 pm
I know this topic is dwindling down, I only wanted to chime in and link the page of a proposed project which fits what I summarily suggested during this discussion (I knew I heard something similar to what I have in mind in the past, finally I've found it!). Here for the curious, it's only a page with the proposal, the project sadly never took off.
Interesting, I will read that :)
Hejnewar wrote:
June 9th, 2019, 5:38 pm
Frankly at this point with so low high level player count statistics might be not as good as some time ago.
Do you think that only high level player statistics should be analyzed? I think it makes sense to analyze all default era 1v1 matches. It is better to consider as much data as possible. At the end we will have an average results.
josteph wrote:
June 6th, 2019, 9:58 pm
There's https://replays.wesnoth.org/1.14/
We have all raw data available. Let's discuss how to analyze it. Who knows, maybe this will lead to something interesting. I am talking about 1v1 below. It is hard to predict changes to multiple factions, it looks better to improve the weakest faction (I think most people agree with that). Additional notes at the end.

Find out which faction wins less. Suppose we have determined it, does it mean it is:
- the weakest
- the hardest to play
- not suitable for the map it was playing on

The next step would be to determine which of the factors above play the most significant role.

The weakest. Comparing to all factions or especially weak to some of them? We might want to determine who are the strongest rival and consider to mitigate that. That means more granular measure - faction vs faction comparison. Suppose faction A is "the weakest" and it is usually defeated by factions B or C. Do we want to buff A or nerf B and C? The easiest solution is to buff A. Otherwise we would need additional analysis of B and C.

The hardest to play. Do we want to buff the faction in such case? I think no. How can we statistically measure the difficulty of playing a faction? I don't have the answer right now.

Not suitable for the map it was playing on. If the faction looses very often on some specific map we could consider rebalancing the map.

Additional notes.
a) What about 2v2 and 3v3? Theoretically 1v1 balance improvement may result in 2v2 balance degradation for some matchups. The easiest solution would be to ignore such risk.
b) If the game was ended after one of the players was defeated we are good. What if player surrendered? Need to check if replay file contains that information. What if player quit? In such case I think we should use some algorithm to determine who was winning and count that in the statistics.

Update. Replay files here https://replays.wesnoth.org/1.14/ don't have information about what side has won. There is no information if unit dies after an attack and no snapshot of the situation on the battlefield at the end. Thus, the task is even harder.
Last edited by sergey on June 9th, 2019, 6:47 pm, edited 1 time in total.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 9th, 2019, 6:14 pm

sergey wrote:
June 9th, 2019, 5:52 pm
Hejnewar wrote:
June 9th, 2019, 5:38 pm
Frankly at this point with so low high level player count statistics might be not as good as some time ago.
Do you think that only high level player statistics should be analyzed? I think it makes sense to analyze all default era 1v1 matches. It is better to consider as much data as possible. At the end we will have an average results.

The hardest to play. Do we want to buff the faction in such case? I think no. How can we statistically measure the difficulty of playing a faction? I don't have the answer right now.

Not suitable for the map it was playing on. If the faction looses very often on some specific map we could consider rebalancing the map.
There are two reasons why i think balancing according to the top is better:
1. On top "easy" and "hard" factions have much less discrepancy in win ratio than no avarage on low lvl and there be much more matches played on average and low lvl.
2. Taking data from everyone means that top is much less important and as you said we have no way to tell if faction is weak or hard (except opinions and common sense), that might lead in case of making wrong decision to creating monster for top players to use and this faction might still be too hard to play for low lvl players.

But as I said I understand it I'm not a fan of this approach anyway.

I totally forgot. Actually probably some maps could use some redesigning. But would even anyone want that?

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 9th, 2019, 8:09 pm

Hejnewar wrote:
June 9th, 2019, 6:14 pm
There are two reasons why i think balancing according to the top is better:
1. On top "easy" and "hard" factions have much less discrepancy in win ratio than no avarage on low lvl and there be much more matches played on average and low lvl.
2. Taking data from everyone means that top is much less important and as you said we have no way to tell if faction is weak or hard (except opinions and common sense), that might lead in case of making wrong decision to creating monster for top players to use and this faction might still be too hard to play for low lvl players.
https://wesnoth.gamingladder.info have their own database of match results. It is better comparing to https://replays.wesnoth.org/1.14/ because:
- They use default era, don't use mods, play 1v1. No need to filter by these criteria.
- It is possible to search for matches with high rating players.
- It is easier to determine who won (still it may require additional work to link players with the factions they played, not sure if there is such info in the database)

Sounds better? You said you are not a fan of this approach, anyway it would be good to have such statistics.

Additional questions:

1) What is the minimum player rating to consider? Rough estimate would be enough at this moment. 2000 is good? Maybe something like 1800 is better in order to get more samples?
Quote from the ladder website:
People that are new to the game or still learning it are expected to have a rating that’s way lower than 1500. A player who has been around for a while and knows the game is expected to be average and have around 1500, while a really skilled veteran would have a higher. As a reference, players that have around 2000 are considered strong, and those beyond 2500 grandmasters.
Search shows that 275 players have more than 1800 points, 83 players have more than 2000 points.

2) What Wesnoth versions are appropriate (i.e. those who have the same unit stats as 1.14)? I will investigate, but maybe someone already knows that.
1.12 - 1.14 comparison:
1.10 - 1.12 comparison:
Last edited by sergey on June 11th, 2019, 3:13 pm, edited 3 times in total.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Pentarctagon
Forum Administrator
Posts: 3980
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Main Factions Balance

Post by Pentarctagon » June 9th, 2019, 9:03 pm

Hejnewar wrote:
June 9th, 2019, 6:14 pm
I totally forgot. Actually probably some maps could use some redesigning. But would even anyone want that?
As long as sufficient reasoning can be given for the change(s), I don't see why not.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code

User avatar
josteph
Developer
Posts: 668
Joined: August 19th, 2017, 6:58 pm

Re: Main Factions Balance

Post by josteph » June 9th, 2019, 9:11 pm

sergey wrote:
June 9th, 2019, 5:52 pm
Do you think that only high level player statistics should be analyzed? I think it makes sense to analyze all default era 1v1 matches. It is better to consider as much data as possible.
I don't think you should consider all default era matches. There are many factors you have to control for. For example, you should probably exclude all games that didn't use random faction assignment for all sides. And games on random maps. And hodor games. And even if none of these things were a problem, most games are played with the default gold values, and it's conceivable that the default gold values cause bias against some faction (the starting gold values, not the unit costs).
Pentarctagon wrote:
June 9th, 2019, 9:03 pm
As long as sufficient reasoning can be given for the change(s), I don't see why not.
I've been thinking that maybe we should do away with the concept of maps that work for any combination of factions. There are 6 factions so that means 30 combinations on each map (6 possibilities for side 1, 5 possibilities for side 2), not counting mirror matches and dunefolk. Maybe we should start to have more specialized maps, for example, imagine Auction X where the center is surrounded by forest and side 1 is always rebels (with nerfed starting gold to compensate for the terrain buff).

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 9th, 2019, 10:04 pm

josteph wrote:
June 9th, 2019, 9:11 pm
most games are played with the default gold values, and it's conceivable that the default gold values cause bias against some faction (the starting gold values, not the unit costs).
I agree. But I can't see how to measure and eliminate the bias. I think it should be considered as minor issue and ignored. Faction units and map have much more significant impact on the balance.
josteph wrote:
June 9th, 2019, 9:11 pm
I've been thinking that maybe we should do away with the concept of maps that work for any combination of factions.
There is an add-on map-pack called "Rushed By Yetis", available for 1.8 - 1.12. Those maps are modified versions of the default maps. They were used for ladder games. I believe they provide better balance. Found here https://wesnoth.gamingladder.info/faq.php section "Game setup" item 8b.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 9th, 2019, 10:50 pm

Pentarctagon wrote:
June 9th, 2019, 9:03 pm
Hejnewar wrote:
June 9th, 2019, 6:14 pm
I totally forgot. Actually probably some maps could use some redesigning. But would even anyone want that?
As long as sufficient reasoning can be given for the change(s), I don't see why not.
Some could probably use just visual improvements. Well everything depends on finding someone who would like to do that and with some actual sense of beauty (so not me [I dont even see a difference between 5 different versions of the same unit that is widely discussed by everyone else..]).
josteph wrote:
June 9th, 2019, 9:11 pm
I've been thinking that maybe we should do away with the concept of maps that work for any combination of factions. There are 6 factions so that means 30 combinations on each map (6 possibilities for side 1, 5 possibilities for side 2), not counting mirror matches and dunefolk. Maybe we should start to have more specialized maps, for example, imagine Auction X where the center is surrounded by forest and side 1 is always rebels (with nerfed starting gold to compensate for the terrain buff).
Well, if we consider only flat, forest, hills, fungus and mountains then this is not that bad.

Flat - pretty much doesn't matter.
Humans, Orcs, Undead, and Drakes have equal defence on forests and hills.
Only Dwarfs, and Elves have different defences on hills and forests (from major races).
Fungus is more important- undead 60%, human 50%, elves 50%, orcs 40%, dwarfs 40%, drakes 40% (no benefit).
Mountains are usually good for everyone (the best for dwarf tho).

I would like to see more different terrains on future maps (there is not a lot to choose from but still).

sergey - 1800 is ok i think. Actually now that i remember there was someone who was doing something like that. He never finished saddly.

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 10th, 2019, 12:50 am

Hejnewar wrote:
June 9th, 2019, 10:50 pm
sergey - 1800 is ok i think. Actually now that i remember there was someone who was doing something like that. He never finished saddly.
Ok, I will try to reach Cackfiend and ask what data is stored in his database and if it can be used for the analysis.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

Mawmoocn
Posts: 97
Joined: March 16th, 2019, 3:54 pm

Re: Main Factions Balance

Post by Mawmoocn » June 10th, 2019, 8:18 am

Xalzar, thank you for sharing (and finding) the idea! I found it interesting that it has similar things on what I have thought.

I think I’m late for posting this...

"Summary" will contain "Final battle statistics" that includes the whole overview result.

In-depth analysis contains verbose details that happened per turn.

Things needed to be defined for automation
Spoiler:

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 10th, 2019, 9:12 am

Mawmoocn wrote:
June 10th, 2019, 8:18 am
Final battle statistics contains

1. Side leader killed within turn number
2. Traits that affected the battle
3. Terrain move speed cost used by units
4. Close win statistics
5. Time of day analysis, fought unfavourably or retreated but lost some units half way
6. Group retreat
7. Individual retreats made
8. Type of victory achieved
9. Map size
Mawmoocn, I have remarks on individual items, however there is a more significant problem as I can see. Ideologically Wesnoth is a non competitive game. All ladders are unofficial. It means that there won't be player rating in the replay statistics. As Hejnewar said, it is essential to consider player rating. I totally agree with him after some thinking.
Hejnewar wrote:
June 9th, 2019, 6:14 pm
There are two reasons why i think balancing according to the top is better:
1. On top "easy" and "hard" factions have much less discrepancy in win ratio than no avarage on low lvl and there be much more matches played on average and low lvl.
2. Taking data from everyone means that top is much less important and as you said we have no way to tell if faction is weak or hard (except opinions and common sense), that might lead in case of making wrong decision to creating monster for top players to use and this faction might still be too hard to play for low lvl players.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 10th, 2019, 11:04 am

josteph wrote:
June 9th, 2019, 9:11 pm
For example, you should probably exclude all games that didn't use random faction assignment for all sides. And games on random maps. And hodor games.
I totally forgot to respond to that. Why "And hodor games."?

Mawmoocn:
Its good they you have enthusiasm but i believe you are overcomplicating things and at the same time you are forgetting about some of more basic stuff.

I wont talk abut good things. There is quite a bit of them (and i still don't know why some of them are needed but it could be fun to just see some of them).

Point 1 to some extent and point 2 can be resolved just by using ladder data.

Comeback from unfavourable results/Close win - comebacks can be very decisive especially if you have good counter and you just didn't recruit it. After few turns if you survived you have that counter and can win easly. I would count that as desive. (This is just an example.)

Traits - you cant always use them as intended. In 1v1 Mp game you usually have very limited resources and you can't bother with leveling even intelligent units if they require more tan 2/3 kills.

What if:
Player kicks his opponent just to win becouse he was losing? It happens from time to time.

Also:
Where is luck? This is one of the most important statistics for this kind of project.

User avatar
josteph
Developer
Posts: 668
Joined: August 19th, 2017, 6:58 pm

Re: Main Factions Balance

Post by josteph » June 10th, 2019, 1:23 pm

Hejnewar wrote:
June 10th, 2019, 11:04 am
I totally forgot to respond to that. Why "And hodor games."?
Including hodor games might skew the score of knalgans. Suppose I told you that in hodor v. Northerners the win rate is 40-60 in favor of Northerners, would that imply that Knalgans v. Northerners is not balanced? I think it wouldn't. (Same goes for the other 5 factions of course)

edit: And by the way, you should also exclude games where a human player droided his side.... the AI doesn't have >1800 ladder ranking :lol:

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 10th, 2019, 1:37 pm

josteph wrote:
June 10th, 2019, 1:23 pm
And by the way, you should also exclude games where a human player droided his side.... the AI doesn't have >1800 ladder ranking :lol:
I decided to contact Cackfiend (the ladder organizer and the #1 multiplayer gamer btw) and ask a copy of the ladder database. No hodor, no random maps, no droided sides on the ladder. They have strict rules. More info why the ladder database if better than the replays storage https://replays.wesnoth.org/ here viewtopic.php?f=15&t=49889&start=45#p643231
Last edited by sergey on July 5th, 2019, 3:01 pm, edited 2 times in total.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

Post Reply