Main Factions Balance

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderators: Forum Moderators, Developers

Post Reply
User avatar
EarthCake
Posts: 348
Joined: March 29th, 2019, 1:57 pm
Location: Dragonstone

Re: Main Factions Balance

Post by EarthCake » June 10th, 2019, 2:01 pm

One question: Do you have to play a game with another player from the ladder to get rating or you can play with anyone?

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 10th, 2019, 2:11 pm

EarthCake wrote:
June 10th, 2019, 2:01 pm
One question: Do you have to play a game with another player from the ladder to get rating or you can play with anyone?
There are registered players who have 0 games and 1500 rating. To get an initial rating you don't have to play at all, just to register. If you want to report a game and gain a rating, both players must have the ladder account.
Last edited by sergey on June 10th, 2019, 3:02 pm, edited 1 time in total.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 10th, 2019, 2:59 pm

josteph wrote:
June 10th, 2019, 1:23 pm
Including hodor games might skew the score of knalgans. Suppose I told you that in hodor v. Northerners the win rate is 40-60 in favor of Northerners, would that imply that Knalgans v. Northerners is not balanced? I think it wouldn't. (Same goes for the other 5 factions of course)
If we don't gather data from all tactics then we might never know if something is op. Hodor has good and bad matchups as any tactic does it's probably better than regular klanga in some of them (also there are many variations of hodor, units like guards, ulfserkers and gryphons are often incorporated into hodor). Not including things like hodor would show statistics for only one tactic most probably and if you are balancing accordingly just to one tactic or playstyle for faction it might easily lead to unintentional things happening. You might be able to balance that one optimal way to play but players might find another way of playing better after changes.

This way of doing things obviously has one big + it's more accurate. But in return you are sacrificing big picture. If only we were able to extract data about units recruited then we could even estimate win ratios for units in matchpus and this wouldn't be a problem.

User avatar
josteph
Developer
Posts: 668
Joined: August 19th, 2017, 6:58 pm

Re: Main Factions Balance

Post by josteph » June 10th, 2019, 4:00 pm

Yeah, if a player chooses to play hodor and loses, that's the same as any bad strategy, sure. I was thinking, however, of matches where the players agreed before the match that they should play hodor. At that point they aren't actually playing the default era any more, and even if it turns out their hodor faction is underpowered that won't imply knalgans in default is underpowered. Hodor is just an example, this goes for any meta rule. This shouldn't be a problem if you use ladder matches, though, right?
Hejnewar wrote:
June 10th, 2019, 2:59 pm
This way of doing things obviously has one big + it's more accurate. But in return you are sacrificing big picture. If only we were able to extract data about units recruited then we could even estimate win ratios for units in matchpus and this wouldn't be a problem.
Extract data from what? If you have the savefile, you can tell how many units were recruited.

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 10th, 2019, 4:40 pm

josteph wrote:
June 10th, 2019, 4:00 pm
Yeah, if a player chooses to play hodor and loses, that's the same as any bad strategy, sure. I was thinking, however, of matches where the players agreed before the match that they should play hodor. At that point they aren't actually playing the default era any more, and even if it turns out their hodor faction is underpowered that won't imply knalgans in default is underpowered. Hodor is just an example, this goes for any meta rule. This shouldn't be a problem if you use ladder matches, though, right?
Hejnewar wrote:
June 10th, 2019, 2:59 pm
This way of doing things obviously has one big + it's more accurate. But in return you are sacrificing big picture. If only we were able to extract data about units recruited then we could even estimate win ratios for units in matchpus and this wouldn't be a problem.
Extract data from what? If you have the savefile, you can tell how many units were recruited.
I was totally convinced that you were talking about hodor and other things like that in general.

From the top of my head I remember:
-1 picked ud player
-1 picked klanga Hodor / 4 gryph rush (Rogue leader) only and 1 normal klanga player
-2 picked elves players
-1 picked loyalist player
on ladder.

I know that i can access that data but going thru let's say 1500 replays manually is not for me as well as im not just focused on default I in my endeavour am focused on balance in wesnoth in general so i wouldnt have much use out of default only data. I can help with some things tho in my limited time that already is consumed by wesnoth in some part .

User avatar
josteph
Developer
Posts: 668
Joined: August 19th, 2017, 6:58 pm

Re: Main Factions Balance

Post by josteph » June 10th, 2019, 5:25 pm

Hejnewar wrote:
June 10th, 2019, 4:40 pm
I know that i can access that data but going thru let's say 1500 replays manually is not for me as well
Write a script to parse savefiles and extract the information you want.

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 10th, 2019, 10:44 pm

josteph wrote:
June 10th, 2019, 5:25 pm
Hejnewar wrote:
June 10th, 2019, 4:40 pm
I know that i can access that data but going thru let's say 1500 replays manually is not for me as well
Write a script to parse savefiles and extract the information you want.
I will pass on that. It was more of a proposal of a solution. Maybe if i was convinced that this is the best approach, its good but i dont think the best.

User avatar
The_Gnat
Posts: 2071
Joined: October 10th, 2016, 3:06 am
Contact:

Re: Main Factions Balance

Post by The_Gnat » June 11th, 2019, 2:34 am

Xalzar wrote:
June 6th, 2019, 6:50 pm
On one side we have a player who advocates many little changes to default factions (but it is known that even small corrections could cause serious unexpected unbalances, expecially when they are so many), based on alleged opinions of expert players (where are they? why don't they intervene in this topic to support his position?). The intentions are noble, the method is a bit amateurish.

On the other side we have other players ready to pounce on every slight proposal of change, insisting that "all is balanced", even resorting to velated insults. They dismiss every proposal, they're ready to judge and they close any space of discussion.

In general, many here speak in absolutes, when the reality is often in the middle.
Wow ^_^ Very well said Xalzar. I completely agree. Apologies if I have partaken on either side of this issue.
I think we need some incontrovertible evidence of these unbalances, to address the sensations that regularly reemerge in the forums only to be shut down by the "guardians" of the balance.
I think we need an automated tool which checks deeply online matches: what factions win more, which units are more used, how do they fare, which maps are more used, how factions behave in them, at all level of skill.
I think we need a balance team which collects and analyzes the data and takes the opportune measures.
We cannot rely on dated, incomplete, or even unproven information like the ones reported in this topic by one side or the other.
I agree. All this is very necessary. 800pgs of balance threads made no change to the DF balance. Which is why we eventually setup a team to test, revise and create balances. Overall balancing the other mainline factions would likely be even harder than balancing the DF. Perfect balance would be aimed for over many more factions which would inevitably taken much longer than the 6 months devoted to balancing and testing the DF.

And now before anyone tells me that 'only small changes are needed which will not take such a time' I will point out that aside from the addition and removal of DF units, every other change was a 'small change' and to this end we spent a considerable time testing its matchups against the other factions.

So in conclusion I am for rebalancing the imbalances of the mainline factions. I am even in support of discussing the mainline factions for long periods of time first to fully establish a perspective of what exactly needs changing in the communities mind. I just don't think that trying to rebalance everything at once will actually happen, and I would like to see balance improved in some areas. :)

Mawmoocn
Posts: 97
Joined: March 16th, 2019, 3:54 pm

Re: Main Factions Balance

Post by Mawmoocn » June 11th, 2019, 5:08 am

sergey wrote:
June 10th, 2019, 9:12 am
Mawmoocn, I have remarks on individual items
If you have questions, feel free to ask. I’ll try to answer, if possible.
sergey wrote:
June 10th, 2019, 9:12 am
All ladders are unofficial. It means that there won't be player rating in the replay statistics. As Hejnewar said, it is essential to consider player rating. I totally agree with him after some thinking.
I disagree on analysing based on player rating.
Beginners and experienced players have different play style, observations allows us to learn from beginner mistakes and learn new strategies.
I want to see the defining line between beginners and experienced, and what makes a beginner learn faster, to be an experienced player.

There are more things to elaborate and it could further be explored.
Hejnewar wrote:
June 10th, 2019, 11:04 am
Point 1 to some extent and point 2 can be resolved just by using ladder data.
They’re are problems that don’t exist within ladder games, how do we define intentions behind strategies used by the player? Is there a difference using a strategy called rushing units by an experienced player vs a beginner player using the same strategy?
Hejnewar wrote:
June 10th, 2019, 11:04 am
Comeback from unfavourable results/Close win - comebacks can be very decisive especially if you have good counter and you just didn't recruit it.
It’s hard to measure close win, as it’s hard to define things that could’ve have made you lost, with a large probability. Even if you can counter, and execute your plan, winning isn’t a 100% chance until you have won the game.

Mistakes can happen at the very end, which makes things very interesting, though that’ll be hard to define, as making mistakes can be avoided, if you learned after quite some time, and the risk of mistake will remain, even if you lessen the chance.

My intent was to find unique replays that does very unfavourable result and either they win or not. It would probably be really fun to watch!
Hejnewar wrote:
June 10th, 2019, 11:04 am
Traits - you cant always use them as intended. In 1v1 Mp game you usually have very limited resources and you can't bother with leveling even intelligent units if they require more tan 2/3 kills.
Well for example, if you had gotten a resilient trait, you can use this unit far longer than a normal unit. That in itself, can help you level units or keep using the unit, after many turns.
I’m trying to measure potential deaths, misplaced positions, level up opportunities that happened.

Traits can magnify the effects of survival, related to mobility and hit points.
Hejnewar wrote:
June 10th, 2019, 11:04 am
What if:
Player kicks his opponent just to win becouse he was losing? It happens from time to time.
There’re several factors and I’ll pick a few to make it short.

If the game records actions like kicking a player, it won’t be considered as their win.

If kicking another player makes them win due to losing to that player, the result will be based on the player that was kicked. If they have the potential to win, the other player wins, and if they don’t, no one wins.
Ideally, this assumes that we can create a way to define winners from stalemate type of games.

Lastly, this needs a new feature to prevent such behaviours. The game needs to prevent kicking a player out, once a game starts. If people tend to abuse a feature, we need proper measures, to solve these issues.

I think the information here could be valuable, we can see if the game turned extremely unfavourable for the player that use kick.
Hejnewar wrote:
June 10th, 2019, 11:04 am
Also:
Where is luck? This is one of the most important statistics for this kind of project.
That will be hard to define as it will branch out to many possibilities, and would clogged identifying problems.

I’ve partially mentioned it’s inner workings on traits and close win.

How do you define luck?

Ok, that’s not easy, so I won’t concentrate on other possibilities without result from replays.

The things in Wesnoth that affects RNG(????) include:
1. Chance to hit
2. Traits
3. Random faction selection
4. Scenarios that use probabilities to change map properties
5. Your potential opponent, depending on circumstances
6. Fog of War / Shroud
7. Random map selection and settings
8. Your potential ally
9. How well you can operate with said ally
10. Kicked by a player

Fog of War and Shroud can be excluded, but as it adds the element of "unknown", you’ll be guessing most of the time without using scouts.

Well currently the game doesn’t support reading a player’s tactics. Game help files also lack tactics on it’s manual. Teaching is a complicated issue, as good teachers are probably hard to find.

Partially, we can try to group units that survived from group attacks, but trying to codify it, to tell us what is happening from the whole map overview, would probably consume more time on perfecting the way it’s understood by the machine.

Finding the point of relevance would need to be defined in a way that’s clear to implement.

I didn’t include "luck”, as I’m basing stats on results based from replays.

Including luck, needs a way to define certain moves, without complications.

If you have questions or clarifications, I’m willing to answer it if I can.

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 11th, 2019, 1:02 pm

Mawmoocn wrote:
June 11th, 2019, 5:08 am
sergey wrote:
June 10th, 2019, 9:12 am
All ladders are unofficial. It means that there won't be player rating in the replay statistics. As Hejnewar said, it is essential to consider player rating. I totally agree with him after some thinking.
I disagree on analysing based on player rating.
Beginners and experienced players have different play style, observations allows us to learn from beginner mistakes and learn new strategies.
I want to see the defining line between beginners and experienced, and what makes a beginner learn faster, to be an experienced player.
We are talking about different things. I am saying that it looks better to exclude novice players when measuring factions balance. It may seem even better if you consider only games in which players have similar rating. But this can lead to a small number of samples, which is also bad. Hejnewar have provided arguments why experienced players are better for the factions balance analysis.

You are saying that novice players should be also taken into account. I agree that their games could be used for some kind of analysis, but not for measuring factions balance. Just in case, by analysis I mean statistics methods, not watching the replays with your eyes. Ladder players have already watched a tons of replays and developed a lot of strategies.
I disagree on analysing based on player rating
I want to see the defining line between beginners and experienced
It sounds like you propose to not take into account the ladder rating, but measure the rating during the game "on the fly". Correct? I am not sure that it makes sense from a technical point of view. Theoretically it is possible to develop some kind of a very sophisticated algorithm that could do that. This also applies to what you said about determining retreat actions, analyzing situation on the map, scoring points to player moves, etc. It is theoretically possible, but not practically. If you don't believe me, ask Wesnoth developers to review your proposal.
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

Mawmoocn
Posts: 97
Joined: March 16th, 2019, 3:54 pm

Re: Main Factions Balance

Post by Mawmoocn » June 11th, 2019, 6:43 pm

sergey wrote:
June 11th, 2019, 1:02 pm
Ladder players have already watched a tons of replays and developed a lot of strategies.
Let’s say for the sake of argument, I disagree. The games can be "farm game" to farm "points", by being the most active ladder player and defeating newbies.

How would you measure these games?

Most competitions in the professional occupations, have at least 3 games or more to prove that their wins are not due to "luck".
sergey wrote:
June 11th, 2019, 1:02 pm
It sounds like you propose to not take into account the ladder rating, but measure the rating during the game "on the fly". Correct?
If you mean while during the game, the answer is no.
If you mean analysing all replays, and making a comparative structure to analyse mass data, the answer is yes.
I’m not measuring player rating, I’m measuring exploits used by using a set amount of definitions and common used actions.

How can you define a losing player’s rating?

For the sake of argument, let’s say it’s flawed since there’s no constructive way to define another player’s effort that failed to win, and heavy amount of data isn’t possible to analyse. Because of the problems, we needed to pick from "2" teams, winning and losing team. Which data is more valuable, the winning team or the losing team? Based on what you picked, Why is their data important?
sergey wrote:
June 11th, 2019, 1:02 pm
It is theoretically possible, but not practically.
I’m suggesting an abstract version on determining replays that have no existing analysis.
I want to limit the scope using categories like map, faction vs faction, disconnection problems, modifications used, eras used, other problems, to even create a better data.
I’m aware that isn’t practical, if definitions on it’s functionalities aren’t clear, it would be impossible to be accepted and if the work load is large.
It’s basically a draft and I’m sure that it isn’t feasible without automation or AI.


If you ask which is more practical at this point, obviously it would be a small sample size.
I want to see a better future to analyse data, so I think what The_Gnat said is the optimal and cost saving solution, at this point of time.

User avatar
sergey
Posts: 378
Joined: January 9th, 2015, 9:25 pm

Re: Main Factions Balance

Post by sergey » June 11th, 2019, 9:54 pm

Mawmoocn wrote:
June 11th, 2019, 6:43 pm
sergey wrote:
June 11th, 2019, 1:02 pm
Ladder players have already watched a tons of replays and developed a lot of strategies.
Let’s say for the sake of argument, I disagree.
I meant that ladder community has already analyzed a tons of replays and developed a lot of strategies. You disagree with that?
Mawmoocn wrote:
June 11th, 2019, 6:43 pm
The games can be "farm game" to farm "points", by being the most active ladder player and defeating newbies.

How would you measure these games?
Are your assumptions based on the ladder game history? Do you know about the ELO system used there?
Mawmoocn wrote:
June 11th, 2019, 6:43 pm
sergey wrote:
June 11th, 2019, 1:02 pm
It sounds like you propose to not take into account the ladder rating, but measure the rating during the game "on the fly". Correct?
If you mean while during the game, the answer is no.
If you mean analysing all replays, and making a comparative structure to analyse mass data, the answer is yes.
I’m not measuring player rating, I’m measuring exploits used by using a set amount of definitions and common used actions.
Anyway I will say the same - ask Wesnoth developers to review this proposal.
Mawmoocn wrote:
June 11th, 2019, 6:43 pm
How can you define a losing player’s rating?

For the sake of argument, let’s say it’s flawed since there’s no constructive way to define another player’s effort that failed to win, and heavy amount of data isn’t possible to analyse. Because of the problems, we needed to pick from "2" teams, winning and losing team. Which data is more valuable, the winning team or the losing team? Based on what you picked, Why is their data important?
Elo rating system defines both looser and winner rating https://en.wikipedia.org/wiki/Elo_rating_system That answers the first part of the quote. Regarding the second part - I am not going to think about that at all, that is your idea to analyze replays by some kind of sophisticated algorithm.
Mawmoocn wrote:
June 11th, 2019, 6:43 pm
sergey wrote:
June 11th, 2019, 1:02 pm
It is theoretically possible, but not practically.
I’m suggesting an abstract version on determining replays that have no existing analysis.
I want to limit the scope using categories like map, faction vs faction, disconnection problems, modifications used, eras used, other problems, to even create a better data.
I’m aware that isn’t practical, if definitions on it’s functionalities aren’t clear, it would be impossible to be accepted and if the work load is large.
It’s basically a draft and I’m sure that it isn’t feasible without automation or AI.
The main questions. Are you going to participate in the work you are proposing? Or you only provide the ideas?
Author of SP scenario Dragon Fight and SP campaign Captured by a Nightmare

User avatar
Hejnewar
Posts: 54
Joined: September 17th, 2016, 11:01 am

Re: Main Factions Balance

Post by Hejnewar » June 11th, 2019, 10:11 pm

Mawmoocn wrote:
June 11th, 2019, 5:08 am
If you have questions or clarifications, I’m willing to answer it if I can.
I'm a bit short on time so i can't respond to each and every issue properly. It seems to me that you are more than balance seeking knowledge. If that is the case then not many things that I said really applies.
They’re are problems that don’t exist within ladder games, how do we define intentions behind strategies used by the player? Is there a difference using a strategy called rushing units by an experienced player vs a beginner player using the same strategy?
I don't define them, but i like to think that winning is the intention.

In wesnoth you can't really rush units. Its bothering me because i was playing a lot of rts games in the past. Yes there is a difference.
I’m trying to measure potential deaths, misplaced positions, level up opportunities that happened.


This is way harder than measuring luck, with luck you just need data about hits and cth. (At least in my definition of luck.) This seems like professional pro player learning tool. The more i look at this, the more i think that you are trying to "simulate" wesnoth replays outside of wesnoth, it reminds me of self-learning ai.
Well currently the game doesn’t support reading a player’s tactics. Game help files also lack tactics on it’s manual. Teaching is a complicated issue, as good teachers are probably hard to find.
Nah, if you are searching to play with good player you will find them, lose to them, and learn from them. I believe that the best way to learn wesnoth is loosing actually. Theory, it's enough to just ask to learn it, it isn't very complicated from my perspective and from what I've seen in past years players usually learn it fast.
Let’s say for the sake of argument, I disagree. The games can be "farm game" to farm "points", by being the most active ladder player and defeating newbies.
There is no such thing. There is no matchmaking, so you need to join to play and you might simply say "I won't play with you.". If I'm not mistaken they also will very quickly start gaining almost no points for that.
Which data is more valuable, the winning team or the losing team?
Can't they just be equaly important? :hmm:

Mawmoocn
Posts: 97
Joined: March 16th, 2019, 3:54 pm

Re: Main Factions Balance

Post by Mawmoocn » June 12th, 2019, 4:08 am

sergey wrote:
June 11th, 2019, 9:54 pm
I meant that ladder community has already analyzed a tons of replays and developed a lot of strategies. You disagree with that?
No.
sergey wrote:
June 11th, 2019, 9:54 pm
Are your assumptions based on the ladder game history? Do you know about the ELO system used there?
No.

I made a mistake assuming that it doesn’t use a system that adjusts, and something can be used to "game the system". My bad.
sergey wrote:
June 11th, 2019, 9:54 pm
Anyway I will say the same - ask Wesnoth developers to review this proposal.
Not now, I haven’t figured a way to explain it without confusing people
sergey wrote:
June 11th, 2019, 9:54 pm
that is your idea to analyze replays by some kind of sophisticated algorithm.
What makes it sophisticated? Explaining abstract ideas isn’t my forte, I need your opinion to see what could be changed or removed.
sergey wrote:
June 11th, 2019, 9:54 pm
The main questions. Are you going to participate in the work you are proposing? Or you only provide the ideas?
If you want my answer now, it’s probably to provide counter arguments or ideas.




Hejnewar wrote:
June 11th, 2019, 10:11 pm
This seems like professional pro player learning tool. The more i look at this, the more i think that you are trying to "simulate" wesnoth replays outside of wesnoth, it reminds me of self-learning ai.
I didn’t think of it like that, but it’s similar to what you said.

Well I’m trying to get a whole overview without watching a replay.

I’ve read a blog about AI phyton development for Wesnoth, they used an algorithm to find unique battle data on AI vs AI matches, to make their AI win something like 95-99% of the time.

I seem to recall that he used thousand of battle data matches and used a graph to find discrepancies from similar battle data using a tree graph.

Well, I think that was amazing in it’s own way... what caught my attention was thousands of battle data analysed using a set algorithm using gold, recruits, probably positions? well hmm can’t seem to recall exactly.

What stood out when he was testing his AI, is using and analysing thousands of battle records.

It helps if I’m able to confirm some of the ideas I have in mind.
Hejnewar wrote:
June 11th, 2019, 10:11 pm
Nah, if you are searching to play with good player you will find them, lose to them, and learn from them. I believe that the best way to learn wesnoth is loosing actually.
I agree the learning from a loss can be a quick way to adapt, but there should be other modes of learning.
Hejnewar wrote:
June 11th, 2019, 10:11 pm
Can't they just be equaly important? :hmm:
Yes they can be equal.
Though I was thinking that losing data has more variables compared to winning.
Comeback or challenging wins however, are in the same position.

foreverfighter
Posts: 3
Joined: November 21st, 2016, 10:17 am

Re: Main Factions Balance

Post by foreverfighter » June 12th, 2019, 6:37 am

as a reasonably strong ladder player (albeit one who only plays northerners), i like the changes and philosophy. thanks for proposing them.

Post Reply