Ladder Council

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderators: Forum Moderators, Developers

User avatar
Horus2
Posts: 407
Joined: September 26th, 2010, 1:05 pm

Ladder Council

Post by Horus2 » January 18th, 2016, 2:08 pm

We are the Ladder Council, a group of dedicated people working on improving your ladder experience. The group consists of five active, high-level players (formerly known as royal advisors) who make the actual decisions and numerous assistants who partake in the testgames and make suggestions. Upon vacancy, remaining "royal advisors" invite and elect new members, but you can also apply for it if you think you have what it takes.

The Council has been reestablished in March 2015, after a long hiatus by ladder players Horus2, Mint, Oook, Krogen and Rigor and our main focus is validating maps for ladder games. Our greatest success is the spread of the new, Sandbox Map Picker, thus retiring the unmaintained Rushed By Yetis. SMP features maps divided to categories according to the result of our votings, including popular maps that are present since years, but never approved for ladder play before (Ruined Passage, Ruphus Isle).

Some of our other activities are:
  • Making of brand new 1v1 maps
  • Organising tournaments and other events
  • Balance discussions
  • Promoting content for mainline
However so far we were secluded from the greater community and i never liked it that way. I opened this thread to show you what we are working on and keep you involved. If you have any question or suggestion, this forum thread is exactly for that purpose, so let us hear you!


Horus2

Kronno
Posts: 1
Joined: January 17th, 2016, 11:12 am

Re: Ladder Council

Post by Kronno » January 18th, 2016, 3:46 pm

Organising tournaments and other events
Wht tournaments you are talking about?

User avatar
Horus2
Posts: 407
Joined: September 26th, 2010, 1:05 pm

Re: Ladder Council

Post by Horus2 » January 18th, 2016, 6:12 pm

Kronno wrote:Wht tournaments you are talking about?
I cannot name it yet, but i can confirm that there will be tournaments, and you definitely do not have to wait for the summer for it.

User avatar
Horus2
Posts: 407
Joined: September 26th, 2010, 1:05 pm

Re: Ladder Council

Post by Horus2 » January 20th, 2016, 1:40 pm

We are currently researching the following 1v1 maps to see if they are suitable for ladder:

Heart Mountains' Caves - a very simple map done by Sulik ages ago, but which has enough potential for not being forgotten
Grand Houde Mountain - Dreadnough's taboo-breaking map with odd number of villages
Hellhole - the map i made when i was possessed by a spooky ghost
2p_Grand_Houde_Mountain.tar
(27 KiB) Downloaded 107 times
Maps can be examined and downloaded by clicking on the links above.


More feedback of these maps would be valuable; if you are a ladder player, please, help our cause by trying them and showing the replays to me/Dreadnough!

User avatar
Cackfiend
Posts: 400
Joined: January 28th, 2007, 7:36 am
Location: Florida, USA
Contact:

Re: Ladder Council

Post by Cackfiend » January 23rd, 2016, 12:42 am

This is going to be really confusing for new players, especially since you linked to the most popular wesnoth ladder.

You can call yourselves whatever you'd like, but you're not the Ladder Council. Consider yourselves a group of players that have good intentions but you have no actual authority over the ladder. The Ladder Council was handed over to me when Doc retired, and I have not reformed it beyond me being the sole member. I am the Ladder administrator and the owner of the Ladder Council forum (as well as TBS). If you're interested in being on the Official Ladder Council you can feel free to message me about it however :)
"There's no love in fear." - Maynard James Keenan

I'm the guy who's responsible for 40% Gliders in all hexes... I can now die a happy man. =D

User avatar
Horus2
Posts: 407
Joined: September 26th, 2010, 1:05 pm

Re: Ladder Council

Post by Horus2 » February 8th, 2016, 7:40 pm

I decided to share a bit of mapmaking voodoo with you: this humble infography shows the results of playing one hundred matches on Ruphus Isle. I am accumulating replays since i took the rights of the old Hornshark, which was almost four years ago. So yes, it takes time, and only reached such a high play count because i was personally partaking in 65 out of those. However, while i will not stop collecting the replays, i leave the next 100 for the ladder community and i devote my time to other causes.
The title texts mention a certain "relevancy" taken into account. This means i personally judged the quality of all the replays, gave them ratings (1, 2 and 3 stars) and weighted them according to it. Three main factors affect the quality of a replay: the version of the map (a significant map change automatically decreases the value of everything prior), the luck, and the players' mistakes. However it is only taken into consideration whether those factors affected the outcome or not. A classic example for such case when a player forgets to recruit in a certain turn; if the flow of the game stalled and the opponent could not attack before it arrived anyway, the replay can still get a three stars rating. In a two stars game, both victory and defeat count only half score, and a one star replay count quarter score.
100 HNX legendary.PNG
Ruphus Isle statistics
Some other interesting statistical facts about the map:
  • Since game 016 no changes were made to the map layout.
  • Without taking the relevancy of replays into account, rebels would have been the top performing faction with a winrate of 58,6% (17 games out of 29)
  • Players have the most trouble to play the map well with Knalga - there is 4% difference between weighted and non-weighted winrate. (This is likely caused by the many players insisting to play the hodorian way, despite the map gives hardly any opportunity for backstabbing and swarming.)
  • The reason why loyalists seem to currently underperform partially lies in the great number of matches played versus rebels. While it is usually considered as a fair and balanced matchup, evidence suggests that on this particular map they might have issues with fighting against merman hunters. Yes, those are the units which are disregarded so passionately even by certain high elo players. Further research must be done in this direction before reaching conclusion.
  • The role of the best faction has been contested by four since the beginning. Initially drakes were the definite leaders, but then undead stole the title and held it for very long. They lost it slowly to rebels around game 075 if i recall correctly, who were formerly 4th, then had a drastic fall. This history illustrates well how even 100 replays are only enough to suspect the imbalance, and pointing out the actual issues is often left to logic and educated guesses based on what they experienced.

Hopefully we can present you another infodump in the near future!

Velensk
Multiplayer Contributor
Posts: 3979
Joined: January 24th, 2007, 12:56 am

Re: Ladder Council

Post by Velensk » February 8th, 2016, 11:40 pm

Allow me to simply express how impressed I am with the data and presentation for this. This is something I wish I could get automatically for my maps but of course it does not come without effort.
"There are two kinds of old men in the world. The kind who didn't go to war and who say that they should have lived fast died young and left a handsome corpse and the old men who did go to war and who say that there is no such thing as a handsome corpse."

User avatar
Aldarisvet
Translator
Posts: 630
Joined: February 23rd, 2015, 2:39 pm
Location: Moscow, Russia

Re: Ladder Council

Post by Aldarisvet » February 9th, 2016, 10:56 am

Horus2 wrote:This history illustrates well how even 100 replays are only enough to suspect the imbalance[/list]
Actually I doubt that 100 replays can really say something.
All that disbalance in statistics could really be just a result of probability, result of many factors.

For example, winrate of 58,6% (17 games out of 29) for rebels is can be just a result of the randomness of any sort.
Really, just 17 games of 29, these numbers are not nearly large to be reliable to judge.

I like the analysis, but to judge seriously, you really need at least 1000 games.

Also the important question is - how many players were participating in these games. The more players participated, the less distorted is statisitcs, even with low number of games. It could be 200 players maximum in 100 games and just 2 players minimum.
facebook.com/wesnothian/ - everyday something new about Wesnoth
My campaign:A Whim of Fate, also see Zombies:Introduction single map campaign
Art thread:Mostly frankenstains

User avatar
tekelili
Posts: 1038
Joined: August 19th, 2009, 9:28 pm

Re: Ladder Council

Post by tekelili » February 9th, 2016, 11:54 am

@Aldarisvet: Your doubts are reasonables, but you are skiping most important factor: Players Skill Difference

No matter how many games are played, even with 1.000.000 games you could have totally biased data. If all those games were played between a very skilled player vs a noob, and they were always picking random faction and side, you would have totally balanced outcomes for factions and sides (about 50% vistory), despite map having some imbalances.
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II

User avatar
Aldarisvet
Translator
Posts: 630
Joined: February 23rd, 2015, 2:39 pm
Location: Moscow, Russia

Re: Ladder Council

Post by Aldarisvet » February 9th, 2016, 3:14 pm

tekelili wrote:@Aldarisvet: Your doubts are reasonables, but you are skiping most important factor: Players Skill Difference

No matter how many games are played, even with 1.000.000 games you could have totally biased data. If all those games were played between a very skilled player vs a noob, and they were always picking random faction and side, you would have totally balanced outcomes for factions and sides (about 50% vistory), despite map having some imbalances.
Well, normally the situation that you discribed will just never happen.
But about a quarter of games would be with noobs vs noobs and this can distort data because some races are harder to play than others. Or some races are easier to play.
Given that rebels actually an easiest faction to play and possibly most beloved and popular in mainline campaigns it is possible that noobs would play better with rebels than with other factions (in battles noobs vs noobs).
facebook.com/wesnothian/ - everyday something new about Wesnoth
My campaign:A Whim of Fate, also see Zombies:Introduction single map campaign
Art thread:Mostly frankenstains

User avatar
tekelili
Posts: 1038
Joined: August 19th, 2009, 9:28 pm

Re: Ladder Council

Post by tekelili » February 9th, 2016, 3:33 pm

Aldarisvet wrote:Well, normally the situation that you discribed will just never happen.
That is not a reasonable asumption from a critic position. If we dont know experiment conditions, we should do educated guess about biased enveiroment. Horus is quite skilled player, and in order to get test games he could very well have acepted lot of unskilled players to fight frustation of lack of similar skill oponents. I dont say it happend, I just say is a reasonable doubt araised from lack of information.
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II

User avatar
iceiceice
Developer
Posts: 1056
Joined: August 23rd, 2013, 2:10 am

Re: Ladder Council

Post by iceiceice » February 9th, 2016, 7:44 pm

Aldarisvet wrote: For example, winrate of 58,6% (17 games out of 29) for rebels is can be just a result of the randomness of any sort.
Really, just 17 games of 29, these numbers are not nearly large to be reliable to judge.

I like the analysis, but to judge seriously, you really need at least 1000 games.
It just depends on exactly what conclusions you are looking to draw.
If you are looking to demonstrate "every factional matchup is balanced to +/- 5%, with 95% confidence" or something very scientific, then yes you would need a lot more data. If you flip a coin 100 times, the standard deviation is 10, so observing only 30 heads or as many as 70 heads is not statistically significant at that level, in regards to the assumption that the coin is fair.

However game balance is not generally so scientific in the end. If you only want to collect what information you have and display it nicely in order to have an informed discussion, well Horus has done a bang-up job of that :)

At the least, data like this refutes a hypothesis like "This map is so broken that for a typical matchup, one faction is the 10:1 favorite over the other in the hands of skilled players" -- since all the observed win rates are much more than two standard deviations from 90%.

That said, definitely agree that more data can't hurt :eng:

User avatar
SigurdFireDragon
Developer
Posts: 389
Joined: January 12th, 2011, 2:18 am
Location: Pennsylvania, USA

Re: Ladder Council

Post by SigurdFireDragon » February 10th, 2016, 3:10 am

With Sandbox Map Picker, I noticed that Rime Grotto and Hellhole isn't in any set. And Cynsaun Battlefield is in both 'Adventurous & Conservative' and 'Museum' Also that 'random start time' isn't checked by default. Are these intentional?

User avatar
Cackfiend
Posts: 400
Joined: January 28th, 2007, 7:36 am
Location: Florida, USA
Contact:

Re: Ladder Council

Post by Cackfiend » February 10th, 2016, 7:07 am

tekelili wrote:@Aldarisvet: Your doubts are reasonables, but you are skiping most important factor: Players Skill Difference

No matter how many games are played, even with 1.000.000 games you could have totally biased data. If all those games were played between a very skilled player vs a noob, and they were always picking random faction and side, you would have totally balanced outcomes for factions and sides (about 50% vistory), despite map having some imbalances.

This is why this type of data has always been invalid unfortunately
"There's no love in fear." - Maynard James Keenan

I'm the guy who's responsible for 40% Gliders in all hexes... I can now die a happy man. =D

Velensk
Multiplayer Contributor
Posts: 3979
Joined: January 24th, 2007, 12:56 am

Re: Ladder Council

Post by Velensk » February 10th, 2016, 12:35 pm

Perhaps you're forgetting something, although any individual match data would definitely be biased, big number math does apply to this situation like anything else.

Remember, most skilled players choose 'random' always. Given a sufficiently large set of data, the more skilled and/or more lucky player has 50% odds of playing either side of a faction match-up. If you did throw 1,000,000 games at this kind of analysis, if you didn't see any distinct trends you'd have reason enough to believe that the map is 'balanced enough'.

Granted 100 games is not a lot of data from a statistics perspective but by the standard of information a wesnoth map designer typically gets it's still a massive amount. Also worth noting that if you inspect each individual data point to weight it's worth, you can offset the small amount of data (and it would be relatively impossible even if you somehow got 1,000,000 games to manually inspect and weight each game).
"There are two kinds of old men in the world. The kind who didn't go to war and who say that they should have lived fast died young and left a handsome corpse and the old men who did go to war and who say that there is no such thing as a handsome corpse."

Post Reply