Quantifying how good/bad units are per-level. (UNFINISHED)

General feedback and discussion of the game.

Moderator: Forum Moderators

Post Reply
SmashingSuccess
Posts: 13
Joined: April 12th, 2018, 5:25 am

Quantifying how good/bad units are per-level. (UNFINISHED)

Post by SmashingSuccess »

As a game designer, I've always been interested in how to quantify how good/bad units are based on their level in this game. To get into this subject, there are a LOT of things that need to broached with a lot of care and tact. As such, I encourage anyone who notices anything I missed or has any questions to voice them.
Method
Every aspect of a unit is to be tallied and added to a total via points. Any negative aspects will take away from these points depending on how it negatively impacts the unit. The goal is to come as close as possible to the general consensus of unit balance while introducing as little bias as possible.

Terrain
Terrain is an integral part of Wesnoth, as it determines a great deal of unit's abilities as well as their defenses and even where they can move. For the purposes of this discussion, I am electing to completely disregard campaign scenarios and focus on a generalized unit value, as in certain scenarios, certain units shine particularly bright where they might not otherwise due to terrain. Fog-of-War and Shroud are ignored for the purposes of this examination, but abilities that alter a foe's ability to see units(such as Nightstalk and Skirmisher) will be factored in.

Defenses - We all know that terrain serves to grant percentage based defenses to units. Due to this fact, units will have differing levels of worth on different tile types(a dwarf in water vs a dwarf in the mountains isn't exactly a fair comparison). These differing scores will be collected and averaged together toward the final unit's score. The per-tile calculation will be the 2nd to last calculation made, as it is a collation of values itself, and the final score is a collation of collations.

Defenses act more as damage reduction through the means of altering hit chances. For instance, over time, a 40% reduction to hit would lead to an overall 40% damage reduction. However, due to the nature of how combat functions in wesnoth, lowering chance to hit on attacks with few hits has far more impact than lowering that same chance to hit on an attack with a large number of hits. This means that individual combat rounds can be substantially altered by a change in defenses.(Not even bringing up attack effects like poison or slow)

Example
25% chance to hit
10-4 Attack VS 40-1 Attack
Round 1
10 damage VS 0 Damage
10 damage VS 0 Damage
20 damage VS 0 Damage
0 damage VS 40 Damage

The average damage between these two attacks is identical over an infinite amount of hypothetical time, but the 10-4 is definitely the better attack because it has the higher chance of dealing SOME damage in a given combat round, and, as such, is more useful when handling multiple units at a time(IE all of the time) and has a higher overall chance of proc'ing attack effects. This is due to how probability works, the more times you roll the dice on the same probability, the more likely that outcome is to happen eventually. That 10-4 attack made 16 attempts at that 25% chance, while the 40-1 only made 4. That's 16 times to hit someone with poison. That's 16 possible killing blows.

You can calculate the percent chances that at least one hit will land from a number of identical attacks with this formula:
(1-P[percentage to miss])^N[number of attempts made] * 100 = Chance to miss every hit
in this case, it'd be (0.75^16)*100 ~= 1% (~= means approximately equals. It's actually closer to 1.0022 %)





Tile Movement Costs -





Crap, I just realized I spent like 20 minutes writing this out when I should have been working. I haven't even properly gotten into the rating system yet or the units themselves. I'll leave this bit here for now and add more to it when I get the time, although I feel I could easily write a small book about the balance aspects of this game and a lot of other strategy games, although at that point it'd be getting closer to game theory than anything.
enclave
Posts: 936
Joined: December 15th, 2007, 8:52 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by enclave »

SmashingSuccess wrote: August 11th, 2018, 3:28 pm The average damage between these two attacks is identical over an infinite amount of hypothetical time, but the 10-4 is definitely the better attack because it has the higher chance of dealing SOME damage in a given combat round, and, as such, is more useful when handling multiple units at a time(IE all of the time) and has a higher overall chance of proc'ing attack effects. This is due to how probability works, the more times you roll the dice on the same probability, the more likely that outcome is to happen eventually. That 10-4 attack made 16 attempts at that 25% chance, while the 40-1 only made 4. That's 16 times to hit someone with poison. That's 16 possible killing blows.
Well if 4 units (4-10) attack a unit (1-40) there is also a chance of this to happen:
Round 1
10 damage VS 40 Damage
10 damage VS 0 Damage
0 damage VS 0 Damage
20 damage VS 0 Damage
Which would lead to a murder of a 40hp unit... so saying that 4-10 is better than 1-40 is a bit strange :) although a nice phylosophy.
Not even mentioning the fact that if 1-40 unit attacks and hits, then he will not get any retaliation from 40hp 4-10 unit.. so how is 4-10 better I don't know.. :D again, a nice phylosophy... the only thing that is not phylosophy here is whether u prefer to risk or not.. I prefer to have a 4-10 unit so I can be confident of hitting at least once and dealing at least some damage.. rather then 1 unit who can kill instantly but u have no idea when it happens..
But comparing 4 units vs 1 is phylosophy :D I don't think it's mathematics :D

PS. sorry to disappoint you but I have no idea how you going to turn wesnoth units into score numbers :D
For example:
A thunderer 1-18 ranged 2-6 melee, vs poacher 4-3? and 2-2? (something like that) which is much weaker coz thunderer also has resistances.. Who has bigger score? :) Hmm... will score be calculated when poacher is in neutral time of day state? :) What is better, a neutral unit or unit with elignment? :) would be a nice phylosophy too...
The thing about these 2 units is that poacher has very low experience to level.. 1+ kills and he is like 6-4 4-4 or something.. which is 3+ for thunderer... Is thunderer better to hold captured villages? should poacher be used only when you low on gold and have enemy near swamps? :D U can't compare units like this... They all are recruited for certain situations... it's not civilization where everything is straightforward... there are million of reasons why somebody would prefer one weaker unit over another much stronger one...
I would rather say that developers of wesnoth just made up the prices and attacks without trying to turn it all into unit score and balance it this way.. its not doable.. balancing probably all happend later by trial/error (like.. "hmm.. looks like drakes too strong vs undead, lets try make adept hit 2-10 cold and drake vulnerable to cold 50%? y lets try".. and it worked..)
Is assasin poison better against ghoul poison? If assasin costs 18g and ghoul is 16g? For some reason i have no doubt that 1v1 ghoul would win, strange :)
Why is assasin more expensive then? :D
pause.
SmashingSuccess
Posts: 13
Joined: April 12th, 2018, 5:25 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by SmashingSuccess »

enclave wrote: August 11th, 2018, 6:06 pm Well if 4 units (4-10) attack a unit (1-40) there is also a chance of this to happen:
Round 1
10 damage VS 40 Damage
10 damage VS 0 Damage
0 damage VS 0 Damage
20 damage VS 0 Damage
Which would lead to a murder of a 40hp unit... so saying that 4-10 is better than 1-40 is a bit strange although a nice phylosophy.
To answer this point, lets put it into more drawn out terms to explain it a bit better.

You have 4 battles, and in each one, 4 rounds of combat, setup so that the 40th damage occurs once per battle in the 1st, 2nd, 3rd, and 4th combat rounds. These battles will each have 4 units on each side, each hitting exactly the same way for the purposes of this.

Battle 1
10 damage VS 40 Damage
10 damage VS 0 Damage
0 damage VS 0 Damage
20 damage VS 0 Damage
Outcome
The best possible outcome for the 40-1 side, and also the most unlikely. 4 units hitting for 40 kills the entire opposition in one swoop if they go first. If the 10-4 side goes first, all the attacks are focused on one side and the 40 side kills only 3 units since one of their dies. After this, the 40-1 side loses due to their units being slowly whittled down over the remaining turns.
40-1 wins one.
10-4 wins one.


Battle 2
10 damage VS 0 Damage
10 damage VS 40 Damage
0 damage VS 0 Damage
20 damage VS 0 Damage
Outcome
First round 40-1 side loses 1 member regardless of whether they go first, and even after the second round, they only kill 3 units if they go first, and 2 if they don't. 10-4 side will kill either 1 or 3 units, depending on whether they go first.
40-1 wins two.
10-4 wins two.

Battle 3
10 damage VS 0 Damage
10 damage VS 0 Damage
0 damage VS 40 Damage
20 damage VS 0 Damage
Outcome
This is where things start to go sour for the 40-1 side. They lose two units without the ability to retaliate, making it so that they cannot kill the opposition. For both sides going first, 10-4 wins.
40-1 wins two.
10-4 wins four.

Battle 4
10 damage VS 0 Damage
10 damage VS 0 Damage
0 damage VS 0 Damage
20 damage VS 40 Damage
Outcome
The worst possible outcome for the 40-1 side. If they go first, they have the same effect as last round, with 1 member remaining against 2 on the 10-4 side. If they go second, it is a complete defeat, and they all die without being able to do anything back.
40-1 wins two.
10-4 wins six.

Out of the 8 total battle situations(including differing turn order), 10-4 still comes out on top overall. In most situations, 10-4 will come out as being the better unit overall. I feel like your post was being heinously condescending without much evidence to support your claim, and I'd ask that you consider all possible situations before pointing out that I am wrong. Mathematically, having a unit with a 10-4 attack is better in almost all situations than having a unit with a 40-1 attack. It's not about preference. There are very few edge cases where it would matter, such as if you were fighting a berserker unit.

That being said, you do bring up a good point regarding experience points and how easy units are to level versus others, so it might be pertinent for me to go into detail on that as well. As stated in the title, this examination is unfinished, and I look forward to putting more time towards it.
enclave
Posts: 936
Joined: December 15th, 2007, 8:52 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by enclave »

SmashingSuccess wrote: August 12th, 2018, 6:17 am Battle 3
10 damage VS 0 Damage
10 damage VS 0 Damage
0 damage VS 40 Damage
20 damage VS 0 Damage
Outcome
This is where things start to go sour for the 40-1 side. They lose two units without the ability to retaliate, making it so that they cannot kill the opposition. For both sides going first, 10-4 wins.
40-1 wins two.
10-4 wins four.
How in the world 10-4 suddenly win 4? :DDDDDD There is no point to go further into this conversation, you show me some kind of magic here, I don't see any real numbers, "things start to go sour for the 40-1 side" :DDDDD Jesus. All i see from what you are showing is that:
10 damage VS 0 Damage
10 damage VS 0 Damage
0 damage VS 40 Damage (here 1-40 kills a unit of 4-10)
20 damage VS 0 Damage (here 4-10 kills a unit of 1-40)
So 1 lost for each side. What bloody magical wins 4 I have no idea, which dimension you come from :D
Don't even reply, I'm not going back here :D Have fun numberizing mr "game designer".
SmashingSuccess
Posts: 13
Joined: April 12th, 2018, 5:25 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by SmashingSuccess »

Good to see someone doesn't know how to math.

First of all, you switch the numbers around on the attacks to make it look like I was incorrect where I wasn't. Second, let's examine your ineptitude further. First round of battle 3:
4 units hit for 10 damage, killing one of the 40-1 units. 40-1 side completely misses, dealing no damage. That leaves 4 remaining 10-4 and 3 remaining 40-1. This is identical no matter the turn order.
Second round of battle 3:
4 units hit for 10 damage, killing another of the 40-1 units. 40-1 side completely misses AGAIN, dealing no damage. That leaves 4 remaining 10-4 and 2 remaining 40-1. This is identical no matter the turn order.
Third round of battle 3:
10-4 goes first
4 units hit for 10, killing another of the 40-1 units. This leaves the 40-1 side with 1 unit left, which attack, and kill only 1 unit. The only unit to be killed this battle. Since no more hits are made by the 40-1 side, they lose in this scenario.
40-1 goes first
2 units hit for 40, killing two 10-4 units. This leaves the 40-1 side with 2 units left, and 2 remaining on the 10-4 side. The 10-4 side then attacks, dealing 20 damage to one of the units and ending the round.
Fourth round of battle 3:
10-4 units hit for 20 total damage, killing one 40-1 unit and wounding the last one. This leaves two 10-4 units remaining and one half dead 40-1 unit remaining, meaning they lose in this scenario. AGAIN.

Spoiler:
This bit was due to the fact that I was tallying up all the scenarios individually. This means all 8 battle situations. The 4 battles, times 2 for the starting position changing between the two sides. I tallied them all up, and spoiler alert, in 6 out of 8 situations, 10-4 is better. I really fail to see why you can't grasp this, and also why you seem so hostile toward me, almost as if you have a kind of grudge against the whole idea of codifying unit balance. This is not an opinion. This is not 'phylosophy'. It is a fact.

I don't care if you reply to me or not at the point, your messages simply being their serves to halt all discussion, and I'd rather remove them for the [censored] stains they are than leave them there. Since I don't have that ability, I'm kinda stuck. Thanks for that.
User avatar
Pentarctagon
Project Manager
Posts: 5496
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by Pentarctagon »

enclave wrote: August 11th, 2018, 6:06 pm Is assasin poison better against ghoul poison? If assasin costs 18g and ghoul is 16g? For some reason i have no doubt that 1v1 ghoul would win, strange :)
Why is assasin more expensive then? :D
That's a silly match up in the first place, since Ghouls can't be poisoned.

I'd also like to remind everyone that:
a) Disagreeing or not understanding someone's point is no reason to be rude.
b) You don't have to reply to people who are being rude - there is a button to report posts if necessary.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
Caladbolg
Posts: 198
Joined: January 1st, 2016, 4:40 pm
Location: Hopelessly trapped within the Submachine

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by Caladbolg »

Nice analysis on the face of it, but I'm not really convinced it is representative of how things would play out in the actual game. For example, your scenarios assume that it's possible for 4 units of one side to gang up on a single enemy which is not the case in the majority of situations due to ZoC. You also did it only for the 10-10-0-20 distribution of damage for the 10-4 side. Generally, putting the weight on the latter end (for example in 0-10-10-20) would make the situation more favorable for the 1-40 side. Though I acknowledge that distributions like 0-0-0-40 on the 10-4 side are rarer so that's not that big of an issue.

Another thing to mention is that you're working with 25% chance to hit, which (ignoring that it's not a multiple of 10 as in-game cth) is really low. Even when attacking elves in forest you have 30% cth. Usually, good terrain provides 60% defense, giving you 40% cth. And if you consider that plains are the most "default" terrain and units there generally have 40% def, you'd have 60% cth.

So let's consider two units, one with 10-4, and one with 1-40 attack, both with 31 hp, ignoring resistances and standing on plain hexes (60% cth), as it commonly happens. The 1-40 unit has 60% cth and deal 40 dmg, which kills the other unit. On the other hand, for the 4-10 unit to kill the 31 hp enemy, it would need to hit all 4 times, which happens (60%)^4~=12.96% of the time. So the 40-1 unit is almost 5 times as likely to kill the enemy in comparison to the 10-4 unit. That's a pretty considerable difference. In fact, the difference in favor of the 40-1 unit is largest when the chance to hit 60% and that's not too uncommon. And in any case, the 40-1 unit will have a greater chance to kill regardless of the cth.

To follow up on that, another thing to consider is the breaking power. Aside from really frail healers or really tough guards, level 1 units have hp in that 31-40 range. Those above 40 won't be considered as neither 10-4 nor 40-1 unit would be able to kill them. For those level 1 units in the 21-30 hp range, the 10-4 unit would need to hit 3 times, and generally the chance to kill will still be lesser than for 40-1 unit. In other words, a 40-1 unit always has a greater chance to kill a level 1 unit. This entails that it also has a greater capacity to deprive the enemy of healing (as it has a greater chance to kill a healer), of leadership, of skirmirsher units, etc. Ignoring units with abilities, it also means a greater chance to break enemy Zones of Control, allowing other units a chance to strike at vulnerable parts of the formation.

In short, whether 40-1 or 10-4 is preferable is situation-dependent to a much higher degree than it might seem on a first glance. While 10-4 will always give you a greater chance to deal some damage, the 40-1 will always give you a greater or equal chance to kill (equal in cases where the enemy has less than 10 hp). And killing is often preferable, especially if you are killing a key unit (the one with a strategically important ability or a strategically important position in the formation, etc.). I certainly enjoy reading these sorts of analyses so I encourage you to continue, but I believe that the values of each individual stat are usually highly context-dependent so I'm doubtful you'll succeed in your desired quantification. Still, I'm open to being surprised :)
SmashingSuccess
Posts: 13
Joined: April 12th, 2018, 5:25 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by SmashingSuccess »

Ahhh yes, finally some solid criticism with thought put into it.

To address your first point, I will admit, the spread of the 10-10-0-20 was kinda random on my part. I was mostly doing it as a response to enclave's post and I needed something fast and effective to prove my point. Seems it failed in the later, but the gist is essentially not wanting to write out every possible permutation of odds for each scenario, and as such I used a generalized one. There are definite failings with it, and I would not fully stand by it for a complete analysis, which will be going in the final post when I'm done. This is also true of your second point, as the 25% was merely for demonstrative purposes.

Your third point is where things get interesting. Two things need to be brought up:
A) This discussion was originally about how to codify attacks such that they could be quantified in relation to defense values. Changing the situation definitely changes how things work, but the attacks will always be the same(not counting leadership and alignment, but that's a whole nother can of worms).
B) This is definitely true, the higher your chances of hitting, the closer a 40-1 and a 10-4 attack(or a 1-40 for that matter) come to being identical. At 99% chance to hit on both sides(of two combat rounds)comes out to only having an overall 0.2% difference in chance to kill.

Your fourth point kinda gets into territory that I can't really get into heavily yet. The 40 overall damage number I came up with was not meant to be based around the Wesnoth units, it was again merely to demonstrate the point. I also implore you to explore the inverse situation, where two units with 40-1 and 10-4 attacks have 41 hit points apiece. Then, things change dramatically, and chance to hit becomes monumentally important.



A great deal of what you have brought up is stuff I haven't touched on yet, but am looking forward to. Zone of Control is an interesting case, as are attack effects like Charge and Poison. The final goal is to find a way of ranking the units by how powerful they are individually, without taking into consideration large amounts of strategy and planning. Iron Maulers and Rogues may be the best tactic since Hannibal's Encirclement, but the point even then, judging that isn't quite the goal of this exercise. Some bits will require it like Nightstalk(god this one is gonna be massively complicated to get into along with its effects in relation to alignment *shudder*). Still, I'm gonna give it my best shot as I go along. I look forward to hearing more from the community as I go along. :)
User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by Sapient »

You may be interested in some of these past attempts, and seeing why they failed, before getting too invested in this project:

viewtopic.php?f=12&t=26558
viewtopic.php?f=12&t=46028
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
SmashingSuccess
Posts: 13
Joined: April 12th, 2018, 5:25 am

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by SmashingSuccess »

Hmmmm.....seems like the first one was trying to build a campaign, and was trying to make his units balanced for his scenarios, so that's not really applicable.

The second one however is more interesting. It seems like he was trying to judge his unit balance based on his own testing rather than the community's, and his testing method was flawed in a number of ways. My method seeks to achieve the opposite, create a system that can(as accurately as possible), judge the value of units to the point where it can predict how good or bad the community might see a unit if it were released today, sans any other changes. It's why I don't just completely ignore terrain values, as every map is different. I plan to give units different scores based on what tile they are on so that players act accordingly.

It's making a model to fit the data that's already available instead of making new data and making justifications based on that premise. That being said, I have my own strategy game to work on, so I won't have a lot of time to update this as much as I'd like to, so it might be better to just to let it go. I'll think on that and come to a decision eventually.
User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by Sapient »

SmashingSuccess wrote: August 13th, 2018, 6:48 am Hmmmm.....seems like the first one was trying to build a campaign, and was trying to make his units balanced for his scenarios, so that's not really applicable.
Not true. Although he eventually came to that realization by the end of the thread, in the beginning his ambition was much the same as what you are attempting. Thus, all the comments by Velensk, myself, and others in that thread apply equally to your endeavor.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
otzenpunk
Posts: 104
Joined: February 11th, 2018, 5:32 pm
Location: Hamburg / Germany

Re: Quantifying how good/bad units are per-level. (UNFINISHED)

Post by otzenpunk »

SmashingSuccess wrote: August 12th, 2018, 6:17 am You have 4 battles, and in each one, 4 rounds of combat, setup so that the 40th damage occurs once per battle in the 1st, 2nd, 3rd, and 4th combat rounds. These battles will each have 4 units on each side, each hitting exactly the same way for the purposes of this.
I guess this is a flawed argument. This is not a very realistic assumption of what typical battles look like. Neither those four 40-1ers all hitting in the first round, nor all in the last is a very probable case. Normally, there would be a more regular distribution of the 40-1ers succeeding about 0–2 times a turn, and the 10-4ers accumulating about 40 damage points per turn as well, with a probable distribution from 20 to 60. The expected damage points are exactly the same (40) in both cases, though! It's just, that the 40-1ers have much higher chances to over- or underperform in a significant way, while 10-4 most likely gives mediocre results.

Neither of them are generally advantageous than the other. It's just, that you have to adapt your strategy accordingly. If you send three 10-4ers after a single unit, for example, you can kind of reliably count on doing enough damage either to kill it or at least deal significant damage and force it to retreat for healing. With three 40-1ers, of course you'd better have a backup plan, in case you miss three times in a row and have your units standing in the open for retaliation. On the other hand, there is a good chance, that you kill this unit with the first strike and still have your other two units at hand to attack further opponents or secure the area or whatever.

Btw I don't think, that there could be a single one-dimensional metric for the strength of units, really, because BfW's battle system is so multi-dimensional with terrain modifiers, daylight times, special abilities, etc. It's like trying to translate a multi-dimensional graph into a single number line. You'll always lose information during the process.

And that units often profit from abilities of other units, doesn't help either. Dwarf units, for example, are often very strong and resilient compared to other units. But that's countered by the fact, that they normally don't have easy access to healers or other special abilities. Elvish fighters are comparatively weak, but elves do have healers, have slow, can easily develop leadership units, etc., and certainly fighter units profit from that. How do you want to fit this into your metric?
Post Reply