Exercises in Formula and Lua AI and AI-demos add-on feedback

[nelson]

Well, a general counter-recruiting rule of thumb is that if your enemy buys units that are vulnerable to a certain type of damage, you want to buy some units that deal that kind of damage. Here cavalry are vulnerable to pierce, so you buy archers and goblins. (This is not obvious, but you want archers before you get goblins, so that the archers can do the initial damage and the goblins can finish without taking too much retaliation.) That should be easy to program in a general sense, and would be a decent first approximation.

A corollary to that is you should also get some units that exploit lesser "vulnerabilities" (aka lower resistances that aren't actually negative vulnerabilities). For example, say your Loyalist opponent likes buying HI. Sure, you want archers to shoot HI, since it is vulnerable to fire and archers shoot fire. But if you have too many archers you don't have enough shields. Get a troll shield, which deals impact retaliation that the HI won't like, even if the HI still has 10% resistance. (HI are also slow and therefore vulnerable to poison since they can't run off to heal.)

A harder part of programming counter-recruiting is dealing with units that aren't vulnerable to any particular type of damage. An example is the Loyalist Spearman. It has no vulnerabilities, and its resistances are all pretty even. How do you counter-recruit against spearmen? Well, as orcs you can buy units that have more HP for a lower price, namely grunts and trolls, so in a slugging match you're at least even if not ahead. Buying units that have more or equal HP for a lower price is a good move if you can't figure out what else to do. Another consideration is to check if you have units that are resistant to the unit's main attack! You have trolls, which have pierce resistance, so some trolls are a decent response to mass spearmen. (When fighting spearmen, you will also want some assassins, to make spearmen run or die slowly when trapped.)

I dunno, does that give you enough to start with? I would have to think for a while to put this into pseudocode.

[Rigor]

well well, the idea of counter recruits might or moght not work too well for the AI here, why? because the one who has to adapt is usually the one behind. it should be the ai forcing the player to counter recruit not the other way around. a lot of grunts is as we have seen a good idea in all cases. you could extract the information about resists from the enemies units and start building counter units as soon as a certain treshold is reached. i imagine this culd work like this:

normal matchup vs loys, loys build some 2 or 3 horses, thats normal and fine. then, they build only cavs and thats it. the ai must realize now the amount of pierce weakness units is >HP ca. (32 to 45) (*3) ca. = 120, lets say 6 cavs are 200 so its a good time to invest in this kind of damage. check with units dealing pierce in the units attacks, and you should have what you need.

however, orcs dont want to buy this kind of unit forever. their main task is to provide grunts, even if they start mixing in archers and gobos. a treshold of not overrecruiting the supporters would then provide the necessary order: 1/3 considering a healthy troop composition would be the limit, and 0.6 the maximum tolerable amount of counter units at any time on the field (that means, the gobo on the left can be neglected as he might affect this balance.

btw i played a game just now and im very content with the ais developments. if you agree i would consider the experiment successful and move on, to further ideas. what about we slowly detach ourselves from freelands and try to adapt the AI parameters to other maps, such as for instance arcanclave citadel? its a map where you can recruit all over the map and the ai doesnt know anything about recruiting in specific locations yet. one idea i had for a long time was to make a calculation of the keep hex to the desired village and recruit the unit at the closest keep position.

[nelson]

Personally, I would prefer to make the Freelands grunt rush AI dangerously good before we put lots of time into generalizing it. We still have a lot of advanced concepts to teach it, like counter-recruiting, advanced trapping (not just 2 units on either side), avoiding traps, etc.

Sure, Fred is doing well, here's Fred crushing Ron the generic grunt rush:

2p_—_The_Freelands_replay_Fred_crushes_Ron.gz

Fred defeats Ron handily.

(15.85 KiB) Downloaded 203 times

And Fred kills the RCA AI easily in even battles. But he still has much to learn.

Here's an interesting game where I pitted Fred against the RCA AI but gave the RCA AI 150 starting gold and +2 income:

2p_—_The_Freelands_replay_RCA_wins_150+2.gz

RCA wins with 150 gold and +2 income

(27.91 KiB) Downloaded 206 times

Note that Fred does really well at killing the unrelenting attackers, even achieving two lvl3 grunts, but he never reclaims his western villages once they are lost, so even those heroes are eventually overwhelmed. Why doesn't Fred do better at reclaiming villages? At some point he should have bought a unit (ideally a fast unit like a wolf or assassin, but even a grunt or goblin might have worked) and sent it on a village-reclaiming mission.

[taptap]

How does Fred against the MLR (see other thread) and GSoC improved AI?

[nelson]

Taptap, I found info on the MLR at http://wiki.wesnoth.org/Machine_Learning_Recruiter but could you tell me more about the GSoC improved AI and provide links? I have not tested that AI.

[Alarantalara]

How does Fred against the MLR (see other thread) and GSoC improved AI?

As I posted in the other thread (or was it this one), the improvements to recruitment are not enough to stop Fred from capturing villages and eventually winning through numerical superiority. This should include any changes to play as a result of GSoC, unless they are currently inactive.

[nelson]

What other thread are you talking about, folks?

[Alarantalara]

This one: http://forums.wesnoth.org/viewtopic.php?f=10&t=36642
The Wesnoth binary needs to be changed to support it, so it hasn't been explored to quite the same extent as Fred.

Here's a sample game against the first version of the MLR. You'll notice that the MLR does a good job of counter recruiting, but the superior village capturing wins out.

2p_—_The_Freelands_Fred_vs_MLR.gz

(21.74 KiB) Downloaded 209 times

Incidentally, I'd like to question a move by Fred on turn 7 of that game. A poisoned archer is adjacent to a village Fred controls, but Fred elects to stay with a grunt in another village rather than move to the adjacent village and attack. Fred only attacks after a village is recaptured by that same unit and at a less favourable time of day.

Edit: nelson, most of the improvements to the AI were to improve support for AIs like Fred. There was also a change to recruitment separate from the one in the thread above. I believe you can test that by playing against the AI in 1.11.0.
http://wiki.wesnoth.org/GSoC_Akihara_Proposal
http://wiki.wesnoth.org/SoC2012_Nephro

[mattsc]

Just a few quick notes for now:

I dunno, does that give you enough to start with? I would have to think for a while to put this into pseudocode.

Yeah, me too. That's why I'd like you to do it so that I can work on some of the other aspects. Actually, it doesn't even have to be pseudo-code, but a couple equations or so would be nice. Something like "if the opponent has X units of type Y, make sure that you have at least Z units of type R" (along the lines of what Rigor wrote, but even more specific), rather than "if the enemy has lots of skeletons, we should think about recruiting some spearmen". Either way, if you have time for it, great. If not, I'll get to it at some point.

Rigor, nelson: I'm kind of torn between moving on and "finishing" Fred first. It would be nice to do something different, but at the same time there are lots of things I still want to do with the Freelands Grunt Rush. I'd also like to do a bit of work on a couple "micro AI's" (and a mechanism for the scenario designer to dial those easily into scenarios with new WML tags). I think I want to keep going with Fred for a little, but then maybe start phasing in some other work as well. I need to consolidate my diverse lists again...

How does Fred against the MLR (see other thread) and GSoC improved AI?

Alarantalara has really answered that already, but just to elaborate a little: Fred does a lot more than different recruiting, so any AI that "only" changes the recruitment pattern but is otherwise the standard RCA AI doesn't stand a chance on the Freelands map. So that includes the MLR and probably Akihara's new recruiting code (I really don't know anything about the latter). Having said that, it would be nice to combine the ML recruiter with the rest of Fred's CA's at some point. I'd expect that to be better than either by itself. It will be interesting to try this sometime, but this is a BfW 1.11 thing (as the MLR uses trunk).

As for Nephro's project, that is almost entirely improvements to the Lua AI, that is, the tools used to write custom AI's. They are not available for Fred yet as they are also mostly for BfW 1.11, but (with a few minor exceptions) they wouldn't change how Fred performs, only how easy it is to code that behavior.

As I posted in the other thread (or was it this one), the improvements to recruitment are not enough to stop Fred ...

It was in an direct chat with me, actually, so I believe this is the first time this has been said in one of these threads.

Besides that, I haven't even looked at the replays yet you all have posted today...

[nelson]

I'm seeing some less-than-ideal behavior from the latest Github version now. Here's me killing Fred while taking zero losses:

2p_—_The_Freelands_replay_Loys_kill_Fred.gz

No losses!

(22.19 KiB) Downloaded 234 times

4. It may be correct for Fred to attack with the lone archer, but everything else should also be rushing to the front as quickly as they can, not hanging back on good terrain while leaving that poor archer hanging out to dry! Fred is a little unlucky here.

5. Fred here should either use 3 grunts on the strong, resilient cav on the left or 2 grunts + assassin on the weak cav on the right to try to get a kill, whichever has the higher ctk. Since grunt attacks deal 8 dmg to cavalry, Fred needs 5/6 grunt hits on the strong, resil cav to kill it. That's a non-zero chance! If he is only going to poison that cav, he should just poison it and use his units to attack something else. The cav on the right would die to 4/4 grunt hits plus 1/3 poison darts. Alternately, Fred could try using two grunts on the fencer, there's a ctk there too. What Fred should definitely not do is attack the village spearman with only one unit when he does not have a ctk and it will heal off the damage.

I would like to propose that the AI should avoid attacking units once they have been poisoned (unless they are going to heal it off e.g. in a village) and focus on attacking other units once the poison has taken. The poisoned unit will either have to retreat to heal or suffer a significant chance of dying in the next couple of turns. If a unit is poisoned or poisoned and attacked, the end result will usually be the same, the unit retreating to heal. If it can retreat in such a way that you can't catch it, then there is no practical difference between these two results.

6. Fred gets really unlucky this turn, sorry Good moving the assassin down the middle to steal a village, although a grunt might be better for stealing purposes since it can probably survive on the village longer. On the other hand, the assassin is guaranteed 6 moves which allows it to take another village on turn 8. But then the assassin dies in one turn, so I still think a grunt would be better, if it is a quick grunt it could make the same moves.

7. Still attacking on the right at dawn, dammit.

11. Why did all of the grunts suicide on the almost invulnerable Swordsman on a village? Try to kill the fencer if you're going to attack, there should be a ctk there. (A better move might be to hang back until you've built up enough units for a serious killing force again.) And on the left, why didn't the full-health grunt try to kill the injured cavalryman to let his mortally wounded comrade escape (to grab another village and die with honor) or set up the dying grunt for a finishing move? The goblin could move up to support this attack. Instead, both units flee, at first watch! If you can't win at 1st watch you can't win at all, attack!

12. 13-0 in kills, this game is over.

[tuggyne]

Something unpleasant happens if I try to run Ron against RCA on Thousand Stings Garrison. Specifically, Wesnoth hangs at full CPU indefinitely. This happens on 1.10.3 and 1.10.4; I don't know whether Fred has the same behavior yet, or the Elvish Grunt Rush (whom I dub Edgar).

[mattsc]

Something unpleasant happens if I try to run Ron against RCA on Thousand Stings Garrison. Specifically, Wesnoth hangs at full CPU indefinitely. This happens on 1.10.3 and 1.10.4

tuggyne: Interesting. I believe that the AI code considers all the petrified scorpions as viable enemies (it just checks for enemies, not for "attackable" enemies), making the number of possible attacks way too large to handle. I'll take care of that in an upcoming version.

All: are there other instances like this where I need to disregard a large number of units on a map for another reason than petrification?

I don't know whether Fred has the same behavior yet

No need to try, it will be even worse for Fred.

or the Elvish Grunt Rush (whom I dub Edgar)

Ok, Ed it is. (The AI's won't be allowed to use two-syllable names until they can beat human players with two-syllable names more consistently than once every 100 games.)

nelson: I'll check out why the current github version does so badly later today.

[mattsc]

Comments on the replays:

nelson: All those attacks happening from dawn to afternoon on the right are not done by the rush_right CA, but by the poisoner attack, attacking enemies with high CTK, etc. CA's. You can see that when, e.g., the assassin and his support unit make the attack - and then everybody around them retreats. Which makes it even more stupid, of course, because now the poor assassin and support grunt are left in the middle of the enemies all by themselves... So, Action Item #1: make sure that poisoners don't do this. I am not equally as sure about it for an attack with a likely kill, but leaving a unit out there all alone is not good for that either. (NB: in the end, the different CA's will, of course, use as much common code as possible, but currently things are still too experimental for that to work.)

As for choosing the wrong attack targets (such as the Swordsman on the village), that's because the evaluation formulas for those attack combinations are still pretty new and not much tested. They give very high priority to taking that village - too high a priority, as is obvious from that example. AI #2: refine those eval functions.

AI #3: advance units not able to attack themselves more aggressively from dusk to second watch. They currently still try to stay on favorable terrain, even then.

As for why Fred did not try to take the western villages back in the game against the over-resourced RCA AI, that's because attacking immediate threats to the leader has currently a higher priority than village grabbing. In general that's correct, I think, but there should be some exceptions, I guess: AI #4.

Not attacking already poisoned units is something I've been thinking about myself. AI #5.

In general, the behavior in the west, and in particular in the SW, is still lagging behind the rest of the AI. For example, that wolf moving next to an occupied village in that replay a while ago was a unit not dealt with by any of Fred's CA's and left to the RCA AI. (Why the RCA AI thought that was a good move is beyond me, but ...) I'll work on that after I have figured out how to deal with things in the east.

Alarantalara: the wrong village grab you pointed out was indeed a bad move. AI #6: refine village grabbing evaluation to take situations like that into account.

Looks like I have a few things left to do...

[Alarantalara]

One option for counter recruiting would be to start with the more powerful counters.
For example, recruit an archer if the number of units with pierce or fire vulnerability and no ranged attack is greater than twice the number of archers/crossbowmen.

[mattsc]

One option for counter recruiting would be to start with the more powerful counters.
For example, recruit an archer if the number of units with pierce or fire vulnerability and no ranged attack is greater than twice the number of archers/crossbowmen.

That sounds reasonable.

Besides that, v0.10.4 is out, featuring a lot of small-ish changes (see changelog below). In particular, I added a bunch of TOD-dependent behavior differences. Most of those should be real, even if incremental improvements, but I am not quite sure about all of them. Most importantly (among those I am not certain about), I have lowered the priority of spreading poison at night, so that more units are available for the rush on the right. I have not yet done enough testing to know whether that's a net improvement or not, so comments on it would be particularly valuable.

Note that I have not done anything specific about the behavior on the left yet, although some of the changes of course have an effect anywhere on the map.

Code: Select all

0.10.4, 30 Aug 2012
- Freelands Side 1 Grunt Rush:
  - Changed day-time behavior (retreat) of rush on right:
    - Now set never to attack from dawn to afternoon
  - Poison attacks and attacks on weak enemies also have TOD-dependent threshold, but are not quite as exclusive
  - In addition, poison attacks have been de-prioritized at night, to free up units for rush on right
  - Poisoner can now move next to AI unit that has attacked already (without need for additional support unit)
  - Combination attack evaluation on right was tuned (hopefully for the better)
  - Units that are out of reach for attacking on right advance more aggressively at night
  - Small change to recruiting: archers have somewhat higher priority now
  - Removed two unused CA dinosaurs
- All grunt rushes:
  - Added the AI's names (Fred, Ron, Ed) to the pop-up menu descriptions :D
  - Added a WML variable containing the AI-Demos version number for debugging
- ai_helper:
  - New function ai_helper.print_eval() to help with debugging