Exercises in Formula and Lua AI and AI-demos add-on feedback

[Rigor]

i emphasize the word slowly, no need to rush but mark it on the horizon already. heres a game vs a very good fred, i mean, more careless players would do one mistake and lose the game in no time, i played a bit sloppily but it turned out alright for me. the ai is very good at taking villages already. one thing that popped in my mind was that you could give fast scout like units the order to, if possible, place themselves in a fork style, so they can threaten to take one or the other village if its left empty. an example would be a wolf who threatens to take the central or the eastern village. or the south western or the western village.

i like the defensive behaviour of both ron and fred now. its not perfect but it did the right thing, at the right time of day, and i gambled that my precious units would make it. lets make it clear you will not have an ai that will ever beat humans who play the game for a very long time because including trickery is beyond the scope not talking about evaluating key units that absolutely must die no matter what. when its alright at beating players regularly, then its the point to say thanks and find a new situation to play and tweak around, right? is one month too short a time or do you have different plans?

[mattsc]

Thanks, Rigor. Yeah, I do quite like what I am seeing, if I do say so myself. Thanks for posting it, although there's also still obvious room for improvement.

lets make it clear you will not have an ai that will ever beat humans who play the game for a very long time

It's funny, pretty early along somebody (it might have even been nelson) asked me (somewhat tongue in cheek, I presume) whether we'll ever have an AI that would beat him consistently. My answer was a single two-letter word, no ifs and buts needed.

when its alright at beating players regularly, then its the point to say thanks and find a new situation to play and tweak around, right? is one month too short a time or do you have different plans?

I agree. I don't want to put a time estimate on it though, because things always take longer than expected and it depends too much on other parts of life. There are some things that I really still want to do, like taking XP and leveling up into account for all attacks, because they are so universal that moving on now would just mean I'll have to do it twice in the end. But anyway, I'd say let's start talking about it now, so that whenever we're ready, we can move ahead. In particular, getting started on the things that are different from the grunt rushes might be a nice diversion, I just want to avoid duplicating too much work. And, of course, I do want to continue making Fred better, but it doesn't have to be 100% of Wesnoth time and whatever we learn from other situations might quite well help with the grunt rushes.

So what do you have in mind?

I'll think about how to do that scout/forking thing, that's a good idea and (probably) not super hard.

[taptap]

Regarding poison, I don't even know whether Fred and friends are doing this - I read this thread just to keep up with this exciting work you do here, one point is lower priority on attacking poisoned units, another is a very high priority on avoiding multiple poison attacks on an already poisoned unit, which is a quite common campaign AI waste of resources. It is always a big help when all those annoying assassins gang up on a single spearman when they could poison and neutralize half your army.

I wonder whether on the horizon there is a possibility for a campaign with less overwhelming enemy numbers and more MP-alike gameplay (and 2 gold per village), but against a toughened customized AI (that still might receive some handicap, but not the ridiculous amount it nowadays often need.)

[Alarantalara]

I wonder whether on the horizon there is a possibility for a campaign with less overwhelming enemy numbers and more MP-alike gameplay (and 2 gold per village), but against a toughened customized AI (that still might receive some handicap, but not the ridiculous amount it nowadays often need.)

Is 50 extra gold a low enough handicap? That approximately represents the current point where I can no longer beat Fred.

[mattsc]

Regarding poison, I don't even know whether Fred and friends are doing this - I read this thread just to keep up with this exciting work you do here, one point is lower priority on attacking poisoned units, another is a very high priority on avoiding multiple poison attacks on an already poisoned unit, which is a quite common campaign AI waste of resources. It is always a big help when all those annoying assassins gang up on a single spearman when they could poison and neutralize half your army.

For the most part, the grunt rush AI's don't do that. There's a candidate action made for the specific purpose of spreading poison to as many enemies as possible. That CA flat out refuses to attack units that are already poisoned or would level up due to the attack. It also strongly disfavors units that regenerate or are on villages. However, other CAs might still use assassins on an already poisoned enemy for other purposes (such as to add another unit to a gang-up attack or to finish off a unit). In that respect, Ron is actually better than Fred, because he doesn't do all that extra stuff.

I wonder whether on the horizon there is a possibility for a campaign with less overwhelming enemy numbers and more MP-alike gameplay (and 2 gold per village), but against a toughened customized AI (that still might receive some handicap, but not the ridiculous amount it nowadays often need.)

That's hard to say, really, but for my own selfish purposes, I am planning to use some of these AIs when I'll get back to working on Grnk Part 2. In fact, I wrote a couple of the custom AIs for that specific purpose. As you might expect, there'll be some surprises there again for the uninitiated and this time not just from the story line perspective ("Come on, why can't I take down those 10 lousy soldiers behind the pass with my 20 orcs?!)

[nelson]

In a sort of response to Rigor, let me give a broad overview of where I would like to take Fred before I consider him "done":

1. Make him the best AI we can within his current limitations. I'll consider this step done when there are no obvious exploits in his gameplay (it's fine if humans beat him, but it shouldn't be easy/straightforward because Fred keeps making the same mistakes) and we've taught Fred all of the fundamental concepts like avoiding traps.
2. Teach Fred to kill Undead.
3. Teach Fred to play Freelands as side 2.
4. Teach Fred all of the other maps in RBY.

Then we can move on to giving Fred siblings who can play as the other races of Wesnoth Despite the existence of Edgar, I would probably target Undead or Loyalists first, since it is usually obvious what TOD they should be retreating and attacking at. It's more complicated to figure out the optimal TOD for races with more of a mix of lawful/neutral chaotic units.

Does this sound reasonable? My goal is to someday have an AI that can defeat casual humans with any race on any map, but I'd settle for a highly lethal orc AI just show how it should be done

[tuggyne]

Just ran an interesting game between Ron and Ed in Den of Onis; several mistakes were made on both sides, but Ron still won handily. One thing visible throughout is the poor placement of recruits, almost always several hexes back from the tip of the castle; this is an artifact of the map, but Den of Onis is not unique in having this castle structure, so the AIs should handle placement better.

3. Advancing assassin+escort into superior enemy forces on unfavorable terrain at bad ToD is quite sloppy for Ron.
4. Ron's choice of assassin target is uninspired, for reasons that have been mentioned; better would have been to spread poison to a unit that won't be ganged up on (e.g. the fighter), although the assassin might still have been lost.
5. Ed's decision to use the fighters' weak bows is intriguingly humanlike, although I'm not convinced it's actually the right choice here.
6. Ron chooses to poison a mage — admittedly, at optimum ToD, but it would have been better to save the assassin attack for after the grunt attempted to kill the same mage, since success on the latter would have been vastly preferable. Ed inexplicably moves a fighter next to an archer in a village, but doesn't attempt to attack (if he wasn't going to attack, he shouldn't have moved there at all).
7. Another attack sequencing problem for Ron; archer first, then grunt to finish off the wounded fighter, not the other way around.
8. Ron should have tried to focus both available grunts on the injured Wose, to prevent it from leveling to great effect (41% CTK); of course, the effort would have failed due to the obstinate fighter, but the initial grunt moving off to village-steal is arguably a bit of a mistake; also, terrain use is subtly wrong, allowing wose or fighter to steal forest.
9. Assassin placement is kinda terrible, but admittedly there aren't many good options.
10. Orcish archer ftw! Except ... Ron moves a heavily injured grunt off defensive terrain without making any last-ditch attempt to kill one last fighter. Bad bad idea.
11. Ed gets lucky and kills an immortal.
12. Ed's use of fighter bow is again intriguing.
13. Good assassin placement, but against three fighters it's not enough. Horrifyingly unlucky crossbowman.
14. And the crossbowman's string of amazingly bad luck continues.
15. Standard, decent tactics on both sides.
16. Ron repeats Ed's mistake, bizarrely moving a critically wounded grunt next to a fighter on worse terrain without attacking. However, the tide is beginning to turn; Ron's village-grabbing apparently works better than Ed's kill advantage.
18. Competing village-grabs, and decent healing retreats. By this point, Ron has negated the long-standing kill advantage.
19. Moving in for the kill with obvious force superiority.
22. Decent retreating, except for critically-injured units; excellent use of marksman to finish off a village unit with properly-sequenced attack.
25. Nice use of poison to doom shaman; pity it's already too late to be relevant.

[mattsc]

tuggyne: Thanks for taking the time to write all this down! Note that neither of the generic grunt rushes (Ron & Ed) have been updated recently. All of the current work is going into the Freelands-specific version (Fred), with the motivation that figuring it out for a specific case and generalizing later is easier than the other way around. So if you don't mind, I will save most of this for when that generalization happens. I know that some of the things you mention will be taken care of by code that is already part of Fred, but others won't, so this will come in very handy then.

Does this sound reasonable?

Is that a question for me or Rigor? I think in general this is a good plan, the part I don't know is whether I want to work exclusively on this until it is "done". (Well, actually, I do know the answer to that question, and I am sure you do too since you are following my github activity ) And of course the details might be slightly different in reality, but we'll see about that when we get there.

I agree that trying to adapt Fred to do something different right now is a bit premature, but doing some other (more or less orthogonal) things "on the side" would be good too. I think it would help me stay motivated better if I had something going on where progress is a bit faster than the incremental changes we're making to Fred at the moment, but I have no intention at all to "abandon" work on the Freelands and orc specific Fred that we've all come to love and hate.
I don't know. Does that makes sense? I think I am saying pretty much the same as in my reply to Rigor, but maybe I'm just babbling gibberish...

My goal is to someday have an AI that can defeat casual humans with any race on any map

I believe I have said before that at that point I'll upgrade the major version number from 0 to 1.

Btw, nelson, have you tried out Fred v0.10.4 yet? Do you think that it is an improvement (however small) over the previous version, or the opposite?

[nelson]

I don't mean you should not work on other things, I just mean that I think Fred is far from finished and I will keep on working on Fred until 1.0 And you now know my priorities.

Yes, I think Fred is improving! In my latest game against him, I see fewer obvious errors. Some of his decisions I think are suboptimal but reasonable people could disagree about them. Other decisions are still flat-out wrong, however. Here's a game where I beat Fred 0.10.4:

2p_—_The_Freelands_replay_Loys_defeat_Fred.gz

Loys beat Fred 0.10.4

(21.35 KiB) Downloaded 199 times

I wonder whether it would be better if the archer were one of the last units to arrive in the rush rather than one of the first units to arrive? I feel it is too squishy, and I'd rather have it strike when it has more supporting units, but on the other hand it takes less retal when it attacks a melee unit in the afternoon than a grunt would. Hard call.

5. At first I questioned the wolf on the left heading south to village steal while the cavalry is still alive on the village (get in your damage at night time!), but the leader kills the cav handily next turn and the wolf successfully takes a village next turn without getting trapped or dying, so maybe Fred is right and I am wrong. I just consider it a little risky to not put as much damage into the cav as possible at night, in case it gets inconveniently dodgy and survives an unexpected extra turn.

On the right Fred gets very unlucky this turn (balancing out his great luck last turn). I'm not sure there was anything truly wrong with his attack. As a human I might decide this position is too difficult to assault and retreat to wait for a better deal, but I'm not sure how I would tell the AI to avoid attacking here during its powerphase. The main problem is that the leftmost attacker is in range of my leader, who is able to help kill units next turn. But my pikeman leader isn't *that* awesome at night, it's not a complete disaster like being in range of a red mage at night, it's a plausible risk. If the cav had died here then it might have been a reasonable trade of a grunt for a cav. Ultimately, I can't fault Fred's decisions here.

At this point I would consider Fred to be ahead on the left (he's doomed a cavalry and is about to steal a village) and behind on the right.

6. I would have used the wolf to grab the outermost village at 10,19, to make it as difficult as possible for me to remove him if he decides to stay there. Grabbing the closer village at 14,17 is a suboptimal move. Generally the AI should prefer to take villages that are further from the enemy keep, because those are the hardest for your opponent to reclaim, and if your opponent does send units to reclaim a far-off village that leaves the other fronts weak. (The only exception to that rule is if you can grab villages unopposed, in which case you should grab closer villages during your powerphase and then retreat to more distant villages when the TOD is bad.) More importantly, my horseman could help trap the wolf on 14,17 but not the outer village, which is a perfectly safe grab.

On the right, due to Fred's bad luck, he has lost too many units to continue the attack. He definitely needs to retreat at this point, it's a crucial game-losing mistake. Fred is now rather outnumbered on that front, he has no anti-horse units left, and he only has one turn of night left. He should flee and try to preserve his units. If he ran away at this point, he would only have a minor kill disadvantage of one unit and an income advantage from the wolf on the left that he could try to make the most of. If the wolf were on the far left the horseman I just recruited would not be able to help trap if I decided to send it to the left b/c Fred is retreating on the right, and the wolf would be able to stay safely for at least one more turn.

How can we teach Fred to retreat in such a situation?

Note that I help Fred out by foolishly suiciding a cavalry

7. Good village steal in the center with the grunt! I think moving the wolf closer is suicide though, it leaves the wolf in range of both the mage and the leader, and the wolf can be trapped even if it avoids dying this turn. The wolf is actually perfectly safe right where it is this turn, it cannot be trapped or killed. There is no good reason to move it. Did the AI calculate the odds the wolf would die next turn in the new village? It must be pretty likely. Bad mistake!

Sadly Fred is retreating too late on the right, those retreating units can't survive on flat and Fred can't get enough full-health units in front to shield them. If I were caught in this situation with doomed units that can't be shielded, I might give up on saving the units that can't be rescued and instead send them sideways to put the Loyalists out of position when they move to kill the mortally injured grunts. Or maybe send one sideways and retreat the other, making the Loys choose. But honestly there are no great moves here, Fred could hope these units get lucky and dodge, in which case he could retreat them to heal and shield them next turn. Fred's choice here is reasonable.

8. With my 6-2 kill advantage Fred is in big trouble now, his best hope is that he can maintain his income advantage long enough to balance out my kill advantage. Putting the grunt on the left next to the fencer at a bad TOD is risky. Force the fencer to choose between holding the village and attacking at day! On the other hand, you can see that this move gives Fred a chance to kill my fencer due to his grunts retaliating when I attack, but on the other other hand the fencer dodges Fred's attempts to one-shot him the next turn. Poor Fred. I would say that this is a bad move in the long run, you do not want to invite attacks from Loyalists at daytime.

On the right, the grunt should have held the outer village while the archer held the more central village. Bringing the archer forward where it could be trapped and killed easily is silly! Now Fred has unnecessarily lost a unit and a village.

10. The game is over at this point. My advantage in income on top of my kill/XP advantage is too much.

[nelson]

For further entertainment, watch Fred annihilate the RCA AI playing as elves with 150 gold and +2 income:

2p_—_The_Freelands_replay_Fred_kills_overresourced_elves.gz

Fred kills over-resourced RCA elves

(22.09 KiB) Downloaded 215 times

More interesting is this one of Fred defeating RCA orcs:

2p_—_The_Freelands_replay_Fred_kills_RCA_orcs.gz

Fred kills RCA orcs.

(20.99 KiB) Downloaded 221 times

4. Why doesn't Fred trap the wolf? Does he think it is likely that the assassin would die while trapping? Only one grunt can reach, and the poisoned wolf might die of retaliation if it tries to follow up on the grunt's attack. It would be perfectly find to trade an assassin for a wolf anyway, they cost the same amount.

5. Wolf mistakenly grabs inside village again. (The chance to damage the assassin isn't worth it, IMHO. If the RCA AI knew how to trap, the wolf would be trapped.)

6. XP considerations: Fred should have let the almost-leveled archer finish the wolf on 25,8. I can't wait to teach Fred about XP!

8. Fred HAS TO take this opportunity to poison the leader! Sadly, Fred instead ABANDONS THE VILLAGE. This makes no sense! Why did Fred do that? Worse yet, he leaves a unit next to the open village, so that an enemy can grab the village AND attack him. That is also a terrible mistake.

10. Fred misses another opportunity to poison the leader! Noooooo.

13. Fred kind of throws away his level2 grunt in the southeast. If he had to make that attack with the supergrunt, I would have done it from the village, since the lvl2 unit is more important and you want to give it the terrain and healing to try to preserve it, and I would have had the lvl2 grunt attack 2nd, so that it might take less retal if it kills the target. The better move would probably have been to retreat the lvl2 grunt to a safer village until it built back up its HP, so that it doesn't die and give the enemy lots of XP.

Fortunately Fred is so superior to the RCA AI that none of these mistakes matter.

[nelson]

Here are two games pitting Fred against a random human from the lobby named ugh!

Fred wins the first game against ugh's Loyalists:

2p_—_The_Freelands_replay_ugh_surrenders_to_Fred.gz

Ugh surrenders to Fred

(25.08 KiB) Downloaded 209 times

Fred completely fails to reclaim his own villages in the west, letting 2 spearmen sit on his villages for most of the game, but fortunately the rush on the right and village stealing on the left are enough to give Fred a long-term village advantage anyway.

9. Note the cocky grunt attacking an HI at daytime! Terrible idea.

Fred loses the 2nd game against ugh's Loyalists, partly because he refuses to finish off injured units:

2p_—_The_Freelands_replay_Fred_doesn't_know_how_to_kill.gz

Fred doesn't know how to kill things

(23.09 KiB) Downloaded 203 times

6. What is the wolf doing in the southwest? Standing next to a village is pointless, it lets the enemy hold a village and attack you at the same time, make him choose. Trapping and attempting to kill the archer in the west village would be a better move, retreating might be better since it is unlikely to die and the orcs can be trapped in return. In general, the AI seems to like to throw units at that archer one by one and have them die individually while it heals off the damage, rather than ganging up on the archer and annihilating it.

7. Rather than poisoning the HI and taking the village next to the injured grunt, the assassin leaves the grunt hanging out to dry and takes a less desirable village closer to the enemy keep! Awful mistake. Attacking on the left at dawn and not even trapping the archer (so that it could get away if it were really in danger of dying) is a mistake.

8. Again the assassin misses an opportunity to poison the HI.

12. Archer should not have attacked by itself on the left.

On the right, a grunt should have claimed the east village that they just killed a unit on. (Just don't move a unit next to the top right village if it is unoccupied.)

14. Here the AI misses a golden opportunity to trap, poison, and possibly assassinate the enemy leader. Alternately it misses a chance to kill 2 fatally injured mages. Not taking either of these opportunities amounts to throwing the game. (A 1HP mage can live forever if you never counter-attack and you have no ranged retaliation.) While you don't want to fight Loyalists at day, if you can poison the enemy leader or trap the leader outside the keep you almost always have to take that opportunity, and if you can trade a grunt for a more expensive unit it is worth attacking at day.

17. AI passes up chance to poison HI.

18. AI doesn't poison spearman.

19. AI doesn't attempt to kill the white mage. Trading a grunt for a white mage is an excellent deal.

20. The white mage is just going to heal everything and kill Fred's units mercilessly, so I surrender on Fred's behalf.

[mattsc]

Something unpleasant happens if I try to run Ron against RCA on Thousand Stings Garrison. Specifically, Wesnoth hangs at full CPU indefinitely. This happens on 1.10.3 and 1.10.4

tuggyne: Interesting. I believe that the AI code considers all the petrified scorpions as viable enemies (it just checks for enemies, not for "attackable" enemies), making the number of possible attacks way too large to handle. I'll take care of that in an upcoming version.

This is solved. I indeed had to do with the petrified units, but the exact cause was different from what I thought. It was not because there were too many units to handle (which is good!), but because it got caught in an infinite loop. It's fixed in my local version already, so you should be able to use Ron/Ed on maps like Thousand Stings Garrison once v0.10.5 is out.

nelson: thanks for all the detailed analysis of the games. This clearly answers my question whether the change to the spread_poison CA was good or bad ... As we discussed yesterday, I have a lot of things to work on now. I'll probably start with de-emphasizing village grabbing at night, although my guess is that this is not as clear-cut as we'd like either, same as the spread_poison thing...

Finally, I have a sad announcement to make: Ed will leave us with the next release! As Ed and Ron are really exactly the same AI (they're not even clones, they use the same file) and Ron was there first, Ed has served his role as a "temporary experiment" and will be released back into the wild. In his place, Ron will be slightly modified to work with all factions automatically (although playing orcs will remain his strength, at least for the time being). At the release of v0.10.5, I ask all of you to observe a moment of silence in hono(u)r of Ed.

[tuggyne]

tuggyne: Thanks for taking the time to write all this down! Note that neither of the generic grunt rushes (Ron & Ed) have been updated recently. All of the current work is going into the Freelands-specific version (Fred), with the motivation that figuring it out for a specific case and generalizing later is easier than the other way around. So if you don't mind, I will save most of this for when that generalization happens. I know that some of the things you mention will be taken care of by code that is already part of Fred, but others won't, so this will come in very handy then.

Good to hear; there's certainly no urgency.

This is solved. I indeed had to do with the petrified units, but the exact cause was different from what I thought. It was not because there were too many units to handle (which is good!), but because it got caught in an infinite loop. It's fixed in my local version already, so you should be able to use Ron/Ed on maps like Thousand Stings Garrison once v0.10.5 is out.

Awesome.

Finally, I have a sad announcement to make: Ed will leave us with the next release! As Ed and Ron are really exactly the same AI (they're not even clones, they use the same file) and Ron was there first, Ed has served his role as a "temporary experiment" and will be released back into the wild. In his place, Ron will be slightly modified to work with all factions automatically (although playing orcs will remain his strength, at least for the time being). At the release of v0.10.5, I ask all of you to observe a moment of silence in hono(u)r of Ed.

Aww, and we just got him named a few days ago. Ah well.

[mattsc]

Aww, and we just got him named a few days ago. Ah well.

Yeah, I know. We might revive him in a different incarnation sometime... And with that:

v0.10.5 is out. Unlike previous versions, this one has very few changes made to the Freelands-specific grunt rush (Fred). Mostly, Fred contains new code for counter recruiting provided by Alarantalara and a new check at the beginning of the game that the right faction, side and map were chosen. Everything else is unchanged.

The main update this time is the absorption of Ed into Ron, and the generalization of the latter into a generic rush that now works with any faction, for any side and on any map. Read the Notes in the changelog below though! The generic rush lags much behind Fred in sophistication, even for the parts that are not Freelands-specific.

Nevertheless, it does already seem to outperform the RCA AI (saying it like that is a little unfair, of course, as it contains all the functionality of the RCA AI, with just a few CAs added). I ran 6 no-gui games for each faction on the Freelands map, having Ron play against the RCA AI, with both AIs using the same faction. A technicality: because of the way the RCA recruitment CA works (which recruits the non-fighter units for all factions but Northerners), I gave both sides 200 gold - except for Northerners, which played with the default 100.

The results: 6:0 Ron over RCA for Northerners, Loyalists and Rebels, 5:1 for Drakes and Undead and 4:2 with Knalgan Alliance.

Also, as announced above, the problem with maps containing petrified units has been fixed.

The full changelog

Code: Select all

0.10.5, 1 Sep 2012
- Generic Rush
  - Replaced the generic grunt and elvish fighter rushes by a single version that can deal with all factions
    - Notes: 
      - The functionality/behavior has not been updated, it's only been generalized
      - We will continue to call the new version Ron.  Ed has been retired (for now)
  - Fixed a bug that hung Wesnoth on maps with petrified units (see below)
- Freelands Side 1 Grunt Rush:
  - Added check for faction, side and map to Fred (stops game if one of them is wrong)
  - Added counter recruiting of archers
- ai_helper:
  - New function ai_helper.get_live_units() that excludes petrified units from selection
  - Changed ai_helper.get_attacks...() functions to exclude petrified unit
    - This means that the generic rush now works on maps that have petrified units on them

[Rigor]

gj! about recruiting: is there any code already except to move the already standing units on the keep? im asking because some maps, such as arcanclave and 1000 scorpions require the ai to understand that connected keeps enable it to recruit at very remote places, which is crucial to winning the game. on most maps this is not the case, however in some scenarios it migth be helpful for the creators to count on a smart recruitment pattern. theres somethign more: if there is just one keep everything is alright, but on many maps you have multiple keeps to recruit in - this means, usually the ai never moves his leader to those keeps and they r good for nothing. ideas?

and something else: i would so love to see this new smarter ai integrated into world conquest, just to see if its owning all those nonstop wcplayers - i was one of them btw and it is so much fun! is it possible to just do a quick and dirty solution for the first scenario?