Exercises in Formula and Lua AI and AI-demos add-on feedback

[The Black Sword]

Hi,
Rigor showed me the Orc AI and asked me to post my thoughts. I'm not sure if I have the replay of our game, if I do, its on my other computer and I'll upload it later for reference, Rigor might also have it. Some thoughts:

-The AI's attack on the right was pretty good
-However it was too open with its villages on the left

Suggestion: If the enemy is in range of your villages(and you know about it), there should be heavy weighting on defending those villages. Probably the only things I would weight higher are a killing blow or hitting an enemy unit on weak terrain during your powerphase. In/before your powerphase: Should prefer moving closer to enemy units and using ToD to block their path to the village if possible. In/before their powerphase: Should prefer just sitting on the village with a grunt.

You could argue that it is often a better move to give your opponent a vill during their powerphase rather than let them attack you but I think that might be aiming too high to start. (*)

-I recruited a lot of cavalry. The AI had 1 archer at the start and then pretty much went soley for grunts and assassins. IIRC the normal AI was better at changing its recruits accordingly. I imagine it shouldn't be too difficult to come up with a formula for this though.

-The AI didn't retreat at all when day came around. Tbh, in this particular game, that wasn't the worst move but in general this is one area where I feel the AI could really improve. Improving it's micro-game is probably difficult but improving its macro game based on ToD should not be so hard I think.

In general you have your front villages and your opponents front villages. I assume your chaotic below, but you can just reverse for lawful.
First Day: You should have a line on your front villages defending them. Again the caveat of (*) above.

Second Day: As P2 move up as quickly as possible, as P1 move up but still hold a line on good terrain. Attack your enemy only if there is some very favourable opportunities.

Dusk: Move up as quickly as possible, aiming to threaten as many vills as possible with at least 1 unit and then have as many units as possible, at least 4, in range of the main village you are attacking. Attack if you can do more damage to the enemy than he can to you, or vs strategic targets, though getting the AI to recognise these is probably difficult.

First Watch: Take any open vills. Attack your opponent pretty much wherever possible, only exception being using less than 3 units vs a full strength enemy in a village.

Second Watch: Probably the biggest question: Stay or fight? If you can do a lot of damage to the enemy then fight. Generally this means if your opponent has a lot of injured units or a lot of units on weak terrain. If however he's given you the village, and formed a good line behind it, you probably have to retreat. As P2 retreat as quickly as possible. As P1 this may not be necessary, hold the village with a quick unit that is also a good vill holder(quick grunt, resil wolf etc.), retreat out of range of your opponent while maintaining a ZOC so that the vill holder can't be trapped next turn. Put quick units in the front line, if they can't be killed this turn, so that if your opponent damages them, they can get to the villages quicker and heal as you retreat.

Dawn: retreat as far as possible, stopping at a line on your vills.

This is the general pattern for a wesnoth MP game I feel and if the AI just followed the ToD like so I think it would be a fair challenge. Rigor mentioned to me about the idea of map specific hints and I think you could almost set down these areas on the map: Defend here First Day, Be Here at Dusk, Attack this village at First Watch.

The main problems I see are
1. teaching the AI to form a defensive line well - though IIRC it did a fair job on the right side in my game
2. When the game begins to deviate from this pattern. The most likely reasons are that the map isn't "standard" and would perhaps need some more specific hints or that the AI gets an advantage. If your opponent doesn't pressure you in this way then stay out of range of his units but as close as possible to the vill you will attack during First Day, Second Day. If the AI gets ahead, stop playing (my vill, middle, enemy 1st vill) and start playing (middle, enemy 1st vill, enemy second line of vill) at (First Day, Dusk, First watch). Assuming "middle" also has some good defensive ground, eg. the 3 castles on Freelands.
3. Choosing what village to attack - I'll give some thought to this but in general the only candidates are those ~3 standard unit moves away from your keep which narrows it down, it could also just be another map hint.

I think I had some more ideas but its late, I might elaborate tomorrow. I hope that's of some help, great work so far.

[mattsc]

The Black Sword: wow, yes this is extremely helpful. Thank you for writing all of this down !! Between what you, Rigor, FAAB and others have told me, I have a lot to think about (and code). As I wrote to Rigor, some of that is easy, other things not so much. I am aiming for making a couple changes, then releasing a new version for testing by those of you who have the time and interest, making more changes, and so on.

I'll need some time to go through everything you all have written, so for now just two general comments. Yeah, the left side is completely ignored by the grunt AI so far. All I do is send two units over there and leave them to themselves. Your (and others') comments are very useful for me to decide what to do about that.

Map-specific hints etc.: I don't think that we'll ever be able to write an AI that can stand up to a half-decent human player. Well, or at least I won't. So, my take is that whenever it is possible for the AI to cheat in order to level the playing field a little, let's go for that. Map-specific hints are one example. Seeing through fog (e.g. don't do a grunt rush when the enemy is undead) is another. So yes, I am all for that. Playing fair is something we can worry about if we'll end up with an AI that can actually beat you guys, but I doubt that that will ever happen.

Thanks much again!

[Rigor]

Dreadnought!

Here we have a game where Dreadnough exploits masterfully all of the AIs little weaknesses. after this debacle i suggest to implement "assassins always attack" no matter where they are. it just pays off to lose them and poison a lot of troops while being slaughtered. the left is as usual the barnyard backdoor to kill the AI pretty easily, but imagine a gobo in the far left and a grunt in the south west - pretty difficult to do anything like that. the grunt is very easy. just stay there. the gobo wold require a kind of "walk there in 3 turns" order issued at a certain turn i guess. the rush on the left can happen as early as turn 4 and could be completely prevented if both units who go to the left stay there until the gobo arrives. apart from that, the right side would definitely have an easier time if the assassins blocked additional hexes for attack.

[mattsc]

All, I have not had time yet for most of the things you all wrote about, but I have made a few simpler changes. From the changelog:

Code: Select all

0.9.9, 3 July 2012
- Freelands Side 1 Grunt Rush:
  - Added basic routine for holding left (west) side (but not great yet)
  - Added fall-back candidate action for when enemy becomes too strong
  - Upped priority of poisoning enemies
  - Adjustment of village grabbing priority (when easy kill is possible instead); other minor changes to this CA
  - Some modifications to recruitment CA (added troll whelps, some other minor changes)
  - A few other minor changes to the weighting functions

If you don't have much time, this version is probably not worth checking out. I am getting the impression that it actually plays worse than the previous version, but I am getting too tired to look into things any more tonight, so I uploaded the code anyway just in case somebody is interested and wants to send me suggestions. I'll pick it up again ... soon, sometime.

Note: This is for the specialized Freelands Side 1 AI only. The more general version has not been changed at all.

[mattsc]

Rigor, All: v0.9.10 of the 'Freelands Side 1' AI might be worth testing. I found a couple bugs in v0.9.9, reorganized and reprioritized the whole thing and added a couple new actions. I still haven't done a lot of what everybody has suggested (there's only so much time...), but I do think that this plays a bit better now. Here's what the code does so far:

• Hard-coded (500,000):
  - T1: Recruit grunt, wolf rider, assassin
  - T2: Move grunt, wolf rider, assassin
  - T3: Move wolf rider
• Attack if high chance to kill (450,000)
• Attack leader threats (420,000)
  - Attack enemies that have made it too far north (maybe a bit too extreme in its current form?)
• Spread poison (400,000)
• Fall back when enemy too strong (350,000)
  - Currently we simply fall back to keep, can be replaced by better algorithm later
• At night, attack village at 27,16 (300,000)
• Grab villages (280,000)
  Rating:
  - if unoccupied: +1000; or if enemy occupied: +2000
  - if can be reached by enemy: +10 per enemy
  - prefer non-assassins: +1; and strong units: +HP/100
  -> need rating >=10 to be considered
• Send units to left (251,000)
  - So that we have at least one more unit on left than the enemy
• Grunt rush right (250,000)
• Hold village on left (249,000)
• Recruitment (181,000)
• Standard RCA AI moves (10,000 - 120,000)

Note: The numbers in () give the priorities with which the moves are considered. Their exact values don't matter, only which one is the largest for any possible move.

[Rigor]

just checked it shortly, it plays great already.

small blunders: turn 12 AI takes its own village to heal instead of one of my two which were equally good.
turn 13: AI exposes the leader to a leaderkill, which is always bad. maybe it should try to stay on good terrain when "max dmg nearby is >= x"

gj!

[mattsc]

just checked it shortly, it plays great already.

Well, on first look, is seems that it lost miserably, never even got its feet on the ground. Yes, yes, I know, no comment necessary...

small blunders: turn 12 AI takes its own village to heal instead of one of my two which were equally good.

Actually, that's not quite what's happening. At that point, the AI has decided that it's going to loose (you've become too strong) and just tries to make it last as long as possible and protect its leader. So it's pulling all units back toward the keep. The fact that it ends up on a village is just a coincidence.

turn 13: AI exposes the leader to a leaderkill, which is always bad. maybe it should try to stay on good terrain when "max dmg nearby is >= x"

Yeah, it does that. I am not controlling the leader at all at the moment. That's left to the default AI and it is rather suicidal at times. I've been thinking about having the leader never attack, but that's not ideal either. Taking enemy retaliation into account is one of the harder things to code, but something approximate (worst-case assumption) like what you suggest should be easier. It's next on the list.

Along the same lines, I also think that the assassins are a bit too aggressive. Poisoning enemies is great, but being killed after poisoning a single unit is maybe not worth it. Again, making it "smart" for real is not easy, but I should be able to regulate it a bit better than it is now with not too much effort (e.g. by changing the priorities how attacks are chosen).

I have gone over all the replays and comments you and others have sent me. They are really helpful, but it looks like there are only a couple things left that are easy to do (such as the things above, putting a gobo onto the far west village, or recruiting more archers when the enemy has a lot of melee-only units). Things like ordered retreats, setting up lines along the AI's villages etc. are non-trivial. I'll work on those too, but if anybody has suggestions for other "easy fixes", that would be much appreciated.

Thanks again!

[Boucman]

I have gone over all the replays and comments you and others have sent me. They are really helpful, but it looks like there are only a couple things left that are easy to do (such as the things above, putting a gobo onto the far west village, or recruiting more archers when the enemy has a lot of melee-only units). Things like ordered retreats, setting up lines along the AI's villages etc. are non-trivial. I'll work on those too, but if anybody has suggestions for other "easy fixes", that would be much appreciated.

maybe it's time to call the grunt rush AI done and work on another AI, the "orderly retreat" one could be interesting... playing against a standard AI with 2x gold so it has to retreat, and trying to make it last as long as possible

moreover "AI loosing to human and trying to make it last as long as possible or make it interesting for the end game" seems to me to be a VERY common use case, and one that would be very good to improve and get mainlined

after that, you could try to assemble these two AI by finding a good way to choose which one to use depending on circumstances

[mattsc]

Boucman: I like the idea of trying to set up things like an "orderly retreat" option separately from the grunt rush. That will make it more generally usable and easier to test as well, and adding it into the grunt rush afterward should be straightforward. There are some things I want to finish up about the grunt rush (both Freelands-specific and general) first though:

I uploaded another incremental update:

v0.9.11 features (Grunt Rush on Freelands Side 1 only):

• The goblin you suggested, Rigor
• The AI leader only attacking when it is "safe". The algorithm isn't perfect (part of that is due to certain things not yet being possible in 1.10 without major effort; they will be easier in 1.11/1.12), but it should be much less suicidal than the RCA AI is at times.
• Some minor reprioritizing and reshuffling of the Freelands CA's
• (It's also a major maintenance release for some of the other stuff not directly related to the grunt rush AI)

So, as far as I can tell, I am done with most everything that is easily doable for the grunt rush. The rest is probably better tackled separately, and then added into the grunt rush later, as Boucman suggests. The exceptions might be some

Questions about the Grunt Rush on Freelands:

• Should recruitment be geared more toward whatever units the enemy has? If so, how?
• Are the assassins too aggressive? If so, what should they do? Do not attack during the day unless it's possible from >=60% terrain, but always spread poison at night ... something like that?
• Is current attacking of units that have made it to the northern part of the map overkill? Should I limit it to, say, 2 units per enemy unit? 1 unit per a certain number of HP? Etc...

Having said all that, here's a

Suggested workplan:

Grunt Rush on Freelands for Side 1:

• Implement whatever simple changes remain
• Clean up code
• Generalize code to work slightly more generally (but still specialized for Freelands Side 1)

General Grunt Rush:

• Use whatever parts of the Freelands code can be used in a more general setting

Orderly Retreat AI

• Put together AI
• Set up a WML tag that can be used to customize this AI in the same way the [side][ai] tag does for the RCA AI.
• Anybody want to suggest an "algorithm"? It doesn't have to be an algorithm per se, description in words is just fine. I have my own ideas, of course...

[Rigor]

brainstorm during game:


beginning: finally the wolf doesnt attack a pointless target. but he could stay on some better terrain except there is some big strategy involved i cant think of to be hit with the gunner just like that.

turn 9: AI blunder, not using leader to remove a footie from a village. apart of that the gobo comes just in time to take the crucial village.
turn 10': i love it how the grunt moves one hex up to give the attackers 4 hexes space.
turn 11: ai attacks villaged guards without assassins and any other support.
turn 14: ai goes back to protect the leader...ok but not necessary here. the nicely coded grunt/gobo on the left go to a place called incomeless. else it would become yet interesting because the ai has still enough troops to be a threat.

ulfs are a tough meat for the ai: attack it or not, and if so, with what? a dilemma not only for mp players.

btw ive followed your assassins orders: they should, for starters, always attack, and if possible, some other units should stand next to it so they can only die by lucky shots "2 out of 2 " etc...you should not forget that a poisoned unit will lose 24 hp in very little time so it can be killed if the attacker goes on with his offensive and fails somewhere. only poisoning at night is vcertainly the way to defeat.

there is some strange logic i can understand behind the ais moves: when i place a unit on a village, and two outside, it poisons the ones outside although the grunts want to storm the village. it has only grunts left to go for the village, which cna be a bit of a gamble. if it works, very good, if not, pretty bad.

here is an example with loys too:

what happens to me here is very bad for me, almost lost you could say. the ai makes a blunder but not because it attacks or defends badly but because of....and maybe thats hard to code here: because of its troop flow!!! more troops should have gone to one of both sides to sustain an edge in village count. now that its day ill certainly attack with all i have. there was a second grunt on the left who could have followed the wolf on the left flank making a comeback very unlikely.

turn 10: ai just attacks everywhere as intended, but now that the ai is so smart, you could prolong the smart play to turn ...16 maybe?
turn 13:my leader has 3 hp and hes poisoned, only for your information how well that games goes for me.

turn 18: income kicks me hard for being asuch a leveler, and thats what orcs love!

gotta go shopping but hers the rp, i saved. very good one!

[alexanderthegre]

Wow, that freelands grunt rush AI is much better than the default one.
I played a rather nice game against it, with me playing drakes.
One thing I noticed about it was that if some of my units are at the end of the left pass, then it won't move any units already there. I think this is a consequence of it putting too much value in high-defence terrain and villages.

[mattsc]

Rigor & alexanderthegre: Thanks much for testing, comments and the replays! Watching the replays, I am actually reasonably pleased by what the AI does. Not perfect by any means, but ... Of course, I am not exactly impartial here.

I just uploaded v0.9.12 which addresses some of the points the two of you raise. Haven't had time for everything I want to do yet, but there should be a few noticeable improvements (I hope). Below follows a rather lengthy discourse what was changed for those of you interested. Or you could just play another game.

Long-winded blathering about the changes made in v0.9.12:

beginning: finally the wolf doesn't attack a pointless target. but he could stay on some better terrain except there is some big strategy involved i cant think of to be hit with the gunner just like that.

Nope, no strategy whatsoever. In fact, that part of the move was still handled by the normal AI in the previous version. I now have changed things about how the left side is handled (see below), which also makes the wolf behave differently. Of course, what he does (join the grunts on the right if there's not threat on the left) might not be ideal either, but one step at a time.

turn 9: AI blunder, not using leader to remove a footie from a village.

The leader now attacks close targets with high priority if it is safe. So that should take care of situations like this.

turn 11: ai attacks villaged guards without assassins and any other support.

I switched the priority of attacking the crucial village with poisoning enemies. This will now make the assassins attack the village as well (if they are in reach), without compromising the spread-poison CA otherwise, I think.

turn 14: ai goes back to protect the leader...ok but not necessary here. the nicely coded grunt/gobo on the left go to a place called incomeless. else it would become yet interesting because the ai has still enough troops to be a threat.

Agreed. The retreat is currently simply coded as: if total HP AI < total HP enemy * 0.75, retreat toward keep.
I will eventually do something more sophisticated that evaluates, among other things, which villages to hold, but I'm planning to develop this in a separate AI (the "orderly retreat" AI) first, and then merge it in here.

btw ive followed your assassins orders: they should, for starters, always attack, and if possible, some other units should stand next to it so they can only die by lucky shots "2 out of 2 " etc...you should not forget that a poisoned unit will lose 24 hp in very little time so it can be killed if the attacker goes on with his offensive and fails somewhere. only poisoning at night is vcertainly the way to defeat.

I've upped the importance of being on good terrain, but besides that this still needs some work.

there is some strange logic i can understand behind the ais moves: when i place a unit on a village, and two outside, it poisons the ones outside although the grunts want to storm the village. it has only grunts left to go for the village, which cna be a bit of a gamble. if it works, very good, if not, pretty bad.

This is because this was handled by two different CA's: one for spreading poison (which does not usually target enemies on villages) and one for taking that crucial village. I've switched the priorities of the two, so this case should be handled better now.

what happens to me here is very bad for me, almost lost you could say. the ai makes a blunder but not because it attacks or defends badly but because of....and maybe thats hard to code here: because of its troop flow!!! more troops should have gone to one of both sides to sustain an edge in village count. now that its day ill certainly attack with all i have. there was a second grunt on the left who could have followed the wolf on the left flank making a comeback very unlikely.

I have changed things so that units stay only on the left when there's a threat from the south (trying to match HP with the enemy), the rest re-joins the rush on the right when the threat goes away (except for the grunt and goblin who stay on the villages indefinitely, because "somebody" told me to do that ). A rush south on the left is not yet supported, but could be added if I can come up with a condition when that might be the right thing to do.

turn 10: ai just attacks everywhere as intended, but now that the ai is so smart, you could prolong the smart play to turn ...16 maybe?

Actually, the condition is simply: if turn >= 10 and AI has at least 50% more total HP than enemy, we go for an all out attack (left to the standard AI, except for things like village grabbing and poisoning). So as long as the AI maintains a definite advantage it will keep attacking. Edit: Oops, I understood that entirely backwards... Too tired, going to bed now. Ok, will extend to turn 16 in the next release.

Wow, that freelands grunt rush AI is much better than the default one.
I played a rather nice game against it, with me playing drakes.

That's great, thanks! I do like what I am seeing there.

One thing I noticed about it was that if some of my units are at the end of the left pass, then it won't move any units already there. I think this is a consequence of it putting too much value in high-defence terrain and villages.

Well, part of that is taken care of, but the grunt and the goblin will stay on the villages forever, unless somebody breaks through the lines. See my comment above.

So, in summary, I did quite a bit of reorganizing and reprioritizing to do all this (with a positive side effect that I could get rid of 2 CA's). Overall I think it has improved the AI some more, but I haven't tested everything extensively yet, so not quite sure yet whether I have introduced new problems.

[mattsc]

Quick release: v0.9.13 is out which now includes moving another unit next to an assassin if the latter is too exposed after a poison attack. I usually wouldn't (quite) make this a separate release and post, but I honestly do not know whether this makes the AI better or worse. On one hand, the assassin is more likely to survive. On the other hand, it takes units away from the grunt rush.

Comments?

Oh, and the AI does not go into full blown offensive mode until Turn 16 now, or earlier if it has at least twice as many total HP as the enemy.

[Rigor]

another one:

this time i checked the AIs priorities by simply placing a HI in the village: both assassins go for the HI although they inflict really low damage while there was another target that was more important - the free spearman. on the other hand, the hi couldnt escape and an archer might have made the difference. alright the code could look like: poisoned village-sitters are not priority no 1 for assassins. then, i noticed that on turn 7 onwards the orc really tries to destroy my village stealer on the left with horrific consequences. even if the HP is super low, the AI goes into berserk mode on a most unfavorable daytime, so that even units die in the attempt to decrease the hp of the offender. i think thats a nice behaviour but could be exploited, for instance by myself smarter behaviour: if hp = less than 20 sit in villages and dont attack, almost all units deal something in that range.

thats all so far, besids, of course i cant finish the other game with loys cuz maybe the ai would win and that would be too shameful. and besides the real ai isnt playing anymore so there is no point in that.

[khorne-flakes]

Hi , by request of a rigorous being i post this succesful defend of human vs ai . I could defend in a long game with a 20% plus receiving in many turns. Things aside.. in this game the assasains shooted units that could them retaliate it very hard and indeed happened( see the leader headshooting one of those bastards ninjas). Then he did some poor management of the "weak side". Cant calculate well still the implication of ulf tricks, and at some point he decided to attack a gryph in flat with grunt without sense. That,s pretty much all the bug/ weakness i found it, also there,s the theme of the leader stacked in defence, here it would have been great to defend villages of weak side , here the replay cyu