Exercises in Formula and Lua AI and AI-demos add-on feedback

[mattsc]

Hey Simons, that's very interesting. I need to try that sometime. Thanks!

[Boucman]

Simon, do you have a link to the actual paper ? that might be usefull...

[Simons Mith]

Hunting algorithm: Behavioural Processes, DOI: 10.1016/j.beproc.2011.09.006
Large packs: Behavioural Ecology, DOI: 10.1093/beheco/arr159

[TickleMe]

This is great work mattsc. You have raised my expectations of an AI by a huge amount. I had to register just to give some well deserved Kudos! Thank you for making this matt.

I finally managed to beat stationary prune cart on the 5th attempt. The first 3 attempts felt like i was put up against unsurmountable odds. The AI beat me without breaking a sweat. The 4th and 5th attempt were quite hard on the AI and I could notice considerable lag during its moves(~3 to 5 seconds).

Unfortunately once I found the recipe to defeat the AI, it was clear how inadequate even the most well conceived AI would be vs a player that can learn and adapt because now it seems that I can beat it consistently and even managed to get a few units promoted.

here is how I beat the AI, in case you are interested in updating the AI further to counter this strategy.

Spoiler:

I built as many melee units as the starting gold + two closest huts allowed before any sort of attack
Then I moved a batch of about 7 units hugging the bottom part of the map towards the right. another batch of 7 units circled north and approached from the top. Rest of the units stayed poised to attack via the pass in the middle leaving one hex between them and the mountains. The AI at this point spent a lot of time moving units up and down as if trying to figure out where to focus. it was a bit too conservative, so it ended up yeilding way to bottom right without any fight and then it was stuck on plains while my troops were on the hills. Also at this point, both healers were moved north. I waited for night and attacked simultaneously from north, south and middle. the AI moved most troops north leaving too little defence near the pass. when the battle ended, it had a couple of promoted units left in north and i was left with about 10 units - mostly in the middle. Scenario was won the next turn. I think the AI needs to split its forces in a better manner - it spent too much force defending north probably because the army it was facing was on hills as opposed to plains in the middle. Also, it needs to place the healers somewhere in the middle and move injured units towards them instead of moving all healers to the north. this prevented cruicial healing in the south and middle for most part. Even the leader was moved north and allowed himself to be killed by getting trapped.

I would be happy to send you a replay if you feel it would improve the AI even further.

[mattsc]

TickleMe: Thank you! I'm glad you are enjoying it and thanks much for the detailed feedback. To be honest, I haven't even tried playing against the Prune Cart AI myself yet (I usually just watch it against the RCA AI), because it is still very much work in progress and has lots of things that I know (even with playing against it) that I want to improve. So that kind of information is really helpful (some of what you write I knew, other things I didn't) and, yes, I would very much like to see the replay.

The 4th and 5th attempt were quite hard on the AI and I could notice considerable lag during its moves(~3 to 5 seconds).

Yeah, working on speed is one of the things I have not done at all yet. That's because I often end up throwing about 3/4 of what I write out (I always come up with these great improvements to the code that make the AI play much worse!), so I don't usually bother with writing efficient code that then gets tossed anyway... I will get to it eventually though.

I'll leave it at that for now and will go over what you wrote again when I'm a bit more awake. Thank you very much again, this does really help me a lot and while I don't have much time for this scenario right now, I will get back to it; soon-ish, I hope. If you notice anything else (now or later), please by all means do let me know.

[Rigor]

hey again,

just now i had this idea that some very basic routines could be implemented in the regular AI already. the first which came in my mind was the "protect mage" scenario: the ai doesnt really care about their units, and giving certain flags to specialists could help the AI to identify crucial units to win the game (heh, ok, to play better at least) on any map.

so heres the idea:


Figure: AI playing smart

you play a game vs the ai, it buys as usual according to some inner mechanism and play the first few turns running around collecting villages. then, the ai could already start behaving special: is there a shaman in the list of recruited units? if yes, use shammies very close to other units, such as fighters, or archers, but no woses.

actually many very good players dont care a bit about losing unimportant units as long as the good ones survive the battle at the end of the day. there might be an algorithm telling the AI which ones are to be protected and which ones expendable. also, sitting on a village as orcish grunt is almost in 99,9% of all cases a very good idea. how much it costs your opponent to get rid of this little pest, and how much time he wastes while doing so makes you understand the point in making it equally beneficial for the AI to do exactly that.

is it too early to try it with one faction and one unit to develop specialized behaviour? i think not. and just imagine where this can lead: the AI having not only one, but many types to behave. there will not be one super-routine how to handle things, there might be a choice, so dont let the width of the topic scare you off from making the first step in the right direction!

[UnwiseOwl]

Every time I re-visit this thread it makes me think that specialist AIs are getting closer and closer thanks to mattsc. I'm with RIgor, I feel like this stuff is almost doable now, even though there are still a lot of drawbacks and I as a terrible coder have no real conception of the issues involved. If I ever get up the courage to try using this stuff it'll be a bot that plays HODOR on a specific map, I'm hoping that that will decrease the number of things that it has to think about, with only 3 units to recruit. Then when it can beat noobs on the multiplayer server I'll know that it's time to try a bigger and more difficult task.

[mattsc]

Rigor, UnwiseOwl: you're too kind! I guess I know too much about all the weaknesses of my AIs to share quite the same level of confidence, but I am certainly planning to test some of these things with MP maps eventually. I don't know about implementing this into the regular AI (you'd have to talk to Crab et al. for that), but it is possible to start Wesnoth on the command line with a custom AI for a MP side.

Now, the closest I have to a general AI is that in Prune Cart, and that is certainly not ready. Setting up some more specialized behavior on the other hand might be doable. So, let's say I'd set up an AI that would do a simple "healer backing" similar to what Rigor suggests, modeled after the 'Protect the Wizard' algorithm: place healers close to other units (and preferably close to those at the front line), but where enemies cannot reach them. Would anybody be interested in testing something like that on MP maps and send me feedback and suggestions for improvement? From my point of view that would be great.

Also, having certain units (e.g. grunts) simply hold villages is quite easy to do - but I'd need some help with setting up the decision process: which units, how do we select relevant villages, do they simply hold and do nothing else, etc? I almost never play MP scenarios (in fact, unfortunately I barely play any Wesnoth at all at the moment), so some pointers here (also which map and faction I should try this with) would be much appreciated.

[Boucman]

how possible would it be to add some info in the map itself ?

if this gets mainlined it would probably be possible to add som AI hints to our MP maps...

[mattsc]

how possible would it be to add some info in the map itself ?

That's trivial and makes writing the AI a lot easier. For example, the 'Pass Defense' scenario uses hard-coded coordinates for the best positions to place defenders, healers and leaders. My recent efforts have been toward getting away from that, but for specific maps it is definitely a possibility.

[UnwiseOwl]

Would anybody be interested in testing something like that on MP maps and send me feedback and suggestions for improvement? From my point of view that would be great.

Interested, yes! Qualified, probably not, but we'll see how it goes.
I'm gonna put this in my (longish) list of Wesnoth projects, because it's definitely something that I'm interested in, but my low standard of multiplayer play will probably mean that I help write a bad AI.

[mattsc]

UnwiseOwl: Cool! And I don't think we have to worry about playing skills, as any half-decent player will always be able to beat an AI in Wesnoth. The trick is to come up with simple concepts that can be added to the existing AI that might improve little bits of it, and then add to that piece by piece. That's why I like this "healer backing" idea - but to be honest, I don't actually know if it will make the AI play better or worse. There's one huge advantage to always attacking: choice of weapon. I can take out an entire army of royal guards with a handful of peasants if the guards never attack... So, I think it's quite possible that taking the healers away from the units available for attacks (without lots of other changes as well) will make the AI play worse originally. But that's why we need to test it.

Another thing I feel compelled to say is that you all seem to think that I know what I am doing, while quite the opposite is the case. I wasn't kidding when I said that I throw 3/4 of my attempts out. At least. And: this isn't going to happen at any speed, as I neither have the skills nor the time to push this ahead quickly. But, if all that's ok with you, I'll try to put together a first attempt of a (very slightly) customized AI that can be used with MP maps. I don't know how long that is going to take as I haven't done it before (and I need to write parts of the AI code as well), a few days would be my guess. I'll let you know when I have something to try out.

And finally, I just want to point out once more that the current AI is really quite good. The more I work on this, the more I learn to appreciate that. We shouldn't be going around saying how bad it is and that we need to replace it (I'm not saying that anybody here did, but I have seen it in other places), but rather make use of its strengths and add to it.

[UnwiseOwl]

Another thing I feel compelled to say is that you all seem to think that I know what I am doing, while quite the opposite is the case. I wasn't kidding when I said that I throw 3/4 of my attempts out.

I've noticed, having read through this thread over and over it's one thing that's pretty obvious. It also gives me hope that even my 'make one change, load campaign, no that didn't work' style of scenario design might one day result in something worth uploading.

And finally, I just want to point out once more that the current AI is really quite good. The more I work on this, the more I learn to appreciate that. We shouldn't be going around saying how bad it is and that we need to replace it (I'm not saying that anybody here did, but I have seen it in other places), but rather make use of its strengths and add to it.

I, and any one else who has had their leader killed in a campaign by the RCA AI is aware of this, Wesnoth AI is good, but I feel like the progress that you've been making in this thread can lead to something more than macros for campaigns (though I like, and plan to use, those too!).

I don't know how long that is going to take as I haven't done it before (and I need to write parts of the AI code as well), a few days would be my guess. I'll let you know when I have something to try out.

Ha! I measure my time for wesnoth changes in months, trying to fit it around a life, work and family that aren't optimised for computer time, if it takes days or even weeks then you're exceeding expectations by a country mile.

[Rigor]

i asked for help form pro players to help you teach the AI their tricks (only those which can be reasonably programmed). personally i find the prospect of teaching the ai about the abusability of the map a very good idea. of course, we, the players should know at least to a certain degree what happens when village x is occupied for a longer time - nothing too good for the assaulted one - and how to get this advantage as early as possible in the game, therefore setting up the first assault force in an appropriate manner. i was thinking about the most basic and almost automatic gameplay strategies which require not even hex counting when recruiting the first units: the grunt rush on freelands.

just imagine your AI not only knows that its crucial standing ante portas at dusk but also attacks while receiving all the boni from playing at the right TOD. what comes next, an orcish player should know that its a very good idea to compose his army of mostly = 50%++ grunts, and not thinking for the moment and building fast wolves who could change the battle thats being fought right now. finally, the orc in us wouldnt like to replace himself from a 60% protected area just like that, mostly he stays put or runs into the next free village when the defending troops are busy. you know what a coin flip it causes if you are grabbing extra income? not only 2 gold. your unit support limit is improved while your opponents is suffering, plus, those 2 gold arent earned anymore. so in total its 6 turns per round thats lost and 2 turns of a grunt sitting there has additional benefits: the army is elsewhere, not using its tod, maybe the leader cant recruit because he has to use some wicked magic...

some of those things have been said already but most ideas would need refinement. i would gladly test the mp ai of any faction on a map of your choice and point out what needs to be pointed out. talking about the scope of the project: i dont mind if its a long term thing, and if it takes weeks to develop it then thats the time it takes. we all have a real life too

[TickleMe]

Sorry for the delay in uploading the replay. I had lost access to the machine I played this before.

I replayed the prune cart on a different machine now. Here is the replay. Things turned out a bit differently from my text blob in the previous post, but the essence and main strategy was about the same.

Spoiler:

Some differences to previous one and key points to observe:
* I ignored time of day completely on this one - causing me to suffer more damage than I liked.
The idea behind attacking at night is to maximize damage and to pick off enemies by focussing to deny their healers any opportunity to use their skill.
* The AI movement is suboptimal, exposing one of the healers for an easy kill fairly early in the battle.
* Leader was managed better than my previous runs - but if you observe his movements, a lot is left to be improved in his usage.
* Note how a few units to the north distracted majority of the troops to move north. When i threatened from the south, the AI was forced to abandon defending the pass and let me on to the juicy hill/mountain hexes. Its northern troops were too far out to turn around.

There seems to be a bug in movement of healer. several times now, i found that the healer or healer+one unit sneaks behind my lines. I find this baffling as the healer by himself behind my lines can achieve almost nothing and actually wastes his most valuable skill - to heal.

Oh and I should probably note that I am a medium skilled player - I still cannot seem to beat Heir to the Throne on hardest difficulty, so I am sure others here can show you much better replays

Edit: I'd like to request an option to play as defender in this scenario with some tunable parameters on how the AI will attack. I think this will achieve one thing - we can observe replays of players defending and pick out any optimizations AI could make to improve its defense capabilities. For example, I do not even have a suggestion on how to defend against my attack strategy - is it better to split defenders in ratio equivalent to attacker split? Is it better to fall back around the cart and make a stand with rotation of injured units? Is it better to counter-attack before the attacker gets a chance to setup his offensive positions? I cannot figure out answers to these questions without actually playing the scenario where I am the defender. Does anyone have an opinion on how to defend against this kind of attack?