Exercises in Formula and Lua AI and AI-demos add-on feedback

Discussion of all aspects of the game engine, including development of new and existing features.

Moderators: Forum Moderators, Developers

Post Reply
name
Posts: 396
Joined: January 6th, 2008, 3:32 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by name » June 16th, 2014, 3:53 pm

Wow, huge performance difference! After several matches I have not encountered any lua errors either.
mattsc wrote: So, please let me know what the worst individual mistakes are, but more useful than knowing about individual blunders would be behavior patterns that he consistently gets wrong.
The most clear mistake I have noticed so far is retreating a unit to a hex adjacent to a village, when the village location has the same or about the same potential attackers and is about equidistant from the friendly keep as the adjacent hex. It only seems to happen about once per game though.

A second possible issue is on the right side of the map Fred uses the small castle to mass forces regardless of how strong and close an opposing lawful/diurnal force is at dawn or morning. Falling back to the nearby hill gives the enemy one less turn of good ToD to attack and brings reinforcements closer. This is less of an issue with the cheap HP northerners, but when Fred starts playing other match ups, especially drakes versus undead...
Make Fred work on different MP maps. Suggestions with which I should start?
It depends on how extreme a difference you want. Freelands is a strongly partitioned, three theater map where little happens in the center theater. Weldyn Channel and Caves of the Basilisk seem about the same as this.

Silverhead Crossing is a similar deal, but it has more going on in the center theater and is partitioned by water allowing for swimming and flying units to play a special role (unsure if Fred use Naga yet?) It also has some castle-hopping.

Den of Onis and Sablestone Delta are largely a single theater for pitched battles and the latter has multiple castles.

Fallenstar Lake might be the biggest jump from Freelands without getting into game changers like great wall style castles, scripted events or multiple players. Most two player maps have biradial symmetry where each player is incentivized to attack opposite where their opponent is attacking. But fallenstar has an unusual bilateral symmetry. It also has few choke points, a special role for swimming and flying units and distant castle hopping. Perhaps a lot of unique concepts for an AI to adapt to.

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 17th, 2014, 2:02 pm

Hi, Cold Steel. Thank you for testing and the very helpful comments! I'm glad that you can see improvement now, because for a long time that wasn't really the case with Fred... A few quick replies:
Cold Steel wrote:The most clear mistake I have noticed so far is retreating a unit to a hex adjacent to a village, when the village location has the same or about the same potential attackers and is about equidistant from the friendly keep as the adjacent hex.
Yeah, Fred has some issues with villages and retreating units. He does consider whether he can be attacked at a village and might move to the hex next to it if that hex is safe while the village is not, but that doesn't sound like what you're observing. I'll keep this in mind and look out for it specifically. Working on the whole retreating/villages thing is definitely on my list.
Cold Steel wrote:A second possible issue is on the right side of the map Fred uses the small castle to mass forces regardless of how strong and close an opposing lawful/diurnal force is at dawn or morning. Falling back to the nearby hill gives the enemy one less turn of good ToD to attack and brings reinforcements closer. This is less of an issue with the cheap HP northerners, but when Fred starts playing other match ups, especially drakes versus undead...
Thanks, that's good advice. I have started testing with other factions and while Fred still generally wins against the RCA AI, it's not as "convincing" as with Northerners. It's going to take some time to sort all of this out and I'll definitely keep this in mind too.

Thanks for the advice on maps too. I want to get the playing of other factions sorted out first, but I'll get to that eventually...

User avatar
Turuk
Sithslayer
Posts: 5283
Joined: February 28th, 2007, 8:58 pm
Contact:

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by Turuk » June 17th, 2014, 11:38 pm

A replay from my middling attempt with Fred, it was Rebels (Fred) vs Northerners (Me).

The few things that jumped out at me were that Fred left his leader (White Mage) fairly undefended even when I was close enough to attack it with a couple units, ToD notwithstanding. Also, when the two keep hexes were held by my units, Fred proceeded to send his leader off to die. His final move placed the leader within a few hexes of 4 of my units... :whistle:
Attachments
2p_—_The_Freelands_replay.gz
(22.9 KiB) Downloaded 190 times
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 18th, 2014, 2:47 am

Thanks for testing and posting the replay, Turuk. I'm not sure that I like what Fred did all that much. There are clearly many things that he isn't dealing with very well (and I'm not just talking about moving the leader right into the middle of the enemy), but I am not really sure right now how to fix them. Anyways, thanks for testing, I'll keep working on this and will definitely do some tests with saves from this replay.

And just for a bit of comic relief from one of my own tests (incidentally also Rebels vs. Northerners), which hex do you think is best for that shaman to move to in this situation, according to Fred:
suicidal_shaman.png
Spoiler:

name
Posts: 396
Joined: January 6th, 2008, 3:32 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by name » June 18th, 2014, 5:07 am

fred_day_attack_turn20.gz
(27.63 KiB) Downloaded 192 times
This is a fairly odd bug I have not been able to reproduce. On turn 19 Fred 'retreats' an injured second tier archer to a village in hostile territory at dawn (it is quickly dispatched). But then the real fun begins the next turn when he doubles down and charges his whole eastern wing into a solid force of drakes during the morning.
mattsc wrote: Yeah, Fred has some issues with villages and retreating units. He does consider whether he can be attacked at a village and might move to the hex next to it if that hex is safe while the village is not, but that doesn't sound like what you're observing.
After closer examination of one of the replays, it turns out you are correct about this. Fred's near village move was to avoid exposing the wounded unit to a second attacker, which is good thinking on his part. So not a bug after all... sorry, Fred.

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 18th, 2014, 3:06 pm

Thanks for posting that replay, Cold Steel. In general, I like what Fred is doing there much better than in Turuk's replay. I mean, there are still plenty of things he could be doing better, but at least they "make sense" (to Fred, at least). One of the differences might be that Fred's playing Northerners here (as opposed to Rebels in Turuk's replay), which is clearly his best faction. He can technically play the other factions, but there's still a lot of work to be done to get those up to speed. So for the time being, please continue testing him as Northerners. I'll post here when I think he is ready for testing with other factions.

As for the bug:
Cold Steel wrote:This is a fairly odd bug I have not been able to reproduce. On turn 19 Fred 'retreats' an injured second tier archer to a village in hostile territory at dawn (it is quickly dispatched). But then the real fun begins the next turn when he doubles down and charges his whole eastern wing into a solid force of drakes during the morning.
Interesting ... Looking into it, this is caused by the same bug that I mentioned in my previous post. It's fixed in my local version (Fred continues playing defensively on both Turns 19 and 20 now). So let me try to clean up a couple things on which I am currently working and I'll release 0.13.1 later today or tomorrow which has this bug fixed.
Technical reason for the bug:
I'm glad that the moving close to a village move turned out to make sense, at least on some level. Thanks again for the testing and feedback, it's much appreciated!

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 19th, 2014, 4:07 am

AI-demos v0.13.1 has been released on the add-ons server.

The main reason for this release is the fix of the bug discussed in the previous posts. I've also started to adapt things a little more to Fred playing factions other than Northerners and "full offensive mode" (when Fred is clearly winning) is not done by the RCA AI any more but by Fred himself. Both are still work in progress.

The version also includes a variety of minor bug fixes and performance improvements.
v0.13.1 changelog

Code: Select all

0.13.1, 18 June 2014
- Main reason for this release: fix for a serious bug in Fred's attack evaluation that
  made Fred do extremely suicidal moves when an enemy unit was close to leveling up
- Other changes (some of them still work in progress):
  - Fred: Tuning of evaluation functions to improve playing of factions other
    than Northerners
  - Fred: Take counter attack rating into account in picking the best attack (this
    makes evaluation times a little longer, but should result in better attacks)
  - Fred: full offensive mode is now done by Fred, not left to the RCA AI any
    more (this is still work in progress)
  - retreat.lua: a variety of bug fixes, ensuring that injured units do, in fact,
    always go to villages or hexes adjacent to healers if possible
  - Miscellaneous performance improvements, minor bug fixes and code cleanup to
    Fred and helper libraries

Duthlet
Code Contributor
Posts: 49
Joined: January 9th, 2007, 8:12 pm

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by Duthlet » June 19th, 2014, 12:06 pm

I got a lua error in my first game against Fred with v0.13.1 :-D
2p_—_The_Freelands_Turn_15maybe.gz
not just maybe, I can definitely reproduce the lua error by ending my turn here
(33.25 KiB) Downloaded 186 times
It seems to me, that his biggest weakness is putting single units or a small group of units to be trapped/killed (with enemy being much stronger at that location and time)
In this game he already lost 3 units that way on second day (turn 6-8) not counting the grunt that stole a village and died for the 3 gold he denied.
Always retreating injured units to villages also didn't work, there's a wounded archer staying at 27,11 for 5 turns before it until it gets attacked.
Two minor notes:
Is this still supposed to be a rush ai? It puts hardly any pressure on the enemy's villages at first night.
Does Fred consider blocking the keep? He had the possibility to do so at turn 11 but preferred taking a village. In this case it's hard to tell which is the better choice, but it can be crucial in other situations.
Attachments
2p_—_The_Freelands_replay2.gz
the replay
(19.63 KiB) Downloaded 190 times

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 19th, 2014, 1:53 pm

Duthlet wrote:I got a lua error in my first game against Fred with v0.13.1 :-D
:D Yay, thanks! I can reproduce it and know what causes it, so it will be fixed in the next version
Duthlet wrote:It seems to me, that his biggest weakness is putting single units or a small group of units to be trapped/killed (with enemy being much stronger at that location and time)
I see what you mean. To be honest, I am not displeased with what Fred's doing in that replay. I mean, obviously he can't match your level of defensive positioning, but compared to what he used to be doing, this isn't so bad. Maybe I just have my standards set to low. ;) I'll definitely see what I can do about though.
Duthlet wrote:Always retreating injured units to villages also didn't work, there's a wounded archer staying at 27,11 for 5 turns before it until it gets attacked.
Yeah, that's certainly not ideal. The problem is getting the balance right between using a somewhat wounded unit for holding an area (or attacking) and retreating it. I mean, this is a pretty clear case, but there's a large gray area where one-size-fits-all evaluation functions don't always work. I'll look into this too.

One other thing I am noting is that he should put more units up against your troops advancing in the west. Getting that balance right has always been a struggle ...
Duthlet wrote:Is this still supposed to be a rush ai? It puts hardly any pressure on the enemy's villages at first night.
No, not really. I've actually been wondering about renaming it. There's nothing in the code any more that specifically rushes units, other than that if there are "spare units" Fred'll move them down the right side but he does so pretty cautiously these days. As background information: there are really no set prescriptions of how to play any more, other than that I define the 3 zones for Fred together with a bunch of parameters (which can be different from zone to zone). He then decides himself when and where to attack, which hexes to pick for setting up a defensive line, etc. In the process, he has lost some (most?) of the rushing quality. I think I can get some of that back simply by changing some of the zone parameters though, at least when he's playing Northerners.
Duthlet wrote:Does Fred consider blocking the keep? He had the possibility to do so at turn 11 but preferred taking a village. In this case it's hard to tell which is the better choice, but it can be crucial in other situations.
No, he does not (yet). He also does not specifically trap the leader off keep. Those are among the things I want to add.

Thank you for the comments. This is great! The feedback I have gotten from you guys in the last few days is already worth a lot more than 100s of games against the default AI.

name
Posts: 396
Joined: January 6th, 2008, 3:32 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by name » June 19th, 2014, 6:34 pm

This match ended in a lua error which seems to somehow have triggered a premature victory screen:
crash_replay.gz
(20.46 KiB) Downloaded 203 times
pre-crash_save.gz
(40.84 KiB) Downloaded 188 times
I was able to continue playing after reloading the last auto save but as I took over Fred's castle, his thinking took longer and longer until reaching a freeze or near freeze:
very_slow_or_frozen_turn20.gz
(38.83 KiB) Downloaded 193 times

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 20th, 2014, 2:18 am

Cold Steel wrote:This match ended in a lua error
Ah, good one. The problem here is caused by a unit being on the edge of the map and the engine trying to find a hex on the opposite side of it (that is, off the map).
Cold Steel wrote:which seems to somehow have triggered a premature victory screen:
Parts of the AI code take units off the map or move them around to play through certain what-if scenarios. If the crash happens while units are off the map they won't get replaced, which at best causes OOS errors and at worst prematurely ends the game (if the leader is off the map at the time).
By contrast, if the crash happens when all units are on the map, the custom AI code gets abandoned and the default AI takes over. Those are sometimes almost imperceptible, which is why I introduced the "please let us know" error message for that case (like the one Duthlet found).
Cold Steel wrote:I was able to continue playing after reloading the last auto save but as I took over Fred's castle, his thinking took longer and longer until reaching a freeze or near freeze:
That's Fred's defense against losing. When he notices that he's in over his head, he slows the game down so much that the opponent is likely to get bored and give up! :mrgreen:

But seriously: Thanks, I can reproduce that too. I'll see what I can do.

Looks like 0.13.2 will have to be released sooner than I thought with fixes to the bugs you and Duthlet found. Thanks!


================== EDIT ==================

v0.13.2 is up on the server featuring fixes for the bugs found by Duthlet and Cold Steel, as well as a cap on the number of counter attack combinations considered. The latter should prevent those slow downs observed by Cold Steel with barely any perceptible change in behavior. Evaluation times might still take a couple seconds in the busier settings, but hopefully they won't be longer than that.
v0.13.2 changelog

Code: Select all

0.13.2, 19 June 2014
- Another bug fix release:
  - Fix a couple bugs in Fred that sometimes caused the AI to crash (with one of
    them even ending the game sometimes)
  - Limit number of counter attack combos considered in order to speed things up again
    after the change in attack evaluation in 0.13.1

User avatar
Rigor
Posts: 941
Joined: September 27th, 2007, 1:40 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by Rigor » June 20th, 2014, 3:28 pm

Image
Figure: crazy solution out of this: kill the troll with low HP by sacrificing two low hp fighters.

Today i had an encounter with the AI and im quite pleased. It works as it should and even though small blunders were done this is quite acceptable. Which ones are the blunders: for once, the western aggressive behaviour is not too well developed, there are rarely 1-turn-grab situations here. The AI doesnt position itself well enough to grab villages it wants to threaten. for the east side the retreat is performed in an orderly fashion and if it wasnt for the western non aggressivity pact this would have meant a steady income and steady supply of expendable grunts who die during each assault. all in all i can recommend this behaviour development.

On another note, Id like to know how well does an integration of critical map specific position influence the behaviour? something like "this is important, try to hold it at more cost than you would any other position"? i think this would be key to improving the behaviour by prior knowledge. every smart player can tell you where a retreat makes no sense or where one should move to squeeze the life out of the opponent.
Attachments
AI_June_2014.gz
AI june2014
(21.46 KiB) Downloaded 195 times

User avatar
Rigor
Posts: 941
Joined: September 27th, 2007, 1:40 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by Rigor » June 20th, 2014, 3:47 pm

and just out of curiosity i tried AI vs AI too. it is pretty scary how your version rocks from turn 3 (first contact with enemy) onwards. i wouldnt be surprised at all if the win-loss ratio was 1.0. it took only 10 turns! gj!

mattsc
Posts: 1105
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by mattsc » June 21st, 2014, 1:31 am

Hi Rigor,

Thank you very much, but, umm, what version did you play this with? The version info in the replay says 1.10.0. Current Fred doesn't work with Wesnoth 1.10, it requires 1.11.7 at the very least. And the behavior doesn't look very much like what he does these days either. :shock: So, did you play with the version on the 1.10 add-ons server (0.11.2), which coincidentally is still much more of a grunt rush than what Fred does these days? The latest version is only available on the Wesnoth 1.11 (or 1.13) server.
Rigor wrote:and just out of curiosity i tried AI vs AI too. it is pretty scary how your version rocks from turn 3 (first contact with enemy) onwards. i wouldnt be surprised at all if the win-loss ratio was 1.0. it took only 10 turns! gj!
Well, in the batch test I did with v0.13.0, it was 299:1. So not quite 1.0, but 0.997 isn't all that bad either.


Also, I found another one of those bugs today that crashes the AI so badly that the game ends. So I released v0.13.3 with the only change being that this bug is fixed.
v0.13.3 changelog

Code: Select all

0.13.3, 20 June 2014
- Fix for bug that prematurely ended the game in rare cases when the enemy
  has units with the hides ability
As a side note, I will be traveling for the next 9 days. I'll still have Internet access most of the time, I think, but probably won't have much time for immediate bug fixes etc.

User avatar
Rigor
Posts: 941
Joined: September 27th, 2007, 1:40 am

Re: Exercises in Formula and Lua AI and AI-demos add-on feed

Post by Rigor » June 21st, 2014, 1:57 pm

oh-ho theres even a better version than this?? :)))

ok i played three games on the newest version of wesnoth, one i tried with double gold but lost because i drew my troops a bit too far apart - unfortunately this game was lost somewhere in the aether :mrgreen:, the other two were much easier.

What i would like to point out is that the ai really plays nicely compared to all other versions ive seen. the gameplay is more than solid, a noob will not stand a chance and intermediate players shouldnt have a too easy time either. Whys that? because the ai really tried to make solid attacks which are based upon reasonable situations. a really good palyer decides a lot of the game in his early moves. Is there a feint, or not, will the attack force be enough for a successful village hold or not, and such are rather hard to compute, for human players less hard :geek: . Although much depends on intuition and experience. Here for instance i saw the ai taking a village but not pressing the point further like any other human would have done. or protecting weak units from slaughter. Whats not so nice is the hard definition, where to start and where to end an attack. often it attacks while drawing back some troops first. doesnt make too much sense. a last thing: the ai must protect the king by making a smart recruit in one of the positions where ZoC is established (drake game) - perhaps very specific but required to not lose the game.

matt, how much prior info for freelands is in that script of yours? could i try it in world conquest i wonder (i tried but cant see the smart AI playing because theres too much of the AI and too few vills)? whats required to make it work? is there some quick fix to make it work quick and dirty style? :lol:
Attachments
AI_J.gz
(20.99 KiB) Downloaded 183 times
AI_Jun_2014.gz
(21.06 KiB) Downloaded 183 times

Post Reply