Help with testing Fred - Freelands MP Custom AI v0.14.15

[pauxlo]

In principle people could even add comments there, but I have no idea if one needs a github account for that and whether I need to set up the permissions differently but, quite honestly, I am way too lazy to try to find out ...

For your curiosity: For commenting one needs a Github account, but not any special settings for your repository.

[mattsc]

For your curiosity: For commenting one needs a Github account, but not any special settings for your repository.

Thanks. The former I thought was the case, the latter I was not sure about.

In the meantime, I am making slow progress on Fred. I think I have the mechanics of protecting the leader down, but it needs some fine-tuning. In general, I am finding yet again that the whole defensive unit positioning continues to be the hardest part for me to get to an acceptable level. I probably should shelf working on the details of that for now and give it some dedicated time for a later release.

Also, just for kicks* I did a bunch of batch testing of Fred (100 gold) vs. the default AI playing drakes (with 160 gold; that's important, at the same gold the default AI just gets trounced). That's the match-up with which Fred usually has the most problems. For previous version, he averaged around 50% win rate. Currently he's winning between 70 and 80%. I know that doesn't necessarily mean that he'll also do better against a human opponent, but at least I don't seem to be making things worse.

*Well, not just for kicks. The other purpose of this is to run through a lot of games to see if there are any errors. I did find a couple that way ...

[name]

What do you think is mainly responsible for his improvement against drakes?

[mattsc]

What do you think is mainly responsible for his improvement against drakes?

I don't think that there's a single change that is the main reason for this, but the overall effect of many gradual changes is that Fred does better defending now, and in particular protecting his leader, when he's up against a stronger force. The main factors contributing to this are defensive unit positioning in the tactics layer (while I am saying that that's still very hard for me to figure out, it is better than it used to be) and better overall orders and unit distribution in the operations layer.

I probably should shelf working on the details of that for now and give it some dedicated time for a later release.

So, I have thought about this some more and put together a rough road map for the next few releases in Issue #12 on the AI-demos site. I think splitting it up like that rather than trying to tackle it all at once will help the overall process.

[mattsc]

Fred / AI-demos v0.14.4 is now on the Wesnoth 1.12 add-ons server

This version addresses several of the issues raised by ForPeace. See
Issue #12: Fred: Tasks for the 0.14.4 release
Issue #13: Fred: Tasks for the 0.14.5 release
for a summary of what was done and a tentative plan for the next couple releases. If you tested Fred v0.14.3, the differences might or might not be noticeable, I'm not sure. Also see the known issues listed below.

v0.14.4 complete changelog

Code: Select all

----- 0.14.4, 3 June 2017 -----

- Fred (Freelands AI for Northerners, Side 1):
  - Redo retreat action:
    - This uses a very different evaluation method from the previous version
    - It now includes multi-move retreats
  - Holding
    - Improve holding for leader protection:
      - Evaluation now mostly based on reduction of threat to leader
      - Combine this with zone holding (overlapping holding zones don't work)
    - Prefer stronger units for village protection
    - Better implementation of the "between map"
    - Some adjustments to other evaluation functions
  - Operations:
    - Reduce resistance to moving units between zones
    - Use utility functions to balance retreating vs. holding vs. scouting
    - Add an all-map unconditional advance action
    - Reset significant_threat mid-turn if needed
  - A variety of small rating adjustments and bug fixes
  - Fred utils: new function inverse_cost_map
  - Fred debug utils: add a menu option to show units info
- Miscellaneous:
  - ai_helper checked actions: output error message only to stdout

Known or expected issues

• Defensive unit positioning (called 'holding' in Fred) still leaves quite a bit to be desired. Improving this will require a dedicated effort and is postponed as per the roadmap shown in the links above.
• Evaluation times when the leader is threatened can sometimes be long in this version. This is due to both the attack and holding evaluations. I will work on that once I have the functionality sorted out (that is, a couple releases down the road, probably).
• While batch testing Fred vs. the default AI, I encountered some errors. I fixed most of those. Others were not fixed directly but did not recur, so I don't know whether they were fixed by some other change I made or whether they are just rare. If you encounter an error, please let me know, ideally together with a save/replay.
• I'm not sure whether I have the balance between actions right yet for some of the newer changes (specifically retreating). Fine tuning this will be a continuous process throughout the next releases.

Interesting batch testing result

At some point I joked that AI-demos will be upgraded to v1.0.0 when Fred wins 100% of games against the default AI with equal gold, on any map, as any faction. So just for fun I decided to test how he does as Northerners on Freelands (this was a little bit before the 0.14.4 release version, but not much). Result from 50 games against each default faction:

Code: Select all

      [ wins (and percentage) / losses / total games ]
Drakes             50 100% / 0 / 50
Knalgan Alliance   50 100% / 0 / 50
Loyalist           50 100% / 0 / 50
Northerners        47  94% / 3 / 50
Rebels             50 100% / 0 / 50
Undead             50 100% / 0 / 50

The three losses against Northerners are interesting. Since batch testing is done in no-gui mode, I cannot go back and see what happened in those. If this trend persists after the planned improvements to attacking and holding, I will look into that more closely.

[The_Gnat]

I think it is great that the AI is being developed further! I am wondering though from the batch result does this mean that Fred as the Northerners lost 3 games against the Northerners using Default AI, but won every other game?

Also if i download the Fred AI can i play against it on other maps beside freelands?

Thank you!

[mattsc]

I am wondering though from the batch result does this mean that Fred as the Northerners lost 3 games against the Northerners using Default AI, but won every other game?

Yes, that's what that means. That's nothing new though, he's been achieving these kinds of win ratios for several years. See for example here.

Also if i download the Fred AI can i play against it on other maps beside freelands?

Not at this time, there is quite a bit of map-specific code. When I get Fred to the point that's "good enough" on Freelands — if that ever happens — then I'll work on adapting this to other maps.

[The_Gnat]

Not at this time, there is quite a bit of map-specific code. When I get Fred to the point that's "good enough" on Freelands — if that ever happens — then I'll work on adapting this to other maps.

Okay. Hopefully that will be able to happen sooner rather than later!

As for the impressive win ratio why do you think Fred lost at all to the northerners?

Also is Fred able to play any other teams?

[mattsc]

Okay. Hopefully that will be able to happen sooner rather than later!

No promises.

As for the impressive win ratio why do you think Fred lost at all to the northerners?

I really don't know. I'd have to look into this, but there's not really a point in doing so until I have worked on some of the obvious current weaknesses as outlined in previous posts.

Also is Fred able to play any other teams?

In principle yes, but it's currently disabled. It's pretty easy to hack around that though. Just remove the first conditional in line 62 here. He won't do as well with other factions though as he has no idea how to use certain abilities (like healing or leadership).

[The_Gnat]

Okay thank you!

[The_Gnat]

I just downloaded the add-on to look at it and i am impressed! I thought it would be simpler but it is certainly complicated and i respect that it will take a long time to turn this into a full usable AI!

[name]

14.4_versus_drakes.gz

(27.93 KiB) Downloaded 305 times

Replay against 14.4 (no gold bonus for this one to make his unit placement decisions more clear) picked random side which turned out to be drakes.

It has been some time since I played a match against Fred, so I may have forgotten how good he was in previous versions, but it felt like he was more responsive in switching between overall attack and defend and his defensive line forming was more apparent. I thought he did quite well given no gold advantage and played very humanly overall. As such, I'll only list the (possible) mistakes he made because listing the things he did right would be too verbose.

Bigger:

His biggest mistake may have been not retreating his western forces further north and into more defensive terrain on turns 19-20. This might be considered the turning point of the battle.

Smaller:

Orcish Archer left in open at day on turn 7 at (15,13) and then again at dawn on turn 18 at (14,14). On turn 19 at day, does something similar with orcish grunt at (24,10).
On turn 21 at (22,8) an assassin is left in the open, although in this case he was pushed out there by an archer that needed his hex to make a kill shot on one of the drakes.

On turn 27, the king moves off keep to less favorable terrain.

The only real evaluation slow down also took place on turn 27.

Fun:

At turn 23 the adventurous orcish grunt has single-handedly taken the enemy castle at (19,20), but witnessing from its battlements the return of the dragon horde, has a sudden crisis of courage.

[mattsc]

Thanks, Cold Steel, for the replay and the useful comments. And for playtesting in the first place!

I watched the replay first and then read your comments (and then I watched parts of it again). I thought it was fun to watch those back and forth dynamics play out with the ToD. Fred's not perfect at it, but he's "making a good effort" by now. I was pleased to see that. And I'm pretty sure that you're right and he wouldn't have been able to do this in previous versions to the same extent. Maybe to a lesser degree in 0.14.3, but not before that.

I had also noticed Fred several times leaving individual units stranded too far in enemy territory, as you point out. That's a "known problem" and caused by a mix of several issues, some in the unit positioning and some in the operations layer (overall coordination). As you can see from the current AI-demos roadmap, I am planning to deal with those for 0.14.6 and 0.14.7. I am going to come back to your replay then, it provides several really good test cases for that!

For 0.14.5, I have started to dig into the attack evaluation. I'm not sure if these will make an significant difference overall, or only in specific cases, but there's a whole bunch of problems with that that I've always pushed off dealing with.

Fun:

At turn 23 the adventurous orcish grunt has single-handedly taken the enemy castle at (19,20), but witnessing from its battlements the return of the dragon horde, has a sudden crisis of courage.

Ha, yeah, that is pretty funny!

[name]

No problem, playing against Fred is a fun change of pace from the zombie rushing of RCA and the 5 minute waits between turns in multiplayer. As more of the community finds this out for themselves you should have quite a few more testers.

I had also noticed Fred several times leaving individual units stranded too far in enemy territory, as you point out. That's a "known problem" and caused by a mix of several issues, some in the unit positioning and some in the operations layer (overall coordination). As you can see from the current AI-demos roadmap, I am planning to deal with those for 0.14.6 and 0.14.7. I am going to come back to your replay then, it provides several really good test cases for that!

Yeah, I saw that under the known issues list and almost didn't report instances of it, but decided it was better to err on the side of being thorough. Glad it turned out to be useful rather than annoying.

For 0.14.5, I have started to dig into the attack evaluation. I'm not sure if these will make an significant difference overall, or only in specific cases, but there's a whole bunch of problems with that that I've always pushed off dealing with.

Indeed, that is quite a list.

I did notice some things that might hopefully be relevant to attack evaluation issues in this next replay:

14.4_versus_knalgans.gz

(24.49 KiB) Downloaded 288 times

Notes:

Force deployment (beginning on turns 5-6) focused a bit too heavily on the center theater. These forces may have done more good threatening/rushing the knalgans in the eastern theater, because defending the center requires very minimal resources. I think he may have gotten spooked by forces I seemed to be amassing in the center, though in reality they were only passing through to reinforce the western theater.

The retreat in the western theater at turn 8 may not have been far enough and ended up adjacent to castle terrain for the enemy to attack from.

It is interesting that in this match up he often puts out defensive screens of assassins rather than trolls on open ground. He might be evaluating the assassin’s superior terrain defense without fully taking into account the resistance, cost and/or regeneration advantages of trolls. There are good examples / test cases of Fred’s assassin lines from turn 10 onwards in the western theater.

He maybe could have counter attacked in the west on turns 17-19. Though the dwarfish line was formidable, its center was on poor terrain and it was dusk, plus Fred’s castle was close by for quick reinforcements. This may be a difficult thing to improve because around the this same time and place in the drake match, Fred suffered for not being defensive enough and here against dwarfs he suffers for not being offensive enough.

On turn 21, the slaying of the dwarf at (13,5) was well executed by Fred, first attacking with the king from strong terrain and then well protecting his flanks with heavies.

[mattsc]

No problem, playing against Fred is a fun change of pace from the zombie rushing of RCA and the 5 minute waits between turns in multiplayer. As more of the community finds this out for themselves you should have quite a few more testers.

That's great to hear, thanks.

Yeah, I saw that under the known issues list and almost didn't report instances of it, but decided it was better to err on the side of being thorough. Glad it turned out to be useful rather than annoying.

Absolutely! Please, never hold back on any comments you have. You may notice something I didn't, you probably have a different perspective or different priorities. And even if you "only" say exactly what I am thinking, it will be good to get the confirmation from somebody who is much better at this than I am. I'm not that good a player in the first place (in MP at least, I seem to be doing semi-ok in SP*).

Indeed, that is quite a list.

Well, I've done a bit of work on the attack code by now. [ You could, in principle, check the commits on github, but I really don't know why anybody would do that to themselves. ;P ] However, so far it's all been just internal refactoring. I am partly cleaning up the code that has "grown" over several years, and partly I am checking that the approximations I am making in the attack evaluations are good enough. Hopefully I will get to things that actually affect behavior soon(ish).

I did notice some things that might hopefully be relevant to attack evaluation issues in this next replay:

Perfect, thanks! It really is fun for me to watch these replays, seeing that Fred more or less does the things I am trying to teach him to do. With mistakes, sure, but overall about as good as I could hope for at this stage. He does like to put assassins on the front line, doesn't he? I'm sure you're right that he is likely putting too much emphasis on the terrain defense.

I'll leave it at that for comments for now. As you say, these provide some great test cases. I'll report back on this when I get to dealing with them. Thank you again!!


* As a side note, coming from my SP background: one of my far-in-the-future goals once I start generalizing Fred for other maps/factions is to set up an AI that can play through and win HttT on medium. Revealing my hubris here, I've decided that doing so on easy is too easy.