Help with testing Fred - Freelands MP Custom AI v0.14.15

[mattsc]

Fred / AI-demos v0.14.6 is available on the Wesnoth 1.12 add-ons server

The emphasis of this version was on improvements to defensive positioning (what I call 'holding'), advancing and retreating. In the process, I realized yet again that holding remains, and likely will remain, the weak link of the AI. I could probably continue to sink a lot of time into this without it making much of a noticeable difference.

So, I've decided to call it good enough and move on. There's one more area that I want to concentrate on, the operations layer, so that will be the emphasis of the 0.14.7 release (see the link to Issue #15 below). After that, I will start to try generalizing Fred to other factions, playing Side 2, other maps, scenarios that have allies or more than one enemy ...
One of my "secret" long-term goals is to turn Fred into an AI that can win HttT on medium difficulty. Previously my main concern with that would have been his ineptness at leveling units and then protecting them, but he's gotten a lot better with that over the last couple releases, so maybe this is not as utopian any more as it once was.

This does, of course, not mean that I will not try to improve Fred further, quite the contrary. In fact, I am hoping that putting Fred into different situations (in particular onto different maps) will make it more apparent what works and what does not, and will result in an overall better AI than if I continued fine-tuning things on Freelands forever.

For all the gory details, see:

Issue #14: Fred: Tasks for the 0.14.6 release
Issue #15: Fred: Tasks for the 0.14.7 release

for a summary of what was done and a tentative plan for the next few releases and:

v0.14.6 complete changelog

Code: Select all

----- 0.14.6, 27 Oct 2017 -----

This release focused on Fred's non-attack actions, mainly defensive positioning
(holding), advancing and retreating. The evaluation functions of these actions
saw many changes, including but by no means limited to those listed below.

- Fred (Freelands AI for Northerners, Side 1):
  - Holding:
    - Added multi-hex pre-rating
    - Improved directional formation rating of hold combos
    - Added counter attack evaluation for the best combos
    - More carefull use of high-XP units
    - Improved combo rating when a unit is unreachable by enemy
    - Fixed (not) assigning extremely injured units as holders
    - Added discarding of holds with high CTD that do not protect assets
    - Holding is now preferentially done with the strongest (highest HP) units
    - Many more small changes to the holding evaluation functions
    - Functional:
      - Separated protect and no-protect holding evaluations
      - Completely changed "between_map" evaluation and application
  - Advancing:
    - Added desperate attacks
    - Fred is now much more careful with high-XP units
    - Based secondary rating on counter attack outcomes
    - Do not move leader off keep for advancers
    - Always give small rating bonus for villages
  - Retreating:
    - Coordinated multi-unit retreats
    - Made retreat direction dependent on enemy threat
    - Improved multi-turn retreat rating
  - Misc other changes:
    - Move execution: improved how units move out of way
    - Minor changes and bug fixes to attack rating
    - Attack utils attack_combo_eval: removed ctd_limit
    - ai_helper move_unit_out_of_way: fixed bug in direction rating
    - Fred utils: new function fgumap_normalize

[The_Gnat]

I am glad to see this progressing! Personally i look forward to the day (far out, though it may be) when Fred is capable of playing any multiplayer scenario, and even winning a campaign.

Since you have uploaded it to the add-on server i assume you would like some people to test it. I can't garentee i will have time, or any useful feedback but what areas in particular would you like tested, and what sort of feedback would you be interested in?

[mattsc]

Thanks, The_Gnat. Yeah, that day may indeed be a long time away, we'll just have to wait and see.

Since you have uploaded it to the add-on server i assume you would like some people to test it. I can't garentee i will have time, or any useful feedback but what areas in particular would you like tested, and what sort of feedback would you be interested in?

Yes, absolutely. Any feedback is welcome, but if you want to see my thoughts on that, check out the first post in this thread. I haven't updated it for 0.14.6 yet, but glancing over it quickly it is still pretty much up to date.

I guess one thing I don't mention there is that I'd be interested in knowing how Fred does if you give him a bit more gold than the player, say 125 vs. 100, maybe even 150 vs. 100. Does that make for an interesting game? I still don't expect Fred to win then, but maybe it's a bit more challenging than against the default AI under the same circumstances?

[Zap-Zarap]

Hello.
Here is a fresh replay for you: replays.wesnoth.org/1.12/20171029/2p__The_Freelands_Turn_35_(14801).bz2

I played very badly on purpose to see what would happen .
That Fred Ai is truely defensive, they just never attack, even when they had many more units than me.
Only around turn 18-20 there were some notable lags on AI turn calculating.

It's fun to see them making those small groups of units

[mattsc]

Hi Zap-Zarap,

Wow, that's really bad what Fred is (not) doing there. Thanks much for testing and posting the replay!

Defensive positioning is what I am struggling with most in this AI. I am specifically trying to include it, but I am finding it hard both to get it working acceptably at all and to get the balance vs. other actions right. It's clear from your replay that it is given too much weight at the moment (I'm also getting the impression that it got a little worse for this last release since I concentrated on these aspects of the AI).

I was aware of the fact that Fred is much to defensive when winning, there's even a note on it in the tasks for the next release:
- More aggressive advancing when winning
but maybe not to the extent seen in your replay. I do have knobs in the code to make things more aggressive or defensive. Just that as as with all balancing, I am finding it difficult to make it robust in different situations.

Anyway, long story short, I'll work on it. Replays like this help a lot, thanks!

[Poison]

Here's mine too (mirror match -random), I gave Fred 150 gold (I think), he went for the luxurious assasins and I went for the cheapo grunts ofc. In the end I fed up, his left side felt kinda weak and my leader helped to also rush through the middle. (Fortunately, the rush worked, Fred didn't respond well there) so I managed to win, but I could have easily lost this so Fred is not bad at all I guess (but then again I'm a terrible player). The start was unlucky, the finish lucky, overall we both were break-even give or take so the match is kinda representative. I think the major leak of the AI is that it is not aggressive enough on it's favourable TOD. Not sure what Fred was trying to do with the isolated units of his, probably steal a village? No idea.

2p_—_The_Freelands_replay_FredvsPoison.gz

(33.48 KiB) Downloaded 329 times

[Zap-Zarap]

Not sure what Fred was trying to do with the isolated units of his...

Maybe Fred loves building groups of units that much, that he simply doesn't know what to do with units, that cannot be included in any group?

[mattsc]

Thanks, Poison. I'm not entirely sure what to say. I see some things I like (as in, I'd expect Fred to do that) and quite a few that I do not like. Overall, it looks like Fred's play's not where I'd like it to be yet. And frankly, I don't know if it'll ever get there, but I am not quite ready to give up yet. In any case, thanks for posting the replay, I'll have to go through it a few more times when I have more time and see what I can do with it. I know that some of the current problems will be addressed in the operations layer work I am planning to do next. Whether that will actually make an appreciable difference in the end is a different question.

Not sure what Fred was trying to do with the isolated units of his...

Maybe Fred loves building groups of units that much, that he simply doesn't know what to do with units, that cannot be included in any group?

Something like that ... I'll have to check it out more carefully, but I think he's trying to move them out of harms way, not noticing that they get trapped that way.

In any case, thanks again to everybody who has (and will be) posting replays. I do get a lot out of them. I hope it's not too painful playing against Fred ...

[singalen]

Hi. I'm thrilled about new Wesnoth AIs and I see a lot of potential here.
I ran into Fred when reading this and would love it to be running in mainline one day.

Trying it on 1.13 now, I get a bunch of errors:

20171029 22:51:02 error scripting/lua: ~/add-ons/AI-demos/lua/ai_helper.lua:212: attempt to call a nil value (field 'check_stopunit')
stack traceback:
~/add-ons/AI-demos/lua/ai_helper.lua:212: in field 'checked_stopunit_moves'
~add-ons/AI-demos/lua/fred.lua:5157: in method 'remove_MP_exec'
[string "(...):remove_MP_exec()"]:1: in main chunk

and it doesn't recruit.
Thanks.

[mattsc]

Thanks for the show of confidence, singalen, but I wouldn't hold my breath regarding mainline. I honestly do not know whether the concepts I am using even work on a general basis. For now, I'd just settle for getting Fred to be an interesting opponent on Freelands.

As for 1.13, I looked into it once and determined that it was not trivial to port the code, but I forgot why. I'll have another look sometime soon.

So, based on feedback and replays of the last couple days, I'll make some adjustments to the plan:

• Instead of working on all tasks of Issue #15, I'll concentrate on those adjusting Fred's aggressiveness first. That's clearly the biggest problem at the moment. If I get that figured out, I'll put out a new release with just those changes. Even that is going to take a while though. I don't just want to increase the value of the aggression parameter(s), as I know that that will cause other problems.
• I'll check out again what porting to 1.13 involves (and hopefully do it).
• Then I'll move on to the other operations layer tasks listed in the issue.

[mattsc]

In a very uncharacteristic move, I released the next version of AI-demos with a rather half-baked fix for one of the issues. I am going to be away for most of the next few weeks and just couldn't leave Fred in that overly scaredy state in which he'd been since the 0.14.6 release. So ...

Fred / AI-demos v0.14.7 is now available on the Wesnoth 1.12 add-ons server

The main change is that Fred now uses a dynamic assessment of the AI vs. enemy power situation, taking time of day into account, to judge how aggressive he should be at any given time. This is applied to all attacks and to defensive positioning. It does not quite fix all of the aggression-related issues (for example, it's not used for deciding which units to retreat yet) and, in fact, it over-compensates a bit when attacking, making Fred too reckless at times. But if anything that's more appropriate for Northerners anyway.

None of the other known issues have been addressed yet, so Issue #15 remains unchanged, just that it applies to the 0.14.8 release now.

v0.14.7 complete changelog

Code: Select all

----- 0.14.7, 2 Nov 2017 -----

This release is mostly a quick and somewhat superficial fix for Fred's
exaggerated defensiveness introduced in the previous release. It does not solve
all the problems and, in fact, introduces a couple new ones, but it should
overall be an improvement.

- Fred (Freelands AI for Northerners, Side 1):
  - Flexible assessment of Fred's aggressiveness:
    - Use time-of-day dependent current power ratio to determine aggressiveness
      of all attacks and holding
    - Lower enemy chance-to-die limit for allowable attack next to village
    - Disallow attacks next to village only if target can reach the village
  - Misc other changes and bug fixes:
    - Add a behavior instructions table in the operations layer (still mostly
      unused)
    - Fix bug causing rare AI crash when assessing village grabbing

[name]

I played some rounds against v14.7 with extra income. His lines seem a good deal stronger with the tactical improvements, as he doesn't leave many good targets open to attack anymore. Here's a replay with +4 income against loyalists, it gets interesting in particular at and beyond turn 10 with good positioning and massing from v14.7:

loyalists_vs_0.14.7_income+4.gz

(26.71 KiB) Downloaded 257 times

Defensive positioning is what I am struggling with most in this AI. I am specifically trying to include it, but I am finding it hard both to get it working acceptably at all and to get the balance vs. other actions right.

Well, positioning does seems to be working well already. It is good to keep in mind that none of the other AIs do this essential thing at all really.

That being said, at some point in the future, you might consider writing up a map editor feature request for an AI "game plan" tool, to make positioning a content-side issue that takes advantage of human preprocessing power. You could even have competitions for each map between Freds using game plans by different authors, see which wins most out of 100 duels.

This might be rendered unnecessary though after the operations layer overhaul, by having such a greater impact at this point versus further improvements to the tactical layer. Holding with minimal units in the northwest while pressing heavy attacks against the player's southeast villages could be surprisingly devastating, for example.

[mattsc]

Thanks, Cold Steel, for yet another round of testing and for the replay and comments!

Well, positioning does seems to be working well already. It is good to keep in mind that none of the other AIs do this essential thing at all really.

You're right, the defensive positioning is not perfect, but it's probably about as good as I can expect it to be. The fact that Fred does this and none of the other AIs is probably the main reason why he has such a high win rate against the default AI. But it is also becoming more and more obvious to me why this is not a common feature in AIs. It's just really hard to get it working well(ish) consistently — the operative word being consistently.

That being said, at some point in the future, you might consider writing up a map editor feature request for an AI "game plan" tool, to make positioning a content-side issue that takes advantage of human preprocessing power. You could even have competitions for each map between Freds using game plans by different authors, see which wins most out of 100 duels.

Hmm, interesting suggestion. It's always been my eventual goal for Fred to be able to handle any (well, most any) map on his own, but with there being configurable options to improve performance on any specific map. I hadn't thought of making that an editor feature though; or about doing tournaments. I'll keep those in mind, it's going to be a while until we get to that point.

This might be rendered unnecessary though after the operations layer overhaul, by having such a greater impact at this point versus further improvements to the tactical layer.

Well, as before, we'll have to see if it makes a noticeable difference. I think the main improvements will come from not exposing small groups of units too much (as Fred did, for example, in your replay in the southeast; although I do like the unkillable troll whelp!) and coordinating actions some more.

Holding with minimal units in the northwest while pressing heavy attacks against the player's southeast villages could be surprisingly devastating, for example.

Funny you should say that. Back a long time ago, when Fred was only 400 lines of code, he started as a grunt rush AI and that's exactly what he was doing. Working on the balance of distributing units between the parts of the Freelands map is certainly on my task list for the operations layer work and there are already hooks in the code for it. Also, those (e.g. being able to instruct the AI on the relative importance of the zones) are the kinds of things that I am planning to leave as configurable to the scenario creator.

[mattsc]

I am going to do a double post here, as release announcements really should get their own posts.

As discussed not too long ago, I ported Fred to Wesnoth 1.13 and released Fred / AI-demos v0.14.8 on the Wesnoth 1.12 and 1.13 add-ons servers. This release features no behavior changes at all, only (a lot of) internal refactoring and porting of code.

I have done some basic testing in 1.13 and things generally seem to work. However, I will continue to do my main work in 1.12 for the time being and 1.13 is a bit less forgiving with sloppy Lua code, so it is not at all unlikely that there might be some error messages popping up in 1.13. If you encounter any of those, please post the error message and, ideally, a save from which this can be reproduced.

The v0.14.8 changelog is very short:

Code: Select all

----- 0.14.8, 8 Nov 2017  (port Fred to Wesnoth 1.13) -----

- Fred (Freelands AI for Northerners, Side 1):
  - Adapt code to work in both Wesnoth 1.12 and 1.13
  - Switch to using external candidate actions
  - Significant amounts of code refactoring to make the above changes possible

[Zap-Zarap]

Wesnoth 1.12.6
Fred 0.14.8

Here goes another Fred replay for you, i was Knalgans, both sides had the same amount of gold.

Fred played much better than 2 versions ago, more agressively.
In the end, i won, because i was lucky, when Fred left his King unprotected.

See the little grunt in the east going south on his own, around turn 17? I guess, he's planning to grab some unattended villages from me, but rethinking his idea quickly, when i started to move my boss into his direction.
For units like him in that situation, it would be fun to have an extra animation, him tiptoeing back north, whistling some innocent tune.