Help with testing Fred - Freelands MP Custom AI v0.14.14

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderators: Forum Moderators, Developers

Post Reply
mattsc
Posts: 1149
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Fred - Freelands MP Custom AI v0.14.14 now available

Post by mattsc »


Fred / AI-demos v0.14.14 is now available on the Wesnoth 1.14 add-ons server

This version contains a huge amount of improvements to the AI evaluation, but the most visible changes are probably that Fred can now also play side 2 of the Freelands map and is not restricted to playing only Northerners any more. There is also a custom option in the MP setup dialog to change the overall aggressiveness.

I could use some help with play testing Fred at this point. The changes in this version result in some clear improvements, but there are, as always, many problems remaining. I have my own opinion on what the biggest issues are, but could use a second (and third, and fourth ...) opinion on that. Also, I'm not sure how much those improvements will actually be visible to others. Since I know exactly what Fred is doing, I "wave away" certain mistakes as "oh, I know what's causing this, it will be taken care of later", but somebody else obviously cannot do that.

So, any replay anybody could send me with a few comments (or even without the comments) would be very valuable! If you have not done so before, see the first post of this thread for information on how to play test and what information is most useful - but really anything helps, so don't worry about it too much.

v0.14.14 changelog

Code: Select all

----- 0.14.14, 12 February 2020 -----

This release includes a large number of improvements touching essentially all
aspects of Fred's gameplay. It also allows Fred to play side 2 on the Freelands
map, lifts the restriction of only playing Northerners and adds a custom option
to change the overall aggression level.

- Fred (Freelands AI):
  - Significant changes to all parts of the AI action evaluations, from the
    operations level to the nitty-gritty details of the individual action
    ratings, with the goal of making Fred overall more aggressive, while
    not putting units at unnecessary risk.
  - Increase the default aggression level
  - Directly player-visible changes:
    - Fred can now play side 2 on the Freelands map
    - Fred can now play any faction
    - Add a custom option to the MP setup dialog to modify the default aggression
    - In the Fred test scenario, Fred can now also be chosen for side 2
    - Remove game start and end messages
- Other:
  - Add external AI control demonstration

name
Posts: 409
Joined: January 6th, 2008, 3:32 am

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by name »

mattsc wrote:
February 12th, 2020, 3:06 pm
This version contains a huge amount of improvements to the AI evaluation, but the most visible changes are probably that Fred can now also play side 2 of the Freelands map and is not restricted to playing only Northerners any more.
Huge improvements indeed!

Playing all factions and both sides of the map certainly are milestones. Playing both sides must also now open the possibility of Fred being able to play against himself rather than against default AI which should make it much easier to track progress against earlier versions (and is a step towards deep learning perhaps?)

But besides the immediately visible changes, I feel like Fred is quite challenging now. Giving him +8 income, I estimate the challenge is about the same level you get playing online against an average human player. He forms tight battle groups, he coordinates them in assassinating specific units (any king found in the open is ruthlessly eliminated), he attacks and retreats quite accurately when he should (maybe just a tad too aggressive with default setting if anything), villages are taken and defended under the conditions they should be and injured plus glass cannon units are given good screening when resources allow.

His recruiting seemed good although I will have to evaluate that over more games since there are now so many more possible factional match ups. When I was playing drakes against his loyalists and recruiting heavily saurians, it felt like he deployed more heavy infantry, cavalrymen, mages and fencers to counter them (unless it just seemed this way because he was using them more effectively).
mattsc wrote:
February 12th, 2020, 3:06 pm
There is also a custom option in the MP setup dialog to change the overall aggressiveness.
I am going to have to experiment with that.
mattsc wrote:
February 12th, 2020, 3:06 pm
I could use some help with play testing Fred at this point. The changes in this version result in some clear improvements, but there are, as always, many problems remaining. I have my own opinion on what the biggest issues are, but could use a second (and third, and fourth ...) opinion on that. Also, I'm not sure how much those improvements will actually be visible to others. Since I know exactly what Fred is doing, I "wave away" certain mistakes as "oh, I know what's causing this, it will be taken care of later", but somebody else obviously cannot do that.
To give more precise feedback than the above, I might have to have more matches against him and review my replays from the previous version, because he has been quite good for a while now and the specific common mistakes are getting harder and harder to spot.

Replays:
v14.14_Undead_vs_Loyalist+8_income.gz
(44.13 KiB) Downloaded 11 times
v14.14_Northerner_vs_Drake+8_Income.gz
(29.06 KiB) Downloaded 11 times
v14.14_Drake_vs_Loyalist+8_Income.gz
(50.51 KiB) Downloaded 11 times

That last replay produces a consistent crash when you end the last turn, with the following errror:

Code: Select all

**** Fred Side 2 (version 0.14.14) *******************************************************
Beginning of Turn 58 (Dusk) stats	[ t = 16519.175 ]
  Side 1: 13 Units (538 HP), 14/16 villages  (Drake Blademaster, 51/80 HP, 4 gold)
  Side 2: 8 Units (274 HP), 2/16 villages  (White Mage, 35/35 HP, 19 gold)
    Leveled units: Dragoon (55/55)  
************************************************************************************
20200213 14:19:39 error scripting/lua: ~add-ons/AI-demos/lua/ca_zone_control.lua:2232: attempt to perform arithmetic on a nil value (local 'forward')
stack traceback:
	~add-ons/AI-demos/lua/ca_zone_control.lua:2232: in upvalue 'get_hold_action'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3254: in function '.get_zone_action'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3475: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3392>
	(...tail calls...)
=> exec: remove_MP CA	[ t = 16522.454, dt = 3.492 ]
20200213 14:19:39 error scripting/lua: ~/add-ons/AI-demos/lua/fred_gamestate_map.lua:34: bad argument #1 to 'next' (table expected, got nil)
stack traceback:
	[C]: in function '.next'
	~/add-ons/AI-demos/lua/fred_gamestate_map.lua:34: in field 'iter'
	~/add-ons/AI-demos/lua/fred_move_leader_utils.lua:297: in field 'leader_objectives'
	~/add-ons/AI-demos/lua/fred_ops_utils.lua:615: in field 'set_ops_data'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3413: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3392>
	(...tail calls...)
20200213 14:19:39 error scripting/lua: ~/add-ons/AI-demos/lua/fred_gamestate_map.lua:34: bad argument #1 to 'next' (table expected, got nil)
stack traceback:
	[C]: in function '.next'
	~/add-ons/AI-demos/lua/fred_gamestate_map.lua:34: in field 'iter'
	~/add-ons/AI-demos/lua/fred_move_leader_utils.lua:297: in field 'leader_objectives'
	~/add-ons/AI-demos/lua/fred_ops_utils.lua:615: in field 'set_ops_data'
	~add-ons/AI-demos/lua/ca_zone_control.lua:3413: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3392>
	(...tail calls...)
20200213 14:19:39 warning engine: Trying to remove non-existent menu item 'm08_show_behavior'; ignoring.
20200213 14:19:39 warning engine: Trying to remove non-existent menu item 'm09_last_behavior'; ignoring.

Also, congratulations on replacing traditional RCA with Ron (Experimental AI) as the new Default AI for the next release! I have been using the Experimental AI in all hosted multiplayer games for quite a few years now, it is so good that this intelligence boost will soon enhance campaigns too!

mattsc
Posts: 1149
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by mattsc »

name: Thank you for all the comments and the replays! This is great to hear and very helpful too. :) I haven't watched the complete replays in detail yet (some are quite long!), but even what I've seen so far is very useful.
name wrote:
February 14th, 2020, 5:22 am
Playing all factions and both sides of the map certainly are milestones. Playing both sides must also now open the possibility of Fred being able to play against himself rather than against default AI which should make it much easier to track progress against earlier versions (and is a step towards deep learning perhaps?)
Yeah, I've watched quite a few games of Fred vs. Fred recently and it's been very instructive. I don't know about deep learning, in part because I don't know much about it (other than the general concept), but also because Fred's quite slow (compared to the default AI, I mean, not necessarily compared to a human player) at the moment and it would take a long time, maybe too long, for the learning process. But even just evaluation of parameter settings or rating contributions is much easier that way.
name wrote:
February 14th, 2020, 5:22 am
(maybe just a tad too aggressive with default setting if anything)
Yeah, I got that impression too, esp. when it comes to not exposing individual units too much. I'm working on the latter though and when that gets a bit better, maybe the overall aggression level should be reduced a little. Or maybe it's not necessary any more. I also found (from Fred vs. Fred games, in fact) that the best value depends a bit on the factions. Northerners, obviously, need to be a bit on the aggressive side, but I also noticed that games of factions of the same alignment on both sides seem to go a bit better with higher aggression than, say, a mostly chaotic vs. a mostly lawful faction. In the end, I might add something to adjust that automatically. For now, there's the slider in the MP setup dialog with which one can play.

BTW, I noticed that you let Fred play side 2. There's an inherent problem (well, an inconsistency more so than a problem) with that in that the time of day will be different on the next enemy turn than on the current turn. That's not (yet) taken into account, but it's also not clear that it is actually a disadvantage.
name wrote:
February 14th, 2020, 5:22 am
His recruiting seemed good although I will have to evaluate that over more games since there are now so many more possible factional match ups. When I was playing drakes against his loyalists and recruiting heavily saurians, it felt like he deployed more heavy infantry, cavalrymen, mages and fencers to counter them (unless it just seemed this way because he was using them more effectively).
Recruiting is actually one thing (pretty much the only thing, I believe) that I did not touch. It is still (almost) exactly the same as in the Experimental AI. So I can't take credit (or blame) for that. ;-)
name wrote:
February 14th, 2020, 5:22 am
To give more precise feedback than the above, I might have to have more matches against him and review my replays from the previous version, because he has been quite good for a while now and the specific common mistakes are getting harder and harder to spot.
Don't worry about this too much. This is already really useful, particularly the replays. I'm planning to put out more frequent releases (I've already made a couple new changes that, I think, should make a difference in some situations, although it's never clear how noticeable any individual change will be when you're not specifically paying attention to it), so hopefully there will actually be progress at a faster than glacial pace now. Of course, I've wanted that before, we'll have to see how real life cooperates...
name wrote:
February 14th, 2020, 5:22 am
That last replay produces a consistent crash when you end the last turn, with the following errror:
Thanks! I managed to reproduce it and know what causes it. It will be fixed for the next release. And it's a rather rare situation causing it, so I am not sure I would have encountered it anytime soon without this!
Also, congratulations on replacing traditional RCA with Ron (Experimental AI) as the new Default AI for the next release! I have been using the Experimental AI in all hosted multiplayer games for quite a few years now, it is so good that this intelligence boost will soon enhance campaigns too!
Thanks! :) It's going to see a lot more use that way, so hopefully we'll find whatever bugs and limitations are left in the code.

Again, thanks a lot, I really appreciate you taking the time for play testing and sending your comments!

name
Posts: 409
Joined: January 6th, 2008, 3:32 am

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by name »

mattsc wrote:
February 14th, 2020, 11:21 pm
I haven't watched the complete replays in detail yet (some are quite long!), but even what I've seen so far is very useful.
Long replays may be the best compliment Fred could receive... it really did take that long to beat him and there were times I wondered if maybe there was no hope against this machine menace!

Let me know if you would like to see replays where he has less gold advantage to shorten the game length.
mattsc wrote:
February 14th, 2020, 11:21 pm
I don't know about deep learning, in part because I don't know much about it (other than the general concept), but also because Fred's quite slow (compared to the default AI, I mean, not necessarily compared to a human player) at the moment and it would take a long time, maybe too long, for the learning process. But even just evaluation of parameter settings or rating contributions is much easier that way.
Sorry, I meant to say just machine learning, not deep learning.

Yeah the only way you might get the processing power you need to run all those matches would be if the process could be distributed over many machines working independently. Assuming we, the wesnoth community, might volunteer to run the AI vs AI matches in the background on our machines.

Though that would mean parallel rather than sequential learning, which might not be useful at all... :hmm:
mattsc wrote:
February 14th, 2020, 11:21 pm
I also found (from Fred vs. Fred games, in fact) that the best value depends a bit on the factions. Northerners, obviously, need to be a bit on the aggressive side, but I also noticed that games of factions of the same alignment on both sides seem to go a bit better with higher aggression than, say, a mostly chaotic vs. a mostly lawful faction.
Very interesting, do you have numbers available for what aggression setting generally works best for each faction?

I wonder if there will come a point where AI results can even aid balancing a bit (a long shot for a complex turn based strategy, I know.)
mattsc wrote:
February 14th, 2020, 11:21 pm
BTW, I noticed that you let Fred play side 2. There's an inherent problem (well, an inconsistency more so than a problem) with that in that the time of day will be different on the next enemy turn than on the current turn. That's not (yet) taken into account, but it's also not clear that it is actually a disadvantage.
Would you like future replays to mix and match which side Fred takes?




BTW, I was lurking on the discord channel a while back and I noticed you have a "Mostly Harmless" avatar... I have to ask, is that an Elite or an Escape Velocity series reference? :geek:

mattsc
Posts: 1149
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by mattsc »

name wrote:
February 16th, 2020, 4:38 am
Long replays may be the best compliment Fred could receive... it really did take that long to beat him and there were times I wondered if maybe there was no hope against this machine menace!

Let me know if you would like to see replays where he has less gold advantage to shorten the game length.
Yeah, that's how I took it. And no, this is great, it gives me a lot to work with. I only mentioned it as an "excuse" as to why I had not gone through them in detail yet.
name wrote:
February 16th, 2020, 4:38 am
mattsc wrote:
February 14th, 2020, 11:21 pm
I also found (from Fred vs. Fred games, in fact) that the best value depends a bit on the factions. Northerners, obviously, need to be a bit on the aggressive side, but I also noticed that games of factions of the same alignment on both sides seem to go a bit better with higher aggression than, say, a mostly chaotic vs. a mostly lawful faction.
Very interesting, do you have numbers available for what aggression setting generally works best for each faction?
It's hard to say, to be honest. I don't really know on a faction by faction basis. I think the current default works pretty well on average (which is why I chose it as the default), maybe a bit on the unnecessarily aggressive side for chaotic vs. lawful matches. So maybe using 0.9 or 0.8 of the default for that would be a little better, but I've seen mixed results with that in the limited number of games I've had time to experiment with myself so far. The extreme ranges of that slider (it goes from 0.1 to 2 times the default) are definitely way too much. They are there mostly for laughs (and a bit to demonstrate what is possible), they certainly don't result in good play. :lol: They might be useful for certain SP scenario settings though, if I ever get to adapting Fred to other maps.

I think the bigger problems at the moment are when Fred moves individual units, or small groups, into enemy range when that really isn't necessary. Reducing the aggression so much that he does not do that any more results in otherwise much too defensive play. So I need to (and am) working on that differently.

name wrote:
February 16th, 2020, 4:38 am
I wonder if there will come a point where AI results can even aid balancing a bit (a long shot for a complex turn based strategy, I know.)
Personally, I doubt that, but one never knows.
name wrote:
February 16th, 2020, 4:38 am
Would you like future replays to mix and match which side Fred takes?
It would be interesting to know whether you (singular and/or plural you) think that there's a difference between him playing side 1 or 2, but it's up to you really. My hope is that, if I can manage to (slowly) improve Fred, I will get more feedback in time which will tell us what we need.
name wrote:
February 16th, 2020, 4:38 am
BTW, I was lurking on the discord channel a while back and I noticed you have a "Mostly Harmless" avatar... I have to ask, is that an Elite or an Escape Velocity series reference? :geek:
Hitchhiker's Guide, actually. :geek:

name
Posts: 409
Joined: January 6th, 2008, 3:32 am

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by name »

mattsc wrote:
February 16th, 2020, 6:16 pm
The extreme ranges of that slider (it goes from 0.1 to 2 times the default) are definitely way too much.
[...]
They might be useful for certain SP scenario settings though, if I ever get to adapting Fred to other maps.
Quite surely; a number of campaign scenarios allow you to set the offensiveness/defensiveness of the AI, but it always plays too recklessly, so I always set it to maximum defense. (Of course this was with old default AI behavior, not Ron tech).

Fred would be an excellent player ally in campaign scenarios for his smart defense in general but even more so with the high defense option available to the player.
mattsc wrote:
February 16th, 2020, 6:16 pm
It would be interesting to know whether you (singular and/or plural you) think that there's a difference between him playing side 1 or 2, but it's up to you really. My hope is that, if I can manage to (slowly) improve Fred, I will get more feedback in time which will tell us what we need.
I will try and mix up his side number then and look out for a difference.

I suspect the answer might be complicated but averages out to "not much difference". There are situations where a final aggressive blow before the ToD becomes unfavorable cripples the enemy's hope to retaliate. Other times, it is best to just get to cover early and sacrifice the attack opportunity. It gets way more complex with more than the two sides of 1v1 maps.
mattsc wrote:
February 16th, 2020, 6:16 pm
Hitchhiker's Guide, actually. :geek:
I just got a notice saying my Geek Card has been revoked!

User avatar
singalen
iOS Port Maintainer
Posts: 266
Joined: January 3rd, 2007, 10:18 am
Location: bay

Re: Help with testing Fred - Freelands MP Custom AI v0.14.14

Post by singalen »

Fred feels very strong now, thanks! This will give me a way to practice offline.
I've lost my shape in the recent years, so Fred with +2 gold, 125 starting has almost beat me twice when I tried him.
Not sure the replays would be useful, I played pretty badly.

mattsc
Posts: 1149
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Fred - Freelands MP Custom AI v0.14.14 now available

Post by mattsc »

Just a couple more quick replies:
name wrote:
February 18th, 2020, 5:50 am
mattsc wrote:
February 16th, 2020, 6:16 pm
It would be interesting to know whether you (singular and/or plural you) think that there's a difference between him playing side 1 or 2, but it's up to you really. My hope is that, if I can manage to (slowly) improve Fred, I will get more feedback in time which will tell us what we need.
I will try and mix up his side number then and look out for a difference.

I suspect the answer might be complicated but averages out to "not much difference". There are situations where a final aggressive blow before the ToD becomes unfavorable cripples the enemy's hope to retaliate. Other times, it is best to just get to cover early and sacrifice the attack opportunity. It gets way more complex with more than the two sides of 1v1 maps.
Yeah, agreed on all of that. I don't think that any specific kind of test/replay is more useful than another (or will be conclusive one way or the other), so just whatever way anybody wants to play is great.
name wrote:
February 18th, 2020, 5:50 am
I just got a notice saying my Geek Card has been revoked!
Hehe, but no worries. I actually had forgotten that it also appears in Elite...

singalen wrote:
February 19th, 2020, 6:55 pm
Fred feels very strong now, thanks! This will give me a way to practice offline.
I've lost my shape in the recent years, so Fred with +2 gold, 125 starting has almost beat me twice when I tried him.
Not sure the replays would be useful, I played pretty badly.
Thanks, this is great to hear! :)

I'm probably going to release another version relatively soon (next week, maybe?). This one won't include any huge changes. I've been working on some of the extreme cases, trying to get unnecessarily aggressive behavior on individual moves somewhat under control without impacting the overall aggression level. So while I would be able to demonstrate different behavior in specific test cases, the changes will not be immediately obvious. But I believe that they will improve the overall behavior, even if only subtly.

User avatar
The_Gnat
Posts: 2120
Joined: October 10th, 2016, 3:06 am
Contact:

Re: Help with testing Fred - Freelands MP Custom AI v0.14.14

Post by The_Gnat »

Hi mattsc! Great to see this coming along. I will definitely try out the newest version, and correct me if I misunderstood but it can now play any default faction (not only Northerners)?
Creator of: The Reign of The Lords Era,The Gnats Franken Dungeon
Co-designer of the (not-wesnoth) space combat video game Planet Bounce.

mattsc
Posts: 1149
Joined: October 13th, 2010, 6:14 pm
Location: Hidden on the hex behind Fred

Re: Help with testing Fred - Freelands MP Custom AI v0.14.14

Post by mattsc »

The_Gnat wrote:
February 20th, 2020, 8:48 pm
Hi mattsc! Great to see this coming along. I will definitely try out the newest version, and correct me if I misunderstood but it can now play any default faction (not only Northerners)?
That's great, thanks. And you understood correctly, Fred can now play any default faction on either side 1 or 2, or both. [It's probably really any faction, rather than any default faction, but I forgot if we built something into the setup that excludes other eras or modifications. That would be easy to change though.] Still restricted to the Freelands map though, that one is much harder to generalize.

Post Reply