Help with testing Fred - Freelands MP Custom AI v0.14.15
Moderator: Forum Moderators
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
Interesting, I might have to take another peak at the code then.mattsc wrote: ↑April 20th, 2019, 4:20 pm And as a note, while that might not seem very different from what he did before, the refactoring of the code means that (I think) I am in a better position to deal with this now. Still doesn't mean that I kow how to do that, but as I told somebody on Discord, figuring that out is part of the fun.
It was near the end, perhaps somewhere in the range between and including turns 21 and 23.
Also, I encountered a new error during another match:
Code: Select all
error scripting/lua: ~add-ons/AI-demos/lua/ca_zone_control.lua:2520: attempt to index a nil value (local 'raw_cfg')
stack traceback:
~add-ons/AI-demos/lua/ca_zone_control.lua:2520: in upvalue 'get_advance_action'
~add-ons/AI-demos/lua/ca_zone_control.lua:3069: in function '.get_zone_action'
~add-ons/AI-demos/lua/ca_zone_control.lua:3257: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3177>
(...tail calls...)
I see, that makes quite a lot of sense. Especially deciding when and where to hold wide and vaguely defined regions seems complicated. Though RCA does next to no holding at all, so Fred has no real competition in that regard.mattsc wrote: ↑April 20th, 2019, 4:20 pm Now, different maps is a different story. I simply don't know yet how well the Fred algorithms would work in different settings. Hopefully they can be adapted to other 2-player maps at least. There are a lot of implicit assumptions about there only being one enemy side (and no allied sides).
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
name: Thanks for the new replay and the information about the errors. I (finally) managed to reproduce the latest error reliable. I have not succeeded with that for the first one yet, since I don't know exactly when it happened. I'll keep looking though, and I can make a guess on its cause simply based on the error message itself for that one. Anyways, thanks, I will have these fixed in the next release.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
For some reason I thought replays contained the chat log and error messages got saved to it as well as the log file.
I will continue testing and see if I can reproduce the error by chance. If it reoccurs I will save a replay on the exact turn it happens.
I will continue testing and see if I can reproduce the error by chance. If it reoccurs I will save a replay on the exact turn it happens.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
They do contain the chat messages, but not the Lua error messages. If you know how to get to your Wesnoth system log, the error message would be in there (and Fred puts a bunch of messages in there, including some stats at the beginning of each turn that indicate what turn it happened on), but don't worry about it too much. Same goes for more testing. If you want to, that's great, but don't worry about it just for reproducing this error. We'll encounter it again sometime.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
Oops, I had not noticed before that the logs report the passage of turns!
So it looks like the error in question occurred on turn 22:
Code: Select all
**** Fred 0.14.12 *******************************************************
Beginning of Turn 22 (Dusk) stats [ t = 4030.996 ]
Side 1: 12 Units (322 HP), 3/16 villages (Orcish Crossbowman, 18 gold)
Side 2: 17 Units (625 HP), 13/16 villages (Swordsman, 2 gold)
************************************************************************
=> exec: top: attack leader threats [ t = 4040.267, dt = 9.642 ]
=> exec: top: attack leader threats [ t = 4041.534, dt = 10.909 ]
=> exec: top: attack leader threats [ t = 4043.475, dt = 12.850 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 4 unit 21,9: 0.00 unit 21,7: 9.08) [ t = 4054.742, dt = 24.117 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 4 unit 21,9: 0.00 unit 21,7: 9.08) [ t = 4055.395, dt = 24.770 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 4 unit 21,9: 0.00 unit 21,7: 9.08) [ t = 4055.843, dt = 25.218 ]
=> exec: leader: recruit [ t = 4060.461, dt = 29.836 ]
=> exec: all_map: retreat (move leader to keep) [ t = 4064.490, dt = 33.865 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 0 unit 20,4: 16.32 unit 21,7: 29.98) [ t = 4068.422, dt = 37.797 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 0 unit 20,4: 16.32 unit 21,7: 29.98) [ t = 4069.092, dt = 38.467 ]
=> exec: top: zone protect (does_protect = yes (leader)
protecting: leader: true castles: 0 unit 20,4: 16.32 unit 21,7: 29.98) [ t = 4069.553, dt = 38.928 ]
20190420 02:40:41 error scripting/lua: ~/add-ons/AI-demos/lua/fred_benefits_utilities.lua:535: attempt to perform arithmetic on a nil value (field '?')
stack traceback:
~/add-ons/AI-demos/lua/fred_benefits_utilities.lua:535: in field 'action_penalty'
~/add-ons/AI-demos/lua/fred_hold_utils.lua:483: in field 'find_best_combo'
~add-ons/AI-demos/lua/ca_zone_control.lua:2328: in upvalue 'get_hold_action'
~add-ons/AI-demos/lua/ca_zone_control.lua:3053: in function '.get_zone_action'
~add-ons/AI-demos/lua/ca_zone_control.lua:3257: in function <~add-ons/AI-demos/lua/ca_zone_control.lua:3177>
(...tail calls...)
=> exec: remove_MP CA [ t = 4070.902, dt = 40.277 ]
=> exec: leader: recruit [ t = 4071.549, dt = 40.924 ]
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
I assume that this is obvious to you, but just stating it for reference for others: that is only true for Fred, not the default AI. I'm having Fred print all this information for reasons exactly like this, so that it is easier to track down rarely occurring errors. Eventually I will remove that (or, more likely, hide it behind a debug flag).
Thanks, this should help a lot with reproducing this. It will still take a bit of experimenting to make it happen as it is pretty deep into that turn and the situation needs to be exactly the same (meaning all, or at least most, of the RNG rolls need to be the same). I do have some experience "faking" that by now though, so I'm sure I'll get there.
The other two errors you encountered are fixed by now.
And ... using one of the replays, I found that there's an issue deep in the attack rating/weighting evaluation. It's not really a bug, as the code is doing exactly what it's supposed to do, but rather a flaw in the logic of how to weight positive vs. negative contributions to the attack outcome. As a result, some fraction of attacks previously would never get an "acceptable" rating, no matter how high the overall aggression factor is set. I'm currently fixing that. It won't by itself suddenly fix all of Fred's timid behavior, but it's one step needed for that.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.12
I played a couple more rounds with a bit more income boost for fred:
It may be that I had forgotten how many clever tricks fred had already adopted in previous versions, but now after playing some rounds against him, I do feel like he is better at protecting and retreating his injured as well as focusing on assassinating particular units of my own before they can retreat by using well coordinated attacks, especially by considering how thoroughly a unit can be surrounded before making it the target of such a focused group attack.
Just needed to make sure I properly reported the bug so that my honor as a tester remains unsullied.
It may be that I had forgotten how many clever tricks fred had already adopted in previous versions, but now after playing some rounds against him, I do feel like he is better at protecting and retreating his injured as well as focusing on assassinating particular units of my own before they can retreat by using well coordinated attacks, especially by considering how thoroughly a unit can be surrounded before making it the target of such a focused group attack.
It isn't a big issue in any case and only occured that one time.mattsc wrote: ↑April 25th, 2019, 2:45 pm It will still take a bit of experimenting to make it happen as it is pretty deep into that turn and the situation needs to be exactly the same (meaning all, or at least most, of the RNG rolls need to be the same). I do have some experience "faking" that by now though, so I'm sure I'll get there.
Just needed to make sure I properly reported the bug so that my honor as a tester remains unsullied.
Thanks!
Interesting, I may have noticed that at certain points; temporary infinite timidness. Other times though his attacks would be pretty savage so I could never be too rash. Looking forward to testing with this fix.mattsc wrote: ↑April 25th, 2019, 2:45 pm And ... using one of the replays, I found that there's an issue deep in the attack rating/weighting evaluation. It's not really a bug, as the code is doing exactly what it's supposed to do, but rather a flaw in the logic of how to weight positive vs. negative contributions to the attack outcome. As a result, some fraction of attacks previously would never get an "acceptable" rating, no matter how high the overall aggression factor is set. I'm currently fixing that. It won't by itself suddenly fix all of Fred's timid behavior, but it's one step needed for that.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.12
Thanks. I've only done a quick run through them so far, but it's already clear that there are some more situations in them that I can use as test cases.
The protecting and retreating are definitely things that are considered more thoroughly in the operations level now and I also think I can see improvements in those. Focused attacks are not something I have specifically worked on specifically, but they could quite well be a side effect of some of the recent attack evaluation changes.name wrote: ↑April 28th, 2019, 4:57 pm It may be that I had forgotten how many clever tricks fred had already adopted in previous versions, but now after playing some rounds against him, I do feel like he is better at protecting and retreating his injured as well as focusing on assassinating particular units of my own before they can retreat by using well coordinated attacks,
I did, after some fiddling, manage to reproduce that one also and while it required a very specific situation, it turned out that it caused OOS errors, so it was an extremely good one to find and fix! Thanks a lot for going through the trouble of finding thst information, that did more than just keeping your tester honor intact, I'd say!name wrote: ↑April 28th, 2019, 4:57 pmIt isn't a big issue in any case and only occured that one time.mattsc wrote: ↑April 25th, 2019, 2:45 pm It will still take a bit of experimenting to make it happen as it is pretty deep into that turn and the situation needs to be exactly the same (meaning all, or at least most, of the RNG rolls need to be the same). I do have some experience "faking" that by now though, so I'm sure I'll get there.
Just needed to make sure I properly reported the bug so that my honor as a tester remains unsullied.
Yeah, Fred's always been a funny mix of being too aggressive and too scared at the same time. For me, knowing the code, that's not really a surprise in some respects, but this particular reason was not something I expected. And as I said, it won't fix all of the problems, but every step counts.
And from one of your earlier posts:
I did have a look at this and while I did not expect to find anything that would be (relatively) easy to improve, I did. Turns out the the super long delays were due to the fact that you had a skeleton very close to leveling in the middle of the units threatening the AI leader. It's the level-up calculation that took a lot of time - which also explains why the extreme slow-downs only happened sometimes when there were lots of enemy units surrounding the leader. Anyway, long story short, I managed to add caching to that and cut down some of the calculation times significantly. It's still not fast at the moment, but having to wait 10s for a move is much better than 30+ seconds. Further improvement on top of that is non-trivial and will have to wait.
Besides that, well, so far I've done a bunch of background work (e.g. I found another bug in the attack evaluation code, although I don't expect that one to make much of a difference in practice). Next I'll try to deal with some of the glaring issues seen in the replays:
- More aggressive attacks on the units threatening the AI leader (mostly toward the end of the game).
- Pull units back to defend the AI leader, even if they cannot get there on the current turn.
- Keeping units from fleeing to the edges/corners of the map and then being stranded there.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.12
I forgot to praise this a while ago when I first saw it- Fred consistently attacks undead with northerners during the day and retreats from them at night. He plays much stronger against undead as a result.
Although, given the layout of the freelands map, a strong leader defense response might act to retake those villages as well.
Or perhaps he is becoming self-aware. You may want to take precautions...
Thanks! This should be plenty fast for future testing (after all he still takes quite a lot less time than a human player does over the server).mattsc wrote: ↑April 29th, 2019, 12:24 am I managed to add caching to that and cut down some of the calculation times significantly. It's still not fast at the moment, but having to wait 10s for a move is much better than 30+ seconds. Further improvement on top of that is non-trivial and will have to wait.
One small side note, at the point where hostile units are close enough to reasonably threaten the leader, the game may be decided already. What really turned the tides was when/if I could hold the four villages to his west without him breaking through and retaking them at the favorable time of day. With that income advantage, victory quickly becomes certain.mattsc wrote: ↑April 29th, 2019, 12:24 am More aggressive attacks on the units threatening the AI leader (mostly toward the end of the game).
[...]
should be relatively simple. Everything needed for that is in place already, I just need to come up with (and test, that's always the time consuming part) a metric for how aggressive Fred should be.
Although, given the layout of the freelands map, a strong leader defense response might act to retake those villages as well.
Speaking of spatial reasoning, I see Fred allocates a lot of units to the central valley region, perhaps because it is the shortest distances between the two keeps. Is there a way to make this area seem less enticing to him based on its lack of villages?mattsc wrote: ↑April 29th, 2019, 12:24 am Keeping units from fleeing to the edges/corners of the map and then being stranded there.
[...]
It might be relatively easy to "fake", or it might be really hard to deal with, as it is part of a bigger, more fundamental problem (of spatial reasoning and the like). So, I might release a new version after the first two are done, or after all three, depending on how difficult that last point turns out to be.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.12
Thanks, that's great to know.
I know...
Spoiler:
Yeah, I agree. I didn't choose this first because I think it will have the biggest impact, but because it is comparatively easy to do. Going for the low hanging fruit first.name wrote: ↑May 1st, 2019, 10:31 pm One small side note, at the point where hostile units are close enough to reasonably threaten the leader, the game may be decided already. What really turned the tides was when/if I could hold the four villages to his west without him breaking through and retaking them at the favorable time of day. With that income advantage, victory quickly becomes certain.
Agree with that too. There is actually a "zone_weight" factor in the AI setup that gives only half the priority to the center as to the east/west regions. But it is currently not being used, if I remember correctly. Ideally I'd like Fred to figure this out himself (e.g. based on the number of villages), but that might simply not work as well as setting it manually.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
The Starting with
wesnoth -tfred
brings following message,
Error loading game configuration files. The game will now exit.
Details:
Macro/file 'campaigns/Heir_To_The_Throne/maps/01_The_Elves_Besieged.map' is missing
at scenario-test.cfg:3453
included from _main.cfg:42
to ensure, I test with the new download from 1.12 Version, is it also ok?
Further more - never heard and seen before, now in many Strings a usage about invalid string: e.G �Path top with 00 and below 04
and a question-mark in chequer (square) - there is no back to the startline in terminal - to much usages.
The new start without -tfred brings the old statement - without invalid strings.
wesnoth -tfred
brings following message,
Error loading game configuration files. The game will now exit.
Details:
Macro/file 'campaigns/Heir_To_The_Throne/maps/01_The_Elves_Besieged.map' is missing
at scenario-test.cfg:3453
included from _main.cfg:42
to ensure, I test with the new download from 1.12 Version, is it also ok?
Further more - never heard and seen before, now in many Strings a usage about invalid string: e.G �Path top with 00 and below 04
and a question-mark in chequer (square) - there is no back to the startline in terminal - to much usages.
The new start without -tfred brings the old statement - without invalid strings.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
The invalid string stuff sounds like a bug that happens when you have >128 textdomains, i.e., when you have many addons installed. I fixed it in 1.14 only.
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
There was also a security issue fixed that affects 1.12, so you really should be using the latest 1.14 version if at all possible.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Help with testing Fred - Freelands MP Custom AI v0.14.11
And that is also why Fred does not work (although it is not the error message I would have expected, but I don't remember all the details). Fred (AI-demos in general) is not supported and won't work on 1.12 any more.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.13
So ... well ... work on Fred is not entirely stalled, but it is proceeding very much at a snail's pace. I would not usually release a version at this stage, but it's been a long time since the last release and I will be away for the next month with very little or no time for Wesnoth. So here it is anyway, even though it only addresses a fraction of the known problems:
Fred / AI-demos v0.14.13 is now available on the Wesnoth 1.14 add-ons server
The main issues I tried to address in this version (see changelog below for more details):
Note: Looking at the changelog, I should maybe point out that "advancing" in Fred jargon does not refer to unit advancements, but to moving units forward (or backward) on the map when not trying to protect other units or terrain.
Fred / AI-demos v0.14.13 is now available on the Wesnoth 1.14 add-ons server
The main issues I tried to address in this version (see changelog below for more details):
- Prevent units from being stranded at the edges/corners of the map. I found that there are actually two separate reasons for this and I have had only time to address one of them so far.
- Don't abandon the leader at the end of the game. As name said, that's mostly only going to really kick in at a time when the game's already lost. I did this because I thought it would be comparatively quick and easy. Turned out not to be all that quick (but still easier than some of the other issues).
- I also made some changes to rating functions that aim at addressing situations when Fred simply refused to attack. That's also still work in progress.
- Fixed all known bugs, but probably introduced a few new ones ...
Note: Looking at the changelog, I should maybe point out that "advancing" in Fred jargon does not refer to unit advancements, but to moving units forward (or backward) on the map when not trying to protect other units or terrain.
v0.14.13 complete changelog
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on