Help with testing Fred - Freelands MP Custom AI v0.14.15
Moderator: Forum Moderators
Re: Help with testing Fred - Freelands MP Custom AI v0.14.8
Here a replay against Rebels
Fred is a coward, a really big coward...
The begining was not so bad, I've waited for him to attack and got a good piece of luck. But when I came to his side, this was awful.
He's made something terribly wrong on turn 23, fleeing on first watch when he was outnumering me.
He gave me villages for free, he doesn't really use his assassins to poison me. He put them on hexes where I could ZoC with only one unit.
I've been able to take the right side with only 5 units. In fact, I recruited only 12 units in the whole game
Each time I advanced my units on good def he was moving back. I've been able to steal his castle without much of a fight.
His leader finally flee to my right side, alone, where I could kill him with 4 units, he didn't even use his sword
In fact, he had an attack EV around 450 for a 42 turns game
As usual for an AI, Fred can't manage "1xp to level up" units.
Maybe, Fred is overrating defense. Just moving forward on good def and Fred moves back... He's not really holding a front line, just optimizing defense of each units.
I've downloaded the add-on today, so I guess I was up to date.
I didn't get lua errors, and, in the end, Fred took some time to play (when he had 17-18 units) but not particulary long (faster than some human player )
Hope this will help you and good luck in your project.
Fred is a coward, a really big coward...
The begining was not so bad, I've waited for him to attack and got a good piece of luck. But when I came to his side, this was awful.
He's made something terribly wrong on turn 23, fleeing on first watch when he was outnumering me.
He gave me villages for free, he doesn't really use his assassins to poison me. He put them on hexes where I could ZoC with only one unit.
I've been able to take the right side with only 5 units. In fact, I recruited only 12 units in the whole game
Each time I advanced my units on good def he was moving back. I've been able to steal his castle without much of a fight.
His leader finally flee to my right side, alone, where I could kill him with 4 units, he didn't even use his sword
In fact, he had an attack EV around 450 for a 42 turns game
As usual for an AI, Fred can't manage "1xp to level up" units.
Maybe, Fred is overrating defense. Just moving forward on good def and Fred moves back... He's not really holding a front line, just optimizing defense of each units.
I've downloaded the add-on today, so I guess I was up to date.
I didn't get lua errors, and, in the end, Fred took some time to play (when he had 17-18 units) but not particulary long (faster than some human player )
Hope this will help you and good luck in your project.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.8
Hi Maboul,
Thanks for the comments and the replay. I have not watched it yet, but I will do so and I am sure I'll get something useful out of it. I always do and appreciate the effort. I likely won't have much time for a couple days, so I just wanted to reply with a couple quick comments (and I'll look into things in more detail later):
Yes, the last release version, v0.14.8, is way too defensive. If you're interested, there's been much discussion on this in this thread over the last few weeks, but the main reasons are:
Thanks for the comments and the replay. I have not watched it yet, but I will do so and I am sure I'll get something useful out of it. I always do and appreciate the effort. I likely won't have much time for a couple days, so I just wanted to reply with a couple quick comments (and I'll look into things in more detail later):
Yes, the last release version, v0.14.8, is way too defensive. If you're interested, there's been much discussion on this in this thread over the last few weeks, but the main reasons are:
- The most recent work has concentrated on defensive positioning, which is therefore over-emphasized at the moment
- Too much testing against the default AI, as rambled about here
Thanks.Maboul wrote:Hope this will help you and good luck in your project.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Based on all your feedback, I changed the emphasis of the immediate operations layer work toward reducing Fred's exaggerated defensiveness. Unfortunately (as always?), I've not had as much time to work on this as I would like. I nevertheless decided to publish the next release now, because I will start traveling on the weekend and will have even less time than usual for at least a week or two. So:
Fred / AI-demos v0.14.9 is now available on the Wesnoth 1.12 and 1.13 add-ons servers
Take this release with the following caveats:
Status of and plan for the current work is here.
Fred / AI-demos v0.14.9 is now available on the Wesnoth 1.12 and 1.13 add-ons servers
Take this release with the following caveats:
- Consider this an intermediate development release with incomplete changes that (probably) introduce some new problems and (definitely) do not address all known issues.
- The emphasis of this release was on improving time-of-day dependent behavior against lawful factions (and only against lawful factions). This probably also works to some extent against neutral factions, but I don't think it's sufficient against chaotic factions. The overall increased aggressiveness might help against the latter a little too, but more work definitely needs to be done for those.
- Because of the limited time I had available (and one technical issue mentioned in the "ramblings" below), I have done even less testing than usual. Thus, it is quite likely that there's some new weird behavior that I do not know about yet.
Status of and plan for the current work is here.
A few more of my usual ramblings on known problems and general observations
v0.14.9 complete changelog
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Well 14.9 does seem a good deal more challenging against loyalists.
In general, I felt he put forward a good balance of offense and defense this time. He launched damaging attacks at times and places that were not ready for them and other times avoided attacking strong defensive lines that would have consumed his available attackers.
Also noticed Fred's assassins do not suffer bad luck silently.
In general, I felt he put forward a good balance of offense and defense this time. He launched damaging attacks at times and places that were not ready for them and other times avoided attacking strong defensive lines that would have consumed his available attackers.
Also noticed Fred's assassins do not suffer bad luck silently.
Oops, I should not have skipped upgrading to 14.8 for so long then, sorry about that. How many modern replays would you want to have on hand before dropping the troublesome older ones?mattsc wrote:One of the unfortunate side effects of porting to Wesnoth 1.13 in the v0.14.8 release is that replays from before 0.14.8 cannot be used for automatic test case creation any more. The replays themselves still work and are extremely useful, of course, but if I want to extract a test case from them, I have to do so manually (which is quite painful). In combination with me having had little time, this means that I've done even less testing than usual so far.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Hi Cold Steel — thank you for yet another replay and the quick turn around!
I kind of like what I saw. Sure, Fred made lots of mistakes and got a resounding beating in the end, but I like the overall dynamics of switching between offense and defense much better than in previous versions. Still lots of room for improvement there too, obviously, but I'm happy to see any progress at all and that the changes are at least going in the right direction.
None of the big problems I see in the replay surprise me. They are all on my list of things to work on. Given that, I think I will continue to work on improving things against lawful factions for now. Making it work against other factions is comparatively simple afterward, but if I tweak things to also work against chaotic factions right now, I'll have to do the whole thing again after fixing the other major problems.
I kind of like what I saw. Sure, Fred made lots of mistakes and got a resounding beating in the end, but I like the overall dynamics of switching between offense and defense much better than in previous versions. Still lots of room for improvement there too, obviously, but I'm happy to see any progress at all and that the changes are at least going in the right direction.
None of the big problems I see in the replay surprise me. They are all on my list of things to work on. Given that, I think I will continue to work on improving things against lawful factions for now. Making it work against other factions is comparatively simple afterward, but if I tweak things to also work against chaotic factions right now, I'll have to do the whole thing again after fixing the other major problems.
He he. Yeah, there are a couple little easter eggs hidden in Fred, but they are set to trigger only a small fraction of the time (even when the conditions for them occur). This is the most "common" one.Cold Steel wrote:Also noticed Fred's assassins do not suffer bad luck silently.
No need at all to be sorry, what you're doing is great. Another replay or so (also against loyalists or drakes) by you or others would be nice, just to see if the impressions from this one hold up. But quite honestly, this replay on its own gives me plenty to work with already. If I notice that other test cases would be good as I go along, I'll speak up here, but my guess is that even then it would probably make sense for me to release the next (dev) version before asking for more replays. (And I still can get test cases out of the old replays in much less time than it takes you to play another game, it just isn't automatic any more.)Cold Steel wrote:Oops, I should not have skipped upgrading to 14.8 for so long then, sorry about that. How many modern replays would you want to have on hand before dropping the troublesome older ones?
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Indeed, he is vastly improved. Tried another round (this time with drakes) and found I had to stick to more conservative strategies to win in the end. Also playing with drakes seemed no less challenging than with loyalists, so he may have overcome his fear of scaly creatures.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Yay! I've been waiting for a long time for somebody to say that. More precisely, I've been waiting for Fred to give somebody a reason to say that.Cold Steel wrote:Indeed, he is vastly improved.
Now, looking at the replay(s), I don't have the offense/defense balance quite right yet and there's a serious issue with coordination of some actions between units or groups of units. But it's progress, so that's good. The work on the operations layer still has a long way to go and as I wrote, there's no way that I'll get this all right the first (or second) time around.
Cold Steel wrote:Also playing with drakes seemed no less challenging than with loyalists, so he may have overcome his fear of scaly creatures.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Yeah, I imagine it should take several "feature" releases to check off all the operations layer tasks you have on the vast 0.14.10 blueprint, followed by or mixed in with the lighter "tweak" releases that adjust the utility weightings to compensate for these new features.
And while I know you want to manage expectations, I have to say I am really excited to watch the operations layer emerge from the development soup. Even if the first steps down this path are uncertain, as it matures, I think it will be a leap ahead of even where Fred is already.
And while I know you want to manage expectations, I have to say I am really excited to watch the operations layer emerge from the development soup. Even if the first steps down this path are uncertain, as it matures, I think it will be a leap ahead of even where Fred is already.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Hah, yeah, I guess you're right, although mostly it's my own expectations. As you know, I've been at this for a long time, and for quite a large fraction of that it felt like I was moving in circles and Fred wasn't going anywhere. But it looks like the "recent" approach of concentrating on the individual actions first and now trying to bind them all together might finally be starting to pay off. So, I'll counter your "excited" with a "cautiously optimistic".Cold Steel wrote:And while I know you want to manage expectations,
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Fair enough.
Just for fun, I did a round with rebels. I thought they might be interesting because rebels are "offensively lawful" since their two main line breaker units are lawful, but they are otherwise mostly neutral. I guess you could say they are technically lawful and in the end what could be more lawful than bending the rules on a technicality?
Just for fun, I did a round with rebels. I thought they might be interesting because rebels are "offensively lawful" since their two main line breaker units are lawful, but they are otherwise mostly neutral. I guess you could say they are technically lawful and in the end what could be more lawful than bending the rules on a technicality?
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Interesting. From a first quick run through the replay, I'd say both the overall behavior and the major mistakes are generally the same. I've had pretty much no time for Fred since the last release, but hopefully that'll change over the next few days. Trying to figure out how to coordinate actions better and not leave units stranded as much remains the highest priority at this time.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
There are the big questions in life that all of us have been pondering essentially since we were in diapers. Such as "Can a nun drive a monster truck?" and "Why does Fred like to attack full-health enemies when there's an easy kill nearby?" While the former was answered in 2004, it took a little longer for the latter, but I am happy to announce that it has finally been figured out.
And as always, the fix's deceptively simple
And as always, the fix's deceptively simple
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Yeah ... And I wasn't even working on attacks, but while checking out something else, I came across an attack choice that just didn't make sense, so I decided to look into what was going on. I figured that I'd have to tweak the evaluation functions, but no, this time it simply was a real, honest bug. And while most bug fixes seem to have a negligible effect at this stage, this one does actually make a difference.* It was very satisfying to finally find it after several hours of digging deep into the attack evaluation code. Like scratching that itch!
That previous post was also my underhanded way of saying that development of Fred, while really slow at the moment, is not quite dead. The 'contributions' chart on the AI-demos github site is really quite telling, with almost no activity for two of the last three months. RL's just really busy (and will continue to be so). Long story short, I just wanted to show a sign of life here and say that work on Fred continues, even if at a snail's pace at the moment.
As for what I am currently working on, well, the previous release was mostly about making Fred overall more proactive and willing to push forward. As a result, some of the actions that were already too aggressive (sending individual units on suicide mission into enemy territory, and the like) got even worse. The main theme of the current work is to get some of those actions under control, while maintaining the overall more aggressive behavior.
*No, it won't suddenly make Fred play a lot better overall, but it's something that is actually noticeable in some situations. Well, would be noticeable if you could see it done both ways, it's hard to notice the absence of a mistake.
That previous post was also my underhanded way of saying that development of Fred, while really slow at the moment, is not quite dead. The 'contributions' chart on the AI-demos github site is really quite telling, with almost no activity for two of the last three months. RL's just really busy (and will continue to be so). Long story short, I just wanted to show a sign of life here and say that work on Fred continues, even if at a snail's pace at the moment.
As for what I am currently working on, well, the previous release was mostly about making Fred overall more proactive and willing to push forward. As a result, some of the actions that were already too aggressive (sending individual units on suicide mission into enemy territory, and the like) got even worse. The main theme of the current work is to get some of those actions under control, while maintaining the overall more aggressive behavior.
*No, it won't suddenly make Fred play a lot better overall, but it's something that is actually noticeable in some situations. Well, would be noticeable if you could see it done both ways, it's hard to notice the absence of a mistake.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.9
Sorry, it broke again on 1.13.12 - dev version 8(
attempt to call a nil value (filed 'maxn')
ca_zone_control.lua:2636: in field 'execution'
[string "local self, params, data = ..."]:2:in main chunk