Help with testing Fred - Freelands MP Custom AI v0.14.15
Moderator: Forum Moderators
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
max_torch: There have been quite a few GSoC projects on AI development over the years, but if we are thinking about the same one (it was called something like "Strategy formulation for the Wesnoth AI") then no, nothing applicable to Fred came out of that. For starters, the project never got to the point where it would have actually improved the Wesnoth AI (the student ran out of time, I think). And second, what Fred does along those lines is actually a lot more complex than what was attempted there. That's the advantage of only dealing with one faction on one map as opposed to trying to do it for all Wesnoth games, it's a much simpler problem.
Horus2: Thanks. I'll continue to post progress here. Unfortunately, it's probably going to be slow going for quite some time ...
Tauriel: That's great. I'm wondering if there's a way you could do this by using a version control system (e.g. making a branch of the AI-demos github repository). That way, if you made a change that would be useful for Fred in general in the future, it would be easy to earmark the respective commits for later inclusion. I'm afraid that large parts of the code will diverge too much (because of what I will be doing next) that simply using 'diff' will quickly become not very useful any more. Of course, you can also just manually keep track of what you're doing and let me know if you think anything might be useful in a general sense.
Horus2: Thanks. I'll continue to post progress here. Unfortunately, it's probably going to be slow going for quite some time ...
Tauriel: That's great. I'm wondering if there's a way you could do this by using a version control system (e.g. making a branch of the AI-demos github repository). That way, if you made a change that would be useful for Fred in general in the future, it would be easy to earmark the respective commits for later inclusion. I'm afraid that large parts of the code will diverge too much (because of what I will be doing next) that simply using 'diff' will quickly become not very useful any more. Of course, you can also just manually keep track of what you're doing and let me know if you think anything might be useful in a general sense.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
If I success to modify / create something big enough / worthy to upload, my plan is to rename my modified Fred into another name. It would be my own AI, but it's based on Fred. That way there would be no confusions.mattsc wrote:Tauriel: That's great. I'm wondering if there's a way you could do this by using a version control system (e.g. making a branch of the AI-demos github repository). That way, if you made a change that would be useful for Fred in general in the future, it would be easy to earmark the respective commits for later inclusion. I'm afraid that large parts of the code will diverge too much (because of what I will be doing next) that simply using 'diff' will quickly become not very useful any more. Of course, you can also just manually keep track of what you're doing and let me know if you think anything might be useful in a general sense.
If I would create something good enough that you could use in Fred, it'd be a honor to share the code. If I will ever get that skilled and come so far, it's a subject we can discuss that day.
For now, I struggle trying to change some of Fred's behaviour, with small progress every time
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
Hey all,
I don’t know if anybody is still interested, but just in case here’s a little update.
Bottom line is, Fred is currently incapacitated but I am working on reviving him. I have no idea whether the new Fred will be any better than the previous best version (which would probably be v0.14.0), but if nothing else he should be more streamlined and less complicated — which, hopefully, will make it easier to adapt him to other maps and scenarios.
Stay tuned, but don’t hold your breath. This is not going to happen quickly.
I don’t know if anybody is still interested, but just in case here’s a little update.
No need to read this
Stay tuned, but don’t hold your breath. This is not going to happen quickly.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
Poor Fred. But it's nice to know you are still working on it.
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
Thanks, doofus-01.
As another update, Fred is now operational again, technically speaking, but he does not do particularly well yet. The main changes are the addition of a more powerful operations layer and the simplification of code that had gotten too complicated. For example:
As another update, Fred is now operational again, technically speaking, but he does not do particularly well yet. The main changes are the addition of a more powerful operations layer and the simplification of code that had gotten too complicated. For example:
- Previously the individual units (or squads of a few units) decided themselves whether to set up a defensive line or not (I call that 'holding'). Now, the operations layer decides whether to hold or not. The squads still get to figure out the details (best units, best hexes), but they get no say in whether this should happen.
- Holding and advancing have been significantly simplified. They were just getting way too complicated, which made them slow, prone to occasional bizarre behavior and much harder to generalize for (future) use on other maps. Of course, currently they are actually too simplistic...
- The operations layer, while functional, is still incomplete. For example, it distributes the units/squads into the three different zones of the Freelands maps, but it needs to give more instructions what exactly to do there (and where).
- The attack code needs to be completely redone and holding and advancing need to be refined.
- Threats against the own leader and the final push against the enemy leader are not dealt with at all yet, other than what happens out of the “advance through the three zones and attack whomever gets into your way” instructions. That works, barely, against the default AI, but is really bad against a human player.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.2
Welcome back, Fred!
Maintainer of the Imperial Era and the campaigns Dreams of Urduk, Epic of Vaniyera, Up from Slavery, Fall of Silvium, Alfhelm the Wise and Gali's Contract.
But perhaps 'maintainer' is too strong a word.
But perhaps 'maintainer' is too strong a word.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Well, here we go... After a long, long time, v0.14.3 of Fred / AI-demos is now on the 1.12 add-ons server. This version features huge changes to Fred, but a lot of them are "under the hood", so while they hopefully will eventually lead to better AI performance, they might not be immediately visible when playing.
I've been reluctant to release this as it is still very much work in progress and it's essentially impossible for anybody else by myself to know whether a strange move means:
I've been reluctant to release this as it is still very much work in progress and it's essentially impossible for anybody else by myself to know whether a strange move means:
- Yeah, I know, I have not gotten to that part yet.
- That's done and this is as good as it's going to get.
- Oops, Fred should not be doing that.
Some of the ifs and buts
v0.14.3 complete changelog
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Hey mattsc, I've tried out Fred. Here are my thoughts:
RNG chose elf for me which is relatively hard to beat as orc (but possible, as many top players' matches proved, for example: http://youtu.be/c0fkuORgHJw. Orc is ShadowVeil, elf Dauntless - both very good players). I'm an averagely skilled ladder player so I don't think Fred was supposed to beat me anyway, but there are a few things that should be improved:
1) long-term planning so that your units are where they should be at the right time
2) protecting wounded/experienced/situationally important units - they can be used to deal damage, but should be covered with grunts/trolls
3) knowing what units should do - you don't send a 4 mp troll to pillage villages - you do it with a wolf. You don't defend a village with a fragile assassin - you do it with a grunt or run away.
Good luck with the project, I'll probably send more replays in near future.
Fred vs Elf:
1) long-term planning so that your units are where they should be at the right time
2) protecting wounded/experienced/situationally important units - they can be used to deal damage, but should be covered with grunts/trolls
3) knowing what units should do - you don't send a 4 mp troll to pillage villages - you do it with a wolf. You don't defend a village with a fragile assassin - you do it with a grunt or run away.
Good luck with the project, I'll probably send more replays in near future.
- Attachments
-
- fred_vs_elf.gz
- (22.85 KiB) Downloaded 343 times
Polish BfW fansite | Polish BfW translation | My Ladder profile
"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Hi ForPeace,
Thank you for the replay and the comments. I am actually not as unhappy watching Fred in the replay as I thought I would be. He's doing more or less what I can expect him to do at this point (and no, I don't think he'll ever win on even terms against a player like you).
The goods news are that I knew of many of the shortcomings you point out, although some I was not (or not as) aware of. Some of those are comparatively "easy" to deal with (such as improving which units to chose for which task), while others (long-term planning or defensive positioning of units relative to each other and distribution across the map) are really hard to do and might simply not happen with Fred. I am not saying that they're impossible, in principle, just that they are much easier for the human brain than for a computer and that I still don't know how to do them.
Well, I'll keep working on Fred for a little. Quite honestly, at some point I'll have to pull the plug. I've been at this on and off literally for years and I am often feeling like I am getting nowhere (new) with this. But your replay gives me some hope that at least the recent changes are working more or less as I'd expect them to. Whether that's good or bad is a separate question.
In any case, the replay will be very useful for continuing the work and setting up some test cases. Thanks!
Thank you for the replay and the comments. I am actually not as unhappy watching Fred in the replay as I thought I would be. He's doing more or less what I can expect him to do at this point (and no, I don't think he'll ever win on even terms against a player like you).
The goods news are that I knew of many of the shortcomings you point out, although some I was not (or not as) aware of. Some of those are comparatively "easy" to deal with (such as improving which units to chose for which task), while others (long-term planning or defensive positioning of units relative to each other and distribution across the map) are really hard to do and might simply not happen with Fred. I am not saying that they're impossible, in principle, just that they are much easier for the human brain than for a computer and that I still don't know how to do them.
Well, I'll keep working on Fred for a little. Quite honestly, at some point I'll have to pull the plug. I've been at this on and off literally for years and I am often feeling like I am getting nowhere (new) with this. But your replay gives me some hope that at least the recent changes are working more or less as I'd expect them to. Whether that's good or bad is a separate question.
In any case, the replay will be very useful for continuing the work and setting up some test cases. Thanks!
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
So, while I've done absolutely nothing on Fred since I put the new version onto the add-ons server, that's given me some time to think and let ideas develop and settle. Here are a few of my usual long-winded ramblings coming out of that.
Hmm, re-reading this I notice it could come across as somewhat defensive in reply to ForPeace's post. It's not meant that way, I really appreciate the effort and the comments! It's just semi-stream-of-consciousness rambling trying to figure out what to do next, and putting it out here in case anybody has comments.
I'll keep muddling along, but don't hold your breath ...
First, a quick explanation ...
What I am going to do next:
What I'll try, but no promises:
What is not going to happen:
I'll keep muddling along, but don't hold your breath ...
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Hi mattsc,
Don't feel like I had been criticizing you or what you have done. I really appreciate your work with Fred as a step to make Wesnoth AI better.
Thanks for your clarification. I'd be happy to see and test the future versions of Fred. Once again, good luck
Don't feel like I had been criticizing you or what you have done. I really appreciate your work with Fred as a step to make Wesnoth AI better.
There's a problem about the testing. Playing against skilled players, Fred will never achieve a successful attack because we are usually well prepared to defend. In fact, in my game Fred was not supposed to attack my defense. Maybe poisoning a scout would be the only thing I would do as orc. Myself, I'd consider playing worse deliberately in order to see how Fred can judge whether he should or not attack as now my good play didn't give him a chance to attack. (Fred was right not to do so )Making Fred push forward harder. That's not as easy as it sounds. If you make him too aggressive, you'll essentially end up with the default AI. If you don't make him aggressive enough, you get a very passive AI. So you need to find some balance, and sometimes that will result in too little aggressiveness and sometimes in too much. I think as a general rule that's unavoidable, but the current behavior can hopefully be improved.
Thanks for your clarification. I'd be happy to see and test the future versions of Fred. Once again, good luck
Polish BfW fansite | Polish BfW translation | My Ladder profile
"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Thanks for the reply, ForPeace.
Yeah, I know about difficulty with testing. As I have mentioned previously, I do a lot of testing against the default AI simply for time reasons, which suffers from the opposite problem. So what I do mostly is have the two AIs set up interesting situations, and then I go back to those in replays and manipulate them in debug mode to set up test cases. So it's a mix of AI and manual testing. The nice thing with your replay was to see that Fred applied some of those lessons correctly (more or less) against a human player as well.
Maybe giving Fred a bit more gold than the player would be an alternative/additional test case to playing worse deliberately (for games against the default AI I do the opposite ), as that is probably the situation one would more commonly encounter in a "normal setting" against the AI. In any case, you might want to wait until I have taken care of the things I list under "what I am going to do next" above as those by itself should have a pretty significant impact (I hope).
But most of all, thank you for your time and effort in testing and posting your thoughts. It really is very useful!
No worries, I didn't take it that way. It's just that, after writing my previous post (which is really some incoherent mix of my own thoughts and a reaction to your post/replay), I noticed that it might come across that way, so that's why I added the disclaimer at the end.ForPeace wrote:Don't feel like I had been criticizing you or what you have done.
Thank you, although so far I seem to be mostly finding what does not work. But seriously, as I said, I am actually reasonably happy with some of the stuff I saw, even though the overall behavior is not (yet?) any better than it was before. Honestly, I've been very close to giving up on Fred recently, and your post pushed me farther over to the let's-keep-trying side.ForPeace wrote:I really appreciate your work with Fred as a step to make Wesnoth AI better.
Thanks for clarifying that, that's good to know. My statement was meant more general than just in reply to your replay. I have definitely encountered situations where Fred does not push hard enough, but getting the defensive behavior right is definitely one of the hardest things I've been trying to do.ForPeace wrote:There's a problem about the testing. Playing against skilled players, Fred will never achieve a successful attack because we are usually well prepared to defend. In fact, in my game Fred was not supposed to attack my defense. Maybe poisoning a scout would be the only thing I would do as orc. Myself, I'd consider playing worse deliberately in order to see how Fred can judge whether he should or not attack as now my good play didn't give him a chance to attack. (Fred was right not to do so )
Yeah, I know about difficulty with testing. As I have mentioned previously, I do a lot of testing against the default AI simply for time reasons, which suffers from the opposite problem. So what I do mostly is have the two AIs set up interesting situations, and then I go back to those in replays and manipulate them in debug mode to set up test cases. So it's a mix of AI and manual testing. The nice thing with your replay was to see that Fred applied some of those lessons correctly (more or less) against a human player as well.
Maybe giving Fred a bit more gold than the player would be an alternative/additional test case to playing worse deliberately (for games against the default AI I do the opposite ), as that is probably the situation one would more commonly encounter in a "normal setting" against the AI. In any case, you might want to wait until I have taken care of the things I list under "what I am going to do next" above as those by itself should have a pretty significant impact (I hope).
But most of all, thank you for your time and effort in testing and posting your thoughts. It really is very useful!
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
After a couple weeks of "creative break," I'm back at working on Fred. I've created an issue on Fred's github site with a checklist of the things I want to do for the 1.14.4 release. Honestely, there's not really a point in that other than playing around with how to do something like that on github, but what the hay. In principle people could even add comments there, but I have no idea if one needs a github account for that and whether I need to set up the permissions differently but, quite honestly, I am way too lazy to try to find out ...
In any case, the whole point is that I am back at it and that there will hopefully be a new release "soon". Whatever that means.
In any case, the whole point is that I am back at it and that there will hopefully be a new release "soon". Whatever that means.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
That's an ambitious task list. If you check all those boxes you might call it v0.15.
Re: Help with testing Fred - Freelands MP Custom AI v0.14.3
Thanks, Cold Steel, but note that I am using words like 'improve' a lot, rather than 'fix' or 'perfect'. That could really mean anything. So, not sure if that will qualify for v0.15 yet, we'll see.Cold Steel wrote:That's an ambitious task list. If you check all those boxes you might call it v0.15.
I'm on to the second major bullet now and found that the cause of Fred not protecting the leader at the end is different from what I had thought (so I updated the wording). It's going to be a bit harder to take care of than I had expected, but at least I know what is going on.
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on