Help with testing Fred - Freelands MP Custom AI v0.14.15

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderator: Forum Moderators

Post Reply
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by mattsc »

max_torch: There have been quite a few GSoC projects on AI development over the years, but if we are thinking about the same one (it was called something like "Strategy formulation for the Wesnoth AI") then no, nothing applicable to Fred came out of that. For starters, the project never got to the point where it would have actually improved the Wesnoth AI (the student ran out of time, I think). And second, what Fred does along those lines is actually a lot more complex than what was attempted there. That's the advantage of only dealing with one faction on one map as opposed to trying to do it for all Wesnoth games, it's a much simpler problem.

Horus2: Thanks. I'll continue to post progress here. Unfortunately, it's probably going to be slow going for quite some time ...

Tauriel: That's great. I'm wondering if there's a way you could do this by using a version control system (e.g. making a branch of the AI-demos github repository). That way, if you made a change that would be useful for Fred in general in the future, it would be easy to earmark the respective commits for later inclusion. I'm afraid that large parts of the code will diverge too much (because of what I will be doing next) that simply using 'diff' will quickly become not very useful any more. Of course, you can also just manually keep track of what you're doing and let me know if you think anything might be useful in a general sense.
Tauriel
Posts: 12
Joined: February 5th, 2016, 7:30 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by Tauriel »

mattsc wrote:Tauriel: That's great. I'm wondering if there's a way you could do this by using a version control system (e.g. making a branch of the AI-demos github repository). That way, if you made a change that would be useful for Fred in general in the future, it would be easy to earmark the respective commits for later inclusion. I'm afraid that large parts of the code will diverge too much (because of what I will be doing next) that simply using 'diff' will quickly become not very useful any more. Of course, you can also just manually keep track of what you're doing and let me know if you think anything might be useful in a general sense.
If I success to modify / create something big enough / worthy to upload, my plan is to rename my modified Fred into another name. It would be my own AI, but it's based on Fred. That way there would be no confusions.

If I would create something good enough that you could use in Fred, it'd be a honor to share the code. If I will ever get that skilled and come so far, it's a subject we can discuss that day. :)

For now, I struggle trying to change some of Fred's behaviour, with small progress every time :P
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by mattsc »

Hey all,

I don’t know if anybody is still interested, but just in case here’s a little update.

No need to read this
Really!:
Bottom line is, Fred is currently incapacitated but I am working on reviving him. I have no idea whether the new Fred will be any better than the previous best version (which would probably be v0.14.0), but if nothing else he should be more streamlined and less complicated — which, hopefully, will make it easier to adapt him to other maps and scenarios.

Stay tuned, but don’t hold your breath. This is not going to happen quickly.
User avatar
doofus-01
Art Director
Posts: 4122
Joined: January 6th, 2008, 9:27 pm
Location: USA

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by doofus-01 »

Poor Fred. But it's nice to know you are still working on it.
BfW 1.12 supported, but active development only for BfW 1.13/1.14: Bad Moon Rising | Trinity | Archaic Era |
| Abandoned: Tales of the Setting Sun
GitHub link for these projects
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by mattsc »

Thanks, doofus-01.

As another update, Fred is now operational again, technically speaking, but he does not do particularly well yet. The main changes are the addition of a more powerful operations layer and the simplification of code that had gotten too complicated. For example:
  • Previously the individual units (or squads of a few units) decided themselves whether to set up a defensive line or not (I call that 'holding'). Now, the operations layer decides whether to hold or not. The squads still get to figure out the details (best units, best hexes), but they get no say in whether this should happen.
  • Holding and advancing have been significantly simplified. They were just getting way too complicated, which made them slow, prone to occasional bizarre behavior and much harder to generalize for (future) use on other maps. Of course, currently they are actually too simplistic...
So what’s next? The most important next steps:
  • The operations layer, while functional, is still incomplete. For example, it distributes the units/squads into the three different zones of the Freelands maps, but it needs to give more instructions what exactly to do there (and where).
  • The attack code needs to be completely redone and holding and advancing need to be refined.
  • Threats against the own leader and the final push against the enemy leader are not dealt with at all yet, other than what happens out of the “advance through the three zones and attack whomever gets into your way” instructions. That works, barely, against the default AI, but is really bad against a human player.
Nevertheless, Fred is operational again and I just watched three quick games against the default AI (Rebels and Drakes with 125 gold vs. Fred’s Northerners with 100 gold) and he won all of them, even with the current very much sub-ideal (to put it nicely) and incomplete behavior. So, there’s still lots to be done, but I am making progress. It’s nice to be back to being able to test Fred in context, rather than writing code in a vacuum without knowing whether it will actually work in-game!
User avatar
UnwiseOwl
Posts: 512
Joined: April 9th, 2010, 4:58 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.2

Post by UnwiseOwl »

Welcome back, Fred!
Maintainer of the Imperial Era and the campaigns Dreams of Urduk, Epic of Vaniyera, Up from Slavery, Fall of Silvium, Alfhelm the Wise and Gali's Contract.
But perhaps 'maintainer' is too strong a word.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

Well, here we go... After a long, long time, v0.14.3 of Fred / AI-demos is now on the 1.12 add-ons server. This version features huge changes to Fred, but a lot of them are "under the hood", so while they hopefully will eventually lead to better AI performance, they might not be immediately visible when playing.

I've been reluctant to release this as it is still very much work in progress and it's essentially impossible for anybody else by myself to know whether a strange move means:
  • Yeah, I know, I have not gotten to that part yet.
  • That's done and this is as good as it's going to get.
  • Oops, Fred should not be doing that.
Nevertheless, I think that I am now at a point where I'd like to hear others' opinions on the greatest shortcomings and, in general, how Fred does against different human players. So, if you want to take Fred for a test drive and let me know what you think, that'd be great. Just keep the "work in progress" bit in mind.

Some of the ifs and buts
Note that I have (somewhat) changed my personal goals of what I want to accomplish with this AI. I am not going for "perfect" any more, meaning that I am no longer trying to tweak the most out of the Freelands map or to code very specialized behavior. Instead, my main objective now is even more so than before to put together an AI that
  • provides an interesting alternative to the default AI and is hopefully somewhat fun to play against,
  • does overall well enough and avoids making too many really stupid mistakes, and
  • can (eventually) be adapted to be used on other maps more easily than previous versions of Fred.
One thing I found (yet again) to be very difficult is getting the balance between too aggressive and not aggressive enough right. I've come to the conclusion that I will not be able to get Fred to always do this right but, well, as I said the goal is for good enough most of the time without too many really bad mistakes.

Things on which I have not worked much since the previous release and known problems:
  • Attacks: the previous attack code is essentially unchanged, incl. that it definitely does not always get the aggressiveness right, that it can sometimes be slow and that Fred does not yet understand some weapons specials (such as drain; which is always funny to watch when he plays against undead)
  • Retreating: Fred's currently really bad at that
  • Fred's a bit fickle when it comes to playing close to either side's leader, but TBH, I am not sure how much more I will be able to improve that.
  • He also does a bad job at protecting units with high XP. I might be able to improve that a bit, but I'm not sure by how much.
  • Oh, and Fred currently assumes that the enemy leader stays close to the starting keep. You can probably throw him for a big loop if you moved your leader somewhere else. ;) That's an obvious thing to work on at some point, it just has not made it high enough on the priority list yet.
v0.14.3 complete changelog

Code: Select all

----- 0.14.3, 25 Apr 2017 -----

- Fred (Freelands AI for Northerners, Side 1):
  - Massive reworking of many parts of the AI, the highlights being:
    - Significant extension of the operations layer
    - Significant modifications to holding, advancing and village actions
    - Some changes to the AI setup and internal structure
  - Minor changes to attacks, recruiting and retreating
- Fred test scenario:
  - Modify for consistency with MP game:
    - Give side 2 a village at scenario start
    - Set village_gold=2
- Miscellaneous:
  - Remove high_xp_attack CA and test scenario as this is in mainline now
  - Add Invisible Unit for testing
  - Modify test scenarios to work in both 1.12 and 1.13, or add comments if
    they only work in one or the other version
User avatar
ForPeace
Posts: 164
Joined: December 12th, 2015, 3:09 pm
Location: Kraków, Poland

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by ForPeace »

Hey mattsc, I've tried out Fred. Here are my thoughts:
Fred vs Elf:
RNG chose elf for me which is relatively hard to beat as orc (but possible, as many top players' matches proved, for example: http://youtu.be/c0fkuORgHJw. Orc is ShadowVeil, elf Dauntless - both very good players). I'm an averagely skilled ladder player so I don't think Fred was supposed to beat me anyway, but there are a few things that should be improved:
1) long-term planning so that your units are where they should be at the right time
2) protecting wounded/experienced/situationally important units - they can be used to deal damage, but should be covered with grunts/trolls
3) knowing what units should do - you don't send a 4 mp troll to pillage villages - you do it with a wolf. You don't defend a village with a fragile assassin - you do it with a grunt or run away.

Good luck with the project, I'll probably send more replays in near future.
Attachments
fred_vs_elf.gz
(22.85 KiB) Downloaded 343 times
Polish BfW fansite | Polish BfW translation | My Ladder profile

"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

Hi ForPeace,

Thank you for the replay and the comments. I am actually not as unhappy watching Fred in the replay as I thought I would be. He's doing more or less what I can expect him to do at this point (and no, I don't think he'll ever win on even terms against a player like you).

The goods news are that I knew of many of the shortcomings you point out, although some I was not (or not as) aware of. Some of those are comparatively "easy" to deal with (such as improving which units to chose for which task), while others (long-term planning or defensive positioning of units relative to each other and distribution across the map) are really hard to do and might simply not happen with Fred. I am not saying that they're impossible, in principle, just that they are much easier for the human brain than for a computer and that I still don't know how to do them.

Well, I'll keep working on Fred for a little. Quite honestly, at some point I'll have to pull the plug. I've been at this on and off literally for years and I am often feeling like I am getting nowhere (new) with this. But your replay gives me some hope that at least the recent changes are working more or less as I'd expect them to. Whether that's good or bad is a separate question. ;)

In any case, the replay will be very useful for continuing the work and setting up some test cases. Thanks!
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

So, while I've done absolutely nothing on Fred since I put the new version onto the add-ons server, that's given me some time to think and let ideas develop and settle. Here are a few of my usual long-winded ramblings coming out of that.
First, a quick explanation ...
... why I said that I am not that unhappy with the posted replay, when Fred's clearly not playing all that well overall. That's because he's actually doing rather well at a couple of the things I've been concentrating on recently, namely the setting up of defensive lines and sending off of units into the different zones of the map. In particular the former is really hard to teach an AI. Well, for me, at least.

Now, he's obviously not doing this perfectly, but at least from my point of view it's much better, and much easier to modify and generalize, than it was before, even if it does not currently translate into better play.
What I am going to do next:
  • First thing is adding in retreating of units. That's the last big piece that's still actively missing from the code base (I think). To some extent this will also include protecting weak or important units.
  • Improve on which units are used for holding terrain, in particular villages. There's actually a reason why Fred used the Assassin on the village in the west, just that that reason is mostly wrong. :P
  • Fix the AI units abandoning the leader at the end. I think that's due to a known problem in the code that I thought was not all that important.
  • After that, I'll go on to trying to improve some of the existing behavior.
What I'll try, but no promises:
  • Making Fred push forward harder. That's not as easy as it sounds. If you make him too aggressive, you'll essentially end up with the default AI. If you don't make him aggressive enough, you get a very passive AI. So you need to find some balance, and sometimes that will result in too little aggressiveness and sometimes in too much. I think as a general rule that's unavoidable, but the current behavior can hopefully be improved.
  • The same goes for moving units from, for example, the right to the left. In ForPeace's replay, the enemy is keeping two units in the SE, so that's why Fred's keeping his units in the NE, to prevent those from stealing the NE villages. Now, he's keeping too many there, but this is a balancing act between too few and too many and I am sure I'll never get to the point where it's always "right". Nevertheless, I will try to improve on the current situation.
  • In the end, these things will be adjustable by tuneable parameters, similar to aggression and caution for the default AI, so the scenario/map designer (or even the player) can tweak the behavior one way or the other. In fact, those parameters are already in the code, but things are still too unstable to open those up for general use. I found Fred's behavior to be incredibly "touchy" to some of the parameters, resulting in almost unstable behavior at times. That probably that means that I have not found the right evaluation functions.
What is not going to happen:
  • Don't expect Fred to play like a human. My goal is to write an AI that can be used on more than the Freelands map. As such, I don't want a scripted or a rule-based algorithm that would have to be changed in fundamental ways to adapt to other maps. Instead, what you do is to write a set of weighting functions and combine those in some way or another. Then, as an example, if the final value is greater than 0 you keep the unit in the east, and if it is smaller than zero you move it to the west. That can be adapted to other situations by changing the weights, but it's never going to be "perfect" in all situations. The trick is to find a general behavior that, in most situations, qualifies as good enough.
Hmm, re-reading this I notice it could come across as somewhat defensive in reply to ForPeace's post. It's not meant that way, I really appreciate the effort and the comments! It's just semi-stream-of-consciousness rambling trying to figure out what to do next, and putting it out here in case anybody has comments.

I'll keep muddling along, but don't hold your breath ...
User avatar
ForPeace
Posts: 164
Joined: December 12th, 2015, 3:09 pm
Location: Kraków, Poland

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by ForPeace »

Hi mattsc,

Don't feel like I had been criticizing you or what you have done. I really appreciate your work with Fred as a step to make Wesnoth AI better.
Making Fred push forward harder. That's not as easy as it sounds. If you make him too aggressive, you'll essentially end up with the default AI. If you don't make him aggressive enough, you get a very passive AI. So you need to find some balance, and sometimes that will result in too little aggressiveness and sometimes in too much. I think as a general rule that's unavoidable, but the current behavior can hopefully be improved.
There's a problem about the testing. Playing against skilled players, Fred will never achieve a successful attack because we are usually well prepared to defend. In fact, in my game Fred was not supposed to attack my defense. Maybe poisoning a scout would be the only thing I would do as orc. Myself, I'd consider playing worse deliberately in order to see how Fred can judge whether he should or not attack as now my good play didn't give him a chance to attack. (Fred was right not to do so ;))

Thanks for your clarification. I'd be happy to see and test the future versions of Fred. Once again, good luck :)
Polish BfW fansite | Polish BfW translation | My Ladder profile

"When I say I've been playing for 10 years people come saying they've played for 15 years and that I know nothing about this game because I didn't use to play when the TRUE pros were playing xD" ~Hejnewar
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

Thanks for the reply, ForPeace.
ForPeace wrote:Don't feel like I had been criticizing you or what you have done.
No worries, I didn't take it that way. It's just that, after writing my previous post (which is really some incoherent mix of my own thoughts and a reaction to your post/replay), I noticed that it might come across that way, so that's why I added the disclaimer at the end.
ForPeace wrote:I really appreciate your work with Fred as a step to make Wesnoth AI better.
Thank you, although so far I seem to be mostly finding what does not work. ;) But seriously, as I said, I am actually reasonably happy with some of the stuff I saw, even though the overall behavior is not (yet?) any better than it was before. Honestly, I've been very close to giving up on Fred recently, and your post pushed me farther over to the let's-keep-trying side. :)
ForPeace wrote:There's a problem about the testing. Playing against skilled players, Fred will never achieve a successful attack because we are usually well prepared to defend. In fact, in my game Fred was not supposed to attack my defense. Maybe poisoning a scout would be the only thing I would do as orc. Myself, I'd consider playing worse deliberately in order to see how Fred can judge whether he should or not attack as now my good play didn't give him a chance to attack. (Fred was right not to do so ;))
Thanks for clarifying that, that's good to know. My statement was meant more general than just in reply to your replay. I have definitely encountered situations where Fred does not push hard enough, but getting the defensive behavior right is definitely one of the hardest things I've been trying to do.

Yeah, I know about difficulty with testing. As I have mentioned previously, I do a lot of testing against the default AI simply for time reasons, which suffers from the opposite problem. So what I do mostly is have the two AIs set up interesting situations, and then I go back to those in replays and manipulate them in debug mode to set up test cases. So it's a mix of AI and manual testing. The nice thing with your replay was to see that Fred applied some of those lessons correctly (more or less) against a human player as well.

Maybe giving Fred a bit more gold than the player would be an alternative/additional test case to playing worse deliberately (for games against the default AI I do the opposite :) ), as that is probably the situation one would more commonly encounter in a "normal setting" against the AI. In any case, you might want to wait until I have taken care of the things I list under "what I am going to do next" above as those by itself should have a pretty significant impact (I hope).

But most of all, thank you for your time and effort in testing and posting your thoughts. It really is very useful!
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

After a couple weeks of "creative break," I'm back at working on Fred. I've created an issue on Fred's github site with a checklist of the things I want to do for the 1.14.4 release. Honestely, there's not really a point in that other than playing around with how to do something like that on github, but what the hay. :P In principle people could even add comments there, but I have no idea if one needs a github account for that and whether I need to set up the permissions differently but, quite honestly, I am way too lazy to try to find out ...

In any case, the whole point is that I am back at it and that there will hopefully be a new release "soon". Whatever that means.
name
Posts: 570
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by name »

That's an ambitious task list. If you check all those boxes you might call it v0.15.
mattsc
Inactive Developer
Posts: 1217
Joined: October 13th, 2010, 6:14 pm

Re: Help with testing Fred - Freelands MP Custom AI v0.14.3

Post by mattsc »

Cold Steel wrote:That's an ambitious task list. If you check all those boxes you might call it v0.15.
Thanks, Cold Steel, but note that I am using words like 'improve' a lot, rather than 'fix' or 'perfect'. That could really mean anything. :P So, not sure if that will qualify for v0.15 yet, we'll see.

I'm on to the second major bullet now and found that the cause of Fred not protecting the leader at the end is different from what I had thought (so I updated the wording). It's going to be a bit harder to take care of than I had expected, but at least I know what is going on.
Post Reply