Help with testing Fred - Freelands MP Custom AI v0.14.15

[mattsc]

What is the procedure to upgrade Fred on my machine?

All you need to do is to go on the add-ons server, look at the AI Demos add-on, and if it tells you that you don't have the latest version, install it again.

You could, in principle, also get the current development version from the AI-demos github repository, but unless you want to actively get involved in the AI development, I would not recommend that. That version is more often than not playing worse than the previous release — because once I get to one that actually does something consistently better than the previous release, I'll make that the next release.

Hope that answers your question.

[sacredceltic]

Hi and thank you for your former reply.
I did what Tauriel advised in order to be able play Fred with other factions, which I was anxious to.
I chose 3 different random factions using Fred
I get an error message in : generic_recruit_engine.lua:673: attempt to index field 'prerecruit' (a nil value)

[sacredceltic]

I also got the following error message:

fred.lua:3096: attempt to concatenate field 'leader_type' (a nil value)

[mattsc]

Hi and thank you for your former reply.
I did what Tauriel advised in order to be able play Fred with other factions, which I was anxious to.
I chose 3 different random factions using Fred
I get an error message in : generic_recruit_engine.lua:673: attempt to index field 'prerecruit' (a nil value)

If I understand you right, you are trying to use Fred for more than one side in the same game? If so, that is not possible a this time. In fact, I believe Fred cannot currently be used simultaneously with any other AI that also uses the generic recruit engine (that is, the Experimental AI in mainline or 'Ron' from AI-demos).

Also, I would be very surprised if Fred could handle multiple opponents well at this stage. He's really meant for 1-vs-1 maps only at this time. Well, no, he's really only meant for playing Northerners as Side 1 on one very specific map so far. Generalizations will come, but I am not there yet.

[Tauriel]

Fred can indeed play simultaneously with the Experimental AI, only issue is that he will stop recruiting units on his second turn. If you're curious how Fred would perform against Experimental, and don't know how to fix this by LUA code, you can simulate recruits by spawning units for each side (or only Fred) in the associated .CFG file for the map.

Fred can also handle multiple opponents very well.

[mattsc]

Hi Tauriel, thanks for that information!

Fred can indeed play simultaneously with the Experimental AI, only issue is that he will stop recruiting units on his second turn. If you're curious how Fred would perform against Experimental, and don't know how to fix this by LUA code, you can simulate recruits by spawning units for each side (or only Fred) in the associated .CFG file for the map.

Well, that's really what I meant with Fred not being able to play against the Experimental AI. The reason for this is the way how the recruiting is implemented in the code (the AIs mix up the recruiting information for the different sides) and we will fix this eventually. Great that you found a work-around for it in the meantime! I assume that you can also spawn these units using debug mode and shift-c.

Fred can also handle multiple opponents very well.

That's good to know, thanks.

[Tauriel]

I assume that you can also spawn these units using debug mode and shift-c.

I didn't know that, sounds a lot more comfortable than messing around in CFGs.

[Horus2]

A long time ago, i promised to join the tests, but before publishing the outcome, my laptop broke down, and i had no access to the files. Five months later i still do not have access.
Whatever, a new Fred version is out. I played against him recently and here are my impressions:

001 - versus my drakes:

1) No wolf?
2) Two assassins on the same side. This is silly.
3) Eventually sent the assassin to the right. This mistake has zero impact on this match, though.
4) Threw the entire rush away? Why? The fork of the glider was not a real threat. It did not advance on left either.
5) What is this solo suicide on left, and on the top of it, on a 60% def? And if you are really into defending, those archers will do no good.
6) Skipping the poisoning entirely because of the fear of 2-4 dmg is a terrible miscalculation. And bought another troll, no excuse for that.
7) Neglected 8,5.
8 ) Both the defense on the left and the sneaking on the right is well-executed.
9) Still doing those back and forth dance moves and left the sneaky assassin exposed.
10) Never will be able to counterattack with these recruits.
11) Another troll. It is obviously over.
After this, turns start to take 5+ minutes, which is a faithful emulation of a low-ranked, whiny human opponent that is about to lose and is secretly hoping that you will accidentally drag your leader into the fray with a series of misclicks.

002 - versus my knalga. This time i admittedly played more sloppily and chose a faction which is weak against northerners to invoke aggressive thoughts in Fred.:

1) Once again no wolves.
2) Not an effective village grabbing. And goblins?
4) This ai stopped rushing entirely.
5) Zero reason to hit gryphon with grunt, and neglected the left side again.
7) Fred got very lucky.
10) Could have blocked the thunderer in 18,9.
11) Allowed
13) Abandoned an assassin.
15) Gave up his whole left side. Archer is always a terrible choice against knalga.
18) if not even that was tempting enough for the orcs...

Tauriel is right. I remember Fred from v.0.14.0 and this most recent version is a huge step back in almost every aspect. Sure, it can defend relatively well, but only because he does not stop defending, so his lines are stuffed with the unused material. Older Fred was reckless as a p1 northerners should, and when luck slightly favoured him, it was able to catch most players off-guard and sometimes did very good plays, even if accidentally. Maybe i have nostalgia glasses, but even his recruits and village grab pattern were more effective, not to mention the 95% less time-consumption in decision-making.

So, i must ask what the intentions of the v.0.14.2 update were, so i can evaluate my encounters with him accordingly. Was the aim solely to make Fred better at defense?
I can imagine this lazy, opportunist Fred version to be incredibly good at beating the default AI, because it basically lets it lose. Default AI sends unit at him blindly, and all Fred has to do is to devour the incoming snacks, and gaining confidence for the assault he launches only when his advantage is far superior. I fear your methods for evaluation (testing en masse vs default ai) are faking the actual state of Fred's strength, and it should not be something to be relied on.

ps: The easiest way to greatly improve Fred's performance would be a built-in guideline for healthy unit compositions against all the different factions. I had an impression once that the reason why Fred is unable to drive me out of his villages because his composition is inherently bad, like 40% of an army consisting of assassins. Message me if you need help in setting a golden standard for each matchup.

[Tauriel]

Tauriel is right. I remember Fred from v.0.14.0 and this most recent version is a huge step back in almost every aspect.

Fred is actually much smarter in 0.14.2, the main problem is that he is way too defensive. As mattsc mentioned at some point, Fred is still not taking advantage of his full potential with the new AI features implemented in 0.14.1 and 0.14.2.

Remember that Fred is still an early alpha, he is still undergoing huge changes that sometimes requires the code to temporarily be worse before he can get even better.

[Horus2]

Tauriel is right. I remember Fred from v.0.14.0 and this most recent version is a huge step back in almost every aspect.

Fred is actually much smarter in 0.14.2, the main problem is that he is way too defensive.

Can you please point it out in which areas you find 0.14.2 much smarter, for the sake of clarification?

[Tauriel]

Can you please point it out in which areas you find 0.14.2 much smarter, for the sake of clarification?

It was quite a long time ago I played with 14.2, so I don't remember exactly and can therefore not give you a detailed description at this moment. Also bear in mind that I experiment with Fred without (or mostly without) the Freelands layer and 1v1 matchup, so I deal with the "true" power of the AI.

Basically, in 14.0 all that Fred is doing is placing his units on advantageous terrain, waiting for a reasonable opportunity to attack, and swaps the injured troops with healthy ones. That's it. It's a very clear pattern that is quite easy to deal with. The real strength here is that he will try to flank his enemy though, which has surprised me a couple of times.

In 14.2 however, there is more depth in how Fred is placing and attacking with his units, he is better at protecting injured units, Fred's overall unit planning appears to be "trickier". But as I said, I can't go into detail here, I need to test 14.2 again first. I can do that next time I have time.

I think your problem with Fred is more about Freelands tactics. It's possible that Fred is currently much worse at Freelands specific stuff. I don't know as I'm not playing Freelands.

[mattsc]

Horus2: Thanks for the replays and the comments. I just got back from traveling and still don't have much time, so for now I'll just make a few general comments:

Maybe i have nostalgia glasses, but even his recruits and village grab pattern were more effective

Yeah, that seems to be the case at least to some extent, because I have not touched recruiting itself. Well, I have added to some of the internal mechanics, but I have not changed the choice of units at all. If you want to compare, you can always get v0.14.0 from the github repository (link's up there a few posts ago). On the other hand, that's probably not worth you time, and waiting for 0.14.3 (or later) might make more sense.

So, i must ask what the intentions of the v.0.14.2 update were, so i can evaluate my encounters with him accordingly.

The goal was to fix an obvious regression in 0.14.1. In that release, I had made the attack/counter attack calculation much more accurate than it had been. Then I released because that must make Fred play better, or at least not worse, right? Well, wrong, for two reasons. 1. There were some subtle but sometimes important bugs in the new code. 2. The weights for choosing between actions were tuned to the old method and didn't work so well with the new, even if more accurate one.

0.14.2 fixed some of those things, and also added some new evaluation of where to place units, when and how to advance or retreat etc. Check out the changelog for the most important changes. But the latter work is definitely not done yet and there are clear problems, which manifest themselves in some of the things you point out. My next step is to work out in a more global sense where to move units on the map in a coordinated way. That's what I am currently working on. Well, I haven't done any work on Fred in a month or so, but when I get back to it it's the next task. Quite honestly, I don't know how to solve that problem yet, but it is the limiting factor for any further real improvements.

After that, I am going to come back to tuning the choices between different actions, between attacking or defending, etc. I am setting up the code so that those can be done via free(ish) parameters in the code. And sure, I could do that now, but it takes a lot of time and testing to get it (somewhat) right, and then when I improve the mechanics, I need to do it all again.

As for the longer evaluation times, that comes from the more sophisticated evaluations Fred is now doing. Some of those are not very efficiently coded. (I am pretty sure I mention this somewhere in this thread, but I am too lazy to look up where.) But as with the above, I am not going to spend time on cleaning up code that might be changed significantly again. Remember, I have no idea what I am doing. I have no AI background whatsoever and only a very tenuous grasp on coding in general. It's a lot of try and error and most of what I write gets thrown out again sometime later.

So, yes, as Tauriel says, Fred is definitely still an alpha release and there will be ups and downs for a while. For right now, I am really only going to work on the unit distribution problem as it is the root source of a lot of the current evil. Or maybe lack of evil. These replays and comments are still useful though, because I can use them as test cases later, to see if the behavior in specific situation has improved. So, thanks again.

[max_torch]

Just wondering since a while back there was a google summer of code project where someone was also working on wesnoth AI, I read his write-up and it involved a lot of math, anyway just wondering if it was useful to you for developing Fred, or if you have any idea what happened to the result of that project?

[Horus2]

@mattsc: Thank you for the detailed answer! I will be around for the next version. Good luck with it!

[Tauriel]

This weekend I decided to start look at Fred v.0.14.2 to see if I can improve / fix some of his issues (for my own pleasure and for fun).

Though I currently need to spend lot of time mapping up the use and content of many variables, functions and their parameters, I have so far made Fred much less defensive (though it needs more work and be more properly done). His evaluation time is also very high, I've tried to improve it for some hour now, but only succeed to shorten it very slightly. But I think I will figure it out eventually.

If I continue to make successful progress in modifying Fred, my goal is to add an option in beginning of the game where you can choose Fred's playstyle. For instance the player could choose if he will be Defensive, Balanced or Super-Aggressive.

so, if I get there I might create an own thread about it and upload the modified Fred.