Help with testing Fred - Freelands MP Custom AI v0.14.11

Discussion of all aspects of multiplayer development: unit balancing, map development, server development, and so forth.

Moderators: Forum Moderators, Developers

Post Reply
mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.9

Post by mattsc » March 5th, 2018, 8:27 pm

singalen wrote:
March 5th, 2018, 6:32 am
Sorry, it broke again on 1.13.12 - dev version 8(
attempt to call a nil value (filed 'maxn')
ca_zone_control.lua:2636: in field 'execution'
[string "local self, params, data = ..."]:2:in main chunk
Huh, that's strange. The message complains about Lua function table.maxn() not existing. That function was indeed deprecated in Lua 5.2 and removed in 5.3 (I think), but I don't understand why this starts producing errors now. I don't see anything in the Wesnoth changelog since the update to 5.3.4 in Wesnoth 1.13.10. Also, I didn't get that error in my own tests. I'm traveling and don't have access to a compiled version of current master right now though (will have one by the end of the week), so I've been testing with 1.13.11, so maybe something's changed since then that I am not aware of.

Well, whatever the cause, the fix is simple and has been committed. Thanks for reporting.

mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by mattsc » May 9th, 2018, 2:38 am

So ....

I took a while off from working on Fred because RL was crazy, because I wanted to concentrate on getting my SP campaigns ported to 1.14 and, well, mostly because I wanted some time off. I've been re-familiarizing myself with the status of the code over the last few days and am back at it now.

First thing I did was committing all the previous changes and releasing this as a new version to both the 1.12 and 1.14 add-ons servers. This means that:

Fred / AI-demos v0.14.10 is now available on the Wesnoth 1.12 and 1.14 add-ons servers

A couple important comments: If you've played against Fred before, there's no need to bother with this version. It's a lot of partial fixes to existing problems, some of which are probably making some aspects temporarily worse. The next thing that needs to happen is some serious work on the operations layer which, I hope, will in the end actually make a real difference.

First, however, I am going to remove all the conditionals and duplicate code that makes AI-demos work in both 1.12 and 1.14 and properly port everything to work correctly and without deprecation warnings in 1.14, and 1.14 only. Thus, starting from AI-demos v0.14.11, support for Wesnoth 1.12 will be discontinued. That part isn't all that much work, I think (hope), so I should have that version out within a week or two, after which I'll be back to working on the AI behavior itself.

Anyway, tl;dr. There remains just this:
v0.14.10 complete changelog

Code: Select all

----- 0.14.10, 8 May 2018 -----

- Fred (Freelands AI for Northerners, Side 1):
  - Operations layer:
    - Calculate fronts from in-zone influence only
    - New full-move versions of influence maps
    - Add individual unit influence maps
    - Change to leader_distance metric
    - Fix bug in base_power calculation
    - New menu options to display behavior instructions
  - Attacking:
    - Fix bug which caused Fred to prefer attacking full-health units over easy
      kills
    - Allow using the leader in attacks when there is a small chance of the
      leader to die in the counter attack.
      - This is configurable and can be (re)set to zero CTD if desired
  - Holding:
    - Do not push too far ahead of main front with few units
    - Do not do normal hold if it is too wasteful
    - Keep already protected hexes in protect_locs
  - Advancing:
    - Discourage advancing behind enemy lines without sufficient support
    - Set advance direction to forward or backward depending on enemy threat
      and own-unit support
    - Full reworking of advancement rating (roughly gold-based now)
  - Functionality and misc. other changes:
    - Also turn shroud off at scenario start (shroud can get the AI to crash)
    - fred_utils: new functions linear_regression(), behind_enemy_map(),
      support_maps(), approx_value_loss()
    - Many changes/additions to rating functions, game state maps, supporting
      code and debugging information to support the changes above
    - Add warning that development will be discontinued on Wesnoth 1.12

name
Posts: 371
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name » May 10th, 2018, 4:52 am

mattsc wrote:
May 9th, 2018, 2:38 am
If you've played against Fred before, there's no need to bother with this version.
The changelog looks pretty enticing so I might have to bother just the same.
mattsc wrote:
May 9th, 2018, 2:38 am
The next thing that needs to happen is some serious work on the operations layer
Huzzah!

mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by mattsc » May 10th, 2018, 2:19 pm

Cold Steel wrote:
May 10th, 2018, 4:52 am
The changelog looks pretty enticing so I might have to bother just the same.
Well, I won't tell you not to, of course, any input is great, but ... The problem is that some of these new changes aren't balanced right yet. Things like "discourage advancing behind enemy lines without sufficient support" do what they say, but they also cause Fred to be (yet again) too hesitant in some situations. Essentially, it comes down to what you comment on next:
Cold Steel wrote:
May 10th, 2018, 4:52 am
mattsc wrote:
May 9th, 2018, 2:38 am
The next thing that needs to happen is some serious work on the operations layer
Huzzah!
I know I have been saying that I'd do this for several releases now, and then I've always gotten side-tracked by problems I found in the tactical layer. This last round of changes really made it obvious though that I am at a dead end here unless I do actually put in the work on the operations layer. So that's next — until I find something else to distract me. :P

[There's also a not super long, but noticeable delay before every single move now, but that's a comparatively minor issue that I will take care of once the functionality is there.]

name
Posts: 371
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name » May 11th, 2018, 6:36 am

Yeah, the operations layer is the holy grail for wesnoth AI.

Very excited. ^_^

name
Posts: 371
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.10

Post by name » May 12th, 2018, 3:47 pm

Just a bit of fun with v14.10:

knalgans_vs_14.10_income+6.gz
(33.61 KiB) Downloaded 52 times
undead_vs_14.10_income+6.gz
(41.52 KiB) Downloaded 56 times

It could be that I am rusty, but I did not notice any new or severe issues from the tactical layer. The utility weightings seemed about right.

mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc » May 17th, 2018, 4:32 am

Thanks, Cold Steel, for continuously doing all that play testing (those are some long games!) and posting the replays! Looks like maybe I did not make things significantly worse in 1.14.10 after all. ;) And I'll get a couple nice new test cases out of this.

In the meantime, I just released v0.14.11 with the announced move to Wesnoth 1.14 only. This release consists almost entirely of internal changes and clean-up (well, and the removal of some outdated or broken code or things that are now in Wesnoth mainline). None of Fred's behavior has been changed.

I will now move on to working on the operations layer. I have no idea how long it is going to take until I have progress that will actually make a difference. I am sure that it will take some time, but I am hoping that it is a question of when, not if.
v0.14.10 complete changelog

Code: Select all

----- 0.14.11, 16 May 2018 (drop support for BfW 1.12 and code clean-up) -----

- Remove all the version checks and only keep Wesnoth 1.14 compatible code
  - This means that AI-demos will not work in Wesnoth 1.12 any more starting
    from this version
  - Replace deprecated functions and macros
- Fred (Freelands AI for Northerners, Side 1):
  - Set up the scenario events, menu items, CA debugging tools etc. so that they
    are only present in scenarios actually using Fred.
  - Fred quitting because he does not know how to play the map, faction or side
    should count as player victory, not defeat
- Other misc. changes and clean-up
  - Remove local ai_helper.lua and battle_calcs.lua and use mainline versions
  - Remove prune cart scenario
  - Remove machine learning recruiter code
  - Remove 'Default + Extra Targets' test AI
  - Lots of internal changes and clean-up of Lua code and scenarios

name
Posts: 371
Joined: January 6th, 2008, 3:32 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by name » May 18th, 2018, 4:36 pm

mattsc wrote:
May 17th, 2018, 4:32 am
doing all that play testing (those are some long games!)
The funny thing is, they are so long because I tried to make them shorter with more aggressive play early on. Fred did a good job of punishing that recklessness as I should by now have expected.
mattsc wrote:
May 17th, 2018, 4:32 am
I will now move on to working on the operations layer. I have no idea how long it is going to take until I have progress that will actually make a difference. I am sure that it will take some time, but I am hoping that it is a question of when, not if.
Looking at your formidable task list for the operations feature, I wonder if it could be broken up over a number of versions? I imagine many of those sub features will heavily influence each other to the point they may be inseparable. But breaking it up allows you to better evaluate the effects and appreciate the progress you are making so that you do not lose your mind in the middle of it. :)

mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc » May 19th, 2018, 2:18 pm

Cold Steel wrote:
May 18th, 2018, 4:36 pm
The funny thing is, they are so long because I tried to make them shorter with more aggressive play early on. Fred did a good job of punishing that recklessness as I should by now have expected.
Well, that's good to know. :)
Cold Steel wrote:
May 18th, 2018, 4:36 pm
Looking at your formidable task list for the operations feature, I wonder if it could be broken up over a number of versions? I imagine many of those sub features will heavily influence each other to the point they may be inseparable. But breaking it up allows you to better evaluate the effects and appreciate the progress you are making so that you do not lose your mind in the middle of it. :)
Yeah, I agree, and that's certainly the plan. I am not entirely sure yet what is needed for that and what steps might make sense for intermediate release, but they always seem to present themselves as I go along. I'll keep you posted. It's too late for my mind though! ;)

denispir
Posts: 62
Joined: March 14th, 2013, 12:26 am

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by denispir » Yesterday, 11:07 am

Hello,

First, thank you very much @mattsc for all your great and hard work on Lua AIs and micro AIs for Wesnoth. (Without all that I would not even start and explore the domain, or only in mind, which cannot go very far...)

I wish here to expose a very general idea. Sorry to post here [*]. And more sorry for the big post. (Also, english is a foreign lang for me so it may be a little hard to understand, but it is not my fault ;) .)

Imagine a very simple scenario: 2-side battle, fair map, possibly designed equivalent or even symetric, few and ordinary unit types, maybe just 1 per side, no special "gifts" (abilities, weapon specials), no levelling thus no issue of XP... The only difference between the sides is that side 1 recruits more but weaker units while side 2 recruits less but stronger units. Those unit types may be level 1 and 2, but tweaked to make them weaker/stronger in a simple way, to make things obvious, say the stronger type just inflicts double damage.
  • How would you play this to win, in either case? (side 1 or 2)
  • How should an AI be designed to (have an optimal probability to) win with side 1? side 2?
  • How should an AI be designed to win whatever the side it controls?
I had thoughts about that (this example is just a way to expose them more easily), but decided to post about them after reading by you (somewhere in this thread?) that Fred's attack decision is basically based on:
Attack when the balance of damage is positive.
(I hope I understood it rightly, because you wrote it out of context.)
If we did so strictly and clearly and simply, as human player or AI designer, we would never win with side 1 (numerous weak units). All else being equal or random, despite our bigger number of units (which lead to local balanced ot advantageous situations) most combat opportunities would be in favor of side 2.

And yet, the scenario is fair and balanced, and any non-novice human player will easily win (with either side for the matter) against the RCA AI. Fred is not directly concerned because of its specialisation (in map, faction, side), but indirectly probably. I started to think at this question after playing several times in a short while with or against only or mainly level 0 units. How would/could/should an AI be designed, in terms of fundamental principles, so that it "naturally" plays well such a game? especially with the side having numerous weak units.

This question radically crushed down my initial idea, more or less the same as yours (IIUC): try to evaluate the balance of damage (albeit I want the eval to remain approximate, but this is another question, related but distinct) (but firstly evaluating probas of death, yet another question, related but distinct). Obviously, since they basically deal half damage, weak units have few(er) opportunites to fight and deal more damage, even widening the evaluation to a "group attack" (my term), meaning including potential attacks by others units (time). [Further widening the evaluation obviously leads to include counter-attacks, which have even greater chances to be disadvantageous in stricts terms or ratio of damage, for the same reasons plus the question of initiative, firstly choice of weapon range but also choice of terrain etc...]

Note: AIs may not be that bad in such a situation for reasons which are clearly distinct from the principles hidden behind this question, such as blindly or exaggerately following ToD for instance (RCA AI not corrected is not that bad with level 0s because it is far too agressive and too few cautionous).

As often happens the answer popped up intuitively in my mind. But I know, also intuitively, that it is only half right, thus half wrong, so that instead of elaborating I will say it as concise as possible, expecting collective intelligence: expose your own reflexions and/or answers, and let us all brass all that into a better understanding. So my present answer is:
Attack when the situation is better than average.
(or than random, or than statistical expectancy...)
(Consider the contrast with: Attack when the balance of damage is favorable.)
(And apply this to the example exposed above.)

I also wish to open another reflexion caused by this one, which I call "meta-criterion". what is the criterion to judge an AI criterion? (a factor of choice, since finally AI as well as human play is all about choosing) My present idea is that an AI criterion is good if ever it makes a favorable difference in average (or it is better than random, or than ignoring it, or than processing it wrongly...)

Thank you,
Diniz

[*] Please someone create an AI forum; for both AI usage & research/developpement, both standard and custom AIs, both general and micro AIS. It would help very much to share and mix and improve reflexion and also knowledge (of existing AIs, their usage, their principles). And basically: where to post AI questions or reflexions?

mattsc
Posts: 1078
Joined: October 13th, 2010, 6:14 pm
Location: Wandering, mostly aimlessly

Re: Help with testing Fred - Freelands MP Custom AI v0.14.11

Post by mattsc » Today, 4:07 pm

Hi denispir,

No problem posting here, as far as I am concerned. This thread or opening your own thread would also be possible, but this is very similar to some of the ideas implemented in Fred, so this is perfectly appropriate.

As for your comment concerning a dedicated AI forum, you could write the forum moderators group, but TBH there aren't all that many AI topics, so they might not consider that worth it. The Coder's Corner forum used to say in its description that it was for AI topics, but I just saw that it does not state so specifically any more. Anyways, that's the place where AI topics usually go.

As for your comments on AI evaluations, I entirely agree. I don't know in what context that statement of mine that you quote was made, but it was either an early version or just a very high-level summary. I mean, that is part of the individual attack evaluation in the tactics layer, but there are also additional considerations in the operations layer that lay out the overall strategy and make the AI more aggressive when the overall situation is good and less so when it is bad. So the damage calculation gets influenced by overall modifiers.

Let me give you another simple example when straight out damage statistics don't work well. Let's say you have a valuable unit that has one strike and 1 HP. It is up against an enemy unit with the same stats and both are on 50% defense terrain. You attack and strike first, so you have a 50% chance of killing the enemy. If you miss, the enemy strikes back and has a 50% chance of killing your unit, making it an overall 25% probability of you losing your unit. So, statistically, things are very strongly in your favor. As a human player, would you do that attack? And the answer is, it depends. If there are a large number of pairings like that on the map: sure I would, because overall (on average) it would come out in my favor. If this were the only pairing like that, and if that unit were important: hell no, 25% chance of losing it is way too high a risk.

So what I am saying is that I agree with you, and what I am trying to do with Fred is to implement more of the kind of thought processes that a human player goes through. What I am learning though is that the devil is in the details, and that finding something that works in general is quite hard to do. General thoughts like these are good and important, but making things work in practice, and in particular in a general setting, is very difficult, at least for me.

I'm afraid I don't really have much more to add at this time. Maybe I missed something you wanted me to comment on specifically. Please remind me if that's the case.


BTW, I have not replied to your other post, because I do not really have anything to add there. What you write there sounds good in general too, but in my experience until you actual try something like that in practice, it's hard to say if it will work as you imagine. Same comment as above, really. :)

Post Reply