Machine Learning Recruiter

[SeattleDad]

I don't say that your recruiter isn't better than the default, just that its performance versus an AI doesn't necessarily indicate its overall quality.

Agreed. I haven't played against it myself very much. We'll have to see when I post it (hopefully with a few days). Hopefully some people can playtest it. I will say that in looking at it, it's doing things that look a lot smarter than, for instance, always recruiting Thunderers.

This is pretty much what I'm concerned about; you need to build new recruiters for virtually every scenario.

I think one simulation run for an entire campaign should suffice. You don't need to build a model for each map. Also, as I mentioned earlier, this only *might* be necessary when the campaign designer introduces units that that the AI hasn't see yet. Also note that 1) I'm not advocating replacing the existing recruiter any time soon, if ever and 2) You can run 600 --nogui runs of a multiplayer game in two hours on one thread on a Mac Mini.

Don't misunderstand me, this whole thing is really awesome stuff

Thanks!

I just have doubts that it'd fit well into the main code. Wesnoth is a rather big project, and they have to take many aspects into consideration when adding new features. Performance might not be the most important one of them.

I guess what I'm aiming for in the short term is for the recruiter to be accepted into Wesnoth where it would appear as an optional recruiter when playing multiplayer. I think it will be found to be a much more challenging opponent. Campaign designers would be free to choose this recruiter or the RCA AI recruiter as the recruiter for their scenarios.

This all should be clearer when I submit the patch.

[SeattleDad]

A couple questions that I've encountered recently as I'm trying to push this patch out:

1. I see ai/lua/extCAexample.lua was recently checked into trunk as an example of how a Lua candidate action should be defined in Wesnoth 1.11. The bulk of my code is in Lua, so it seems like I should use this. Could someone point me to documentation of how I can call extCAexample as a candidate action?
2. My recruiter needs to write out a fair number of logging messages when in "training mode" (as opposed to when used in game play, when I want it to be silent). Am I right that wesnoth.message is the best way to do this? It seems like that is the intent from looking at this: http://wiki.wesnoth.org/LuaWML:Display#wesnoth.message

[mattsc]

1. I see ai/lua/extCAexample.lua was recently checked into trunk as an example of how a Lua candidate action should be defined in Wesnoth 1.11. The bulk of my code is in Lua, so it seems like I should use this. Could someone point me to documentation of how I can call extCAexample as a candidate action?

Luaai_new_syntax. This is a new way of defining a CA for 1.11 that Nephro implemented recently. I've tested it and for all I can tell, it works without problems. The old method still will work though (no plans to deprecate it anytime soon), so whichever way you choose is fine.

2. My recruiter needs to write out a fair number of logging messages when in "training mode" (as opposed to when used in game play, when I want it to be silent). Am I right that wesnoth.message is the best way to do this? It seems like that is the intent from looking at this: http://wiki.wesnoth.org/LuaWML:Display#wesnoth.message

I guess it depends on where you want the output. wesnoth.message() will get you a message in the chat area of the window, and in the log (apparently, I haven't checked that). Personally, I launch Wesnoth from a terminal window for debugging and simply use print() - which will show up in the terminal and is easily redirected to a file.

[SeattleDad]

Luaai_new_syntax. This is a new way of defining a CA for 1.11 that Nephro implemented recently. I've tested it and for all I can tell, it works without problems. The old method still will work though (no plans to deprecate it anytime soon), so whichever way you choose is fine.

This is working in the gui for me, but is not working in --nogui mode. Has anyone gotten this working with --nogui?

For instance, the following does not work for me where I set up new_lua_demo.cfg to include extCAexample:
/Applications/Wesnoth1.11.app/Contents/MacOS/Wesnoth --nogui --multiplayer --controller 1:ai --controller 2:ai --ai-config 1:ai/ais/new_lua_demo.cfg --scenario multiplayer_Sablestone_Delta

[mattsc]

This is working in the gui for me, but is not working in --nogui mode. Has anyone gotten this working with --nogui?

I'd give it a very good chance that you and I are the only people who have used external CAs so far (well, except for Nephro). So I gave it a shot with a very simple toy AI and it seems to work for me. Here's what I did:

File: ai_extCA-test.cfg

Code: Select all

#textdomain wesnoth

{core/macros/ai_candidate_actions.cfg}

[ai]
    id=extCa_test
    description="External CA test"
    version=10710
    [stage]
        id=main_loop
        name=testing_ai_default::candidate_action_evaluation_loop
        [candidate_action]
            engine=lua
            name=external_example
            location="~/add-ons/LuaAI_tests/other/extCA-test.lua"
        [/candidate_action]
    [/stage]
[/ai]

File: extCA-test.lua

Code: Select all

example_ca = {}

function example_ca:eval(ai)
        print("External eval says hi!")
        local units = wesnoth.get_units { side = wesnoth.current.side, formula = '$this_unit.moves > 0' }

        if units[1] then return 10000 end
        return 0
end

function example_ca:exec(ai)
        print("External CA goes south, side:", wesnoth.current.side)
        local units = wesnoth.get_units { side = wesnoth.current.side, formula = '$this_unit.moves > 0' }
        for i,u in ipairs(units) do
            print('before:', u.id, u.x, u.y)
            ai.move_full(u, u.x, u.y+1)
            print('after: ', u.id, u.x, u.y)
        end
end

return example_ca

Launched from the command-line with:

Code: Select all

/home/misc/Wesnoth/trunk/projectfiles/Xcode/build/Debug/Wesnoth.app/Contents/MacOS/Wesnoth -d -m --parm 1:fog:no --parm 2:fog:no --parm 1:village_gold:2 --parm 2:village_gold:2 --ai-config 1:~add-ons/LuaAI_tests/other/ai_extCA-test.cfg --controller 1:ai --controller 2:ai --side 1:Northerners --side 2:Undead --nogui

Gives this output in the terminal window:

Code: Select all

Turn 1:
External eval says hi!
External CA goes south, side:   1
before: Northerners     19      4
after:  Northerners     19      5
External eval says hi!
 Player 1: 0 Villages
 Player 2: 2 Villages
Turn 2:
External eval says hi!
External CA goes south, side:   1
before: Northerners     19      5
after:  Northerners     19      6
External eval says hi!
 Player 1: 0 Villages
 Player 2: 2 Villages
Turn 3:
External eval says hi!
External CA goes south, side:   1
before: Northerners     19      6
after:  Northerners     19      7
External eval says hi!
 Player 1: 0 Villages
 Player 2: 3 Villages

So everything seems to work (and I confirmed in gui mode that I do get the expected behavior).

[JaMiT]

Thanks for the tip. I've given this a try and unfortunately adding:
--parm 1:seattle:hello
gives me the error message
20120709 22:44:31 warning unit: Unknown attribute 'seattle' discarded.

I was afraid that might happen. Is the attribute preserved under the [ai] subtag, though? Still kind of hacky I guess.

Still, it would be nice if I could pass in arbitrary attributes or there was even one free parameter called, for instance, "dev_parm", that could be used for this kind of thing.

Maybe a subtag of [side] that only exists if the game was started from the command line? A subtag that would simply store all the --parm settings, preserving them from being stripped? I'm thinking a really basic implementation that would not even be preserved over saved games, so it would be up to the WML author (you) to process this data early, probably in prestart.

I think that would be fairly simple to do. Not sure if it is appropriate for the official game, though.

[SeattleDad]

So I just uploaded the patch, which you can see at https://gna.org/patch/index.php.

Some still evolving documentation for the patch can be found at http://wiki.wesnoth.org/Machine_Learning_Recruiter. Key take-away from that page is that this patch beats the default RCA AI 70% of the time, a major win considering that only the recruiting is affected. Also, I haven't really focused on pushing accuracy as high as possible, delaying this until I had it running end to end. I expect the win rate to go up over time.

I'd be very interested in getting feedback on this patch. Can you build it? Are you seeing that it wins with roughly the same frequency that I do? How could it be improved? How can this patch be included in Wesnoth?

[lipk]

Well, I tried to apply the patch but it didn't exactly go seamlessly:

Code: Select all

Skipped missing target: 'b/RELEASE_NOTES'
Skipped missing target: 'data/ai/ais/ml_ai_faction_specific.cfg'
A         b
A         b/data
A         b/data/ai
A         b/data/ai/ais
A         b/data/ai/ais/ml_ai_faction_specific.cfg
Skipped missing target: 'data/ai/ais/ml_ai_faction_specific_weighted_random.cfg'
A         b/data/ai/ais/ml_ai_faction_specific_weighted_random.cfg
Skipped missing target: 'b/data/ai/ais/testing_ai_default.cfg'
A         data/ai/dev/class.lua
A         b/data/ai/dev
A         b/data/ai/dev/class.lua
A         data/ai/dev/default_ai_with_recruit_log.cfg
A         b/data/ai/dev/default_ai_with_recruit_log.cfg
A         data/ai/dev/ml_ai_events.lua
A         b/data/ai/dev/ml_ai_events.lua
A         data/ai/dev/ml_ai_faction_specific_hybrid.cfg
A         b/data/ai/dev/ml_ai_faction_specific_hybrid.cfg
A         data/ai/dev/ml_ai_features.lua
A         b/data/ai/dev/ml_ai_features.lua
A         data/ai/dev/ml_ai_general.lua
A         b/data/ai/dev/ml_ai_general.lua
A         data/ai/dev/ml_ai_random.cfg
A         b/data/ai/dev/ml_ai_random.cfg
Skipped missing target: 'data/ai/models/Drakes/.DS_Store'
Skipped missing target: 'data/ai/models/Drakes/default_model.json'
A         b/data/ai/models
A         b/data/ai/models/Drakes
A         b/data/ai/models/Drakes/default_model.json
Skipped missing target: 'data/ai/models/Knalgan/.DS_Store'
Skipped missing target: 'data/ai/models/Knalgan/default_model.json'
A         b/data/ai/models/Knalgan
A         b/data/ai/models/Knalgan/default_model.json
Skipped missing target: 'data/ai/models/Loyalists/default_model.json'
A         b/data/ai/models/Loyalists
A         b/data/ai/models/Loyalists/default_model.json
Skipped missing target: 'data/ai/models/Northerners/default_model.json'
A         b/data/ai/models/Northerners
A         b/data/ai/models/Northerners/default_model.json
Skipped missing target: 'data/ai/models/Rebels/default_model.json'
A         b/data/ai/models/Rebels
A         b/data/ai/models/Rebels/default_model.json
Skipped missing target: 'data/ai/models/Undead/default_model.json'
A         b/data/ai/models/Undead
A         b/data/ai/models/Undead/default_model.json
Skipped missing target: 'b/projectfiles/Xcode/Wesnoth.xcodeproj/project.pbxproj'
Skipped missing target: 'b/src/CMakeLists.txt'
Skipped missing target: 'b/src/SConscript'
Skipped missing target: 'b/src/ai/lua/core.cpp'
A         src/ai/waffles
A         src/ai/waffles/GActivation.cpp
A         b/src
A         b/src/ai
A         b/src/ai/waffles
A         b/src/ai/waffles/GActivation.cpp
A         src/ai/waffles/GActivation.h
A         b/src/ai/waffles/GActivation.h
A         src/ai/waffles/GBitTable.cpp
A         b/src/ai/waffles/GBitTable.cpp
A         src/ai/waffles/GBitTable.h
A         b/src/ai/waffles/GBitTable.h
A         src/ai/waffles/GDom.cpp
A         b/src/ai/waffles/GDom.cpp
A         src/ai/waffles/GDom.h
A         b/src/ai/waffles/GDom.h
A         src/ai/waffles/GError.cpp
A         b/src/ai/waffles/GError.cpp
A         src/ai/waffles/GError.h
A         b/src/ai/waffles/GError.h
A         src/ai/waffles/GHeap.cpp
A         b/src/ai/waffles/GHeap.cpp
A         src/ai/waffles/GHeap.h
A         b/src/ai/waffles/GHeap.h
A         src/ai/waffles/GHolders.cpp
A         b/src/ai/waffles/GHolders.cpp
A         src/ai/waffles/GHolders.h
A         b/src/ai/waffles/GHolders.h
A         src/ai/waffles/GLearner.cpp
A         b/src/ai/waffles/GLearner.cpp
A         src/ai/waffles/GLearner.h
A         b/src/ai/waffles/GLearner.h
A         src/ai/waffles/GMatrix.cpp
A         b/src/ai/waffles/GMatrix.cpp
A         src/ai/waffles/GMatrix.h
A         b/src/ai/waffles/GMatrix.h
A         src/ai/waffles/GNeuralNet.cpp
A         b/src/ai/waffles/GNeuralNet.cpp
A         src/ai/waffles/GNeuralNet.h
A         b/src/ai/waffles/GNeuralNet.h
A         src/ai/waffles/GRand.cpp
A         b/src/ai/waffles/GRand.cpp
A         src/ai/waffles/GRand.h
A         b/src/ai/waffles/GRand.h
A         src/ai/waffles/GTokenizer.cpp
A         b/src/ai/waffles/GTokenizer.cpp
A         src/ai/waffles/GTokenizer.h
A         b/src/ai/waffles/GTokenizer.h
A         src/ai/waffles/GTransform.cpp
A         b/src/ai/waffles/GTransform.cpp
A         src/ai/waffles/GTransform.h
A         b/src/ai/waffles/GTransform.h
A         src/ai/waffles/GVec.cpp
A         b/src/ai/waffles/GVec.cpp
A         src/ai/waffles/GVec.h
A         b/src/ai/waffles/GVec.h
Skipped missing target: 'b/utils/ai_test/ai_test2.cfg'
Skipped missing target: 'b/utils/ai_test/ai_test2.py'
A         utils/ai_test/analyze_log.py
A         b/utils
A         b/utils/ai_test
A         b/utils/ai_test/analyze_log.py
A         utils/ai_test/analyze_recruitment.py
A         b/utils/ai_test/analyze_recruitment.py
A         utils/ai_test/copy_predict_only_waffles.py
A         b/utils/ai_test/copy_predict_only_waffles.py
A         utils/ai_test/csv_normalizer.py
A         b/utils/ai_test/csv_normalizer.py
A         utils/ai_test/csv_to_arff.py
A         b/utils/ai_test/csv_to_arff.py
A         utils/ai_test/ml_utilities.py
A         b/utils/ai_test/ml_utilities.py
A         utils/ai_test/process_output_for_ml.py
A         b/utils/ai_test/process_output_for_ml.py
Skipped missing target: 'b/utils/ai_test/readme.txt'
Summary of conflicts:
  Skipped paths: 19

Of course, after that it was no suprise that it didn't work at all.

[SeattleDad]

Sorry about that.

Try it with patch -p1 to strip away the a/ and b/ prefixes.

[Crab]

It's good to see some working code making the AI better.

How can this patch be included in Wesnoth?

If it is consistenty better than the old recruitment (when trained for a particular map), I would include the patch in wesnoth when all the issues would be sorted out. Since the algorithm requires per-map training, it is not possible to set it as default recruitment; yet, it is possible to add somethings like "RCA AI with Machine Learning Recruiter" to the AI selection list; also, it is possible to enable it on some of the mainline campaigns by default.

A question - have you modified the source of Waffles or you're using unmodified source?

Also, if there are any bugs with --nogui, then you're welcome report them via bugs.wesnoth.org

[lipk]

Now I'm getting a compiler error:

Code: Select all

/home/lipk/projects/Wesnoth/wesnoth/src/ai/waffles/GTokenizer.cpp: In constructor ‘GClasses::GTokenizer::GTokenizer(const char*)’:
/home/lipk/projects/Wesnoth/wesnoth/src/ai/waffles/GTokenizer.cpp:75:27: error: ‘access’ was not declared in this scope

Additionally, note that patches must not trigger any warnings either.

[SeattleDad]

Now I'm getting a compiler error:


Additionally, note that patches must not trigger any warnings either.

What platform are you building on? Where are you seeing warnings? I'm not getting warnings in Xcode on OS X 10.7.

[lipk]

I'm on Linux, building with gcc. I get a lot of complaints about unused variables and parameters in src/ai/waffles. However, that isn't really important right now, those could be easily fixed. The problem at hand is the compile error. It fails to resolve an invoke to a function (apparently affiliated with file I/O). Probably other packages are needed to build the code as well (although I believe that #include-s wouldn't be resolvable in that case), or even worse, access() is an OSX specific function.

[SeattleDad]

I'm on Linux, building with gcc. ... The problem at hand is the compile error. It fails to resolve an invoke to a function (apparently affiliated with file I/O).

Okay, thanks for the info. I'm checking with Mike Gashler, the maintainer of Waffles on this one. Will get back to you when I hear from him.

[SeattleDad]

It's good to see some working code making the AI better.

Thanks!

How can this patch be included in Wesnoth?

Since the algorithm requires per-map training,

This is not exactly correct. The current status is that I explicitly trained it on four maps. When tested on those four maps with the "pure" AI, win percentage is 71.5%. I then tested it on those four maps plus eight other maps I hadn't trained it on and win percentage was 68.9% with the Pure AI. So there is some loss, but it's not that bad. I expect generalization to get better when I start training it on more maps.

At the moment, the limitation is more around factions than maps. My expectation is that it will do fine when both sides are a subset of one of the standard six factions. When you introduce a new unit, that's when you need to retrain. My guess is that a single new enemy unit won't be that big a deal, but it definitely needs to be trained on all of its recruitable units so it can learn how well they do in different situations.

it is not possible to set it as default recruitment; yet, it is possible to add somethings like "RCA AI with Machine Learning Recruiter" to the AI selection list;

This label seems fine and the general approach seems fine, although couldn't we use it as the default on two-player multiplayer games on which we had tested it?

also, it is possible to enable it on some of the mainline campaigns by default.

Sounds good.

A question - have you modified the source of Waffles or you're using unmodified source?

All of the changes I've made to Waffles have been folded back into the Waffles code base. In addition, the Waffles code maintainer (Mike Gashler) and I added #ifndef MIN_PREDICT statements to Waffles to comment out sections of Waffles we didn't need for executing a model in Wesnoth. I wrote a small Python script to strip out this code from what's included in Wesnoth, on the assumption that we want the patches to be as short as possible.