Machine Learning Recruiter
Moderator: Forum Moderators
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
Agreed. I haven't played against it myself very much. We'll have to see when I post it (hopefully with a few days). Hopefully some people can playtest it. I will say that in looking at it, it's doing things that look a lot smarter than, for instance, always recruiting Thunderers.lipk wrote: I don't say that your recruiter isn't better than the default, just that its performance versus an AI doesn't necessarily indicate its overall quality.
I think one simulation run for an entire campaign should suffice. You don't need to build a model for each map. Also, as I mentioned earlier, this only *might* be necessary when the campaign designer introduces units that that the AI hasn't see yet. Also note that 1) I'm not advocating replacing the existing recruiter any time soon, if ever and 2) You can run 600 --nogui runs of a multiplayer game in two hours on one thread on a Mac Mini.lipk wrote: This is pretty much what I'm concerned about; you need to build new recruiters for virtually every scenario.
Thanks!lipk wrote: Don't misunderstand me, this whole thing is really awesome stuff
I guess what I'm aiming for in the short term is for the recruiter to be accepted into Wesnoth where it would appear as an optional recruiter when playing multiplayer. I think it will be found to be a much more challenging opponent. Campaign designers would be free to choose this recruiter or the RCA AI recruiter as the recruiter for their scenarios.lipk wrote: I just have doubts that it'd fit well into the main code. Wesnoth is a rather big project, and they have to take many aspects into consideration when adding new features. Performance might not be the most important one of them.
This all should be clearer when I submit the patch.
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
A couple questions that I've encountered recently as I'm trying to push this patch out:
1. I see ai/lua/extCAexample.lua was recently checked into trunk as an example of how a Lua candidate action should be defined in Wesnoth 1.11. The bulk of my code is in Lua, so it seems like I should use this. Could someone point me to documentation of how I can call extCAexample as a candidate action?
2. My recruiter needs to write out a fair number of logging messages when in "training mode" (as opposed to when used in game play, when I want it to be silent). Am I right that wesnoth.message is the best way to do this? It seems like that is the intent from looking at this: http://wiki.wesnoth.org/LuaWML:Display#wesnoth.message
1. I see ai/lua/extCAexample.lua was recently checked into trunk as an example of how a Lua candidate action should be defined in Wesnoth 1.11. The bulk of my code is in Lua, so it seems like I should use this. Could someone point me to documentation of how I can call extCAexample as a candidate action?
2. My recruiter needs to write out a fair number of logging messages when in "training mode" (as opposed to when used in game play, when I want it to be silent). Am I right that wesnoth.message is the best way to do this? It seems like that is the intent from looking at this: http://wiki.wesnoth.org/LuaWML:Display#wesnoth.message
Re: Machine Learning Recruiter
Luaai_new_syntax. This is a new way of defining a CA for 1.11 that Nephro implemented recently. I've tested it and for all I can tell, it works without problems. The old method still will work though (no plans to deprecate it anytime soon), so whichever way you choose is fine.SeattleDad wrote:1. I see ai/lua/extCAexample.lua was recently checked into trunk as an example of how a Lua candidate action should be defined in Wesnoth 1.11. The bulk of my code is in Lua, so it seems like I should use this. Could someone point me to documentation of how I can call extCAexample as a candidate action?
I guess it depends on where you want the output. wesnoth.message() will get you a message in the chat area of the window, and in the log (apparently, I haven't checked that). Personally, I launch Wesnoth from a terminal window for debugging and simply use print() - which will show up in the terminal and is easily redirected to a file.SeattleDad wrote:2. My recruiter needs to write out a fair number of logging messages when in "training mode" (as opposed to when used in game play, when I want it to be silent). Am I right that wesnoth.message is the best way to do this? It seems like that is the intent from looking at this: http://wiki.wesnoth.org/LuaWML:Display#wesnoth.message
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
This is working in the gui for me, but is not working in --nogui mode. Has anyone gotten this working with --nogui?mattsc wrote: Luaai_new_syntax. This is a new way of defining a CA for 1.11 that Nephro implemented recently. I've tested it and for all I can tell, it works without problems. The old method still will work though (no plans to deprecate it anytime soon), so whichever way you choose is fine.
For instance, the following does not work for me where I set up new_lua_demo.cfg to include extCAexample:
/Applications/Wesnoth1.11.app/Contents/MacOS/Wesnoth --nogui --multiplayer --controller 1:ai --controller 2:ai --ai-config 1:ai/ais/new_lua_demo.cfg --scenario multiplayer_Sablestone_Delta
Re: Machine Learning Recruiter
I'd give it a very good chance that you and I are the only people who have used external CAs so far (well, except for Nephro). So I gave it a shot with a very simple toy AI and it seems to work for me. Here's what I did:SeattleDad wrote:This is working in the gui for me, but is not working in --nogui mode. Has anyone gotten this working with --nogui?
File: ai_extCA-test.cfg
File: extCA-test.lua
Launched from the command-line with:
Gives this output in the terminal window:
SP campaigns: Galuldur's First Journey (1.12 & 1.14) & Grnk the Mighty (1.10 & 1.12)
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
AI experiments: Micro AIs (wiki, forum thread, known/fixed bugs), Fred, AI-demos add-on
Re: Lua recruiting example
I was afraid that might happen. Is the attribute preserved under the [ai] subtag, though? Still kind of hacky I guess.SeattleDad wrote:Thanks for the tip. I've given this a try and unfortunately adding:
--parm 1:seattle:hello
gives me the error message
20120709 22:44:31 warning unit: Unknown attribute 'seattle' discarded.
Maybe a subtag of [side] that only exists if the game was started from the command line? A subtag that would simply store all the --parm settings, preserving them from being stripped? I'm thinking a really basic implementation that would not even be preserved over saved games, so it would be up to the WML author (you) to process this data early, probably in prestart.SeattleDad wrote:Still, it would be nice if I could pass in arbitrary attributes or there was even one free parameter called, for instance, "dev_parm", that could be used for this kind of thing.
I think that would be fairly simple to do. Not sure if it is appropriate for the official game, though.
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
So I just uploaded the patch, which you can see at https://gna.org/patch/index.php.
Some still evolving documentation for the patch can be found at http://wiki.wesnoth.org/Machine_Learning_Recruiter. Key take-away from that page is that this patch beats the default RCA AI 70% of the time, a major win considering that only the recruiting is affected. Also, I haven't really focused on pushing accuracy as high as possible, delaying this until I had it running end to end. I expect the win rate to go up over time.
I'd be very interested in getting feedback on this patch. Can you build it? Are you seeing that it wins with roughly the same frequency that I do? How could it be improved? How can this patch be included in Wesnoth?
Some still evolving documentation for the patch can be found at http://wiki.wesnoth.org/Machine_Learning_Recruiter. Key take-away from that page is that this patch beats the default RCA AI 70% of the time, a major win considering that only the recruiting is affected. Also, I haven't really focused on pushing accuracy as high as possible, delaying this until I had it running end to end. I expect the win rate to go up over time.
I'd be very interested in getting feedback on this patch. Can you build it? Are you seeing that it wins with roughly the same frequency that I do? How could it be improved? How can this patch be included in Wesnoth?
Re: Machine Learning Recruiter
Well, I tried to apply the patch but it didn't exactly go seamlessly:
Of course, after that it was no suprise that it didn't work at all.
Code: Select all
Skipped missing target: 'b/RELEASE_NOTES'
Skipped missing target: 'data/ai/ais/ml_ai_faction_specific.cfg'
A b
A b/data
A b/data/ai
A b/data/ai/ais
A b/data/ai/ais/ml_ai_faction_specific.cfg
Skipped missing target: 'data/ai/ais/ml_ai_faction_specific_weighted_random.cfg'
A b/data/ai/ais/ml_ai_faction_specific_weighted_random.cfg
Skipped missing target: 'b/data/ai/ais/testing_ai_default.cfg'
A data/ai/dev/class.lua
A b/data/ai/dev
A b/data/ai/dev/class.lua
A data/ai/dev/default_ai_with_recruit_log.cfg
A b/data/ai/dev/default_ai_with_recruit_log.cfg
A data/ai/dev/ml_ai_events.lua
A b/data/ai/dev/ml_ai_events.lua
A data/ai/dev/ml_ai_faction_specific_hybrid.cfg
A b/data/ai/dev/ml_ai_faction_specific_hybrid.cfg
A data/ai/dev/ml_ai_features.lua
A b/data/ai/dev/ml_ai_features.lua
A data/ai/dev/ml_ai_general.lua
A b/data/ai/dev/ml_ai_general.lua
A data/ai/dev/ml_ai_random.cfg
A b/data/ai/dev/ml_ai_random.cfg
Skipped missing target: 'data/ai/models/Drakes/.DS_Store'
Skipped missing target: 'data/ai/models/Drakes/default_model.json'
A b/data/ai/models
A b/data/ai/models/Drakes
A b/data/ai/models/Drakes/default_model.json
Skipped missing target: 'data/ai/models/Knalgan/.DS_Store'
Skipped missing target: 'data/ai/models/Knalgan/default_model.json'
A b/data/ai/models/Knalgan
A b/data/ai/models/Knalgan/default_model.json
Skipped missing target: 'data/ai/models/Loyalists/default_model.json'
A b/data/ai/models/Loyalists
A b/data/ai/models/Loyalists/default_model.json
Skipped missing target: 'data/ai/models/Northerners/default_model.json'
A b/data/ai/models/Northerners
A b/data/ai/models/Northerners/default_model.json
Skipped missing target: 'data/ai/models/Rebels/default_model.json'
A b/data/ai/models/Rebels
A b/data/ai/models/Rebels/default_model.json
Skipped missing target: 'data/ai/models/Undead/default_model.json'
A b/data/ai/models/Undead
A b/data/ai/models/Undead/default_model.json
Skipped missing target: 'b/projectfiles/Xcode/Wesnoth.xcodeproj/project.pbxproj'
Skipped missing target: 'b/src/CMakeLists.txt'
Skipped missing target: 'b/src/SConscript'
Skipped missing target: 'b/src/ai/lua/core.cpp'
A src/ai/waffles
A src/ai/waffles/GActivation.cpp
A b/src
A b/src/ai
A b/src/ai/waffles
A b/src/ai/waffles/GActivation.cpp
A src/ai/waffles/GActivation.h
A b/src/ai/waffles/GActivation.h
A src/ai/waffles/GBitTable.cpp
A b/src/ai/waffles/GBitTable.cpp
A src/ai/waffles/GBitTable.h
A b/src/ai/waffles/GBitTable.h
A src/ai/waffles/GDom.cpp
A b/src/ai/waffles/GDom.cpp
A src/ai/waffles/GDom.h
A b/src/ai/waffles/GDom.h
A src/ai/waffles/GError.cpp
A b/src/ai/waffles/GError.cpp
A src/ai/waffles/GError.h
A b/src/ai/waffles/GError.h
A src/ai/waffles/GHeap.cpp
A b/src/ai/waffles/GHeap.cpp
A src/ai/waffles/GHeap.h
A b/src/ai/waffles/GHeap.h
A src/ai/waffles/GHolders.cpp
A b/src/ai/waffles/GHolders.cpp
A src/ai/waffles/GHolders.h
A b/src/ai/waffles/GHolders.h
A src/ai/waffles/GLearner.cpp
A b/src/ai/waffles/GLearner.cpp
A src/ai/waffles/GLearner.h
A b/src/ai/waffles/GLearner.h
A src/ai/waffles/GMatrix.cpp
A b/src/ai/waffles/GMatrix.cpp
A src/ai/waffles/GMatrix.h
A b/src/ai/waffles/GMatrix.h
A src/ai/waffles/GNeuralNet.cpp
A b/src/ai/waffles/GNeuralNet.cpp
A src/ai/waffles/GNeuralNet.h
A b/src/ai/waffles/GNeuralNet.h
A src/ai/waffles/GRand.cpp
A b/src/ai/waffles/GRand.cpp
A src/ai/waffles/GRand.h
A b/src/ai/waffles/GRand.h
A src/ai/waffles/GTokenizer.cpp
A b/src/ai/waffles/GTokenizer.cpp
A src/ai/waffles/GTokenizer.h
A b/src/ai/waffles/GTokenizer.h
A src/ai/waffles/GTransform.cpp
A b/src/ai/waffles/GTransform.cpp
A src/ai/waffles/GTransform.h
A b/src/ai/waffles/GTransform.h
A src/ai/waffles/GVec.cpp
A b/src/ai/waffles/GVec.cpp
A src/ai/waffles/GVec.h
A b/src/ai/waffles/GVec.h
Skipped missing target: 'b/utils/ai_test/ai_test2.cfg'
Skipped missing target: 'b/utils/ai_test/ai_test2.py'
A utils/ai_test/analyze_log.py
A b/utils
A b/utils/ai_test
A b/utils/ai_test/analyze_log.py
A utils/ai_test/analyze_recruitment.py
A b/utils/ai_test/analyze_recruitment.py
A utils/ai_test/copy_predict_only_waffles.py
A b/utils/ai_test/copy_predict_only_waffles.py
A utils/ai_test/csv_normalizer.py
A b/utils/ai_test/csv_normalizer.py
A utils/ai_test/csv_to_arff.py
A b/utils/ai_test/csv_to_arff.py
A utils/ai_test/ml_utilities.py
A b/utils/ai_test/ml_utilities.py
A utils/ai_test/process_output_for_ml.py
A b/utils/ai_test/process_output_for_ml.py
Skipped missing target: 'b/utils/ai_test/readme.txt'
Summary of conflicts:
Skipped paths: 19
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
Sorry about that.
Try it with patch -p1 to strip away the a/ and b/ prefixes.
Try it with patch -p1 to strip away the a/ and b/ prefixes.
Re: Machine Learning Recruiter
It's good to see some working code making the AI better.
A question - have you modified the source of Waffles or you're using unmodified source?
Also, if there are any bugs with --nogui, then you're welcome report them via bugs.wesnoth.org
If it is consistenty better than the old recruitment (when trained for a particular map), I would include the patch in wesnoth when all the issues would be sorted out. Since the algorithm requires per-map training, it is not possible to set it as default recruitment; yet, it is possible to add somethings like "RCA AI with Machine Learning Recruiter" to the AI selection list; also, it is possible to enable it on some of the mainline campaigns by default.SeattleDad wrote:How can this patch be included in Wesnoth?
A question - have you modified the source of Waffles or you're using unmodified source?
Also, if there are any bugs with --nogui, then you're welcome report them via bugs.wesnoth.org
Re: Machine Learning Recruiter
Now I'm getting a compiler error:
Additionally, note that patches must not trigger any warnings either.
Code: Select all
/home/lipk/projects/Wesnoth/wesnoth/src/ai/waffles/GTokenizer.cpp: In constructor ‘GClasses::GTokenizer::GTokenizer(const char*)’:
/home/lipk/projects/Wesnoth/wesnoth/src/ai/waffles/GTokenizer.cpp:75:27: error: ‘access’ was not declared in this scope
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
What platform are you building on? Where are you seeing warnings? I'm not getting warnings in Xcode on OS X 10.7.Now I'm getting a compiler error:
Additionally, note that patches must not trigger any warnings either.
Re: Machine Learning Recruiter
I'm on Linux, building with gcc. I get a lot of complaints about unused variables and parameters in src/ai/waffles. However, that isn't really important right now, those could be easily fixed. The problem at hand is the compile error. It fails to resolve an invoke to a function (apparently affiliated with file I/O). Probably other packages are needed to build the code as well (although I believe that #include-s wouldn't be resolvable in that case), or even worse, access() is an OSX specific function.
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
Okay, thanks for the info. I'm checking with Mike Gashler, the maintainer of Waffles on this one. Will get back to you when I hear from him.lipk wrote:I'm on Linux, building with gcc. ... The problem at hand is the compile error. It fails to resolve an invoke to a function (apparently affiliated with file I/O).
-
- Posts: 74
- Joined: March 4th, 2012, 6:09 pm
Re: Machine Learning Recruiter
Thanks!Crab wrote:It's good to see some working code making the AI better.
SeattleDad wrote:How can this patch be included in Wesnoth?
This is not exactly correct. The current status is that I explicitly trained it on four maps. When tested on those four maps with the "pure" AI, win percentage is 71.5%. I then tested it on those four maps plus eight other maps I hadn't trained it on and win percentage was 68.9% with the Pure AI. So there is some loss, but it's not that bad. I expect generalization to get better when I start training it on more maps.Crab wrote: Since the algorithm requires per-map training,
At the moment, the limitation is more around factions than maps. My expectation is that it will do fine when both sides are a subset of one of the standard six factions. When you introduce a new unit, that's when you need to retrain. My guess is that a single new enemy unit won't be that big a deal, but it definitely needs to be trained on all of its recruitable units so it can learn how well they do in different situations.
This label seems fine and the general approach seems fine, although couldn't we use it as the default on two-player multiplayer games on which we had tested it?it is not possible to set it as default recruitment; yet, it is possible to add somethings like "RCA AI with Machine Learning Recruiter" to the AI selection list;
Sounds good.also, it is possible to enable it on some of the mainline campaigns by default.
All of the changes I've made to Waffles have been folded back into the Waffles code base. In addition, the Waffles code maintainer (Mike Gashler) and I added #ifndef MIN_PREDICT statements to Waffles to comment out sections of Waffles we didn't need for executing a model in Wesnoth. I wrote a small Python script to strip out this code from what's included in Wesnoth, on the assumption that we want the patches to be as short as possible.A question - have you modified the source of Waffles or you're using unmodified source?