WML syntax tree analysis
Moderator: Forum Moderators
WML syntax tree analysis
Repository: https://github.com/ProditorMagnus/WML_tree_tools
I am creating a tool, that can read addon folder, and return information on where certain defined WML patterns exist. My first use case is finding events without id.
I use wesnoth -p and wmlparser3.py.
Idea for this is from
Nothing works correctly yet, but it has already been useful in finding some real errors in ageless.
I am creating a tool, that can read addon folder, and return information on where certain defined WML patterns exist. My first use case is finding events without id.
I use wesnoth -p and wmlparser3.py.
Idea for this is from
Code: Select all
20170528 20:08:35< zookeeper> well, it seems really hard to get a sense of how often death animations have weapon filtering in UMC
..
20170528 21:28:11< DeFender1031> zookeeper, I'm wondering if there's a simple way to grep for that... (Probably not.)
..
20170528 21:29:43< celticminstrel> DeFender1031: It's impossible to grep for.
..
20170528 21:47:56< DeFender1031> what we really need is some kind of tool that is capable of parsing WML that you can basically give parameters to saying "search for anywhere where a WML tag that matches these criteria exists"
20170528 21:48:46< celticminstrel> That sounds doable with the C++ WML API.
20170528 21:48:51< DeFender1031> (That actually shouldn't be that hard when there's already a WML parser... The biggest issue is having some clean way to define the criteria")
20170528 21:48:55< celticminstrel> Well probably even with Lua.
20170528 21:49:25< celticminstrel> Except that Lua can't access the unfiltered game config.
20170528 21:49:39< DeFender1031> I'd suggest a separate tool.
20170528 21:49:46< celticminstrel> (By which I mean the root config loaded from data/_main.cfg)
20170528 21:49:55< DeFender1031> right
20170528 21:50:03< celticminstrel> (I guess the addon configs are also merged into it.)
Re: WML syntax tree analysis
didn't iceiceice at some point wrote a wml parser that was somhow better then wesnoths internal wml parser? Maybe you coudl use that?
EDIT: this is the thread i think viewtopic.php?f=10&t=44584
EDIT: this is the thread i think viewtopic.php?f=10&t=44584
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
Re: WML syntax tree analysis
I dont know how to compile wesnoth, so that does not help me much.
I found that normal core preprocessing skips some macros, therefore I need to use SKIP_CORE, and first collect all core macros.
Current progress is support for most basic querys, like finding all unit types with attack of 25 damage.
I found that normal core preprocessing skips some macros, therefore I need to use SKIP_CORE, and first collect all core macros.
Code: Select all
call([wesnoth_path, "--preprocess-defines", "MULTIPLAYER,SKIP_CORE", "-p", core_path, core_out_path, "--preprocess-output-macros"])
call([wesnoth_path, "--userdata-dir", userdata_path, "--preprocess-defines", "MULTIPLAYER,SKIP_CORE", "--preprocess-input-macros", join(core_out_path, "_MACROS_.cfg"), "-p", input_path, addon_out_path])
Re: WML syntax tree analysis
Now supports:
query without full path;
attribute check with comparison operators, implemented as lambda, so could be used with any function;
way to select which attributes to include in output
Some selection examples:
Select id, experience, level from [unit_type]s that are under toplevel [units] and have attribute experience=100
Select all [movetype] names
Select used movetypes
query without full path;
attribute check with comparison operators, implemented as lambda, so could be used with any function;
way to select which attributes to include in output
Some selection examples:
Select id, experience, level from [unit_type]s that are under toplevel [units] and have attribute experience=100
Code: Select all
[units]/[unit_type]/experience==100~id,experience,level
Code: Select all
//[movetype]/name~name
Code: Select all
[units]/[unit_type]/movement_type~movement_type
Re: WML syntax tree analysis
Now supports query with attributes from different tag levels
Supports some negation. Currently trying to say with multiple conditions that attribute should not exist in non-terminal(from the first query point of view) node passes when there is any node in path where such attribute does not exist.
https://github.com/ProditorMagnus/WML_t ... out_id.cfg
Code: Select all
parsed_query = [parse_wml_query("[units]/[unit_type]/[attack]/damage>15"),
parse_wml_query("[units]/[unit_type]/hitpoints==70")]
output_keys = ["id", "cost", "hitpoints"]
Code: Select all
parsed_query = [parse_wml_query("[units]/[unit_type]/[event]/id!")]
output_keys = ["id", "name", "first_time_only"]
Re: WML syntax tree analysis
Continuing with this to find possible typos. https://github.com/ProditorMagnus/WML_t ... 53c03701R8 reports uncommon damage types.
- Celtic_Minstrel
- Developer
- Posts: 2209
- Joined: August 3rd, 2012, 11:26 pm
- Location: Canada
- Contact:
Re: WML syntax tree analysis
I notice you've hardcoded your personal Wesnoth path into those files... and what the heck is "ghype"?
That aside, this definitely seems useful.
That aside, this definitely seems useful.
Re: WML syntax tree analysis
ghype is developing new era for future inclusion for ageless, using that to refer to his version.
compile_ageless.py is supposed to hardcode paths, it is used to build two command line calls that would be too long normally. find_patterns.py can be changed though.
compile_ageless.py is supposed to hardcode paths, it is used to build two command line calls that would be too long normally. find_patterns.py can be changed though.
Re: WML syntax tree analysis
Added some changes to make it easier to use, and with different addons as well.
Now only one file has absolute paths, others use relative paths.
Now only one file has absolute paths, others use relative paths.
Re: WML syntax tree analysis
Nice
I usually use wmlparser3 in json output mode and then pipe the output to jq.
I usually use wmlparser3 in json output mode and then pipe the output to jq.
Ravana wrote: ↑June 5th, 2017, 8:24 pm Select id, experience, level from [unit_type]s that are under toplevel [units] and have attribute experience=100Code: Select all
[units]/[unit_type]/experience==100~id,experience,level
Code: Select all
data/tools/wesnoth/wmlparser3.py -w ./wesnoth -i data/_main.cfg -j | jq '.units[0].unit_type | map(select(.experience = 100)) | map({id, experience, level})'
Ravana wrote: ↑June 5th, 2017, 8:24 pm Select all [movetype] namesCode: Select all
//[movetype]/name~name
Code: Select all
data/tools/wesnoth/wmlparser3.py -w ./wesnoth -i data/_main.cfg -j | jq '.units[0].movetype | map(.name)'
Ravana wrote: ↑June 5th, 2017, 8:24 pm Select used movetypesCode: Select all
[units]/[unit_type]/movement_type~movement_type
Code: Select all
data/tools/wesnoth/wmlparser3.py -w ./wesnoth -i data/_main.cfg -j | jq '.units[0].unit_type | map(.movement_type) | map(select(. != null)) | sort | unique'
Re: WML syntax tree analysis
Interesting. Looks like it would support all normal cases. Technically I support selecting with stateful function too, but it is not what I use in practice. Select all units that have more health than the last match - https://github.com/ProditorMagnus/WML_t ... 6b0c2ecfda gives
Code: Select all
[[['units', 'unit_type'], ['hitpoints', <function inc at 0x053A00C0>], ['id']]]
found match at ['units', 'unit_type'] [{}, {'id': 'AE_arc_ukians_Royal_Constable', 'hitpoints': '70'}]
found match at ['units', 'unit_type'] [{}, {'id': 'AE_arc_ukians_Royal_FireKnight', 'hitpoints': '88'}]
found match at ['units', 'unit_type'] [{}, {'id': 'AE_arc_menagerie_Green_Mage', 'hitpoints': '444'}]
found match at ['units', 'unit_type'] [{}, {'id': 'AE_feu_khaganate_Rigid', 'hitpoints': '555'}]
Re: WML syntax tree analysis
You could probably this with
with_entries
or reduce
, but if you don't want to learn a new DSL just for this, you could use Python for the stateful part:Code: Select all
jq '.units[0].unit_type | map({ id, hitpoints })' | jq -c '.[]' | perl -MJSON::PP -nE '$h = decode_json $_; say $h->{id} if $h->{hitpoints} > $prev; $prev = $h->{hitpoints}'
Re: WML syntax tree analysis
I will remain with python. Even just being able to add breakpoints makes it worth it.