WML syntax tree analysis

Discussion of all aspects of the game engine, including development of new and existing features.

Moderators: Forum Moderators, Developers

Post Reply
User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

WML syntax tree analysis

Post by Ravana » June 1st, 2017, 9:24 pm

I am creating a tool, that can read addon folder, and return information on where certain defined WML patterns exist. My first use case is finding events without id.

I use wesnoth -p and wmlparser3.py.

Idea for this is from

Code: Select all

20170528 20:08:35< zookeeper> well, it seems really hard to get a sense of how often death animations have weapon filtering in UMC
..
20170528 21:28:11< DeFender1031> zookeeper, I'm wondering if there's a simple way to grep for that... (Probably not.)
..
20170528 21:29:43< celticminstrel> DeFender1031: It's impossible to grep for.
..
20170528 21:47:56< DeFender1031> what we really need is some kind of tool that is capable of parsing WML that you can basically give parameters to saying "search for anywhere where a WML tag that matches these criteria exists"
20170528 21:48:46< celticminstrel> That sounds doable with the C++ WML API.
20170528 21:48:51< DeFender1031> (That actually shouldn't be that hard when there's already a WML parser... The biggest issue is having some clean way to define the criteria")
20170528 21:48:55< celticminstrel> Well probably even with Lua.
20170528 21:49:25< celticminstrel> Except that Lua can't access the unfiltered game config.
20170528 21:49:39< DeFender1031> I'd suggest a separate tool.
20170528 21:49:46< celticminstrel> (By which I mean the root config loaded from data/_main.cfg)
20170528 21:49:55< DeFender1031> right
20170528 21:50:03< celticminstrel> (I guess the addon configs are also merged into it.)
Nothing works correctly yet, but it has already been useful in finding some real errors in ageless.

gfgtdf
Developer
Posts: 992
Joined: February 10th, 2013, 2:25 pm

Re: WML syntax tree analysis

Post by gfgtdf » June 1st, 2017, 9:35 pm

didn't iceiceice at some point wrote a wml parser that was somhow better then wesnoths internal wml parser? Maybe you coudl use that?

EDIT: this is the thread i think viewtopic.php?f=10&t=44584
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.

User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

Re: WML syntax tree analysis

Post by Ravana » June 2nd, 2017, 1:46 pm

I dont know how to compile wesnoth, so that does not help me much.

I found that normal core preprocessing skips some macros, therefore I need to use SKIP_CORE, and first collect all core macros.

Code: Select all

call([wesnoth_path, "--preprocess-defines", "MULTIPLAYER,SKIP_CORE", "-p", core_path, core_out_path, "--preprocess-output-macros"])

call([wesnoth_path, "--userdata-dir", userdata_path, "--preprocess-defines", "MULTIPLAYER,SKIP_CORE", "--preprocess-input-macros", join(core_out_path, "_MACROS_.cfg"), "-p", input_path, addon_out_path])
Current progress is support for most basic querys, like finding all unit types with attack of 25 damage.

User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

Re: WML syntax tree analysis

Post by Ravana » June 5th, 2017, 8:24 pm

Now supports:
query without full path;
attribute check with comparison operators, implemented as lambda, so could be used with any function;
way to select which attributes to include in output

Some selection examples:

Select id, experience, level from [unit_type]s that are under toplevel [units] and have attribute experience=100

Code: Select all

[units]/[unit_type]/experience==100~id,experience,level
Select all [movetype] names

Code: Select all

//[movetype]/name~name
Select used movetypes

Code: Select all

[units]/[unit_type]/movement_type~movement_type

User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

Re: WML syntax tree analysis

Post by Ravana » June 18th, 2017, 6:59 pm

Now supports query with attributes from different tag levels

Code: Select all

parsed_query = [parse_wml_query("[units]/[unit_type]/[attack]/damage>15"),
                parse_wml_query("[units]/[unit_type]/hitpoints==70")]
output_keys = ["id", "cost", "hitpoints"]
Supports some negation. Currently trying to say with multiple conditions that attribute should not exist in non-terminal(from the first query point of view) node passes when there is any node in path where such attribute does not exist.

Code: Select all

parsed_query = [parse_wml_query("[units]/[unit_type]/[event]/id!")]
output_keys = ["id", "name", "first_time_only"]
https://github.com/ProditorMagnus/WML_t ... out_id.cfg

User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

Re: WML syntax tree analysis

Post by Ravana » March 21st, 2018, 12:16 am

Continuing with this to find possible typos. https://github.com/ProditorMagnus/WML_t ... 53c03701R8 reports uncommon damage types.

User avatar
Celtic_Minstrel
Developer
Posts: 1137
Joined: August 3rd, 2012, 11:26 pm
Contact:

Re: WML syntax tree analysis

Post by Celtic_Minstrel » March 21st, 2018, 5:17 am

I notice you've hardcoded your personal Wesnoth path into those files... and what the heck is "ghype"?

That aside, this definitely seems useful.
Author of The Black Cross of Aleron campaign and Default++ era.
Maintainer of Steelhive.

User avatar
Ravana
Moderator
Posts: 1745
Joined: January 29th, 2012, 12:49 am
Location: Estonia

Re: WML syntax tree analysis

Post by Ravana » March 21st, 2018, 1:52 pm

ghype is developing new era for future inclusion for ageless, using that to refer to his version.

compile_ageless.py is supposed to hardcode paths, it is used to build two command line calls that would be too long normally. find_patterns.py can be changed though.

Post Reply