Option limit?

The place to post your WML questions and answers.

Moderator: Forum Moderators

Forum rules
  • Please use [code] BBCode tags in your posts for embedding WML snippets.
  • To keep your code readable so that others can easily help you, make sure to indent it following our conventions.
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Ravana wrote: March 1st, 2020, 7:11 pm Too many too large too nested macros.

When you have the 68MB file you can check what parts take most space there, and then try to optimize/rewrite that part.
nested macros are macros inside of other macros, yes?
how can I rewrite those though? it would be an obscene amount of work to have to re-write those... There are just so many...
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
Ravana
Forum Moderator
Posts: 2452
Joined: January 29th, 2012, 12:49 am
Location: Estonia
Contact:

Re: Option limit?

Post by Ravana »

Thats why find one part to rewrite, not all of it.
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Ravana wrote: March 1st, 2020, 8:07 pm Thats why find one part to rewrite, not all of it.
okay so do I need to rewrite the macro or reduce the amount of macros I have and write them the standard way?
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
Pentarctagon
Project Manager
Posts: 4507
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Option limit?

Post by Pentarctagon »

So, just to establish a starting point, do you have an understanding of how the preprocessor handles macros and how that ends up creating a cache entry that's 68 MBs?
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Pentarctagon wrote: March 1st, 2020, 9:34 pm So, just to establish a starting point, do you have an understanding of how the preprocessor handles macros and how that ends up creating a cache entry that's 68 MBs?
unfortunately I actually dont have any understanding of it. I just guess by the name preprocessing it does things before the game begins such as during the prestart.
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
Pentarctagon
Project Manager
Posts: 4507
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Option limit?

Post by Pentarctagon »

The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

Code: Select all

#define EXAMPLE
1
2
3
4
5
#enddef
#define A
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
#enddef
#define B
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
#enddef
#define C
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
#enddef
{C}
That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
Ravana
Forum Moderator
Posts: 2452
Joined: January 29th, 2012, 12:49 am
Location: Estonia
Contact:

Re: Option limit?

Post by Ravana »

The example is not WML, but if changed to WML syntax, for example

Code: Select all

#define EXAMPLE
[a]
a=1
b=2
c=3
d=4
e=5
[/a]
#enddef
then you can run wesnoth -p _main.cfg out to get the preprocessed content. That is what I tried on your addon when I got out of memory.
User avatar
Pentarctagon
Project Manager
Posts: 4507
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Option limit?

Post by Pentarctagon »

Yeah, that wasn't meant to be something that works, just something simple to use as an example.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Pentarctagon wrote: March 1st, 2020, 11:02 pm The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.
Okay thank you for explaining that! It makes a lot of sense now! Would using [load_resource] work better for separate add-ons that make use of macros and files from my original add-on? I was reading in wml optimization and was wondering if it would prevent my second add-on from forcing the engine to create a cache and load the same thing twice and potentially speed things up?
Last edited by LordAwsomeness on March 3rd, 2020, 6:43 pm, edited 2 times in total.
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Ravana wrote: March 1st, 2020, 11:22 pm The example is not WML, but if changed to WML syntax, for example

Code: Select all

#define EXAMPLE
[a]
a=1
b=2
c=3
d=4
e=5
[/a]
#enddef
then you can run wesnoth -p _main.cfg out to get the preprocessed content. That is what I tried on your addon when I got out of memory.
I think I've never gotten an issue with running out of memory because I have a ridiculously powerful PC so that I can run games like FIFA and pubg at ultra settings without any frame rate drops. As well as I have an extremely powerful processor so that I can stream while running these games.
But I didn't realize that I can use wesnoth -p _main.cfg out to get the preprocessed data
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
Ravana
Forum Moderator
Posts: 2452
Joined: January 29th, 2012, 12:49 am
Location: Estonia
Contact:

Re: Option limit?

Post by Ravana »

You might need to replace all 3 paths with absolute paths depending on your configurations - full path to wesnoth, full path to addon _main.cfg, and full path to folder where output is created. For multiplayer addon also --preprocess-defines=MULTIPLAYER is needed. (wesnoth --help if interested in more of the preprocessing options)

Not sure if load_resource would help, but it would split some of the logic away. Which provides way to disable that part to see if there are improvements.
User avatar
Pentarctagon
Project Manager
Posts: 4507
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Option limit?

Post by Pentarctagon »

LordAwsomeness wrote: March 3rd, 2020, 6:26 pm
Pentarctagon wrote: March 1st, 2020, 11:02 pm The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.
Okay thank you for explaining that! It makes a lot of sense now! Would using [load_resource] work better for separate add-ons that make use of macros and files from my original add-on? I was reading in wml optimization and was wondering if it would prevent my second add-on from forcing the engine to create a cache and load the same thing twice and potentially speed things up?
I'm not familiar with [load_resource]. The reason I mentioned lua originally though is that lua functions don't suffer from the problem WML macros do, that being, lua functions don't get touched by the preprocessor - that example I gave would stay 44 lines if it were in lua instead of getting expanded out to multiple thousands of lines.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
lhybrideur
Posts: 145
Joined: July 9th, 2019, 1:46 pm

Re: Option limit?

Post by lhybrideur »

BTW, you should take a look at LotI item management system.
I modified it for my add-on (Legens of Idaamub), so maybe you can modify it too for your purpose.
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

lhybrideur wrote: March 4th, 2020, 12:30 pm BTW, you should take a look at LotI item management system.
I modified it for my add-on (Legens of Idaamub), so maybe you can modify it too for your purpose.
Thank you for the suggestion. I'll definitely download it and see what I think after I get off of work today. Are you suggesting it because you have seen my current inventory system or just because you see that I have an inventory and weapon swapping system?
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
User avatar
LordAwsomeness
Posts: 198
Joined: August 12th, 2013, 2:20 pm
Location: U.S.A.

Re: Option limit?

Post by LordAwsomeness »

Pentarctagon wrote: March 4th, 2020, 2:03 am
I'm not familiar with [load_resource]. The reason I mentioned lua originally though is that lua functions don't suffer from the problem WML macros do, that being, lua functions don't get touched by the preprocessor - that example I gave would stay 44 lines if it were in lua instead of getting expanded out to multiple thousands of lines.
How would I use lua? I'm honestly not familiar enough with it to make options and message tags with it
- Been playing Wesnoth since 2004 and the 1.0.x versions.
- Creator of Undead Invasion MP Scenario Pack.
- Creator of Valeria MP Adventure
- Creator of LA_RPG ERA
Post Reply