Option limit?

[LordAwsomeness]

Too many too large too nested macros.

When you have the 68MB file you can check what parts take most space there, and then try to optimize/rewrite that part.

nested macros are macros inside of other macros, yes?
how can I rewrite those though? it would be an obscene amount of work to have to re-write those... There are just so many...

[Ravana]

Thats why find one part to rewrite, not all of it.

[LordAwsomeness]

Thats why find one part to rewrite, not all of it.

okay so do I need to rewrite the macro or reduce the amount of macros I have and write them the standard way?

[Pentarctagon]

So, just to establish a starting point, do you have an understanding of how the preprocessor handles macros and how that ends up creating a cache entry that's 68 MBs?

[LordAwsomeness]

So, just to establish a starting point, do you have an understanding of how the preprocessor handles macros and how that ends up creating a cache entry that's 68 MBs?

unfortunately I actually dont have any understanding of it. I just guess by the name preprocessing it does things before the game begins such as during the prestart.

[Pentarctagon]

The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

Code: Select all

#define EXAMPLE
1
2
3
4
5
#enddef
#define A
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
{EXAMPLE}
#enddef
#define B
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
{A}
#enddef
#define C
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
{B}
#enddef
{C}

That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.

[Ravana]

The example is not WML, but if changed to WML syntax, for example

Code: Select all

#define EXAMPLE
[a]
a=1
b=2
c=3
d=4
e=5
[/a]
#enddef

then you can run wesnoth -p _main.cfg out to get the preprocessed content. That is what I tried on your addon when I got out of memory.

[Pentarctagon]

Yeah, that wasn't meant to be something that works, just something simple to use as an example.

[LordAwsomeness]

The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.

Okay thank you for explaining that! It makes a lot of sense now! Would using [load_resource] work better for separate add-ons that make use of macros and files from my original add-on? I was reading in wml optimization and was wondering if it would prevent my second add-on from forcing the engine to create a cache and load the same thing twice and potentially speed things up?

[LordAwsomeness]

The example is not WML, but if changed to WML syntax, for example

Code: Select all

#define EXAMPLE
[a]
a=1
b=2
c=3
d=4
e=5
[/a]
#enddef

then you can run wesnoth -p _main.cfg out to get the preprocessed content. That is what I tried on your addon when I got out of memory.

I think I've never gotten an issue with running out of memory because I have a ridiculously powerful PC so that I can run games like FIFA and pubg at ultra settings without any frame rate drops. As well as I have an extremely powerful processor so that I can stream while running these games.
But I didn't realize that I can use wesnoth -p _main.cfg out to get the preprocessed data

[Ravana]

You might need to replace all 3 paths with absolute paths depending on your configurations - full path to wesnoth, full path to addon _main.cfg, and full path to folder where output is created. For multiplayer addon also --preprocess-defines=MULTIPLAYER is needed. (wesnoth --help if interested in more of the preprocessing options)

Not sure if load_resource would help, but it would split some of the logic away. Which provides way to disable that part to see if there are improvements.

[Pentarctagon]

The preprocessor takes all your macros, and substitutes in the actual text of the macros. So for example:

That's just 44 lines of text there, but the preprocessor expands that out so that there are no macros present. The {C} is instead replaced by 1000 copies of the contents of EXAMPLE, which means the 44 lines you coded becomes 5000 lines in the cache. The large amount of load time you get after changing something is the preprocessor redoing all of that substitution.

Okay thank you for explaining that! It makes a lot of sense now! Would using [load_resource] work better for separate add-ons that make use of macros and files from my original add-on? I was reading in wml optimization and was wondering if it would prevent my second add-on from forcing the engine to create a cache and load the same thing twice and potentially speed things up?

I'm not familiar with [load_resource]. The reason I mentioned lua originally though is that lua functions don't suffer from the problem WML macros do, that being, lua functions don't get touched by the preprocessor - that example I gave would stay 44 lines if it were in lua instead of getting expanded out to multiple thousands of lines.

[lhybrideur]

BTW, you should take a look at LotI item management system.
I modified it for my add-on (Legens of Idaamub), so maybe you can modify it too for your purpose.

[LordAwsomeness]

BTW, you should take a look at LotI item management system.
I modified it for my add-on (Legens of Idaamub), so maybe you can modify it too for your purpose.

Thank you for the suggestion. I'll definitely download it and see what I think after I get off of work today. Are you suggesting it because you have seen my current inventory system or just because you see that I have an inventory and weapon swapping system?

[LordAwsomeness]

I'm not familiar with [load_resource]. The reason I mentioned lua originally though is that lua functions don't suffer from the problem WML macros do, that being, lua functions don't get touched by the preprocessor - that example I gave would stay 44 lines if it were in lua instead of getting expanded out to multiple thousands of lines.

How would I use lua? I'm honestly not familiar enough with it to make options and message tags with it