Code optimization

[Dunno]

I couldn't find anything on this topic. My campaign is slowly becoming very sophisticated and this results in a long wait before turn start. I'm wondering if there are any general tips as to how I should write my wml to make it run faster? Is dividing into different files advised or the contrary? Should I make and use as many macros as I can? Which tags are known to cause the most trouble?
Thanks!

[Dugi]

If you are executing something at the start of the scenario, your problem is probably in ActionWML or InternalActionsWML. Here, the main time eaters are things with graphical representation, creating numerous units, filter_wml if not used to search for variables in [variables] or [status], and cascades of [while] cycles. Usually, just a while cycle inside another while cycle, where each cycle cycles through several dozens of variables, is enough to create visible delays. I think that creating numerous nested events can slow the game down too.

It can also easily be a broken [while] cycle, because if nothing stops it, it stops after 216 cycles (to identify it easier, you can change this in wml-tags.lua to another value to see if the time it takes changed).

It is a popular belief that using as many macros as possible, even for things that appear only once, makes the code faster, but it is completely wrong. The macros only make it easier to write and read for humans, nothing else. If fact, they motivate people to write codes ineffectively. Macros are replaced by their content before WML is parsed, so if you use a long macro a lot of times, all the long code will be loaded each time you use the macro. The size that the parsed WML takes in RAM is proportional to the time it needs to load. And it takes exactly the same time to execute, because it is the same when loaded.

One false positive I am experiencing is that if I load a scenario for the first time, it makes me wait for about 15 seconds till something happens, after executing all start and turn refresh events, but before showing the objectives screen. But it does not appear if I simply restart the scenario, and it happens also when I load a save file from the middle of a scenario. It seems to appear every time I launch the game after that it was closed for some time, and it is not related to the WML, because if I hit F5 and load it again, it does not make me wait. I don't know what is exactly the case of it, but I believe it is related to the quantity of save files.

Also, you might try to locate the place where the trouble happens, place some [chat] or [message] tags in various places and watch which parts took much time.

HTH

[Dunno]

In my case it's just the time before player's turn starts. I guess I'll have to reduce the amount of variables that do "cool" things but aren't really necessary. It's interesting what you've said about macros, though, I too believed they speed up the code a little
I'm still curious if dividing the code into a couple of files affects the speed at all.

[Anonymissimus]

It is a popular belief that using as many macros as possible, even for things that appear only once, makes the code faster, but it is completely wrong. The macros only make it easier to write and read for humans, nothing else. If fact, they motivate people to write codes ineffectively. Macros are replaced by their content before WML is parsed, so if you use a long macro a lot of times, all the long code will be loaded each time you use the macro. The size that the parsed WML takes in RAM is proportional to the time it needs to load. And it takes exactly the same time to execute, because it is the same when loaded.

Use custom events instead and macros only at such places where event wml doesn't work.

One false positive I am experiencing is that if I load a scenario for the first time, it makes me wait for about 15 seconds till something happens, after executing all start and turn refresh events, but before showing the objectives screen. But it does not appear if I simply restart the scenario, and it happens also when I load a save file from the middle of a scenario. It seems to appear every time I launch the game after that it was closed for some time, and it is not related to the WML, because if I hit F5 and load it again, it does not make me wait. I don't know what is exactly the case of it, but I believe it is related to the quantity of save files.

I suspect it's the operating system, keeping things a program needs in memory, unless that memory needs to be freed for some other process.
Or the wesnoth feature "wml cache"; but there was a bug making it unused until recently, and it also cannot be used in case the wml was changed.

Also, you might try to locate the place where the trouble happens, place some [chat] or [message] tags in various places and watch which parts took much time.

There's set_variable's time=stamp.

What hasn't been much mentioned yet is the most common problem; "too much wml" and thus slowdown while saving. Custom events and lua instead of wml help.

The one thing I noticed to cause a "processing time lag" is store_location's radius= (which makes sense, as it has an inefficient implementation). SoW is calling it multiple times when calculating the longest road, that functionality can last several seconds when the map is in an advanced stage. (It is in lua on my side and about as efficient as I could make it...)

[Dugi]

@Dunno
Dividing the code into a couple of files has no effect on the speed. It just makes the files easier to read for humans. Sorry about forgetting to answer this. It results in the same text in RAM when loaded.

As I said, place some [chat] tags between the steps of the new turn or turn refresh events and watch what will take a lot of time and what not. It can be easier than using timestamp as Anonymissimus suggested, but it is not very effective if the cause is not in one or a very few parts, as I don't expect.

As Anonymissimus simultaneously said, another possibility is autosaving, if there is too much code (usually resulting from the dangers of usage of macros, like big macros in larger numbers within other macros, but there was a bug in some 1.11 version causing this too), the save files become excessively large, making the autosaving take a lot of time. Try to disable autosaving to see if this is the cause. Save files larger than 2 megabytes are very likely to be the cause.
In most cases, it can be done by optimalising WML, but there is no standard way to use it, just things like usage of variables serving as database instead of switch tags, fire_event instead of macros (this can cause a segmentation fault, that will always happen under some conditions on all versions, but I have not learned the exact conditions yet).
How much RAM does wesnoth take when playing your campaign and how many scenarios are there?

@Anonymissimus

I suspect it's the operating system, keeping things a program needs in memory, unless that memory needs to be freed for some other process.
Or the wesnoth feature "wml cache"; but there was a bug making it unused until recently, and it also cannot be used in case the wml was changed.

Turning the game off and then relaunching it will not make it happen again (or pressing F5, which will reload WML for sure), so I suppose it is the first possibility. I don't know what is stored there, though, because the images seem to be loaded before.

[Anonymissimus]

I don't know what is stored there, though, because the images seem to be loaded before.

The cache doesn't store binary data. You can open those files, similar to saves. They contain preprocessed wml and such, so wesnoth doesn't need to do that again, unless the wml changed, of course.

[Pentarctagon]

Leaving a loop as soon as possible can help a bit, depending on the size of the array. Also if you are sorting an array, its much much better to keep it pre-sorted since then you only have to loop through the array once to insert new data.

[Max]

wesnoth UI and backend aren't properly separated. when the display isn't updated wesnoth is probably busy with some code that doesn't trigger ui refreshes.

to find out what's going on you could increase the log level (--log-debug=all) and have a look in the stderr.txt (if you're on windows).

how large are your uncompressed save games?

[Iris]

One false positive I am experiencing is that if I load a scenario for the first time, it makes me wait for about 15 seconds till something happens, after executing all start and turn refresh events, but before showing the objectives screen. But it does not appear if I simply restart the scenario, and it happens also when I load a save file from the middle of a scenario. It seems to appear every time I launch the game after that it was closed for some time, and it is not related to the WML, because if I hit F5 and load it again, it does not make me wait. I don't know what is exactly the case of it, but I believe it is related to the quantity of save files.

I suspect it's the operating system, keeping things a program needs in memory, unless that memory needs to be freed for some other process.
Or the wesnoth feature "wml cache"; but there was a bug making it unused until recently, and it also cannot be used in case the wml was changed.

Note that “recently” was versions 1.11.0 (tagged August 25, 2012) and 1.10.3 (tagged May 18, 2012).

As an aside:

In most cases, it can be done by optimalising WML, but there is no standard way to use it, just things like usage of variables serving as database instead of switch tags, fire_event instead of macros (this can cause a segmentation fault, that will always happen under some conditions on all versions, but I have not learned the exact conditions yet).

It would be nice if you could determine the conditions required to trigger that. Some campaigns (in particular one of my own) make heavy use of custom named events, and it would be a pity if that was failing on some systems or under other specific conditions because of a bug that hasn’t been reported yet.

[Dugi]

It would be nice if you could determine the conditions required to trigger that. Some campaigns (in particular one of my own) make heavy use of custom named events, and it would be a pity if that was failing on some systems or under other specific conditions because of a bug that hasn’t been reported yet.

I have filled a report on gna, but I could not provide any reliable information, so it was ignored. I can't tell what was its cause reliably, because I was using the same custom event all over the campaign and it never appeared elsewhere. I learned that it has no relation to surrounding ActionWML, no relation to the setting of variables and no relation to the contents of the even fired. I have worked it around by removing the fancy fire_event and used the classical macros (it was only in one scenario, it did not matter as much), and now I don't have an example of code where it happens (because it was in the first scenario, that should be flawless). This is all I can say:
- it happened when I used fire_event inside a menu item that was previously edited by a menu item with the same id (this might be the cause, but I am far from sure with it)
- it was preceeded by strange behaviour (a character said several empty messages, the number varied, and then it crashed)
- it happened when the code was returning from the code of the event fired into the code that fired it

[Dunno]

Holy crap! My uncompressed save is 57 MB big
I have much to do then...
Thanks for all your replies

[Dugi]

This is caused either by a lot of units with a lot of AMLA options (that I don't think to be too likely), a bug a 1.11 version (but this causes trouble rather after having completed a lot of scenarios) or by ineffective macro usage. I think it is the third possibility. All of the problems are uneasy to avoid.

Upload the save file, it is the best way to learn what is the problem.

Ineffective macro usage can be avoided by a large reconstruction of the whole system, replacing switches and ifs by iterations over some variable-databases. This is especially problematic when items are used, Heindal is facing these problems too, I faced it too, but I realised early enough that this is not the perfect way to do it. (see the World Conquest MP campaign to find an easy way to do it without too many macros). Another way is to replace most large macros by fire_event and custom events, arguments will be a bit harder to handle, but not really hard.

[Dunno]

Yes, yes, I'm replacing some macros by fire_events as we speak, and I've already managed to decrease the save size by a couple of megs. Thanks again
If you're just curious, I've attached a monstrous macro (1549 lines, duuuude) that is most likely the biggest problem

[Anonymissimus]

custom events, arguments will be a bit harder to handle, but not really hard.

Note that custom events can be passed any sort of data that can be put into a wml table/a block of wml, this is a side effect of the way [fire_event][primary_arrack] is implemented. The advantage over setting some global variables as parameters and clearing them after the fire_event is that the parameters are automatically cleared exactly at the point they're no longer needed. They also don't overwrite, instead hidden variables of the same name become usable after the call, as in a real programming language.
See the "EDIT" part in this post for a straightforward example:
http://forums.wesnoth.org/viewtopic.php ... 66#p510066

[Dugi]

@Dunno
That is not the problem, unless you use it multiple times in other macros that appear many times in scenarios (it is 28 kiB large, and even if it is ten times in each scenario, it will create only like 600 kilobytes of bloat). Long macros themselves aren't really a problem, the problem is the usage of macros many times in other macros, that are used many times. Just open the code inside a save file, look at some random points and you will find quickly which structure is causing the bloat.

And sorry for this dumb question, if you have many scenarios like this, isn't it causing trouble by filling the RAM memory? Macbooks usually have good hardware, but still...

@Anonymissimus
Interesting, I didn't know that any variables could be passed as fire_event arguments.