Save game files becoming excessively large

[chak_abhi]

I don't know whether this is the right place to post this question, but this is troubling me very much.
In a particular scenario of a campaign created by me the save game files have become very large suddenly (from <100KB in the previous scenario to >250KB), and increases highly as the game progresses. As a result the game loads very slowly. Whats worse that after the scenario is over the option to end the level appears nearly 20 minutes later and when it is clicked the program hangs and the next scenario doesn't load at all. What are the problems that may cause this thing? Please suggest a possible solution. Thanks in advance.
N.B I use BfW 1.11.1

[Dugi]

Usual generators of size (or at least those that troubled me so far):
1. Walking Corpses and Soulless. This cannon fodder accumulates on the recall list (because it is useless to recall), and its unit code is very long because all variations are contained in every unit's code in save files. This is my actual problem, you know how will it look after 50 scenarios with plague-using units.

2. AMLA. If there is too many units with AMLA, it might become a problem. All the AMLA possiblities of all units are stored quite uselessly in save files (because unstore_unit reloads it anyway). If you have 20 units of the same type with AMLA, all the AMLA will be there 20 times. I have coded a workaround for this, but the way it works is oblivious to me.

3. Macros. Each macro is substituted in the save file, so if you have a long macro and it appears a lot of times in every scenario, it is useful in most cases to use some variables as arguments and use fire_event instead (WARNING: in rare cases, it can cause segmentation faults, so if it starts crashing, just try to code it differently).

4. Garbage. Always delete the variables you don't need anymore, especially variable arrays.

Also, you can actually open the file (look for a program that can open that type of archive, it is a relatively common type of archive), skim through its contents to see which parts of the code are taking so much space.

But, the save files don't slow down the loading too much. At some point, I had excessive problems with point 2, resulting in save files larger than 2 megabytes, and even 40 megabytes when uncompressed. But, it was still loading relatively quickly (back to the start of the turn usually took only like 2 seconds, that wasn't much more than usually). It was causing various short freezes and fps drops, also froze for like 2 seconds when it wanted to autosave, but it still didn't delay the end scenario message or proceeding into another scenario. At the moment, my campaign produces save files that are like 200 kilobytes large, and it does not cause any slowdowns. The only thing that takes a lot of time is the loading for the first time when your turn the game on, because there is so many scenarios (trying to load a single scenario shortens it significantly).

Therefore, I am quite sure that the cause of your problem is not the size of save files, but something else. While cycles are the most probable option. WML is not C, when you fire an event inside a while cycle and use the same name for the iterator, an infinite loop will arise. If you have an infinite loop there, it will break after 65k cycles (if the contents of the [do] tag is short, it may take like 5 seconds), so you might not realise that you've a bug there. Use [chat] or [message] to see how it is iterating.

[chak_abhi]

I do not have any walking corpses/soulless units in my recall list, but there are some units which have AMLAs (most of them are leaders and have to be kept). There are quite a few macros too, and they also must stay. I have also tried to clear the variables once their use is over as far as possible. And in addition to this I have upgraded to BfW 1.11.2 yesterday, which is a bit faster than 1.11.1.
So if the problem is not with the size of the save game files then it won't be easy to locate it. In any case if files about 2 MB size create no problem with save loading then those with 200-400 KB can hardly be held responsible. Anyway, thanks for your help.

[beetlenaut]

You are using the version in development, so there are some severe bugs in it. It sounds like you are triggering one: It is supposed to be impossible for any WML file to make BfW hang or crash. You should report this because they probably don't know about it.

[chak_abhi]

I can report it, but the problem is there is no save-file at this moment which I can upload as a specimen. In that case the report would lose its value. I will try to retrieve the file if I can.

[Dugi]

Why don't you open the save file and look what is taking so much space, or upload it so that we could have a look? A few units with AMLA are not a trouble, and macro systems that aren't a macro used several times inside another macro that is used several times in another macro should not be a problem neither. Every scenario's statistics are saved inside the save files, but this isn't extremely large even after 100 scenarios.

[Anonymissimus]

There are quite a few macros too, and they also must stay

If they are used for action wml, you could probably replace them with custom events. Custom events are basically like functions in a real programming language, so not copied each time they're called.

[chak_abhi]

I have retrieved one save file and I am uploading it. It is from scenario 40 of my campaign. If anyone can please find out if anything can be changed for the better then I will be grateful. Thanks in advance.

[Dugi]

Like 100000 sides are carried over. No recruits, no player, 0 gold, no save ID, red colour. Never seen anything like that. And because the compressed file is 42 MiB large, it is the cause of the slowdowns (the final archive size is not so large, because it is highly repetitive).

[chak_abhi]

So what is your suggestion? Should I add a save_id?

[Anonymissimus]

Looks like a bug, probably triggered by some unusual circumstances in your scenario and/or campaign.
What does the savefile look like at the start of the scenario ? (If there aren't so many sides, then obviously some useless [side]s get added at every turn or so.) Is it during scenarios in particular, that the savefiles increase, or rather during the campaign ?
You could post everything related to the side definitions as they are in the cfg file of this scenario, so we can see whether there's something unusual.
What type of scenario/campaign is that (SP/MP) ?

EDIT
I'd recommend manually setting a save_id for all persistent sides. If you don't do that it could trigger stuff.
Do know something about that this didn't happen in 1.10 yet ? It could be related to the recent scenario transition refactoring.

[Pentarctagon]

The save had 345,600 extra side tags which took up 3,124,922 lines
Without all that its 1.69 MBs uncompressed.

[chak_abhi]

Looks like a bug, probably triggered by some unusual circumstances in your scenario and/or campaign.
What does the savefile look like at the start of the scenario ? (If there aren't so many sides, then obviously some useless [side]s get added at every turn or so.) Is it during scenarios in particular, that the savefiles increase, or rather during the campaign ?
You could post everything related to the side definitions as they are in the cfg file of this scenario, so we can see whether there's something unusual.
What type of scenario/campaign is that (SP/MP) ?

EDIT
I'd recommend manually setting a save_id for all persistent sides. If you don't do that it could trigger stuff.
Do know something about that this didn't happen in 1.10 yet ? It could be related to the recent scenario transition refactoring.

I didn't add a save_id as I hadn't foreseen the consequences. A large number of scenarios has no leaders for the players side at the beginning (for side 1 "no_leader=yes") for a special purpose, but it seems it has backfired. This is a campaign, and tested only in 1.11x. I will soon add a save_id manually and see how it goes.

[AI]

Sounds like a bug in wesnoth too. My interpretation:

• Sides without a save_id get the leader's id as a fallback
• Without a leader, the save_id is empty
• Non-human sides should default to not having carryover, but this seems to be failing somehow

The result seems to be that every leaderless side ends up in the savefile.

[Anonymissimus]

for side 1 "no_leader=yes"

Well, this is certainly very unusual and perhaps the trigger of the bug. Normally, there always needs to be a leader on the human side. You could put a dummy leader and kill it in the same event where the real human leader is placed. You can use [hide_unit] or so to hide the dummy leader if neccessary.