Adding support to replay old saves (1.4-1.12 1v1) in BFW1.14

Discussion of all aspects of the game engine, including development of new and existing features.

Moderator: Forum Moderators

User avatar
egallager
Posts: 245
Joined: November 19th, 2020, 7:27 pm
Location: Concord, New Hampshire
Contact:

Re: Adding support to replay 1.14 saves in BFW1.16

Post by egallager »

demario wrote: October 27th, 2021, 10:40 am First version of the backward support for 1v1 replays in BFW1.16. It is still quite an early release (it has been tested on about 30 saves).

I cut the following corners for this release:
  • commented the whole checksum on recruit in the code (src/synced_checkup.cpp) as I didn't find a way to disable it by modifying save content.
  • I kept the 1.14 unit types under data/core (instead of a standalone add-on) as the cpp kind of expect the unit definition to be present (maybe there is a way to check the presence of the add-on in the code, but I haven't look at it)
  • no support for ladder era
  • no support for dunefolk
The good news is that the RNG hasn't changed at all (assuming the intelligent trait is not RNG related) and no old RNG is needed. The consequence is that you can replay 1.14 saves and play 1.16 games without limitations.
Hi, could you submit this patch as a pull request on GitHub and then iterate on it there? It would be great to get this merged so that everyone can benefit from it.
Wesnoth-related GitHub repos:
General mods collection, SotBEEE, AToTBWaTD, The Earth's Gut, A Little Adventure, FtF
Social media: Twitter: @cooljeanius, Steam: egallager
demario
Posts: 64
Joined: July 3rd, 2019, 1:05 pm

[mod] Adding support to replay 1.12-1.14 saves in BFW1.16

Post by demario »

Third Fifth Sixth version of the backward support for 1v1 replays in BFW1.16.

I cut the following corners for this release:
  • I still haven't worked out how to skip the recruit checksum
  • no support for dunefolk
I have decided to split the code in different patches, each one digging deeper in the changes required to the engine to support replaying old saves. You will have to apply the changes for the highest version first (working backward from 1.14)

It is split in different bunches of changes:
  • support 1.14: [Updated on Dec 12 to unitless rev 3]
    • it could have been pretty straightforward (only unit stat changes) if not for the issue of differences between version on intelligent trait. So it requires in the conversion to know the SPDE add-on era, launching the RNG, generating the random numbers to check for intelligent trait on new recruits to fix the xp rounding.
      The fix for xp rounding is done by catching recruit and advancement events for the impacted units with intelligent trait.
    • Good thing is that the RNG is unchanged from BFW1.16
    • It also starts to rollback of context-free grammar reduce the expected number of random numbers needed for names from gender-less races
    • Support some specific save format from replay server
  • support 1.14 Ladder era: [Updated on Dec 12 to unitless ladder rev 1]
    • Support for Ladder era 0.1.1-0.1.17
  • Support 1.12: [Removed for now]
    • it uses its own RNG and the memory of the desired engine version needs to be saved in a global variable (cause since BFW1.14, the RNG is not reset but is recreated from scratch for each new seed [and doesn't support any initialization data]).
    • So a config file must be passed the first time using an RNG to select the right version (at this time the global variable is not reset when switching back to RNG 1.16).
    • It also fixes the different numbers of random numbers required for nameless recruits. It starts getting fun in combination with plague attacks to create nameless walking corpses :geek:
    • It also completes the rollback of context-free grammar started in support 1.14 for names from gendered races
  • Support 1.8-1.10 [Removed for now]
    • (1.10) this is the first version where in some cases, the random numbers cannot be recreated (seed is lost). Replaying them requires the [replay] to be loaded into the RNG and have the RNG read the numbers from a stack populated with this data rather than generating its own numbers (called simulation code)
    • (1.10) the RNG simulation must be activated/disabled at the appropriate time
    • (1.10) turn the seed back and forth from integer to hexa-decimal
    • (1.10) maintain the leader's position to fill it into the recruit commands
    • (1.10) fill the seed for recruit commands from the previous attack, maintain the number of random calls
    • (1.10) also maintain the number of random calls when pre-recruited units are involved (statues, loyal in Hornshark Island)
    • (1.10) fix the Rush By Yetis (early random map picker) specific settings
    • (1.8 ) this version uses the same RNG as BFW1.10, with the only difference is that the random pool can be a negative number
    • (1.8 ) fix the petrified units (statues in Sulla's Ruins, ...)
    • (1.8 ) fix different terrain codes
    • (1.8 ) fix the hp rounding difference with BFW1.10 when some units are strong/intelligent and quick
  • Support 1.4-1.6
    • (1.6) the RNG at this step is much simpler and only one seed is used for the whole game
    • (1.6) at the time, compatibility between RNG versions was already broken and a no-check mode supported dropping the actual numbers for those in the saves
    • (1.6) the WML for the [replay_start] is different from now (eg. the side leaders were not defined as [units]s)
    • (1.6) the random numbers for traits were drawn after those for names
    • (1.6) a random number was drawn even for gender-less units
    • (1.6) one additional number was drawn for each unit
    • (1.6) fix for this issue on maps with controller=ai side (statues)
    • (1.6) different behavior for healthy and feral traits
    • (1.4) identical RNG
    • (1.4) difference in rounding hitpoints for some units with quick trait
    • (1.4) the format of the move command in replay is different
    • (1.4) different behavior for healthy trait
    • (1.4) different behavior for drains ability
Most unitless regressions should be fixed by now except known problem with Hornshark Island saves.
Last edited by demario on January 12th, 2022, 12:02 pm, edited 25 times in total.
"simply put, it's an old game"T — Cackfiend
User avatar
egallager
Posts: 245
Joined: November 19th, 2020, 7:27 pm
Location: Concord, New Hampshire
Contact:

Re: Adding support to replay 1.14 saves in BFW1.16

Post by egallager »

demario wrote: October 27th, 2021, 10:40 am First version of the backward support for 1v1 replays in BFW1.16.

Known issue: differences between version on intelligent trait
egallager wrote: October 28th, 2021, 4:51 am Hi, could you submit this patch as a pull request on GitHub and then iterate on it there?
Hi, this is a non-official mod to the engine for the wesnoth community so I would rather keep the development progress visible on the forum.
I think it should become official though; the community would be able to benefit from it more if it were merged upstream and then shipped to everyone. Also I find checking out a pull request to be easier than applying patches. And you can always link back to the PR to keep it visible from here, as well.
Wesnoth-related GitHub repos:
General mods collection, SotBEEE, AToTBWaTD, The Earth's Gut, A Little Adventure, FtF
Social media: Twitter: @cooljeanius, Steam: egallager
demario
Posts: 64
Joined: July 3rd, 2019, 1:05 pm

[mod] Adding support to replay 1.14 saves in BFW1.16

Post by demario »

New (and possibly last) iteration on support 1.14 replay in BFW1.16 in attachment.

This version of the code (to apply on clean BFW1.16) doesn't require the SPDE add-on anymore.
Most of the changes are now done by WML in the events added to the era which mean no more additional unit types are created. That is why I label it the unitless era. This does reduce the risk of a slow down on the engine (which I never noticed) due to overhead in processing/storing/accessing more type definitions.
Second change is that the era is defined right inside the patch under a new data/add-ons sub-directory so that is doesn't mix with default macros but is always available when the modified code is used. There is no unit type definition under data/add-ons/Support_Past_Default_Era_Unitless for 1.14 support.

[edit] New patch to replay ladder-1.14 saves in BFW1.16. All units changed in ladder era still have a ladder-1.14 specific type (as identified by their own unit type when saves were recorded)[/edit]
[edit] I removed the 1.8-1.10 and 1.12 unitless patches. The reason is that this design is broken on the long term as I will try to explain below.

The design of unitless support is to add WML [event]s to make corrections to original (ie. new version) unit stats as the units appears on the map.
The issue is that the [event]s are executed after some engine actions are already executed. For example, if you want to restore a couple of hitpoints to some unit type, the [event]s will be applied each time such a unit is recruited but after the unit has got its traits. So the correction will be applied to the hitpoints after traits are taken into account. For rounding reasons, for some trait combination the correction may be wrong and it needs to be corrected again.
An example
Troll Rocklobber changed from 49 to 51hp at version 1.16. So the raw correction would be to remove 2hp to restore 1.14 stats.

If this Rocklobber is quick (-5% hp), applying this correction after the -5%hp reduction leads to 46hp while applying the -5%hp on original 49hp lead to 47hp.
So a final correction is required to add one hitpoint.
These final corrections depend on the hitpoints at the latest version and the amount of the corrections for older versions.
So that at each new version, any correction to support past versions could be possibly broken.
As I believe that the support for oldest versions should be unchanged with a new version (ie. we only need to add support for old stable version), the only solution is to go back to version specific unit types (where the hitpoints are set to an absolute value, not through a correction).[/edit]

You can either apply all the patches in order and compile once after they're applied (remember to run cmake), or you can apply the patch one by one with compilation after each patch (apply cmake+make at each step).

The SPDE add-on is still on the add-on server for use with the old patches (it should not conflict with this patch if installed).
Attachments
troubles-1.14.md
Known problems with 1.14 and ladder-1.14 replays and status
(9.37 KiB) Downloaded 1 time
BFW1.16.load_ladder1.14_replay_backward_support_UNITLESS-noSPDE.v1.diff.gz
Apply over BFW1.16 1.14_replay_backward_support unitless v3.for ladder-1.14 replays, patch -p1, build with cmake (no add-on needed) [apply second]
(14.92 KiB) Downloaded 11 times
BFW1.16.load_1.14_replay_backward_support_UNITLESS-noSPDE.v3.diff.gz
Apply over clean BFW1.16 for 1.14 replays, patch -p1, build with cmake (no add-on needed) [apply first]
(8.63 KiB) Downloaded 13 times
"simply put, it's an old game"T — Cackfiend
Post Reply