Experiment in loading past eras/campaigns in cores

Discussion of all aspects of the game engine, including development of new and existing features.

Moderator: Forum Moderators

Post Reply
demario
Posts: 95
Joined: July 3rd, 2019, 1:05 pm

Experiment in loading past eras/campaigns in cores

Post by demario »

I am looking into the core as a way to load past translated content.

The core is a lightweight mechanism for isolating an add-on from the game configuration. In this way we can see it as sandboxing.
The two limits to the independence to the game engine are:
  • in the top layer, you still need to meet the requirements that are set by the C++ engine. It means that you need the right amount of settings, otherwise the core will behave badly. That includes initializing terrains, game config and lua among others.
  • in the bottom layer, you still need to rely on the default parser and scripting (so not following the current lua API will end with errors).
Basically everything between these two can be configured differently, but anything that is left unchanged must be referenced explicitly in the core (see it like a wall around the core that prevent other settings from being reached).

So for a core to support units from an old version, you need to define locally:
  • the old unit types, but it is important to leave the descriptions unchanged so you don't break past translations
  • I think it is better to leave unit stats unchanged as that doesn't break "balance" in the campaign
  • abilities are better kept the same because using a new macro of an ability will introduce new untranslated strings
  • the translations supporting the type definitions (domain #wesnoth-units...)
  • if you want to reduce the size of the core, you can modify unit definitions to use the current images, animations...
  • make direct reference to the new macros (you're really never sure where they are needed) and right after reference a local copy of the old macros (matching the version of the unit definitions). If they are conflicting, the last definition will take over.
    in case of failure with old macros, you will have to edit the files to use the new macro and remove the old one.
With the target to run an old version of campaigns to restore the past translations, my current knowledge is:
  • change id for campaigns and scenario to append a suffix
  • update the campaign _main.cfg so that it meet the expectations of the current C++ engine (no [tutorial], no difficulties=...)
  • change all the paths (to maps, images...) to point to the local copy defined in the add-on
  • the old translations must be stored inside the add-on and be explicitly referenced in each campaign _main.cfg
  • add dependency to your core (in _info.cfg or _server.pbl). At this time, it seems you also have to explicitly import _main.cfg from the core. [that would define each trait twice, leading to "strong,strong" units]
  • you may again avoid distributing the old images from the campaigns by referencing the current ones from the game (data/campaigns/.../images/)
  • You need to keep the AI up-to-date to make sure the settings in the old scenarios work with the underlying functions from the current engine.
The core is not checked when:
  • joining a server, which makes it possible to run games with past default era on 1.16 server ;)
  • observing a game, which could lead to either OOS (if unit stats are different) or scripting error (if an add-on is installed but related to another core)
  • loading a save (using the wrong core for a campaign will break on next scenario, as you have suffixed their id, remember?)
Last edited by demario on May 18th, 2022, 2:15 am, edited 11 times in total.
"simply put, it's an old game"T — Cackfiend
User avatar
hermestrismi
Posts: 313
Joined: February 6th, 2016, 11:28 pm
Location: Tunisia
Contact:

Re: Changing user interface based on WML core

Post by hermestrismi »

I didn't found the thread for the new addon-modification "core 1.12"
after installing it, this message keep appers

Code: Select all

Error validating data core.

Details:

    Core ID: core_1_12
    Core Path: ~add-ons/Campaigns_1_12_Core/core-1.12/
    File not found.
    Skipping the core.
User avatar
Pentarctagon
Project Manager
Posts: 5112
Joined: March 22nd, 2009, 10:50 pm
Location: Earth (occasionally)

Re: Changing user interface based on WML core

Post by Pentarctagon »

hermestrismi wrote: May 6th, 2022, 7:22 pm I didn't found the thread for the new addon-modification "core 1.12"
after installing it, this message keep appers

Code: Select all

Error validating data core.

Details:

    Core ID: core_1_12
    Core Path: ~add-ons/Campaigns_1_12_Core/core-1.12/
    File not found.
    Skipping the core.
Moved to the thread.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
User avatar
hermestrismi
Posts: 313
Joined: February 6th, 2016, 11:28 pm
Location: Tunisia
Contact:

Re: Changing user interface based on WML core

Post by hermestrismi »

Pentarctagon wrote: May 6th, 2022, 7:23 pm
hermestrismi wrote: May 6th, 2022, 7:22 pm I didn't found the thread for the new addon-modification "core 1.12"
after installing it, this message keep appers

Code: Select all

Error validating data core.

Details:

    Core ID: core_1_12
    Core Path: ~add-ons/Campaigns_1_12_Core/core-1.12/
    File not found.
    Skipping the core.
Moved to the thread.
Thank you Pentarctagon
It is a great job. Hope you will succeed on it Demario
demario
Posts: 95
Joined: July 3rd, 2019, 1:05 pm

Re: Changing user interface based on WML core

Post by demario »

At this time, it appears that it can help replace some of the text domains with old ones: wesnoth-units, wesnoth-help, wesnoth-multiplayer and all domains related to campaigns. The hope to be able to load old wesnoth textdomain is slim (that's the domain for messages from C++, lua, macros).
Both wesnoth-manpages and wesnoth-manual are not loaded into the game IIRC.

The main problem is wesnoth-lib as it contains the strings for the user interface (button labels...). To support a good user experience they need to be loaded in the version that is translated. It seems there are some limitations that prevent that:
  • The data/gui wml files can't be loaded into a core. So it doesn't seem possible to change the layout of the screens to "delete" the buttons that add new features having untranslated strings.

    It matches my experience that any slight change in a data/gui wml file will generally lead to a C++ exception if the corresponding cpp changes are missing.
  • Loading themes in a core is possible. But I didn't see any example where themes are "removing" (aka making them invisible) widgets completely. Themes are usually tweaking font size, adding overlays, changing/adding menus...
  • Lastly I am still looking for themes that apply changes beyond the main screen of the game (minimap, status...). For example the title screen or the dialogs for game creation. Beside removing some new widgets (RNG selection in campaign startup screen...), it would be required to change the labels of some buttons (like "Campaign" instead of "Campaigns" in the title screen)
When all that is sorted out, then I would need a system that loads to right core based on the chosen language. I don't remember seeing any code that checks the active language to drive decisions about behaviors (except displaying the translated strings obviously).
hermestrismi wrote: May 6th, 2022, 7:22 pm

Code: Select all

    Core Path: ~add-ons/Campaigns_1_12_Core/core-1.12/
    File not found.
I fixed the typo at core version 0.1.6. You'll need to update.
"simply put, it's an old game"T — Cackfiend
demario
Posts: 95
Joined: July 3rd, 2019, 1:05 pm

Re: Experiment in loading past eras/campaigns in cores

Post by demario »

First thoughts about using cores in wesnoth.

There could be 2 ways for wesnoth to implement cores that I can think of, both have advantages and disadvantages.
  • the core must be selected at start. The choice of the core leading to different contents shown to the user
    This is how it is currently implemented in wesnoth.
    Advantages:
    • as the core is selected at start, the whole user experience can be dependent on the core.
      A spacenoth core could replace swords by laser sabres in wesnoth logo, an after-the-fall core could replace the map shown in title screen.
    • for two cores linked to the same world, the selection of the core leads to show only a selection of the installed content, making the choice easier
    • two copies of the same scenario could be linked to different cores, and they would not show at the same time to avoid confusion for the user
    • as tips and help can be accessed from title screen, the loaded core can provided all the relevant data.
    Disadvantages:
    • first and foremost: the user need to know some specifics about cores (how to load one, how it is reloading each time)
    • it works best if wesnoth was distributed with different cores, but there is no alternative to the default core in wesnoth
    • by filtering everything that is not listing the selected core, most of the add-ons will not be available outside of the default core
    • it doesn't allow the user to compare and make a choice between scenarios in different cores based on general information (latest version, mainline vs add-on version, version with full translation...)
    • this pretty excludes cores to have any role in multiplayer. You could only connect to the server "related to your core" (relationship that doesn't exist at this time)[edit] Actually the current behavior is that the MP server is neutral towards cores. All cores can connect to it, which has its own benefits, like not splitting the player community.[/edit]
    The benefits of this option are also reduced from limitations in wesnoth cores:
    • while an early selection of the core could allow full different looks of the game, the [gui] configuration cannot be set in a core (both from lack of WML to modify GUI and possible conflicts with C++ GUI code). So the tittle screen will still look the same for all cores (only tips can be changed).
    • the isolation of add-ons in cores is still incomplete: the add-on id, path and scenario id for campaigns still have to be unique across all cores. So putting a copy of an add-on in new core still requires a lot of renaming. That renaming can be easily extended to names and descriptions to let the user know the specifics of each versions if they were listed at the same time.
  • the core is loaded as a result of a selection of a scenario: when the selected content requires a core, the core is loaded before the scenario is started
    Advantages:
    • as the core are loaded based on the selected scenario, the user doesn't need to know about it
    • as all add-ons are available for selection, a secondary criteria could be used for listing (mainline/UMC, version number, translation status...)
    • possibility to select modifications linked to default core (or no core) to be active with scenario bound to another core (as they are not filtered-out)
    • the core would behave more like a super-resource
    Disadvantages:
    • if the core is loaded based on the selected add-on, each time a game is ended, the engine would need to load again the default core until a new scenario is started
    • complications for knowing which add-ons are compatible with a core. It would be left to the user to sort it out
    Engine limitations:
    • if all add-ons are listed together, additional information should be listed to help telling them apart (translation status...)
    • [edit] at this time the concept of a core being a super-resource is limited by the fact that core can't load resources. If a core needs to copy events to the era/scenario it needs to define a modification that must be hand-selected when running the core.[/edit]
"simply put, it's an old game"T — Cackfiend
demario
Posts: 95
Joined: July 3rd, 2019, 1:05 pm

Re: Experiment in loading past eras/campaigns in cores

Post by demario »

Here is a summary of my findings and the thoughts I got about using wesnoth cores to provide some backward support to old materials:

The support that can be provided can be split into 5 categories:
  • Macro
    With each version, the syntax of some macros changes: either the macros gets dropped (MAGENTA_IS_THE_TEAM_COLOR) or they are changed to get more parameters. Using an existing macro with the wrong number of parameters will lead to a parser error.

    So at each version existing UMC must be updated to be compatible with new version. It is done by the author, a new maintainer that takes over or be part of crowd effort.

    The core can help by offering the old syntax for the macros. As the number of macros is known, this kind of support can be written once for all.
    All add-ons can then just link to that specific core rather than the default one and the unchanged add-on can be used in the new version. The most basic one-faction add-on and simple multiple-faction era can be recovered with this mechanism. There can be one core for each past version of wesnoth and have each core to provide the original default era for compatibility.

    We could also consider connecting to the past version add-on server when the matching core is loaded, download these add-ons and append the loaded core to the core="..." inside the _info.cfg.
  • WML actions
    The WML actions can be changed to be renamed or the keys can be changed to be made mandatory or renamed to more self-explanatory. Using the wrong key will be ignored or generate a WML parsing error.

    The core can help by providing a bridge between old WML actions keys and the new ones. This is done by catching the WML action in lua, rename the keys to the new version or fill the missing ones and fire the WML action with the new data. As the number of WML action is known, this kind of support can be written once for all. However given that it involves firing events, it must be put inside a modification because core can't define events at this time. The modification must be link to the core (to benefit the support for macro too) and be selected when using the add-on coming from a past version.

    In the long run, it would be easier to allow wesnoth cores to define their own events and to have them added to the scenario when one core is loaded.
  • Other WML and lua
    There is some WML that cannot be caught as WML action. They can be also changed in a way that generates an error when add-ons are loaded in new version. One typical example is the old way of defining difficulty in campaigns:
    The old way (example)

    Code: Select all

        difficulties=EASY,NORMAL,HARD
        difficulty_descriptions={MENU_IMG_TXT2 "units/legion/legionnaire.png~TC(1,magenta)" _"Legionnaire" _"(Easy)"} +
                         ";*" + {MENU_IMG_TXT2 "units/legion/propugnator.png~TC(1,magenta)" _"Decurion" _"(Medium)"} +
                         ";*" + {MENU_IMG_TXT2 "units/legion/centurion_b.png~TC(1,magenta)" _"Centurion" _"(Hard)"}
    
    The new way (another example)

    Code: Select all

        {CAMPAIGN_DIFFICULTY EASY   "units/elves-wood/fighter.png~RC(magenta>red)" ( _ "Fighter") ( _ "(Beginner)")}
        {CAMPAIGN_DIFFICULTY NORMAL "units/elves-wood/lord.png~RC(magenta>red)" ( _ "Lord") ( _ "(Normal)")} {DEFAULT_DIFFICULTY}
        {CAMPAIGN_DIFFICULTY HARD   "units/elves-wood/high-lord.png~RC(magenta>red)" ( _ "High Lord") ( _ "(Challenging)")}
    
    There is no way to "catch" this part of WML in lua and turn the old definition to the new format.
    Since most add-on authors do not care for deprecation warnings. most campaigns in past version still use the old version and as such are broken when running the new version. A manual edit is required. Which limits the "automatic" way to load old add-ons by just appending the matching core id in _info.cfg of the downloaded add-on.

    Another example (that is not SP specific) is the replacement of [choice] option definition by [combo] that again was enforced in BFW 1.16. While the [choice] is dropped silently (not an error), if that option is defining a variable used later in the code, that variable will be missing, leading to wrong behavior. That is the only manual change I had to make to the "(unofficial) Half Civ (1.14)" that I put up on the 1.16 add-on server after linking it to 1.14 core from "Support 1.10-1.14 cores".
  • Engine
    Beside changes in the WML, some changes to the C++ engine might also lead to different behaviors that lead to different experience to the player. One of the most recent example is how rounding is done when applying traits. While these problems could be considered as regression bugs, this is not how the devs think and they won't be fixed.

    These problems must be sorted on case-by-case basis. For that specific bug, the solution is to apply a post-recruit correction. But it is only applied to all unit types known to have the problem and so it will have to be extended whenever an era is defining a unit having the same problem.

    The need to have the exact same XP for a recruited spearman when playing the 1.14 version of the default era may seems to be only interesting to purists but is actual is quite significant when we are interested in replaying old saves (see next section).

    Another error of this type is that while in 1.14, the prestart events defined in sequence are executed in the same sequence. It seems that in BFW 1.16 the events will be executed in a different sequence. So if one event is using the result of another event prior in the sequence, when the sequence is shuffled, the expected data is not present and there is an error. The solution would be to make the sequence explicit by calling the different events by id.
    That is one problem that I encountered when porting "(Good old) World Conquest II"
  • Replays
    Trying to replay old saves in the current version of BFW opens its own can of worms.
    The two critical things to keep in mind when trying to allow replaying old saves are:
    • The engine must resolve the old save data exactly the same way as the original engine did. Otherwise OOS are thrown (and replays often ends badly).
    • The data in the replay is self contained (except references to external lua files) and no event will be added by the engine before the replay is started.
    As the engine must performs exactly as in the old version and as the most common way to fix engine behavior by appending events cannot be used when replaying saves, it is impossible to solve the situation within these constraints. The only way to support this feature currently is to hack in the C+ code and provide the feature as a patch that require people to compile their own version of the game.


    The selection of an additional modification (as described before) to offer some WML support is also impossible as the only modifications that will be loaded during replay are the ones that were enable when the save was generated (usually none). So offering some support from wesnoth core to replay saves must be through allowing cores to define their own events that will be added to the scenario where the core is loaded.

    When replaying a save, the core events must also be injected into the game data. That is a change in the design in the replay controller. The way to limit possible impact of such a change would be to only append core events when loading a save for replay. It will be then up to the developer of the core to make sure the events from the core is not doing anything "funny" with the data.
    We could also enforce that the core only defines preload events (but other events can be defined inside preload events).

    I believe that if cores are allowed to define their own events and that they can be injected into save data before replay, there is a good shot at supporting replay of saves from version 1.14.

    For 1.12 and beyond, there is a need to select different RNG based on the save to replay. Again, the engine could be changed so that is gets the information about the RNG to use from the loaded core (matching current RNG for default core). But at this time, there are too many missing features to entertain such a solution.

    We could also make sure that the selected item in the directory-selector to the file dialog is defined inside the core (and matching the current directory for the default core).
"simply put, it's an old game"T — Cackfiend
Post Reply