Issue with distribution of Lua code in a multiplayer scenario

Discussion of Lua and LuaWML support, development, and ideas.

Moderator: Forum Moderators

Post Reply
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

I encountered an unexpected issue with hosting a custom multiplayer scenario on the official multiplayer server.

# Context

The layout of the scenario's (the add-on's) directory:

Code: Select all

~/add-ons/myscenario/src/myscenario.lua # Important segment

~/add-ons/myscenario/maps/myscenario.map
~/add-ons/myscenario/scenarios/myscenario.cfg
~/add-ons/myscenario/_main.cfg

_main.cfg:

Code: Select all

#ifdef MULTIPLAYER
  {~add-ons/myscenario/scenarios}
#endif
[lua]
    code=<<wesnoth.dofile('~add-ons/myscenario/src/myscenario.lua')>>
[/lua]
The Lua code snippet defines a custom WML tag for the scenario. __It executes in single player as expected__.

# Use case

A user, that is me in this case, hosts the scenario on the official __multiplayer__ server and __not the add-ons server__. A tester joins the host normally. The scenario starts normally. However, the identities expected to be produced by the Lua code are present in the host game, but absent in the tester's game. I believe there was a Lua error displayed during the game to them about unrecognized tag, but not entierly sure if it happened in all cases we tested. Eventually the session has to end due to OOS errors, predictably.

Eventually two adjustments are made to the scenario and the use case. First, Lua snipped is moved from src/ to lua/. Second, the tester receives the scenario data in an archive and puts the content in their ~/add-ons/ directory explicitly. Note that previously they simply joined a hosted game on the multiplayer server, with no additional preparation (no installation of add-ons from server or manually).

After adjustments are made it worked as expected.

# Problem
Most likely I missed some documentation. Still I ask for clarification on the following. The last question is the most important.

Does the initial layout break the convention for distributing the Lua files? What is this convention? Are subdirectories allowed? Does it actively affect how the scenario is installed on the peer's system? Where does Wesnoth stores the scenario data that is not downloaded from add-ons server but from multiplayer server? How is it then accessed? Is it possible to reference the code snippet files in _main.cfg file with path relative to the add-ons current directory?

How can I make sure that the Lua snippets are available to the user regardless of distribution method?
Shiki
Developer
Posts: 311
Joined: July 13th, 2015, 9:53 pm
Location: Germany

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Shiki »

wesnoth.dofile('X')
The scenario data is transfered over the network from the host to other players. For this case, it transfers the literal Lua statement, which means read file X. But other players do not have the add-on and not file X.

But it's also possible to send the contents of file X over the network, by using the preprocessor to embed it.

Code: Select all

[lua]
    code={X}
[/lua]
Notice the absence of the quotes. … But where are the quotes then?

Code: Select all

-- The content of file X looks like this:
-- << 
… Lua code is here …
-- >>
The preprocessor will ignore everything between << >>. The game engine on the other hands will ignore the Lua comments which start with --.
That way you can appease both. The drawback for you as developer however is that changes to the Lua files will not come into effect immediately — like WML changes they will require you to press F5 in the main menu.
Try out the dark board theme.
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

What I understood is that Lua files are not transmitted over the network, only the scenario WML and map data. The work-around you suggest is to embed the Lua code into the scenario WML descriptor.

Why can't the game simply transfer the Lua file as well? Is there some kind of manifest I can use? In the use case that I described, will the game parse the _server.pbl file?
User avatar
Ravana
Forum Moderator
Posts: 2464
Joined: January 29th, 2012, 12:49 am
Location: Estonia
Contact:

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Ravana »

I load lua with dofile while developing, but for people who download addon and then host, it will be preprocessed into wml. https://github.com/ProditorMagnus/Oroci ... cfg#L1-L15
gfgtdf
General Code Maintainer
Posts: 1344
Joined: February 10th, 2013, 2:25 pm

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by gfgtdf »

Bladget wrote: September 14th, 2020, 11:12 pm
Does the initial layout break the convention for distributing the Lua files? What is this convention? Are subdirectories allowed?
wesnoth itself does not impose any restrictions on your directory structure, if it didn't work with src/ then you probably did something wrong during testing. However most people use lua/ for lua files.

Bladget wrote: September 14th, 2020, 11:12 pm Does it actively affect how the scenario is installed on the peer's system?
Not sure what you mean here.
Bladget wrote: September 14th, 2020, 11:12 pm
Where does Wesnoth stores the scenario data that is not downloaded from add-ons server but from multiplayer server? How is it then accessed?

It doesn't. the multiplayer server only sends a 'save file' (that contains only the [snapshot]/[scenario]) to the other clients that is then played by all clients simultaneously, any information that is not part of a savefile, such as .lua files loaded via dofile, [unit_type]s or image files, is epxected to be present on the clients already.
Bladget wrote: September 14th, 2020, 11:12 pm Is it possible to reference the code snippet files in _main.cfg file with path relative to the add-ons current directory?
Iirc there is the {CURRENT_DIRECTORY} macro that expands to the curent directory so you can do something like code = "dofile({CURRENT_DIRECTORY}/lua/main.lua)"
Bladget wrote: September 14th, 2020, 11:12 pm
How can I make sure that the Lua snippets are available to the user regardless of distribution method?
If you don't want to put your addon on the addon server, then Shikis suggestion is the best you can do. however i still reccomend to just put your addon on the addon server.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

Thank you. Your responses were enlightening.
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

I discovered an important option that is related to the issue. To prevent players from running corrupt scenario that is missing the Lua script, the developer should set "require_scenario" property of "multiplayer" tag. [Reference](https://wiki.wesnoth.org/ScenarioWML#require_scenario).
User avatar
Celtic_Minstrel
Developer
Posts: 1739
Joined: August 3rd, 2012, 11:26 pm
Location: Canada
Contact:

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Celtic_Minstrel »

Bladget wrote: September 15th, 2020, 12:06 am Why can't the game simply transfer the Lua file as well?
The reason is that the game has no way to know which Lua files will be needed. You could for example have some complicated code that builds up the name of a Lua file from components and then calls it, and there would be no way for the server to know from inspecting the WML which file will end up being called.

As you discovered, the best solution is probably require_scenario, especially since I believe that will automatically download it to anyone who hasn't installed your add-on. Of course, that does rely on you putting it on the add-on server.
Last edited by Celtic_Minstrel on October 20th, 2020, 12:43 pm, edited 1 time in total.
Author of The Black Cross of Aleron campaign and Default++ era.
Maintainer of Steelhive.
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

> The reason is that the game has no way to know which Lua files will be needed.

Thank you for the attention.

To me it's a matter of implementing a table-of-contents file that lists all of the files that the scenario requires. (It would have been usable for deployment too. I ended up uploading Git repository to the add-ons server at least three times.) Personally, in the context of Wesnoth, I would have extended the scenario WML for this purpose. Note, I suggest nothing, I only state this for the sake of discussion.
User avatar
Celtic_Minstrel
Developer
Posts: 1739
Joined: August 3rd, 2012, 11:26 pm
Location: Canada
Contact:

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Celtic_Minstrel »

There is no way for the game to automatically build a table of contents of every file that the scenario requires. It's because WML doesn't know what Lua is - to the WML parser, your Lua code tags are nothing but a blob of text. Even the << >> tags have nothing to do with Lua - they just tell the WML parser that this is a quoted string that should not permit macro expansion with it. In theory, it would be possible to trawl the parsed WML object for [lua] tags and search those tags for require or dofile calls, and that would probably get most of the Lua files required, but it might miss some, and it certainly isn't a simple task.

There could of course be a way for the designer to add such a table of contents, but of course that risks you forgetting one. Or the game could err on the side of caution and just send every Lua file it finds in your addon's directory, which would probably work most of the time (but what if you're using Lua files from another addon, such as Wesnoth Lua Pack?).

Long story short: what you're asking for is a lot trickier than it sounds.
Author of The Black Cross of Aleron campaign and Default++ era.
Maintainer of Steelhive.
User avatar
Bladget
Posts: 22
Joined: August 23rd, 2020, 9:47 pm
Location: Eastern Europe

Re: Issue with distribution of Lua code in a multiplayer scenario

Post by Bladget »

I am aware of the problems you described. I did mean that a developer should list the dependencies, or rather the contents o their scenario, explicitly. Works for World of Warcraft add-ons.

Sure, developer can forget to upgrade their table-of-contents or manifest file. Developer can also make a mistake in Lua code. Manifest is easier to debug than guessing the internal workings of the engine like I did here.

Again, I suggest nothing, it was just interesting for me to ponder this issue. Feel free to close the thread, since the original topic is exhausted and the solution found. I thank every participant for valuable input.
Post Reply