Persistent Worlds
Moderator: Forum Moderators
Persistent Worlds
What, if anything, is currently missing from the existing multiplayer server system that would prevent the deployment of persistent worlds?
It occurred to me that I do not actually know what the specific no-go-theorems are for that.
What I am trying to work-around is the complexity that would be involved in implementing persistent worlds by means of the the single-player campaigns system as sketchily outlined in parts (specifically the Space Cadet and Time Cadet mission-control area's story screens) of my Between the Worlds campaign portmanteau... Which is basically to require people to upload replay and/or savegame files documenting what happened to the small part of the world that they chose to play with. (And choosing whose version of events actually happened probably as simply as by who uploaded a (successful?) completion of the scenario/campaign soonest.)
I see that all observable games' reply files are archived, so I guess requirement that people upload could be skipped by instead requiring them to play in observable mode.
So maybe the existing infrastructure already has the minimum necessary feature-set? All I'd need to do would be (1) have a method of figuring out which replays in the archive pertain to one of my persistent worlds; (2) be able to parse the replay file to retrieve the recalls list and gold on hand of each side, possibly even their income; (3) be able to generate a new scenario to make available for play that contains that gold and recalls?
Is changing a savefile or replay file recalls list into a portion of a scenario specification fraught with problems or basically a straightforward CABD ?
Edit: the method considered for using single-player campaigns supposed if would need an add-ons server that has user login thus giving specific users specific scenarios, so left open the possibility the players would maybe download savegame files instead of scenario or campaign files. Would it be easier to have players load games than start new scenarios/campaigns?
-MarkM-
It occurred to me that I do not actually know what the specific no-go-theorems are for that.
What I am trying to work-around is the complexity that would be involved in implementing persistent worlds by means of the the single-player campaigns system as sketchily outlined in parts (specifically the Space Cadet and Time Cadet mission-control area's story screens) of my Between the Worlds campaign portmanteau... Which is basically to require people to upload replay and/or savegame files documenting what happened to the small part of the world that they chose to play with. (And choosing whose version of events actually happened probably as simply as by who uploaded a (successful?) completion of the scenario/campaign soonest.)
I see that all observable games' reply files are archived, so I guess requirement that people upload could be skipped by instead requiring them to play in observable mode.
So maybe the existing infrastructure already has the minimum necessary feature-set? All I'd need to do would be (1) have a method of figuring out which replays in the archive pertain to one of my persistent worlds; (2) be able to parse the replay file to retrieve the recalls list and gold on hand of each side, possibly even their income; (3) be able to generate a new scenario to make available for play that contains that gold and recalls?
Is changing a savefile or replay file recalls list into a portion of a scenario specification fraught with problems or basically a straightforward CABD ?
Edit: the method considered for using single-player campaigns supposed if would need an add-ons server that has user login thus giving specific users specific scenarios, so left open the possibility the players would maybe download savegame files instead of scenario or campaign files. Would it be easier to have players load games than start new scenarios/campaigns?
-MarkM-
Re: Persistent Worlds
Leaving aside issues of whether or not this is a good idea, I will simply suggest one way you might go about it.
1. Start an alternate wesnoth MP server running the standard "wesnothd" daemon
2. Host each distinct gameworld area as a seperate map
3. Use a variation of the "Debug Era" that would allow only the host side to create units, assign their traits, abilities, etc.
4. Whenever an observer joins a map, the host can slot them into one of the available sides and create their units using the Debug Era commands
5. To speed up and automate this whole process, write a "portmanteau" bot to host the maps, slot people in, etc.
1. Start an alternate wesnoth MP server running the standard "wesnothd" daemon
2. Host each distinct gameworld area as a seperate map
3. Use a variation of the "Debug Era" that would allow only the host side to create units, assign their traits, abilities, etc.
4. Whenever an observer joins a map, the host can slot them into one of the available sides and create their units using the Debug Era commands
5. To speed up and automate this whole process, write a "portmanteau" bot to host the maps, slot people in, etc.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Re: Persistent Worlds
Thanks! I had not tried wesnothd yet as my GUI install-packages thing did not show the wesnoth-server RPM in the games section. Maybe Fedora chose to hide it in a system services section or somesuch instead of filing it under games, as Google found they do have one and yum was able to install it for me.
It's man page has stuff about the syntax of a config file but no default location for one and the etc/init.d/wesnothd start/stop script also has no specifying of a config file. There is no /etc/wesnoth dir or /etc/wesnothd.conf or suchlike so I am at a loss for where to find its config until my nightly locate-database catalogues my disk drive so that locate can maybe find it. So if anyone happens to know where Linux / Fedora 13 wesnothd RPM puts the config file that'd be handy to know.
I recently made a map for a scenario so my very own map was offered first on the list of maps when I told wesnoth to host a network game, so I finally have some idea how this could look.
Immediately I realised that if the map was going to basically sit there online to join 24/7 people would want to be able to either droid their side (probably other than their leader, whom they would probably want to "inn" - stash at an inn or some such place, a building maybe, that it would pop out of again when they join again) or "inn" their side.
Does anything along those lines exist, or is that the point where a bot or something would have to step in or the whole thing would have to be saved and fed to a parser that would extract their side into some kind of saved-sides database then start up the map again with that side-number now available for use by the next player to step out of the "inn" ?
EDIT: maybe lua could add to keeps an actual "keep" command, which on the host machine could have write access to disk to write out the characters that are to be saved and their gold, but for everyone else it just tells them those characters all moved to the keep and vanished?
-MarkM-
It's man page has stuff about the syntax of a config file but no default location for one and the etc/init.d/wesnothd start/stop script also has no specifying of a config file. There is no /etc/wesnoth dir or /etc/wesnothd.conf or suchlike so I am at a loss for where to find its config until my nightly locate-database catalogues my disk drive so that locate can maybe find it. So if anyone happens to know where Linux / Fedora 13 wesnothd RPM puts the config file that'd be handy to know.
I recently made a map for a scenario so my very own map was offered first on the list of maps when I told wesnoth to host a network game, so I finally have some idea how this could look.
Immediately I realised that if the map was going to basically sit there online to join 24/7 people would want to be able to either droid their side (probably other than their leader, whom they would probably want to "inn" - stash at an inn or some such place, a building maybe, that it would pop out of again when they join again) or "inn" their side.
Does anything along those lines exist, or is that the point where a bot or something would have to step in or the whole thing would have to be saved and fed to a parser that would extract their side into some kind of saved-sides database then start up the map again with that side-number now available for use by the next player to step out of the "inn" ?
EDIT: maybe lua could add to keeps an actual "keep" command, which on the host machine could have write access to disk to write out the characters that are to be saved and their gold, but for everyone else it just tells them those characters all moved to the keep and vanished?
-MarkM-
Re: Persistent Worlds
The WML of the scenario itself would be responsible for removing units from play.
The host (bot) would be responsible for remembering their statistics, in case they return.
There are many ways the host (bot) could memorize unit statistics.
Method 1:
When units are removed from play they are stored to a temporary variable, the host saves his game, and copies the contents of that variable from the saved-game file to a backup file.
Method 2:
The host (bot) memorizes all unit statistics at the beginning of every side turn.
Etc...
Saving files with Lua on each client is not an option. The File I/O routines are disabled for obvious security reasons.
The host (bot) would be responsible for remembering their statistics, in case they return.
There are many ways the host (bot) could memorize unit statistics.
Method 1:
When units are removed from play they are stored to a temporary variable, the host saves his game, and copies the contents of that variable from the saved-game file to a backup file.
Method 2:
The host (bot) memorizes all unit statistics at the beginning of every side turn.
Etc...
Saving files with Lua on each client is not an option. The File I/O routines are disabled for obvious security reasons.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Re: Persistent Worlds
I wasn't thinking of saving files with lua on clients other than the host.
For example maybe have a lua file that in the distribution has no function or only innocuous functions but on the host's machine (which might perforce also have to be the host's client's machine too for this to work if it is actually their client not their server that runs the lua?) it includes a write to disk of some info. That should not cause out of synch errors as it is only writing not reading.
When it comes to reading, where the info exists becomes important. Does only the server know everything, so that each client does not know other people's units that are out of sight, how much money other people have etc? Or do all the clients have all the info? (In which case why is there a server instead of one of the clients acting as a host? Or, oops, is that what I am actually seeing when I run a game: my client hosting a game and the so-called server merely acting as a way for other clients to get in touch with the client that is hosting?)
-MarkM-
For example maybe have a lua file that in the distribution has no function or only innocuous functions but on the host's machine (which might perforce also have to be the host's client's machine too for this to work if it is actually their client not their server that runs the lua?) it includes a write to disk of some info. That should not cause out of synch errors as it is only writing not reading.
When it comes to reading, where the info exists becomes important. Does only the server know everything, so that each client does not know other people's units that are out of sight, how much money other people have etc? Or do all the clients have all the info? (In which case why is there a server instead of one of the clients acting as a host? Or, oops, is that what I am actually seeing when I run a game: my client hosting a game and the so-called server merely acting as a way for other clients to get in touch with the client that is hosting?)
-MarkM-
Re: Persistent Worlds
No offense, but I think if you aren't able to answer those types of questions for yourself then this might be too difficult for you to implement in the way I described.
I don't have any interest in coding it, and I was only suggesting one possible approach (among many).
The advantage of my approach is that it relies on WML and bots, and thus would not require the players to use a modified client for that version of Wesnoth.
I don't have any interest in coding it, and I was only suggesting one possible approach (among many).
The advantage of my approach is that it relies on WML and bots, and thus would not require the players to use a modified client for that version of Wesnoth.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Re: Persistent Worlds
I am not sure what you mean by a bot. I have heard of bots for the multiplayer server lounge but was not aware of bots being able to observe interior details of games such as how much money everyone has what they currently have on their recall list what their list of possible recruits is or anything like that. Are there any skeletons frameworks examples etc of such bots? Or are they a new concept you have come up with as the elegant solution to this but that are such a new idea nothing even close to them has yet been attempted?
(Are you maybe thinking something like an IRC bot or "expect" program or MUD macro-capable client listening to and typing back to the server's FIFO? Are there already existing scripts/programs that do that?)
-MarkM-
(Are you maybe thinking something like an IRC bot or "expect" program or MUD macro-capable client listening to and typing back to the server's FIFO? Are there already existing scripts/programs that do that?)
-MarkM-
Re: Persistent Worlds
There is a sample lobby bot written by norbert (http://www.wesnoth.org/forum/viewtopic. ... 95#p425995)markm wrote:I am not sure what you mean by a bot. I have heard of bots for the multiplayer server lounge but was not aware of bots being able to observe interior details of games such as how much money everyone has what they currently have on their recall list what their list of possible recruits is or anything like that. Are there any skeletons frameworks examples etc of such bots? Or are they a new concept you have come up with as the elegant solution to this but that are such a new idea nothing even close to them has yet been attempted?
I am not aware of any in-game bots. So, yes, I suppose that this hypothetical bot I am describing is a new concept. The bot would need to be gamestate-aware. In other words, it would need to reuse much of the existing codebase for the wesnoth client so that it can maintain a consistent gamestate.
IMO, it would be much easier to write it as a modification to the existing wesnoth client than to try to use norbert's version.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Re: Persistent Worlds
Thank you, that makes sense. But if I am therefore to do it by means of the client maybe a first step could be to simply run a client in a save game every turn kind of mode, and have a separate task on the machine run a makefile loop so each time it sees a new savefile it decides it is time to rebuild the things that depend upon savefiles. Such as a database of sides remembering what units each side on each map has in it's recall list, how much gold it has, what it's name/username is, and what password the player wishes to have to be given in order to load that side.
Then make in the WML of the scenarios or era or whatever a routine players can use to designate a new leader, moving their current leader to their recall list; and a command to move a specified unit to their recall list; and a command to use on keeps or inns or some such places for logging off leaving your recall list stashed at that specific keep/inn and turning your units still on the map over to an AI.
The initial goal would be to enable a player to join a map, specify a username and password of a side inned/keeped/kept on that map, and load that side.
-MarkM-
Then make in the WML of the scenarios or era or whatever a routine players can use to designate a new leader, moving their current leader to their recall list; and a command to move a specified unit to their recall list; and a command to use on keeps or inns or some such places for logging off leaving your recall list stashed at that specific keep/inn and turning your units still on the map over to an AI.
The initial goal would be to enable a player to join a map, specify a username and password of a side inned/keeped/kept on that map, and load that side.
-MarkM-
Re: Persistent Worlds
Some of the pieces which are needed for persistent words are now being developed ( by upthorn ), as part of summer of code project.
The first part is the support of 'persistent WML variables' for wesnoth, to allow any wesnoth client to save/load state to local disk.
SP prototype has been already done by Upthorn, MP variant of it would be done, too.
(see http://wiki.wesnoth.org/SoC_Ideas_Persistent_Gameworld http://wiki.wesnoth.org/PersistenceWML and http://wiki.wesnoth.org/SummerOfCodePro ... odyNorthup for more info)
Then, as another summer of code project, we're ( billynux ) developing an alternate network library for wesnoth which would allow a client to join to wesnoth's MP server multiple times. This will allow the things to work that way:
* everyone is using stock wesnoth mp server
* everyone is using stock wesnoth client
* we will code and launch a special 'client', called 'the game master', which will connect to wesnoth's mp server, and act like a 'chat bot' in a dedicated channel, creating wesnoth's games on demand. Of course, it would be able to create games based on the 'state of the world' it would store locally. such a gamemaster would either take part in each and every game related to that persistent work, or grab replays via a side channel (e.g. web upload form or something else).
This would allow us to store a gameworld state on one machine (say, in a SQL db), and use it to host games on demand, amending the world state depending on the results of games.
The first part is the support of 'persistent WML variables' for wesnoth, to allow any wesnoth client to save/load state to local disk.
SP prototype has been already done by Upthorn, MP variant of it would be done, too.
(see http://wiki.wesnoth.org/SoC_Ideas_Persistent_Gameworld http://wiki.wesnoth.org/PersistenceWML and http://wiki.wesnoth.org/SummerOfCodePro ... odyNorthup for more info)
Then, as another summer of code project, we're ( billynux ) developing an alternate network library for wesnoth which would allow a client to join to wesnoth's MP server multiple times. This will allow the things to work that way:
* everyone is using stock wesnoth mp server
* everyone is using stock wesnoth client
* we will code and launch a special 'client', called 'the game master', which will connect to wesnoth's mp server, and act like a 'chat bot' in a dedicated channel, creating wesnoth's games on demand. Of course, it would be able to create games based on the 'state of the world' it would store locally. such a gamemaster would either take part in each and every game related to that persistent work, or grab replays via a side channel (e.g. web upload form or something else).
This would allow us to store a gameworld state on one machine (say, in a SQL db), and use it to host games on demand, amending the world state depending on the results of games.
Re: Persistent Worlds
Nice! Thanks for the links!
Looks like I can continue right along with my current method (storing persistent data in savegames of two scenarios you can freely move back and forth between so as to be able to move from old version of one to new version of the other then back to the new version of the one you started that little dance in) until 1.9 or 1.10 or whatever brings the ability to smoothly migrate the data accumulated in those frequently-updated scenarios over to the newfangled persistence system. Cool.
I guess once the persistence is in place everyone will maybe start wanting to store pretty much their entire campaign (actually, entire world or many worlds in some/many cases) outside of their campaign/scenario entirely, so their so called campaign or scenario becomes actually pretty much just a client interfacing to their persistent world stored elsewhere entirely...
Edit: I guess I'd better start trying to find out what has to change to turn a single-player such campaign into multi-player whilst preserving each player's unique backstory even in multiplayer mode. Simplest likely will be to first migrate the data to the persistence system I suspect...
-MarkM-
EDIT:
Okay, so far so good. Once the persistence system is in place we should be able to know which maps are currently being played, whereupon we will need a way of resolving contention over maps. If two maps that are neighbours are being played and both players desire to enter multiplayer mode to interact, we need to be able to go to multiplayer. Given enough worlds and maybe more importantly enough spots of specific interest / usefulness to players we could find vast numbers of players are perfectly happy to play on maps that are in little or no danger of being encroached upon by other players for at least an hour or few. But the need will always exist to accomodate people both of whom wish to go to multiplayer mode and also to handle situations where one of more of the players prefers not to go to multiplayer mode.
One approach I have long been considering is to maintain buffer zones around all the maps. Basically each map would be surrounded by maps that are not normally played. The terrain that is normally, in singleplayer campaigns, crossed in storyscreen mode.
That way if for example there is a player on the map northeast of you, in practice we would treat it as three away northeast, with you having one map northeast of you as bufferzone and them having one map southwest of them as bufferzone.
These bufferzones could be used in singleplayer mode by deploying NPC (Non-Player Character) troops to them to slow down anyone who might try to approach. You could pick one of your units to send there as a leader, giving them a budget and possibly being able to limit their recruiting choices to only some of the types of units you have available in the region. (This is one of the places Freeciv might come into it: each Freeciv unit is very very large by Wesnoth standards, usually being probably at least several hundred Wesnoth units in size/strength. The presence of a Freeciv unit on a tile of the worldmap would provide Wesnoth-scale players (of that nation/faction and with the required military rank or other qualifications - such as gold in the case of Pirate or Barbarian units quite likely) the ability to recruit units of the Wesnoth-unit types that correspond to the type of nearby Freeciv units. Thus one of the functions of the Freeciv-level players will be designating the districts in which the types of troops they have units of are available for recruiting by their Wesnoth-scale leaders/players.)
These bufferzones would actually be a kind of "scenario creation by means of play" system, because in effect by sending troops out to your bufferzones you are creating scenarios that would have to be played through by anyone attempting to reach the map you are on.
I have not noticed any WML relating to actual earth-time clock/calendar; presumably though lua has access to system time? I am thinking it might be desireable to do some synchronisation to the player's calendar, so they could know for example no-one is going to be able to complete more than X number of moves by noon next sunday, stuff like that...
-MarkM-
Looks like I can continue right along with my current method (storing persistent data in savegames of two scenarios you can freely move back and forth between so as to be able to move from old version of one to new version of the other then back to the new version of the one you started that little dance in) until 1.9 or 1.10 or whatever brings the ability to smoothly migrate the data accumulated in those frequently-updated scenarios over to the newfangled persistence system. Cool.
I guess once the persistence is in place everyone will maybe start wanting to store pretty much their entire campaign (actually, entire world or many worlds in some/many cases) outside of their campaign/scenario entirely, so their so called campaign or scenario becomes actually pretty much just a client interfacing to their persistent world stored elsewhere entirely...
Edit: I guess I'd better start trying to find out what has to change to turn a single-player such campaign into multi-player whilst preserving each player's unique backstory even in multiplayer mode. Simplest likely will be to first migrate the data to the persistence system I suspect...
-MarkM-
EDIT:
Okay, so far so good. Once the persistence system is in place we should be able to know which maps are currently being played, whereupon we will need a way of resolving contention over maps. If two maps that are neighbours are being played and both players desire to enter multiplayer mode to interact, we need to be able to go to multiplayer. Given enough worlds and maybe more importantly enough spots of specific interest / usefulness to players we could find vast numbers of players are perfectly happy to play on maps that are in little or no danger of being encroached upon by other players for at least an hour or few. But the need will always exist to accomodate people both of whom wish to go to multiplayer mode and also to handle situations where one of more of the players prefers not to go to multiplayer mode.
One approach I have long been considering is to maintain buffer zones around all the maps. Basically each map would be surrounded by maps that are not normally played. The terrain that is normally, in singleplayer campaigns, crossed in storyscreen mode.
That way if for example there is a player on the map northeast of you, in practice we would treat it as three away northeast, with you having one map northeast of you as bufferzone and them having one map southwest of them as bufferzone.
These bufferzones could be used in singleplayer mode by deploying NPC (Non-Player Character) troops to them to slow down anyone who might try to approach. You could pick one of your units to send there as a leader, giving them a budget and possibly being able to limit their recruiting choices to only some of the types of units you have available in the region. (This is one of the places Freeciv might come into it: each Freeciv unit is very very large by Wesnoth standards, usually being probably at least several hundred Wesnoth units in size/strength. The presence of a Freeciv unit on a tile of the worldmap would provide Wesnoth-scale players (of that nation/faction and with the required military rank or other qualifications - such as gold in the case of Pirate or Barbarian units quite likely) the ability to recruit units of the Wesnoth-unit types that correspond to the type of nearby Freeciv units. Thus one of the functions of the Freeciv-level players will be designating the districts in which the types of troops they have units of are available for recruiting by their Wesnoth-scale leaders/players.)
These bufferzones would actually be a kind of "scenario creation by means of play" system, because in effect by sending troops out to your bufferzones you are creating scenarios that would have to be played through by anyone attempting to reach the map you are on.
I have not noticed any WML relating to actual earth-time clock/calendar; presumably though lua has access to system time? I am thinking it might be desireable to do some synchronisation to the player's calendar, so they could know for example no-one is going to be able to complete more than X number of moves by noon next sunday, stuff like that...
-MarkM-