why do we need 2 random generators?

[gfgtdf]

hi i think everyone knows about this: there are 2 synced random generators in wesnoth: one is used for [ser_variable]rand=, traits and more and the second is used for attack results + more, the first one is deterministic while the second one is not and will be changed with save + reload.

i would appreciate to remove the second use the first whenever possible for the following reasons:

• i dont like the non deterministicness: when i forgot something during my last move i reload the game and the result is eigher that i get worse random result making me fell betrayed by the game or getting better random results making me fell like i cheated.
• i think it would make the c++ code easier.

a drawback would be that save and reload for changing attack resulty would be impossible (in such a case everything that gives the player less options is a drawback), but i think that could be very easily fixed with a "allow player reload to change random results" addon at least in sp/local games.

note that this would not mean having less random during normal game.

so here is my question:
for players: do you think making the attacks random determintic woud be good?
for devs: are there other techincal reasons i haven't seen for the second random generator to exist?
ofc. everyone can answer both questions.



EDIT:
state of disussion:
the main problem for seeeded random variables is, that players could save a game while playing a mp game and load them in a singleplayer game to find out "what would happen if".
this is currently (=1.11.6) possible for traits but not for attack results,
so the current suggestion is, to have both traits and attack results nondeterminisitc. Similar to the current attack results.


EDIT2:

Spoiler:

there is currently a bug with the attack events which is very related with this topic: http://gna.org/bugs/?20871
i think if i change all randoms to behave like the "attacks random", the bug might also apply for ALL events, that would be a big desaster.
i originaly came to this topic while investigating the bug 20871
but it's better do it in the opposite order meaning fixing 20871 before handeling this issue.

[Dixie]

Well I wouldn't be against the game being more deterministic, seeing how a while back I tried creating a mod that would prevent save-loading. But I could imagine how many people would be against it.

[tekelili]

I am not sure if I understood it well: Do your proposal means than when a game is saved to be continued later (like in a tgt tournament), players can load game to check what attacks outcomes they will get in next turn following some secuence? If thats case, and I am for it, of course.

[gfgtdf]

I am not sure if I understood it well: Do your proposal means than when a game is saved to be continued later (like in a tgt tournament), players can load game to check what attacks outcomes they will get in next turn following some secuence? If thats case, and I am for it, of course.

good point, yes i tihnk it would mean that. I havent thought about that.
but that argument is true aswell for the traits and [set_variable]rand= , while traits have a rather little influence on the game compared to attack results, the
[set_variable]rand= can have a big influence in certain umc scenarios.
i still think it is a good idea to have only one random number generator. Maybe let the players decide when loading load game weather to reinitialize the radom generator or not, that would also solve the other problem.

[taptap]

It would merely change saveloading style. People will try different sequencing of actions. (As they do in games with this kind of RNG.) Depending on implementation it may even make saveloading easier as you can collect information about the random numbers by saveloading. You also have no way to check whether occasional saveloads (say when you reload after you forget moving half your army - this can happen fairly easy in Northern Rebirth or similar campaigns) affected your game by way of statistics. Similarly you wouldn't be able to check saves from other players whether they may have changed their fate by saveloading.

Isn't the first one new? Saveloading for traits used to work at some point.

[iceiceice]

Before I begin let me say that I have not read the source code and do not have dev-level knowledge.

hi i think everyone knows about this: there are 2 synced random generators in wesnoth: one is used for [ser_variable]rand=, traits and more and the second is used for attack results + more, the first one is deterministic while the second one is not and will be changed with save + reload.

(Actually I guess that there is a third, the one built into lua which is separate from these two. Sometimes this is necessary because there are some known bugs with [set_variable] rand=.)

Isn't this related to this post? I guess that it means that traits being "deterministic" is new to 1.11. Are you sure that set_variable rand and traits are on the same rng?

Actually I think it makes slightly more sense from an engineering point of view if the rand available to UMC content is distinct from the rand available to built-in features. But yeah it seems better to me also if they are both "deterministic" ("synced" i.e. synced with the replay might be better term?).

If moving traits to the synced RNG is new to 1.11 then maybe your request is already in the works, to move the combat randomness to the synced RNG as well.

i would appreciate to remove the second use the first whenever possible for the following reasons:

• ...
• i think it would make the c++ code easier.

So I would not be so sure about this. The random generation code is very old and I am told there are not so many people still active who have messed around with it. Random generation can really be a pain in software engineering because it is very difficult to test if it is working properly, it is very easy to create "silent" bugs. So most likely the instinct of a dev is to not want to touch this.

But the change you propose is just to replace instances of one RNG with the other so maybe its not so bad.

for players: do you think making the attacks random determintic woud be good?

Yes I think it would be very good for competitive play if you could reload because of a misclick in middle of turn and be guaranteed to get the exact same RNG if you repeat the prior attacks in the same order. At least it would be a "nice feature", possibly others will have their opinions about whether reloading for a misclick is good or bad.

On the other hand, obviously there are campaign players like the OP in the post I linked to who view save-scumming as a feature.

Let me also say that I don't think it is bad if newbies to the game save scum a bit when they first play campaigns, obviously it is not intended in some sense but if you don't really understand how the game works it is best if you can back up when you make a big mistake without realizing it. So maybe the unsynced RNG is there in order to support this essentially.

Edit: Maybe a good solution is to change this over for [multiplayer] type games but not [campaign] type games. In fact while we are at it,

Spoiler:

it would be even better if in multiplayer games, random bits are generated not only by the host but also by the clients, and these results are all xor'd together whenever a bit is needed by any party. As I understand, currently all the randomness is generated by the host, so if the host has hacked his wesnoth client he can cheat by controlling the randomness, and it would be extremely difficult to catch him because he might make it normal except at critical moments. If random bits from host and clients are xor'd then as long as there is one honest party the randomness will be good. (Actually I guess the proper solution is that the host should make a cryptographic bit commitment which he discloses, before the clients give their bits, then he reveals the commitment. I guess that there is a famous paper about this by Manuel Blum.)

But also I guess this conflicts with the current push to put multiplayer and sp campaigns "under the same roof" so to speak...

[gfgtdf]

It would merely change saveloading style. People will try different sequencing of actions. (As they do in games with this kind of RNG.) Depending on implementation it may even make saveloading easier as you can collect information about the random numbers by saveloading. You also have no way to check whether occasional saveloads (say when you reload after you forget moving half your army - this can happen fairly easy in Northern Rebirth or similar campaigns) affected your game by way of statistics. Similarly you wouldn't be able to check saves from other players whether they may have changed their fate by saveloading.

Isn't the first one new? Saveloading for traits used to work at some point.

My intenteion was not to do something against saveloading, in fact i think everyone can play this game how he wants to. I have said my inention i the first post.
(ok my original intention was to fix http://gna.org/bugs/?20871, but i looked at the source and this topic realy bothers me.)
Yes saveloading for traits worked at 1.10 but not in 1.11 anymore.

Before I begin let me say that I have not read the source code and do not have dev-level knowledge.


(Actually I guess that there is a third, the one built into lua which is separate from these two. Sometimes this is necessary because there are some known bugs with [set_variable] rand=.)

yes there is third, but i am only talking about synced random generators (which retun the same value on all clients).

Isn't this related to this post? I guess that it means that traits being "deterministic" is new to 1.11. Are you sure that set_variable rand and traits are on the same rng?

yes they are they are the same, its clear from the sourcecode.

Actually I think it makes slightly more sense from an engineering point of view if the rand available to UMC content is distinct from the rand available to built-in features.

why?

But yeah it seems better to me also if they are both "deterministic" ("synced" i.e. synced with the replay might be better term?).

If moving traits to the synced RNG is new to 1.11 then maybe your request is already in the works, to move the combat randomness to the synced RNG as well.

maybe, that was changed in 2010, so quiet some thime ago. I doubt someone it working 3 years on one part of the code.

So I would not be so sure about this. The random generation code is very old and I am told there are not so many people still active who have messed around with it. Random generation can really be a pain in software engineering because it is very difficult to test if it is working properly, it is very easy to create "silent" bugs. So most likely the instinct of a dev is to not want to touch this.

But the change you propose is just to replace instances of one RNG with the other so maybe its not so bad.

i just don't see not reason to have 2 random generators, and there are some parts of the code that are quiet confusing.

Yes I think it would be very good for competitive play if you could reload because of a misclick in middle of turn and be guaranteed to get the exact same RNG if you repeat the prior attacks in the same order. At least it would be a "nice feature", possibly others will have their opinions about whether reloading for a misclick is good or bad.

On the other hand, obviously there are campaign players like the OP in the post I linked to who view save-scumming as a feature.

Let me also say that I don't think it is bad if newbies to the game save scum a bit when they first play campaigns, obviously it is not intended in some sense but if you don't really understand how the game works it is best if you can back up when you make a big mistake without realizing it. So maybe the unsynced RNG is there in order to support this essentially.

it would be even better if in multiplayer games, random bits are generated not only by the host but also by the clients, and these results are all xor'd together whenever a bit is needed by any party. As I understand, currently all the randomness is generated by the host, so if the host has hacked his wesnoth client he can cheat by controlling the randomness, and it would be extremely difficult to catch him because he might make it normal except at critical moments. If random bits from host and clients are xor'd then as long as there is one honest party the randomness will be good. (Actually I guess the proper solution is that the host should make a cryptographic bit commitment which he discloses, before the clients give their bits, then he reveals the commitment. I guess that there is a famous paper about this by Manuel Blum.)

But also I guess this conflicts with the current push to put multiplayer and sp campaigns "under the same roof" so to speak...

the deterministic random is generated on all cleints independently which is possible because it is deterministic.

the non deterministic random for attacks is currently generated at the server (if i remember correctly).

[iceiceice]

Actually I think it makes slightly more sense from an engineering point of view if the rand available to UMC content is distinct from the rand available to built-in features.

why?

Because then if the UMC uses the rng badly or creates or problem it won't mess up built-in behaviors, and will only mess up UMC content? This will cause more useful error messages for one. It is better generally in software if different modules share only what they need to for this reason.

For example I remember reading that OOS can be caused by UMC if one machine queries from rand= more times than another machine. If the UMC and built-in use the same rng then this OOS may occur when some arbitrary unit attacks another unit, as opposed to being prompted by something the UMC was trying to do. So it would be harder to debug.

It might be that separating these will sometimes prevent OOS. If the UMC rng's get out of sync at some point by mistake but the UMC is done affecting the game state, then the game can still continue since the built-in rng are still in sync.

[gfgtdf]

Because then if the UMC uses the rng badly or creates or problem it won't mess up built-in behaviors, and will only mess up UMC content? This will cause more useful error messages for one. It is better generally in software if different modules share only what they need to for this reason.

For example I remember reading that OOS can be caused by UMC if one machine queries from rand= more times than another machine. If the UMC and built-in use the same rng then this OOS may occur when some arbitrary unit attacks another unit, as opposed to being prompted by something the UMC was trying to do. So it would be harder to debug.

It might be that separating these will sometimes prevent OOS. If the UMC rng's get out of sync at some point by mistake but the UMC is done affecting the game state, then the game can still continue since the built-in rng are still in sync.

the case that "UMC is done affecting the game state" is quiet rarely.
plus, note that even in 1.10 where they are spereatd, the OOS error messages arent that useful in most cases.
plus, i think if you realy want, you can add some debug features for better oos detection, like remembering how often random was called.
note that currenty they arent seperated.

[iceiceice]

You are right that it is quite rare that this would matter. I don't think it is very important, all I am saying is that if it were my personal project I would likely code it this way, because I would want to sandbox UMC content away from core like this to some extent. Not to make the game harder to mod, just so that it is easier for my part of the code to make guarantees etc.

[Alarantalara]

I am not sure if I understood it well: Do your proposal means than when a game is saved to be continued later (like in a tgt tournament), players can load game to check what attacks outcomes they will get in next turn following some secuence? If thats case, and I am for it, of course.

good point, yes i tihnk it would mean that. I havent thought about that.
but that argument is true aswell for the traits an [set_variable]rand= , while traits have a rather little influence on the game compared to attack results, the
[set_variable]rand= can have a big influence in certain umc scenarios.
i still think it is a good idea to have only one random number generator. Maybe let the players decide when loading load game weather to reinitialize the radom generator or not, that would also solve the other problem.

Determinism would also allow players to load autosaves of multiplayer games in alternate clients and try out attacks. While preventing cheating is impossible, I'm not sure it should become that easy.

[gfgtdf]

good point, yes i tihnk it would mean that. I havent thought about that.
but that argument is true aswell for the traits an [set_variable]rand= , while traits have a rather little influence on the game compared to attack results, the
[set_variable]rand= can have a big influence in certain umc scenarios.
i still think it is a good idea to have only one random number generator. Maybe let the players decide when loading load game weather to reinitialize the radom generator or not, that would also solve the other problem.

Determinism would also allow players to load autosaves of multiplayer games in alternate clients and try out attacks. While preventing cheating is impossible, I'm not sure it should become that easy.

but thats why i suggeted (in the post you quoted, sry i have been unclear) to give the player the option to reinitalise the random generator at scenario load or not, meaning that the player can choose weather to continue with the old random generator or the random generator a new ramdomly picked seed when loading a (mp) scenario.
so if the player coose the second that wouldn't be possible anymore.

Note that in the current impementation the same problem exists with [set_variable]rand= and traits.


EDIT:
I just did some investigations and found out that the functions that calculates teh atack reults also calculates the unit names, but unlike the attack results these are not the same, i normaly would suggest that after the attack they both have the same seed and therefore would return the same value if called for unit names. , it seems like this is not the case i'll have to investigate more.

[Alarantalara]

Do you mean at the start of the scenario? Because if the choice is when loading the saved game, then choosing not to randomize by the person loading the game is exactly the problem I'm describing.

[gfgtdf]

Do you mean at the start of the scenario? Because if the choice is when loading the saved game, then choosing not to randomize by the person loading the game is exactly the problem I'm describing.

well i dont know exactly how mp saveloading is handeled, but the intention was ofc. that that other players can see wether the person loading the game did choose to randomize.

[iceiceice]

Determinism would also allow players to load autosaves of multiplayer games in alternate clients and try out attacks. While preventing cheating is impossible, I'm not sure it should become that easy.

gfgtdf, I don't understand your answer.

Alarantara is suggesting that, if I am in an mp game, and attacks are generated by a synced rng, if it is my turn then I can save and load in a second copy, then attempt an attack I am contemplating and see if I get hits. If I don't then I won't do it in the real game, or I might try a different order or something. The way it currently works this is impossible because when I save load I have a different seed.

Your suggestion that the player have the option on load whether to reset the seed won't help, because the cheater will choose not to reset it. Edit: And if the player from original copy is not in the save loaded copy they won't see this choice either.

Actually this is a big issue -- if the idea was that the two players could reload with the same randomness if they wanted to undo a misclick, then sometimes they will legitimately want to use same rng. So the game would have to know when they are save loading legitimately and when one player is trying to cheat, which seems impossible.

Also Alarantalara's issue applies right now to traits, it may be that cheaters who want to gamble for quick recruits will now save load to see if they will get the quick ones before they spend the money. So maybe this should be fixed somehow, since it is a new and undetectable kind of cheating in 1.11.