Optimisation Discussions Advises Experiences Ideas Etc
Moderator: Forum Moderators
Optimisation Discussions Advises Experiences Ideas Etc
Some of us have created eras or modifications. They work.
But there may be not obvious problems.. The code is working fine! But...
Maybe you can not reload your saved game because save file code is taking too much space..
Maybe your random map generator takes ages to generate map and most users leave before map was even loaded. Maybe something works just too slow.. Why?
There are at least 3 ways to do same things when you code the era or mod.
1) Using Macros
2) Using Fire Events
3) Using Lua
In my opinion they all have problems.. Lua might be problem-free if you know it at expert level, which I don't even after spending couple years coding some wesnoth eras/mods. And I still would not use lua alone to code anything..
So In here I will throw my experiences and comparison when trying to optimise something using these 3 methods.. Real speed difference.. real megabite per savefile difference.. etc
But there may be not obvious problems.. The code is working fine! But...
Maybe you can not reload your saved game because save file code is taking too much space..
Maybe your random map generator takes ages to generate map and most users leave before map was even loaded. Maybe something works just too slow.. Why?
There are at least 3 ways to do same things when you code the era or mod.
1) Using Macros
2) Using Fire Events
3) Using Lua
In my opinion they all have problems.. Lua might be problem-free if you know it at expert level, which I don't even after spending couple years coding some wesnoth eras/mods. And I still would not use lua alone to code anything..
So In here I will throw my experiences and comparison when trying to optimise something using these 3 methods.. Real speed difference.. real megabite per savefile difference.. etc
Re: Optimisation Discussions Advises Experiences Ideas Etc
MACRO vs FIRE_EVENT in random map generation cycle.. version 1.12 of course..
TIMING MACRO MMBE_RANDOMIZE_CURRENT_HEX
When hosting a game, the difference between check time before and after was 3112, 3058 and it felt quick..
TIMING FIRE EVENT mmbe_randomize_current_hex
When hosting a game, the difference between check time before and after was 23563, 25496 and it felt SLOW..
So hosting 2 times Isar Cross with macro and fire_event has shown that same code with macro is nearly 10 times quicker than same code with fire_event when used in cycle.. FOREACH, where mmbe hex is every hex of Isar cross with some exclusions.. (for example hexes with castles or villages not getting randomized so they are not included into mmbe_hex array.. as per settings)
And that's the code for fire_event {FIRE_EVENT mmbe_randomize_current_hex} or {MMBE_RANDOMIZE_CURRENT_HEX} macro. If you having difficulties to read it due to bad coding culture, please don't, I code bad, this is a long macro.. and i will spend a lot of time changing it to your preference. While it really doesn't matter... the only thing that matters in my opionion is that you should use fire_events everywhere you can EXCEPT when you doing a HUGE calculation that requires FAST SPEED.
If anyone has questions, for example would like me to provide all macros I used, feel free to ask. For now it seems to be obvious that macro wins speed race vs fire_event at least in certain circumstances (I think most of the time macro is faster, specially in case if there are not too many macros used and if the whole code of whole era is not too big). I will modify my codes into lua some time later and we will see how much quicker it would do same thing in lua.. But I already seen how same things may take 1 sek in lua, while macros would last at least 5 sek.. so I expect to see a big speed gain, but let's not talk about it until it's tested..
TIMING MACRO MMBE_RANDOMIZE_CURRENT_HEX
Code: Select all
[set_variable]
name=check_time_before
time=stamp
[/set_variable]
{FOREACH mmbe_hex i}
{MMBE_RANDOMIZE_CURRENT_HEX}
{NEXT i}
[set_variable]
name=check_time_after
time=stamp
[/set_variable]
TIMING FIRE EVENT mmbe_randomize_current_hex
Code: Select all
[set_variable]
name=check_time_before
time=stamp
[/set_variable]
{FOREACH mmbe_hex i}
{FIRE_EVENT mmbe_randomize_current_hex}
{NEXT i}
[set_variable]
name=check_time_after
time=stamp
[/set_variable]
So hosting 2 times Isar Cross with macro and fire_event has shown that same code with macro is nearly 10 times quicker than same code with fire_event when used in cycle.. FOREACH, where mmbe hex is every hex of Isar cross with some exclusions.. (for example hexes with castles or villages not getting randomized so they are not included into mmbe_hex array.. as per settings)
And that's the code for fire_event {FIRE_EVENT mmbe_randomize_current_hex} or {MMBE_RANDOMIZE_CURRENT_HEX} macro. If you having difficulties to read it due to bad coding culture, please don't, I code bad, this is a long macro.. and i will spend a lot of time changing it to your preference. While it really doesn't matter... the only thing that matters in my opionion is that you should use fire_events everywhere you can EXCEPT when you doing a HUGE calculation that requires FAST SPEED.
Spoiler:
Re: Optimisation Discussions Advises Experiences Ideas Etc
Main benefit of fire_event is not duplicating your code to all places where you call it. If you only use the code from one place, then macro doesnt hurt you.
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Optimisation Discussions Advises Experiences Ideas Etc
More specifically: not duplicating your code after preprocessing is done. If all you care about is not having duplicate code for the same functionality everywhere, then macros are fine. The risk then becomes that your WML ends up taking up MBs of space after preprocessing is done, but if you don't care about that, then it isn't much of a risk.Ravana wrote:Main benefit of fire_event is not duplicating your code to all places where you call it. If you only use the code from one place, then macro doesnt hurt you.
What would be somewhat interesting though, is if there was a way to load a particular chunk of WML from the cache, rather than keeping all WML in the save file. That would at least prevent the save files themselves from taking up a huge amount of space.
-edit-
Also, my understanding as far as lua goes, is that it generally isn't any more performant than WML. It's strength is that it can do a lot more than WML in certain cases (things involving string manipulation, for example), and that it can be used to create new WML tags.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Optimisation Discussions Advises Experiences Ideas Etc
This greatly depnds on what you are trying to do, exspecialyl very basic tags ([set_variable], [while] etc.) are much slower than their lua conterparts, you can see this easily since thoe tags are implemntted in lua, for example [set_varialbe] name,add=a,5 [/set_varialbe] is equivalent to the lua codePentarctagon wrote: Also, my understanding as far as lua goes, is that it generally isn't any more performant than WML.
Code: Select all
function wesnoth.wml_actions.set_variable(cfg)
local name = cfg.name or helper.wml_error "trying to set a variable with an empty name"
if cfg.value ~= nil then -- check for nil because user may try to set a variable as false
wesnoth.set_variable(name, cfg.value)
end
if cfg.literal ~= nil then
wesnoth.set_variable(name, helper.shallow_literal(cfg).literal)
end
if cfg.to_variable then
wesnoth.set_variable(name, wesnoth.get_variable(cfg.to_variable))
end
if cfg.add then
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) + (tonumber(cfg.add) or 0))
end
if cfg.sub then
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) - (tonumber(cfg.sub) or 0))
end
if cfg.multiply then
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) * (tonumber(cfg.multiply) or 0))
end
if cfg.divide then
local divide = tonumber(cfg.divide) or 0
if divide == 0 then helper.wml_error("division by zero on variable " .. name) end
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) / divide)
end
if cfg.modulo then
local modulo = tonumber(cfg.modulo) or 0
if modulo == 0 then helper.wml_error("division by zero on variable " .. name) end
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) % modulo)
end
if cfg.abs then
wesnoth.set_variable(name, math.abs(tonumber(wesnoth.get_variable(name)) or 0))
end
if cfg.root then
if cfg.root == "square" then
local radicand = tonumber(wesnoth.get_variable(name)) or 0
if radicand < 0 then helper.wml_error("square root of negative number on variable " .. name) end
wesnoth.set_variable(name, math.sqrt(radicand))
end
end
if cfg.power then
wesnoth.set_variable(name, (tonumber(wesnoth.get_variable(name)) or 0) ^ (tonumber(cfg.power) or 0))
end
if cfg.round then
local var = tonumber(wesnoth.get_variable(name) or 0)
local round_val = cfg.round
if round_val == "ceil" then
wesnoth.set_variable(name, math.ceil(var))
elseif round_val == "floor" then
wesnoth.set_variable(name, math.floor(var))
else
local decimals, discarded = math.modf(tonumber(round_val) or 0)
local value = var * (10 ^ decimals)
value = helper.round(value)
value = value * (10 ^ -decimals)
wesnoth.set_variable(name, value)
end
end
-- unlike the other math operations, ipart and fpart do not act on
-- the value already contained in the variable
-- but on the value assigned to the respective key
if cfg.ipart then
local ivalue, fvalue = math.modf(tonumber(cfg.ipart) or 0)
wesnoth.set_variable(name, ivalue)
end
if cfg.fpart then
local ivalue, fvalue = math.modf(tonumber(cfg.fpart) or 0)
wesnoth.set_variable(name, fvalue)
end
if cfg.string_length ~= nil then
wesnoth.set_variable(name, string.len(tostring(cfg.string_length)))
end
if cfg.time then
if cfg.time == "stamp" then
wesnoth.set_variable(name, wesnoth.get_time_stamp())
end
end
if cfg.rand then
wesnoth.set_variable(name, helper.rand(tostring(cfg.rand)))
end
local join_child = helper.get_child(cfg, "join")
if join_child then
local array_name = join_child.variable or helper.wml_error "missing variable= attribute in [join]"
local separator = join_child.separator
local key_name = join_child.key or "value"
local remove_empty = join_child.remove_empty
local string_to_join = ''
for i, element in ipairs(helper.get_variable_array(array_name)) do
if element[key_name] ~= nil or (not remove_empty) then
if #string_to_join > 0 then
string_to_join = string_to_join .. separator
end
string_to_join = string_to_join .. element[key_name]
end
end
wesnoth.set_variable(name, string_to_join)
end
end
wesnoth.wml_actions.set_variable {name="a", add=1}
a = a + 5
Another case is when you are changing units stats, in wml this often requires storing and unstoring the unit which is a very slow process since all the attributes need to be first saved and then the units is complteley rebuild, even if you just wanted to change one attribute (like hitpoints). In lua you can changw those attributes directly witout havng to unstore/store anything.
A case where it won't match that much are tags that are more or less quivelent to their internal lua function, for example [store_locations], whether lua or wml, the filter will usually be applied to all locations on the initial set and the effecieness of the filter is much more important then the potential overhead of using wml here.
EDIT: It would be nice to have a [wmlcode] phpbb tag that would bhave like
Code: Select all
but automatically run wmlindent on it.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
Re: Optimisation Discussions Advises Experiences Ideas Etc
That isn't about "MACRO vs FIRE_EVENT". You're using a macro in both cases in the first place.enclave wrote:MACRO vs FIRE_EVENT in random map generation cycle.. version 1.12 of course..
As was already said, almost always the point of using [fire_event] "instead of a macro" is to avoid code duplication. In your example there is no code duplication. Your example is a demonstration only of how [fire_event] does have an inherent performance cost, meaning that it might not be a good idea to use it in a loop. Which is certainly a good thing to be aware of when writing your code, but it doesn't have anything to do with macros.
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Optimisation Discussions Advises Experiences Ideas Etc
It sounds like what would be a more interesting test then, would be the difference between using the VARIABLE macro vs inline lua usinggfgtdf wrote:...words...
wesnoth.set_variable
.99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Optimisation Discussions Advises Experiences Ideas Etc
Or would it make sense to define VARIABLE to use lua in the first place.
Re: Optimisation Discussions Advises Experiences Ideas Etc
Pentarctagon wrote:It sounds like what would be a more interesting test then, would be the difference between using the VARIABLE macro vs inline lua usinggfgtdf wrote:...words...wesnoth.set_variable
.
well inline [lua] would need to compile the lua at runtime first, so it's unclear whether [lua] woudl be faster than [set_variable] for that, but note that you (or rather i) usually don't replace single tags with lua but the whole functions/event . That's also why i compared it to a simple
a = a + 5
since it's quite likleley that a
would become a local variable if you port the whole event/function to lua.Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Optimisation Discussions Advises Experiences Ideas Etc
Which is why it would make an interesting test. So, something like:gfgtdf wrote:well inline [lua] would need to compile the lua at runtime first, so it's unclear whether [lua] woudl be faster than [set_variable] for that, but note that you (or rather i) usually don't replace single tags with lua but the whole functions/event . That's also why i compared it to a simplea = a + 5
since it's quite likleley thata
would become a local variable if you port the whole event/function to lua.
- VARIABLE
- inline lua
- some sort of WML tag like [wesnoth_set_variable] that only calls
wesnoth.set_variable
rather than doing all the other checks like [set_variable] does
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Optimisation Discussions Advises Experiences Ideas Etc
wow thats a big conversation here anyone would like to share results? I wouldnt be able to do many lua tests, because i'm too bad at them. I will only write my particular MACRO vs FIRE_EVENT code with a bit of lua soon when I get a time for it. If somebody could share your example would be really cool
I think I demonstrated quite good a particular case when macro and fire_event are used to represent the same code as alternative to a function/procedure. Obviously they all contain macros.. and I think it's even more obvious that if I replaced every single macro of that code with [fire_event] then performance would be decreased progressively. I think you agreed with that in your text.. But I still don't understand what you were trying to say..
Sorry I can't understand your text.. or point behind your text.zookeeper wrote: That isn't about "MACRO vs FIRE_EVENT". You're using a macro in both cases in the first place.
Your example is a demonstration only of how [fire_event] does have an inherent performance cost, meaning that it might not be a good idea to use it in a loop. Which is certainly a good thing to be aware of when writing your code, but it doesn't have anything to do with macros.
I think I demonstrated quite good a particular case when macro and fire_event are used to represent the same code as alternative to a function/procedure. Obviously they all contain macros.. and I think it's even more obvious that if I replaced every single macro of that code with [fire_event] then performance would be decreased progressively. I think you agreed with that in your text.. But I still don't understand what you were trying to say..
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Optimisation Discussions Advises Experiences Ideas Etc
It's also probably worth mentioning that there's no runtime performance penalty to using macros vs just writing out the WML yourself. The preprocessor does all the macro substitution beforehand and caches it. This is why, for example, the WML I've written for my add-on is ~9.8k lines, while the cached WML from my add-on after the preprocessor runs is over 63k lines.enclave wrote:wow thats a big conversation here anyone would like to share results? I wouldnt be able to do many lua tests, because i'm too bad at them. I will only write my particular MACRO vs FIRE_EVENT code with a bit of lua soon when I get a time for it. If somebody could share your example would be really cool
Sorry I can't understand your text.. or point behind your text.zookeeper wrote: That isn't about "MACRO vs FIRE_EVENT". You're using a macro in both cases in the first place.
Your example is a demonstration only of how [fire_event] does have an inherent performance cost, meaning that it might not be a good idea to use it in a loop. Which is certainly a good thing to be aware of when writing your code, but it doesn't have anything to do with macros.
I think I demonstrated quite good a particular case when macro and fire_event are used to represent the same code as alternative to a function/procedure. Obviously they all contain macros.. and I think it's even more obvious that if I replaced every single macro of that code with [fire_event] then performance would be decreased progressively. I think you agreed with that in your text.. But I still don't understand what you were trying to say..
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Optimisation Discussions Advises Experiences Ideas Etc
I had serious optimization problems with my add-on written without fire events.. People could not load the savefile because it was exceeding 40mb maximum allowed size.. So basically I had to convert a lot of my macros into fire events to try to reduce the savefile size. After reading about optimization in wiki it felt like fire_event is the answer to everything.. but turns out that if you need speed, then fire_event plays against you in certain circumstances and it was a big surprise for me to find it out.. therefore I think it's very important to talk about it
- Pentarctagon
- Project Manager
- Posts: 5564
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: Optimisation Discussions Advises Experiences Ideas Etc
In which case, it'd be good to specify which type of optimization you're talking about - speed or size. Macros are better in terms of speed, but can take up additional space. [fire_event] reduces the amount of space needed for save files, but has a performance penalty.enclave wrote:I had serious optimization problems with my add-on written without fire events.. People could not load the savefile because it was exceeding 40mb maximum allowed size.. So basically I had to convert a lot of my macros into fire events to try to reduce the savefile size. After reading about optimization in wiki it felt like fire_event is the answer to everything.. but turns out that if you need speed, then fire_event plays against you in certain circumstances and it was a big surprise for me to find it out.. therefore I think it's very important to talk about it
-edit-
Did a quick test:
Code: Select all
[set_variable]
name=start
time=stamp
[/set_variable]
[for]
start=0
end=500000
step=1
[do]
{VARIABLE test 5}
[/do]
[/for]
[set_variable]
name=end
time=stamp
[/set_variable]
[message]
message="$start - $end"
[/message]
7017
and an end of 11502
, for a difference of 4485
.Code: Select all
[set_variable]
name=start
time=stamp
[/set_variable]
[for]
start=0
end=500000
step=1
[do]
[lua]
code=<<
wesnoth.set_variable("test", 5)
>>
[/lua]
[/do]
[/for]
[set_variable]
name=end
time=stamp
[/set_variable]
[message]
message="$start - $end"
[/message]
6609
and an end of 10308
, for a difference of 3699
.Code: Select all
function wesnoth.wml_actions.set_test_variable(var)
wesnoth.set_variable(var.variable, 5)
end
Code: Select all
[set_variable]
name=start
time=stamp
[/set_variable]
[for]
start=0
end=500000
step=1
[do]
[set_test_variable]
variable=test
[/set_test_variable]
[/do]
[/for]
[set_variable]
name=end
time=stamp
[/set_variable]
[message]
message="$start - $end"
[/message]
6765
and an end of 9171
, for a difference of 2406
.So directly calling
wesnoth.set_variable
via a simpler WML tag was close to 2x faster, but given that it took 500,000 calls to create a difference of ~2 seconds, actually writing code this way seems beyond even micro-optimization in most cases.99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: Optimisation Discussions Advises Experiences Ideas Etc
Nice
Have you tried multiple times? Because there is always a chance that your computer was performing some other tasks at the background.. and at the point of testing the performance of wesnoth could be decreased by tiny bit, enough to change the results. I will actually copy your code and test the same, just curiosity of
Have you tried multiple times? Because there is always a chance that your computer was performing some other tasks at the background.. and at the point of testing the performance of wesnoth could be decreased by tiny bit, enough to change the results. I will actually copy your code and test the same, just curiosity of