Optimisation Discussions Advises Experiences Ideas Etc
Moderator: Forum Moderators
Re: Optimisation Discussions Advises Experiences Ideas Etc
Before it took wmlunits between 20 sec and 60 sec to create unit trees for ageless. So while it had timeout of 60 sec, it worked, but this summer timeout was lowered to 20 sec, so I had to make it load faster.
With current ageless you can see ageless_4_18.preprocessed.cfg which is conditionally loaded instead of usual folder structure(which is still included in addon as zookeeper asked it be so). Such file is generated with wesnoth -p, followed by some regex replaces to make it valid unprocessed wml ("" to <<>> and >><< to ").
Some references
wesnoth -p viewtopic.php?p=613516#p613516
irc https://www.wesnoth.org/irclogs/2017/06 ... -06-03.log
With current ageless you can see ageless_4_18.preprocessed.cfg which is conditionally loaded instead of usual folder structure(which is still included in addon as zookeeper asked it be so). Such file is generated with wesnoth -p, followed by some regex replaces to make it valid unprocessed wml ("" to <<>> and >><< to ").
Some references
wesnoth -p viewtopic.php?p=613516#p613516
irc https://www.wesnoth.org/irclogs/2017/06 ... -06-03.log
Re: Optimisation Discussions Advises Experiences Ideas Etc
Comparing
In previous posts match_locations worked much much much faster than get_locations, when there were complicated filters used.
This time I will try to compare simple filters and speeds..
So this gave me a count of villages on map in about 1-3 miliseconds on Isars Cross... or around 20-50 on Valeria.
This gave me a count of villages on Isar in about 9-30 miliseconds.. and around 210-230 on Valeria
So with very very basic filter it worked nearly 10 times faster to use:
This is actually giving similar values as lua.. 1-5 on isar and 40-50 on Valeria
NEXT: I will try to add more terrains to filter and all the testing will now be on 180x90 Valeria map.
SO..
TEST1
TEST2
TEST3
TEST4
600-630 ms
70-100ms
80-110ms (funny.. max is less than test3.. i guess my PC was under less load during test4)
Ok.. so from these examples get_locations or store_locations working much quicker than iterating over each hex of the map with match_location
Lets try to filter just a 20 square hex area with same filters as in TEST4
So
TEST5
40-70ms
40-70ms
This time iterating over each hex of x,y from 20 to 40 with
In next examples I will try to compare whole map match_locations vs ipairs of get_locations to change each hex of filter matching part of map into something.. For example remove all forest only from hills.. or remove all villages except from water.. or something similar..
To be continued...
wesnoth.get_locations
vs wesnoth.match_location
vs [store_locations]
In previous posts match_locations worked much much much faster than get_locations, when there were complicated filters used.
This time I will try to compare simple filters and speeds..
Code: Select all
local all_villages = wesnoth.get_locations { terrain="*^V*"}
wesnoth.set_variable("counter_66",#all_villages)
Code: Select all
[message]
message="$counter_66"
[/message]
Code: Select all
[lua]
code = <<
local ww,hh,bb = wesnoth.get_map_size()
local counter_66 = 0
local ybegin = 0
while (ybegin<=hh+1)
do
local xbegin = 0
while (xbegin<=ww+1)
do
local bool ns_check_if_village = wesnoth.match_location(xbegin, ybegin, { terrain="*^V*"})
if ns_check_if_village == true then
counter_66 = counter_66 + 1
end
xbegin = xbegin + 1
end
ybegin = ybegin + 1
end
wesnoth.set_variable("counter_66",counter_66)
>>
[/lua]
[message]
message="$counter_66"
[/message]
So with very very basic filter it worked nearly 10 times faster to use:
local all_villages = wesnoth.get_locations { terrain="*^V*"}
Code: Select all
[store_locations]
terrain=*^V*
variable=counter_66
[/store_locations]
[message]
message="$counter_66.length"
[/message]
NEXT: I will try to add more terrains to filter and all the testing will now be on 180x90 Valeria map.
SO..
TEST1
wesnoth.match_location(xbegin, ybegin, { terrain="*^V*"})
210-230 milisecondswesnoth.get_locations { terrain="*^V*"}
20-50 miliseconds[store_locations]terrain="*^V*"
40-50 milisecondsTEST2
wesnoth.match_location(xbegin, ybegin, { terrain="*^V*,*^F*"})
240-280 milisecondswesnoth.get_locations { terrain="*^V*,*^F*"}
20-60 miliseconds[store_locations]terrain="*^V*,*^F*"
60-100 milisecondsTEST3
wesnoth.match_location(xbegin, ybegin, { terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*"})
390-430 milisecondswesnoth.get_locations { terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*"}
30-60 miliseconds[store_locations]terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*"
80-120 milisecondsTEST4
wesnoth.match_location(xbegin, ybegin, { terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*", {"not", {terrain="H*,H*^*,G*,G*^*"}}})
600-630 ms
wesnoth.get_locations { terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*", {"not", {terrain="H*,H*^*,G*,G*^*"}}}
70-100ms
[store_locations]terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*"[not]terrain="H*,H*^*,G*,G*^*"[/not]
80-110ms (funny.. max is less than test3.. i guess my PC was under less load during test4)
Ok.. so from these examples get_locations or store_locations working much quicker than iterating over each hex of the map with match_location
Lets try to filter just a 20 square hex area with same filters as in TEST4
So
TEST5
wesnoth.match_location
20-40 msSpoiler:
wesnoth.get_locations { x="20-40", y="20-40", terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*", {"not", {terrain="H*,H*^*,G*,G*^*"}}}
40-70ms
[store_locations]x=20-40 y=20-40 terrain="*^V*,*^F*,*^Q*,*^G*,*^E*,*^W*,*^D*"[not]terrain="H*,H*^*,G*,G*^*"[/not]
40-70ms
This time iterating over each hex of x,y from 20 to 40 with
match_location
was 2 times quicker than using get_locations
or store_locations
... while lua get_locations
has same speed as WML store_locations
.In next examples I will try to compare whole map match_locations vs ipairs of get_locations to change each hex of filter matching part of map into something.. For example remove all forest only from hills.. or remove all villages except from water.. or something similar..
To be continued...
Re: Optimisation Discussions Advises Experiences Ideas Etc
Remove all villages from map, 3 ways.. comparison [terrain] vs wesnoth.get_locations (wesnoth.set_terrain) vs wesnoth.match_location (wesnoth.set_terrain) vs wesnoth.get_villages (wesnoth.set_terrain)
Somehow
Nice comparison.. Conclusions:
If you working with villages, definitly better use
if filtering smaller area, match_location probably would fit better and may be quickest..
filtering whole map is much better with get_locations..
Might make sense to consider using match_location after narrowing the area by get_locations, depends..
And of course match_location is a more advanced tool than get_locations, if you need to check adjacent hexes or radius for a specific place on map.
[terrain]
130-150ms Spoiler:
wesnoth.match_location & wesnoth.set_terrain
210-240ms Spoiler:
wesnoth.get_locations { terrain="*^V*"} & wesnoth.set_terrain
20-30ms Spoiler:
wesnoth.get_villages() & wesnoth.set_terrain
2ms Spoiler:
wesnoth.get_villages()
working much faster than any other lua filter for villages, might be because of stars used in *^V* filter.. idk.. why..Nice comparison.. Conclusions:
If you working with villages, definitly better use
wesnoth.get_villages()
,if filtering smaller area, match_location probably would fit better and may be quickest..
filtering whole map is much better with get_locations..
Might make sense to consider using match_location after narrowing the area by get_locations, depends..
And of course match_location is a more advanced tool than get_locations, if you need to check adjacent hexes or radius for a specific place on map.
Re: Optimisation Discussions Advises Experiences Ideas Etc
5k times {REPEAT} testing some basic filters for speed (being said that [filter_wml] is slowest filter so I decided to test if it's true..) wesnoth 1.12
-------
1)
2)
3)
4)
Wth... I dont understand the results... I will do same test in WML, its confusing me.. filter_wml was even faster than basic "side=1" filter..
So exactly same tests in WML ([store_unit]):
1) 757 746 752
2) 742 741 744
3) 674 672 666
4) 3034 2999 3030
LOL that is funny... the more simple filter is the slower it seems to be... or maybe it depends on how much data it needs to write inside the WML variable... idk.. In this case the speed of store_unit would depend least on efficiency of the filter... most on efficiency of speed of creation of array.
These above ideally need more experiments, but I cant be bothered at this point of my life sorry..
Unfortunately the tests were carried with non-existant abilities and there was only 9 sides with 1 unit leader on field. So the accuracy is questionable, although the results are not what I expected to see at all. (The filters itself are 99% sure working fine, I tested them on another era, but without timing)
NOW just 20 times {REPEAT} testing some basic store locations filter. (50k too long)
--------
1)
2)
3)
4)
No difference at all with all the stars or without... I mean the difference is nearly non-existant..
then I wonder why lua wesnoth.get_villages is much much more faster than normal terrain filter..
-------
1)
local units_with_upkeep = #wesnoth.get_units { side = 1, canrecruit="no", {"not",{ability="hc_build_new_city"}} }
854 868 880 877 8762)
local units_with_upkeep = #wesnoth.get_units { side = 1, canrecruit="no", {"filter_wml",{upkeep=1}} }
190 188 177 184 1923)
local units_with_upkeep = #wesnoth.get_units { side = 1, canrecruit="no", ability="hc_build_new_city"}
739 742 764 742 7584)
local units_with_upkeep = #wesnoth.get_units { side = 1 }
776 770 774 784 774Wth... I dont understand the results... I will do same test in WML, its confusing me.. filter_wml was even faster than basic "side=1" filter..
So exactly same tests in WML ([store_unit]):
1) 757 746 752
Spoiler:
Spoiler:
Spoiler:
Spoiler:
These above ideally need more experiments, but I cant be bothered at this point of my life sorry..
Unfortunately the tests were carried with non-existant abilities and there was only 9 sides with 1 unit leader on field. So the accuracy is questionable, although the results are not what I expected to see at all. (The filters itself are 99% sure working fine, I tested them on another era, but without timing)
NOW just 20 times {REPEAT} testing some basic store locations filter. (50k too long)
--------
1)
Spoiler:
terrain=Ch*,Kh*
89 88 85 72 762)
terrain=C*,K*
89 76 71 86 753)
terrain=Ch,Kh,Chw
69 77 83 76 824)
terrain=C*^*,K*^*
80 109 87 75 78No difference at all with all the stars or without... I mean the difference is nearly non-existant..
then I wonder why lua wesnoth.get_villages is much much more faster than normal terrain filter..
Last edited by enclave on May 27th, 2018, 1:37 pm, edited 1 time in total.
Re: Optimisation Discussions Advises Experiences Ideas Etc
A funny find on 1.12:
The code starts on select event:
This will be timed and will take 100 mileseconds or whatever units the time stamp is using...
While if I just change $side_number into $unit.side the code will start to take 700-900 miliseconds... almost 10 times slower.
Without any massive change... like so:
[store_locations]
[not]
find_in=qc_visible_villages
radius=$qc_system_variables.city_min_distance
[/not]
[not]
terrain=$hc_system_variables.illegal_city_building_terrain
[/not]
[filter_vision]
visible=yes
respect_fog=no
[/filter_vision]
variable=qc_allowed_build
[/store_locations]
Also just to be more helpful if I set {VARIABLE temp_side $unit.side} before storing locations and using $unit.side in filter, and then use side=$temp_side in filter then speed will again be only 100 ms or whatever time stamp units.
Magic, but that is how it is...
The code starts on select event:
Code: Select all
[store_locations]
[not]
find_in=qc_visible_villages
radius=$qc_system_variables.city_min_distance
[/not]
[not]
terrain=$hc_system_variables.illegal_city_building_terrain
[/not]
[filter_vision]
visible=yes
respect_fog=no
side=$side_number
[/filter_vision]
variable=qc_allowed_build
[/store_locations]
While if I just change $side_number into $unit.side the code will start to take 700-900 miliseconds... almost 10 times slower.
Without any massive change... like so:
[store_locations]
[not]
find_in=qc_visible_villages
radius=$qc_system_variables.city_min_distance
[/not]
[not]
terrain=$hc_system_variables.illegal_city_building_terrain
[/not]
[filter_vision]
visible=yes
respect_fog=no
side=$unit.side
[/filter_vision]
variable=qc_allowed_build
[/store_locations]
Also just to be more helpful if I set {VARIABLE temp_side $unit.side} before storing locations and using $unit.side in filter, and then use side=$temp_side in filter then speed will again be only 100 ms or whatever time stamp units.
Magic, but that is how it is...
Re: Optimisation Discussions Advises Experiences Ideas Etc
well this is somehow expected, the first time that
$unit
is used it needs to be created first, that is, the game silently performs a [store_unit] for the primary unit that is invonved in that event, and that takes some time.Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
Re: Optimisation Discussions Advises Experiences Ideas Etc
thanks gfgtdf... thats interesting.. Ill keep it in mind using unit variables in future...
Is it same thing for $x1 $y1? or it's only about $unit $secondary_unit etc?
Is it same thing for $x1 $y1? or it's only about $unit $secondary_unit etc?
Re: Optimisation Discussions Advises Experiences Ideas Etc
this is only about unit, it has some unxpected side effects becasue if you change the primary unit (the original unit, for example via [modify_unit], not using $unit) the content of $unit might be different depending onn when your event used $unit for the first time: if you used $unit before your change the $unit variable will contain the state of the original unit, even when you access it again after your change, but you did not use $unit at all before your change it will contain the 'new' state of the unit.
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.