Dugi's lua questions

[Dugi]

Lately, I was used to do a lot of variable-related stuff with WML, that resulted in long calculations, than resulted sometimes in visible times of execution (I mean like, 0.3 second, the code was like 200 KiB long, but contained many cycles and macros), while creating and scattering a hundred units around a map took shorter.

As it is visible on the forums, me and some other people are planning to make a new RPG add-on, that is meant to have quite vast system of mechanics, and almost all of its calculations would be variable operations. I don't know much about the power of variables of wesnothian lua (as the core lua is really hard to decipher, it has a lot of custom functions with unobvious purpose, and also it is not commented like wml macros).

So, is lua much faster than WML in this (is it even worth the effort)?
Are there easy or relatively easy lua equivalents for these wml possibilies (examples included)?

Spoiler:

Splitting lists of things into variable arrays, joining them, like this:

Code: Select all

			[set_variables]
			    name=description
			    mode=replace
			    [split]
				list=$advanced2.modifications.object[$i].description
				key=line
				separator="
"
				remove_empty=yes
			    [/split]
			[/set_variables]

Numerous variable substitutions, including insert_tag, like this:

Code: Select all

	[unit]
		side=$advanced2.side
		x=$advanced2.x
		y=$advanced2.y
		experience=$advanced2.experience
		canrecruit=$advanced2.canrecruit
		variation=$advanced2.variation
		type=$advanced2.type
		id=$advanced2.id
		moves=$advanced2.moves
		hitpoints=$advanced2.hitpoints
		gender=$advanced2.gender
		name=$advanced2.name
		facing=$advanced2.facing
		extra_recruit=$advanced2.extra_recruit
		underlying_id=$advanced2.underlying_id
		unrenamable=$advanced2.unrenamable
		animate=no
		[insert_tag]	
			name=status
			variable=advanced2.status
		[/insert_tag]
		[insert_tag]	
			name=modifications
			variable=advanced2.modifications
		[/insert_tag]
		[insert_tag]
			name=variables
			variable=advanced2.variables
		[/insert_tag]
		to_variable=advanced
	[/unit]

Writing directly a variable array to be inserted somewhere, with subtags, something like this:

Code: Select all

[set_variables]
    name=anim_object
    mode=replace
    [value]
	sort=temporary
	silent=yes
	[filter]
		x,y=$advanced.x,$advanced.y
	[/filter]
	[effect]
		apply_to=new_animation
		name=aa$advanced.modifications.advance[$i].effect[$j].name
		[insert_tag]
			name=attack_anim
			variable=unit_style.attack_anim[$anim_index]
		[/insert_tag]
	[/effect]
    [/value]
[/set_variables]

Numerous variable substitutions taking place in creating the name of the desired variable, usage of the .length possibility, like this:

Code: Select all

[set_variable]
	name="index"
	to_variable="advanced.attack[$($advanced.attack.length-1)].specials.$count|.length"
[/set_variable]

[Alarantalara]

The simplest way to put it is that Lua takes a completely different approach to everything. Most variables you use temporarily in a single event will be much faster, but permanent storage becomes more awkward.

Your first example would probably end up about the same if you wanted to store it permanently, and likely wouldn't even be done if you wanted to just go over the values (since you can look at them directly with string functions and not have to store it somewhere else first).

Insert tag is completely unneeded unless you are creating WML events to run later.
Your second example simplifies to:

Code: Select all

   local advanced = wesnoth.copy_unit(advanced2)

Then you modify the copy as desired.
This is likely not how you'd usually do the substitutions, but it does show how differently somethings are done.

For the third one, you'd likely build the structure then copy it where you wanted it.

For the last, there would likely be more temporary variables used to store intermediate results. Variable substitutions are more or less meaningless. specials.$count|.length becomes #specials[count] for example, assuming that specials isn't a WML tag, in which case the syntax is different. Generally you want to avoid referencing WML tags as much as possible in the Lua code as they slow you down and instead find sources that you can reference directly.

[Dugi]

I meant to use it to create a wml tag that would store a unit, import a load of wml variable arrays as local variables, perform a lot of variable operations on the variable created by storing that unit, using the imported wml variable arrays, and then unstore the unit and possibly also export a few wml variable arrays.

When thinking about your reply:
1. I am glad to read that I can read variables by lines without splitting them.

2. You misunderstood what was that code for. It was supposed to create a unit with some properties adjusted explicitly, but most properties determined from its modifications and unit_type.
But if I understood the use of inner variables in lua correctly (and assuming lua's variable arrays have the same structure as wml variables) I think it might be equivalent to something like:

Code: Select all

local unit = {}
unit.side = advanced2.side
unit.s = advanced2.x
...
unit.variables =advanced2.variables
unit = wesnoth.create_unit(unit)

3. I was already forced in WML to make a structure that was building the structure and then using set_variable to move its contents elsewhere, because variable substitution cannot be done if it is substituted by a variable whose name contains another substitution (that is the 4th question).
This means that the lua equivalent for this code would be longer, because I would have to set each variable in the variable array by another command? Then I would add that object with wesnoth.add_modification ?

4. Because I have seen nothing like variable substitutions in wml_tags.lua, I assume that there is a string operation that connects a few pieces of code together, and then something that recalls a variable whose name is determined by the contents of a variable, is that correct?

An additional question:

Code: Select all

for ... in ipairs {...} do

Is this something like {FOREACH ...} in wml?

[Elvish_Hunter]

Splitting lists of things into variable arrays, joining them, like this:

In that case you'll need the string.gmatch function:

Code: Select all

for item in string.gmatch( my_string, "[^\n]" ) do

That means: for every part of the string that is not a newline. The square brackets mean a set of characters, ^ means every character that's not in set, and \n is a newline in form of escape sequence. Lua, unlike WML, supports escapes.

Numerous variable substitutions, including insert_tag, like this:

Variable substitution does not exists in Lua. You just do this:

Code: Select all

unit.hitpoints = unit.max_hitpoints

However, not all of a unit's keys can be accesses in this way. For some keys you'll need to read the .__cfg field, as they don't have a proxy field. And if you want to modify such keys, you'll need a wesnoth.put_unit afterwards.

Writing directly a variable array to be inserted somewhere, with subtags, something like this:

In that case, the function that you're looking for is helper.set_variable_array.

Numerous variable substitutions taking place in creating the name of the desired variable, usage of the .length possibility, like this:

Lua has two length operators. The first one is the # symbol, and you must place it immediatly before a string or table variable name, while the other is the string.len() function, that - as the name suggests - works only for strings.

(and assuming lua's variable arrays have the same structure as wml variables)

They don't. At first, it took me some time to understand it, but every WML array is structured in this way in Lua:

Code: Select all

{ array_name, { key1, key2, key3 } }

where array_name is the array_name (for example, "status", "variables"), while the table with key1, etc. is the content of the array. To have a simpler structure, like ordinary Lua tables, you can either go for the helper.set_wml_var_metatable or helper.get_variable_proxy_array.

Because I have seen nothing like variable substitutions in wml_tags.lua, I assume that there is a string operation that connects a few pieces of code together, and then something that recalls a variable whose name is determined by the contents of a variable, is that correct?

Yes and no. There isn't a specific function for this, but you can use the string.format function:

Code: Select all

local my_var = wesnoth.get_variable( string.format( "%s_%s", v1, v2 ) )

where %s means a string and v1 and v2 are strings, that in turn can be read from WML variables. In fact, you build a new string that will be the variable name.

Is this something like {FOREACH ...} in wml?

Not exactly, as FOREACH is really a [while] cycle, and a for cycle can be used also to iterate over range of numbers:

Code: Select all

-- from http://lua-users.org/wiki/ControlStructureTutorial
> for count = 1,3 do print(count) end  -- numerical iteration
1
2
3

But for most purposes, for .. in ipairs() can be considered equivalent.

[Dugi]

So, please correct me if I'm getting it wrongly, but does it work like this?
This:

Code: Select all

local object = {sort=temporary , silent=yes , { "effect" , ( apply_to=new_animation )}}
object.filter.x = advanced.x
object.filter.y = advanced.y
object.effect.name = "aa" + advanced.modifications.advance[i].effect[j].name
object.effect.animation = unit_style.attack_anim[anim_index]

means:

Code: Select all

[set_variables]
    name=anim_object
    mode=replace
    [value]
   sort=temporary
   silent=yes
   [filter]
      x,y=$advanced.x,$advanced.y
   [/filter]
   [effect]
      apply_to=new_animation
      name=aa$advanced.modifications.advance[$i].effect[$j].name
      [insert_tag]
         name=attack_anim
         variable=unit_style.attack_anim[$anim_index]
      [/insert_tag]
   [/effect]
    [/value]
[/set_variables]

Am I right? Or are there just some petty errors that can be corrected? It is to totally wrong?

I understood the other stuff like array length or splitting strings, there were some examples in wml_tags.lua.

(If that matters, from programming languages, I know WML and (the basic logic) of C and Visual Basic.)

[Alarantalara]

It would look like this:

Code: Select all

anim_object = {
    sort = "temporary",
    silent = true,
    { "filter", { x = advanced.x, y = advanced.y } },
    { "effect", {
        apply_to = "new_animation",
        name = "aa" .. advanced.modifications.advance[i].effect[j].name,
        { "attack_anim", unit_style.attack_anim[anim_index] }
    } }
}

This assumes that unit_style.attack_anim is a WML style table and that all tables referenced in advanced.modifications.advance.effect[j].name are Lua style tables. If not, then it would be different.
Edit: It also assumes that both variables have already been retrieved by Lua. Variables accessible by WML all have to be retrieved with wesnoth.get_variable then stored again with wesnoth.set_variable if you want to use them. If all you're doing is copying values from one WML variable to another repeatedly, I suspect it's unlikely you'll notice any speed improvement. The benefits of Lua come more from being able to not use WML variables for large parts of execution. Your reinterpretation of the unit creation is probably a good example here.

Code: Select all

    u = wesnoth.create_unit( advanced2 )

This will create a unit with all the values specified in advanced2 if that variable is not already a unit. There is no need to do another copy.

Another way to look at it is this. I've been spending time writing code to recruit units in Lua recently. The first version of it took about a second per unit recruited on some maps. However it was possible to rewrite it so that only the first unit took any significant time and all the rest took less than 0.1 seconds total. How it was written made a huge difference and the language did not change. Speed in code is usually more related to how something is written than what language it was written in.

Another speed example, this one in WML. Look at the flooding algorithm in Out of the Frying Pan in 1.10 and 1.11. The code is different for both of them. The code used in 1.11 is much slower than that in 1.10 if it were used in 1.10. I believe it's faster than the code from 1.10 if used in 1.11.

[Xudo]

snippet 1:

Code: Select all

do
local weapons = {}
weapons[1] = {name = 'One-handed sword', type = 1}
weapons[2] = {name = 'Spear', type = 2}
weapons[3] = {name = 'Two-handed sword', type = 2}
end

snippet 2:

Code: Select all

CREATE TABLE weapons
{
    id    int,
    name   varchar(20),
    type    int
};
INSERT INTO weapons (id, name, type) VALUES (1, 'One-handed sword', 1);
INSERT INTO weapons (id, name, type) VALUES (2, 'Spear', 2);
INSERT INTO weapons (id, name, type) VALUES (3, 'Two-handed sword', 2);

snippet 3:

do
local type_of_spear = weapons[2].type
end

Is snippet 1 logicaly equivalent to snippet 2?
In snippet 3, I tried to get type of spear. Is this syntax correct?

[Luther]

I'm jumping in here to suggest that if you want to make a complex add-on like an RPG, you should write as much of it as possible in Lua. That would help you organize your code better, and it reduces WML bloat. Here are some rules of thumb I've written in my own private set of coding standards:

Lua is easier to read and maintain then ActionWML. If a piece of ActionWML consists of more than one tag, or if that tag is [if], [switch], or [while], write a custom tag in Lua, instead. As an exception, you may write any number of [message] tags in WML. This is because [message] tags often appear in series, and it's probably not worth it to have a separate Lua function for every piece of dialog you write. (This rule might sound rather extreme, but I find Lua far easier to work with than ActionWML code.)

Never use macros for ActionWML. Custom tags are easier to read because the attributes and subtags have explicit names. Also, macros lead to WML bloat. As an exception, you may use macros like ON_DIFFICULTY, since they use #ifdef to bundle up information that you can't otherwise get in Lua.

Never hard code x,y coordinates in Lua. Always take them as cfg attributes. That way, you can change your map without having to change your Lua file.

Other suggestions:

Get the Lua Globals add-on (which I wrote), and read the README and globals.lua. (There's very little code to read.) If you can understand how to use it, I believe it can greatly simplify your code.

Remember that you can store constant data (like weapon stats, for example) in Lua. That way, it won't have to get stored in the save files.

You sound like you might be just getting started with Lua, so I (and I'm sure others) will be happy to answer any questions you have.

Can you please post a link to where you're talking about the RPG?

[Anonymissimus]

Luther advertising "never use macros" reminded me of this.
Another (good) alternative to macros are often custom events. They can also take sort-of parameters ($unit, $second_unit, stuff queried for in [filter_condition], ...) and reduce wml bloat just as well.

[Sapient]

Another thing to consider is that blindly converting everything to Lua may reduce the number of potential future contributors/maintainers. Unless you just want the experience of rewriting it in Lua for your own preference or education, a more sensible approach would be to identify what part is the real bottleneck. Then you can see if the WML itself could have been written more efficiently, or if it really is a case where you need to move some computationally complex routine into Lua. The TIMED_ACTIONS macro from debug-utils.cfg (or similar) may help you to identify the bottleneck.

[Elvish_Hunter]

Is snippet 1 logicaly equivalent to snippet 2?

For what little I know about databases (some LibreOffice), I'd say yes, with the exception that Lua tables do not require a primary key (and if you want to give one, it'll be a normal variable anyway). Out of curiosity, why did you decide to compare Lua to SQL?

In snippet 3, I tried to get type of spear. Is this syntax correct?

If it's inside a for or while cycle yes, otherwise remove the do .. end structure.

[Xudo]

Can you please post a link to where you're talking about the RPG?

See my signature.

Another (good) alternative to macros are often custom events. They can also take sort-of parameters ($unit, $second_unit, stuff queried for in [filter_condition], ...) and reduce wml bloat just as well.

Is it possible to create variables in special namespace instead of $unit ?

Out of curiosity, why did you decide to compare Lua to SQL?

I wanted to know whether I understand lua tables right or wrong. As long as SQL designed to operate data, I choose it in this case.

Thank you all for answers.

[Alarantalara]

Is snippet 1 logicaly equivalent to snippet 2?

Snippet 2 is closer to an INDEX on id from snippet 1 by behaviour. If you included id= fields in snippet 2, and used table.insert to add the data so that you didn't specify their location in the Lua table, then they'd be identical logically.

[Dugi]

@Aralantara

It would look like this:
...

So it's more practical. Would my version work too (though a bit slower), anyway?

This assumes that unit_style.attack_anim is a WML style table and that all tables referenced in advanced.modifications.advance.effect[j].name are Lua style tables. If not, then it would be different.

One was meant to be created by store_unit_type and the second to be the variable being worked on, so one would be created in lua and the ther one converted to lua and after numerous operations taken back to WML. I wrote that I meant to take all data variables needed into lua before the process, and then released them into WML (but that would be just a small fractions of the operations done).

I didn't expect it to be possibly even 10 times faster than WML. Good.

@Luther

Remember that you can store constant data (like weapon stats, for example) in Lua. That way, it won't have to get stored in the save files.

Would it look like WML? Read bellow why I need to know this, please.

Can you please post a link to where you're talking about the RPG?

Here.But we haven't properly started yet, I want to write the core mechanics, but I need first to know what am I about to write.

I have downloaded your lua pack, though I have forgotten most of the things I have read there, but it seems to be quite useful.

@Anonymissimus

Luther advertising "never use macros" reminded me of this.
Another (good) alternative to macros are often custom events. They can also take sort-of parameters ($unit, $second_unit, stuff queried for in [filter_condition], ...) and reduce wml bloat just as well.

Learned this from my own experience. Replaced the largest macro by custom event and suddenly the largest thing in save files were variations of useless walking corpses created by plague...

@Sapient
I didn't plan to write everything in lua, just the parts that contain a lot of variable operations, usually invoked at once, and extremely complicated even in WML anyway.

Another thing to consider is that blindly converting everything to Lua may reduce the number of potential future contributors/maintainers.

That is why I want to keep scenarios, item lists, skills' properties and so on in WML, because it would be much more understandable and easier to edit, save this stuff as variables and load it with lua. Because Luther mentioned that it might be stored somehow with lua outside save files, could it be done without making it look to weird and hard to understand?

@Xudo A big thanks for hijacking my topic with some SQL.

[Alarantalara]

@Aralantara

It would look like this:
...

So it's more practical. Would my version work too (though a bit slower), anyway?

Assuming the final result has to return to WML, it would not work. See Elvish_Hunter's description of how WML data is represented in Lua several posts earlier.

Remember that you can store constant data (like weapon stats, for example) in Lua. That way, it won't have to get stored in the save files.

Would it look like WML? Read bellow why I need to know this, please.

It would look like the sample code I wrote for assigning anim_object above except that there would be fewer references to other variables. Tags end up looking like "{ 'tag_name', {" for opening tags and "} }" as a closing tag. You could add comments to the end of each closing pair and write all values one per line if you wanted it to look more like WML.