Why not LUA?

Discussion among members of the development team.

Moderator: Forum Moderators

User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Why not LUA?

Post by Sapient »

I think the question of "Why not LUA" has been answered enough here. And I don't see anything new being added to this discussion. Nonetheless, if anyone wants to start work on integrating LUA to Wesnoth, then you are free to do so. If the patch is clean and it offers real solutions to real problems, then it has a good chance for inclusion.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
User avatar
Jetrel
Posts: 7242
Joined: February 23rd, 2004, 3:36 am
Location: Midwest US

Re: Why not LUA?

Post by Jetrel »

Dave wrote:
turin wrote:Incidentally, in the unlikely circumstance that we did decide to switch to Lua, how easy would it be to convert all the WML files to Lua files?...
I don't think we would ever "switch" to Lua.

What I think is suggested would be to add Lua as a new language that is used, so some 'moderately complex' functionality is carried out in Lua. Simple things would still be done in WML, complex things in C++, but things 'in between' could be done in Lua.

Removing WML altogether and using Lua completely for everything would be a total non-starter. As much or moreso than if someone suggested we re-write Wesnoth in Java.
I apologize to sapient for breaking the soft-lock; I hope I'm either saying something new, or if I'm not, reiterating something someone else has already stated.

If we needed to add some language to support "moderately complex" functionality, we should probably use python instead. The only real advantages lua offers over python for integration are apparently in "ease of binding lua to the C++", and since we've already got the tougher issues of that worked out for python, and already went through the pain of dealing with "another dependency" when we added python, it annuls that advantage.

There are apparently a lot of things that python does better on the side of "being a programming language" - lua tends to have strengths as the "configuration file" language it was originally intended to be, but for that, WML beats its pants off. Lua's kinda in an awkward middle ground - WML's a better config language, and Python's a better programming language, so there's no point in using Lua if we've already got the stronger extremes implemented.

It might be a tenable idea if we were starting a program from scratch, but we're not doing that.

Geos: and BTW, I say this as a guy who is writing a game which uses lua exactly as you describe.
Play Frogatto & Friends - a finished, open-source adventure game!
User avatar
appleide
Posts: 1003
Joined: November 8th, 2003, 10:03 pm
Location: Sydney,OZ

Re: Why not LUA?

Post by appleide »

I know there is a soft-lock on this discussion but I think I have something new to add. Forgive me if it isn't.

Wesnoth's macro language has some basic functionality that is really useful. But the macro 'language' is rather limited. There is no looping, for example.

What if, we replaced WML macro language with just Lua? Lua is powerful enough to do loops and generating code. But not too powerful it takes more than a fraction of a second to load.

Implementation 3 is what we're interested in.
All three is supposed to be run before scenario starts, though the first one could be run during the scenario too, I suppose.

Code: Select all

--[[
Example:
Implementation 1: DOM/js-like approach:
add("#textdoman wesnoth-multiplayer")
add(function() --function add adds arg as string to the file
	local scen=wMultiplayer()
	scen.appendWML("
	type={TYPE}
	user_description={USER_DESCRIPTION}
	x={X}
	y={Y}
	unrenamable=yes
	random_traits=no")
	scen.add( --function node.add adds arg as string to the node's text data
	function()
		local event=wNode("event")
		event.add("name=start
		"..wMath("x=$y+(4*$z)")
		..message('narrator'
			,'This is a wml scenario generated from lua. test x=$x'))
		return tostring(event)
	end)
	return tostring(scen)
end)
save("myscenario.cfg")

Comment: It is longer than WML scenario, 
	just as writing webpage in js takes longer than doing in HTML

Implementation 2: macro like lua:
add("#textdoman wesnoth-multiplayer") --add adds a line to the file
push("multiplayer") --push adds an open tag like "[multiplayer]" to file
	add("type={TYPE}
		user_description={USER_DESCRIPTION}
		x={X}
		y={Y}
		unrenamable=yes
		random_traits=no")
	push("event")
		add("name=start"
		..wMath("x=$y+(4*$z)")
		..message('narrator'
			,'This is a wml scenario generated from lua. test x=$x'))
	pop("event") --Pops only if top==arg, else print error; arg is optional
pop() --adds [/multiplayer]"

Comment: It is basically writing WML and using macros,
	except lua is much much richer than WML's macro language
	and can easily turn infix math expressions to WML.
	Other WML cliches can also be shortened into lua
	expanding functions, certainly more efficiently than using
	WML macro.
	If we turn the inline WML to a lua table of data
	(eg. an associative array with key-value pairs),
	can do even more stuff.

Code: Select all

Implementation 3: true macro like lua:
#textdoman wesnoth-multiplayer
[multiplayer]
	type={TYPE}
	user_description={USER_DESCRIPTION}
	x={X}
	y={Y}
	unrenamable=yes
	random_traits=no
	[event]
		name=start
		{wMath("x=$y+(4*$z)")} 
		{message('narrator'
			,'This is a wml scenario generated from lua. test x=$x'))}
	[/event]
[/multiplayer]

Comment: Lua used exactly like macroes, but Lua is more powerful 
	you can actually do some recursion and looping with it. 
	e.g 'repeat this section of code 3 times'.
	Easily implemented into current Wesnoth.

Code: Select all

Output:
#textdoman wesnoth-multiplayer
[multiplayer]
	type={TYPE}
	user_description={USER_DESCRIPTION}
	x={X}
	y={Y}
	unrenamable=yes
	random_traits=no
	[event]
		name=start
		{VARIABLE x $y} --This is the bane of WML; doing arithmetics; even with macros is a chore.
		{VARIABLE z1 4}
		{VARIABLE_OP z1 multiply $z)
		{VARIABLE_OP x add $z1}
		[message]
			speaker=narrator
			message='This is a wml scenario generated from lua. test x=$x'
		[/message]
	[/event]
[/multiplayer]
]]--
Why did the fish laugh? Because the sea weed.
User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Why not LUA?

Post by Sapient »

I think what you may not understand about macros is that they are a text preprocessing language. As such, they have no meaning at run time, but only when the WML is first parsed. Thus, a REPEAT_N_TIMES macro would kind of make no sense for a variable $n, unless the {CONTENT} was itself actionWML, in which case it could be executed in a do/while loop, which is already quite possible with the current macro language.

The nature of a text preprocessing language also means that preprocessor arithmetic with WML variables also makes no sense because it cannot pre-generate the necessary config output unless it is in an ActionWML context where set_variable is allowed.

The other examples where you are adding additional syntax requirements to textdomains and side definitions don't appear to be an improvement either. So I am afraid this solves nothing. But I do thank you for spelling out your proposal with the sufficient level of detail and examples, as we requested, so that I was able to evaluate it.

I agree that a solution for inline arithmetic would be nice if we can integrate it easily into the existing model. However, I don't think this is the answer, and I also think bringing in a giant Lua dependency would be overkill for such a modest goal.

Also, a note to follow-up posters: the soft lock is still in effect
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
silene
Posts: 1109
Joined: August 28th, 2004, 10:02 pm

Re: Why not LUA?

Post by silene »

(Disclaimer: I wrote the LUA engine of The Mana World, so I did actually experiment with using LUA for designing scenarios and how to interface it in a C++ game.)

I think this thread is missing some real examples of LUA code versus WML code. In order to show what LUA is, I will take some parts from chapter 5 of UTBS.

The first example is ugly on purpose. It is meant to appeal to people who have a bit of programming background. For more practical examples, see the latter ones.

Code: Select all

    [event]
        name=moveto

        [filter]
            x=18-24
            y=26-28
            side=1
        [/filter]

        [message]
            speaker=Nym
            message= _ "You know, if all we discover down here are insects, I'll be very disappointed. "
        [/message]

        [if]
            [have_unit]
                id=Elyssa
            [/have_unit]

            [then]
                [message]
                    speaker=Elyssa
                    message= _ "Spiders aren't insects."
                [/message]

Code: Select all

  [lua]
    event("moveto",

      -- event filter: argument u points to the unit that moved, return true if the event should apply
      function(u)
        return u.x >= 18 and u.x <= 24 and u.y >= 26 and u.y <= 28 and u.side == 1
      end,

      -- event action
      function()
        message("Nym", L["You know, if all we discover down here are insects, I'll be very disappointed."])
        if have_unit("Elyssa") then
          message("Elyssa", L["Spiders aren't insects."])
        end
      end
    )
  [/lua]
Notice the "lua" tag. Removing WML would be a mistake in my opinion; If Lua was to be used, it should be embedded inside WML. (Fwiw, TMW uses XML for describing maps/scenarios, and scripting languages like Lua are embedded into it, so the situation is similar.) The current campaigns should still be allowed to work, and one should be able to mix Lua and WML freely depending on the situation.

I think the biggest advantage of LUA is that it reduces the verbosity of logic operations:

Code: Select all

        [if]
            [variable]
                name=battle_turn_counter
                greater_than=0
            [/variable]

            [then]
                [set_variable]
                    name=battle_turn_counter
                    add=1
                [/set_variable]
            [/then]
        [/if]

        # on turn 3 have winning enemy launch special attack
        [if]
            [variable]
                name=battle_turn_counter
                numerical_equals=4
            [/variable]

            [then]
                {ENEMY_ATTACK}
            [/then]
        [/if]

        # on turn 6 have losing enemy bring reinforcements
        [if]
            [variable]
                name=battle_turn_counter
                numerical_equals=7
            [/variable]

            [then]
                {ALLY_REINFORCEMENTS}
            [/then]
        [/if]

        # at a random time later, the assassin should appear and challenge Kaleh

        [if]
            [variable]
                name=battle_turn_counter
                numerical_equals=$assassin_turn
            [/variable]

            [then]
                [set_variable]
                    name=call_assassin
                    value=1
                [/set_variable]
            [/then]
        [/if]

Code: Select all

        if battle_turn_counter > 0 then
          battle_turn_counter = battle_turn_counter + 1
        end

        -- on turn 3 have winning enemy launch special attack
        if battle_turn_counter == 4 then
          enemy_attack() -- it doesn't have to be a function, but that's just to keep the same structure as above
        end

        -- on turn 6 have losing enemy bring reinforcements
        if battle_turn_counter == 7 then
          ally_reinforcements()
        end

        -- at a random time later, the assassin should appear and challenge Kaleh
        if battle_turn_counter == assassin_turn then
          call_assassin = 1
        end
Note that LUA is based on associative tables, so it can be made to look a lot like a markup language without stressing the C++ interface:

Code: Select all

        [unit]
            type=Dwarvish Explorer
            id=Nori
            name= _ "Nori"
            canrecruit=yes
            x=45
            y=10
            side=5
            [modifications]
                {TRAIT_STRONG}
                {TRAIT_QUICK}
            [/modifications]
        [/unit]

Code: Select all

        {
            type = "Dwarvish Explorer",
            id = "Nori",
            name = L["Nori"],
            canrecruit = true,
            x = 45,
            y = 10,
            side = 5,
            modifications = { predefined_traits.strong, predefined_traits.quick }
        }
silene
Posts: 1109
Joined: August 28th, 2004, 10:02 pm

Re: Why not LUA?

Post by silene »

See https://gna.org/patch/index.php?1120 for a Lua-enabled version of Wesnoth. I expected it to be a lot more work to get something up and running; But in the end, i only had to add 3 lines to game_events.cpp.
User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Why not LUA?

Post by Sapient »

hey that's pretty cool for a quick hack. :wink:

although there is definitely a part of me that weeps to see another dependency being added to Wesnoth. :cry:
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
User avatar
Iris
Site Administrator
Posts: 6797
Joined: November 14th, 2006, 5:54 pm
Location: Chile
Contact:

Re: Why not LUA?

Post by Iris »

They can always be made optional, you know*.

It would be extremely nice if Lua could expose functions to WML as well; e.g. a Lua function being exposed as a WML tag, passing arguments as attributes or children tags.

* worried about your build environment as usual? ;-)
Author of the unofficial UtBS sequels Invasion from the Unknown and After the Storm.
silene
Posts: 1109
Joined: August 28th, 2004, 10:02 pm

Re: Why not LUA?

Post by silene »

Shadow Master wrote:It would be extremely nice if Lua could expose functions to WML as well; e.g. a Lua function being exposed as a WML tag, passing arguments as attributes or children tags.
As a matter of fact, it already works somehow. Since the current patch implements Lua code as event actions, you can do:

Code: Select all

[event]
    name=lua_dosomething1
    [lua]
        # note that the ... below are not an ellipsis because of I being lazy, that's really the arguments sent from WML
        code="dosomething1(...)"
    [/lua]
[/event]
[event]
    [filter]
    # from now on I'm being lazy though
    ...
    [/filter]
    [fire_event]
        name=lua_dosomething1
        [primary_unit]
            ...
        [/primary_unit]
        ...
    [/fire_event]
[/event]
Only locations can be sent, but that's not really a limitation of Lua nor WML, it's just the way events work in Wesnoth. Anyway, it's quite verbose, but not much more than usual. But I must admit I don't really see the point: If you can place a fire_event tag, you can also place a lua tag, so you could directly call a lua function instead of going through an event.
CIB
Code Contributor
Posts: 625
Joined: November 24th, 2006, 11:26 pm

Re: Why not LUA?

Post by CIB »

Before some time, I have been working on some advanced WML variable handling together with Sapient and I have the feeling that the lua extension could benefit from the ideas we had there.

You see, normally, WML is static. Once parsed by the preprocessor, it doesn't get modified anymore. Yet, WML features two dynamic aspects:

1. EventWML(I like to call it ActionWML): EventWML is parsed quite dynamically.. for example in [if][then] blocks, [while] loops, nested [event]s, there are nested levels of EventWML, so you can make infinitely complex WML structures out of different nested EventWML combinations.

Whenever the engine reaches an EventWML node, it will interpret the contents dynamically, similar to an interpreted language. Now, this might set up some people, but this dynamic way to parse WML makes an advanced technique possible: eval.

Yeah, yeah, "eval is evil" and all, but it's the reason that it is so easy to plug in a Lua interpreter into EventWML - It's basically some sort of eval, just with Lua used instead of WML. Important note: This means that the Lua interpreter is called dynamically and can create >any< WML output, based on what the Lua interpreter allows(erm, does it even allow to generate WML?). Fortunately, the WML engine can handle this just perfectly.

2. Variables(vconfig): In the WML engine, there is an alternative to the normal config - vconfig. When in C++ you request an attribute of vconfig, it will check whether the attribute contains a variable and parse its content in. This means that you can have dynamic WML anywhere where vconfigs are used.

Consider:

Code: Select all

[side]
   type=$dynamic
[/side]
I have no idea whether [side] actually uses vconfig, but if it does, once the WML engine reads the SideWML in, it will substitute the type dynamically with its value at that point. So, if this would be the second scenario of a campaign, you could actually set the leader type based on an event in the last scenario without EventWML. Of course, vconfigs are also essential for EventWML, but their power can extend further.

OK, so now I introduced the two ways how WML can be dynamic and how the Lua interpreter makes use of the first way. However, we discovered that the first way alone doesn't suffice for all situations.

Let's assume we have a moveto event and we want lua to calculate the x,y range for it. That means, that we don't want to make the whole event Lua, only the filter should be. So, it should look something like this:

[event]
name=moveto
[lua]
x = foo
y = bar
WML filter{ #create a WML tag that will be inserted as "[filter]" tag in place of the "[lua]" tag when parsing
attributes { "x" : x, "y" : y}
}
[/lua]
[event]

Well, now there are two things here: For one, using lua to directly generate WML to be used. I think this is the best way to have lua interface with WML, as it will allow Lua to use WML's full feature set.

Next thing: The lua tag is not EventWML! This means that the interpreter wouldn't even recognize it there. It would probably just ignore it. Yet, we thought, being able to insert dynamic WML would be useful in more places, specifically anywhere where variables are recognized(that is, in vconfigs). So Sapient went right ahead and extended the vconfig parser to actually take whole config objects as variables and not only $variables, look for "insert_tag" in variable.cpp for this. It might make sense to use the same technique for [lua] tags, so the lua interpreter could be called anywhere where $variables are being recognized(i.e. in my example above).

Anyway, this post is a bit messy, sorry about that, but I hope you get the basic ideas.
silene
Posts: 1109
Joined: August 28th, 2004, 10:02 pm

Re: Why not LUA?

Post by silene »

CIB wrote:I think this is the best way to have lua interface with WML, as it will allow Lua to use WML's full feature set.
Note that the current patch already allows to use WML's full feature set. For instance, the following excerpt is the part of the tutorial where Konrad's gender changes:

Code: Select all

W.kill { id="student", fire_event="no", animate="no" }
local x,y = student_start_loc()
W.unit { type="Fighteress", side=1, x=x, y=y, id="student", name=name, canrecruit=true }
W.redraw()
Just to stress it again: All the event tags and all their attributes are supported by the current patch. None of them are hardcoded; If someone adds a new WML tag, it gets automagically supported by Lua scripts.
CIB wrote:look for "insert_tag" in variable.cpp for this. It might make sense to use the same technique for [lua] tags, so the lua interpreter could be called anywhere where $variables are being recognized(i.e. in my example above).
As a matter of fact, my local version of the patch as an "insert_lua" tag which is handled at the same place "insert_tag" is. But I haven't found a use for this feature yet, that's why I haven't published it.
User avatar
Sapient
Inactive Developer
Posts: 4453
Joined: November 26th, 2005, 7:41 am
Contact:

Re: Why not LUA?

Post by Sapient »

One of our GSoC students, "Crab" is refactoring the AI code this summer. Now that Lua is becoming more influential in the codebase, I asked him "how difficult will it be to allow Lua AI's in your system?" Here was the log of our conversation:
#wesnoth-dev wrote: <Crab_> Sapient: the core of the system we're trying to create is a 'candidate action execution loop' written in c++. it uses pieces called 'candidate actions' which have a simple c++ interface. implementation details of those 'candidate actions' doesn't matter to 'candidate action execution loop' - so they can be written in lua with a simple c++ proxy implementing said 'candidate action' interface.
<Sapient> of course, this isn't part of your project, but I just want to make sure you are keeping that future development possibility in mind
<Crab_> Sapient: also, src/ai/game_info.hpp must be exposed to lua to allow lua to 'see' the state of the game.
<Sapient> ok, that is useful to know
<Crab_> Sapient: also, helper functions in src/ai/contexts.hpp must be exposed to lua or re-implemented in it (functions such as 'move unit' or 'calculate_possible_moves')
<Sapient> mind if I post these comments in the Lua development thread?
<Crab_> Sapient: ok
<Crab_> Sapient: note that if anyone seriously starts implementing an ai in lua, they should contact me to make sure they're not hit by my refactoring removing some piece of code they come to depend on.
<Sapient> it occurs to me that we may need a "Lua Workshop" subforum eventually
<Crab_> Sapient: I expect that I'll have a prototype of 'default ai as a candidate action AI' next week.
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Post Reply