Anonymissimus' lua thread

[silene]

So you think that it's worth trying, too.

Until you mentioned it, I didn't even imagine that remote debugging was a possibility. Once I had looked a bit and learned that remdebug would allow it without even modifying the engine, it would have been a shame not to give it a try. WML is in serious need of a proper debugger and it finally looks like we have a solution at hand.

Sure, this only really works for LuaWML, so it is not quite optimal yet. But it doesn't take long to write a Lua script that allows to do real-time remote debugging of plain WML; just register trivial Lua forwarders for the WML tags you are interested in and set breakpoints in them.

Will these modifications to the engine be backported to 1.8 ?

Possibly. The bug fix will, at least.

As for the "package" thing, I'm not quite sure yet. A command-line option doesn't seem enough to me. Some users may put it in their wesnoth shortcut one day and forget it. Perhaps, a message should be displayed each time a scenario is started/loaded while this option is enabled; something like "You are about to open a security breach in Wesnoth. Are you sure you want to continue? If you click 'yes', a malicious add-on will immediately take over your computer and post your bank details all over the Internet. You have been warned." Or perhaps it should be an in-game command, again with a dialog box to warn curious users.

Regarding eclipse (without knowing what your problems were), ... Still not sure whether eclipse+lua is good idea, though. One of the lua plugins (luaeclipse) is outdated, didn't yet try lunareclipse.

My problem was with luaeclipse in particular. After starting remote debugging of LuaWML, a Java error message would soon be displayed, preventing me to test any further. So perhaps I was just using it wrongly, I don't know. As for lunareclipse, I didn't try it, as it doesn't seem to support remote debugging.

[Anonymissimus]

Sure, this only really works for LuaWML, so it is not quite optimal yet.

I was guessing that due to the way wesnoth handles wml, debugging it "directly" isn't possible, other than lua - I should have asked earlier obviously, I was wondering about SciTE and its (remote) debugger feature months ago already...

just register trivial Lua forwarders for the WML tags you are interested in and set breakpoints in them.

How would this look like, then ?

As for the "package" thing, I'm not quite sure yet. A command-line option doesn't seem enough to me. Some users may put it in their wesnoth shortcut one day and forget it. Perhaps, a message should be displayed each time a scenario is started/loaded while this option is enabled; something like "You are about to open a security breach in Wesnoth. Are you sure you want to continue? If you click 'yes', a malicious add-on will immediately take over your computer and post your bank details all over the Internet. You have been warned." Or perhaps it should be an in-game command, again with a dialog box to warn curious users.

A :debug mode command to enable it would be great. Normal users don't enter debug mode.
In which way does wesnoth become unsecure, then ? People using the feature will probably only be the devs and advanced wml authors anyway, but we should know it.

Tried lunareclipse btw - it didn't work at all for me. Both eclipse add-ons don't seem to be so well-developed, so to say, although lunareclipse appears to be at least still under development...

[silene]

just register trivial Lua forwarders for the WML tags you are interested in and set breakpoints in them.

How would this look like, then ?

Let's suppose you are interested in what happens at the point of tags [store_locations], [kill], and [unstore_unit] when side 2 is playing. Set a breakpoint on the dummy= line below, load the file (possibly dynamically from the internal command line), and play. The game will freeze when it reaches such a tag, and you can then mouse over the "tag", "literal", and "parsed" variables to know which tag is executed and what are its parameters (before and after variable substitution).

Code: Select all

local tags = { "store_locations", "kill", "unstore_unit" }
local old_handlers = {}
local dummy
for i,v in ipairs(tags) do
  old_handlers[i] = wesnoth.register_wml_action(v, function(cfg)
    if wesnoth.current.side == 2 then
      local tag = tags[i]
      local literal = cfg.__literal
      local parsed = cfg.__parsed
      dummy = #literal + #parsed + #tag
    end
    old_handlers[i](cfg)
  end)
end

In which way does wesnoth become unsecure, then ? People using the feature will probably only be the devs and advanced wml authors anyway, but we should know it.

Lua scripts gain the same privileges as the Wesnoth binary itself: ability to access most of the files of the computer, ability to use a network interface, and so on. For instance, an addon could connect to a distant website, download a binary file, and execute it on the local computer.

[Anonymissimus]

Gave it a try and came much farer than hoped. I've created a wesnoth.exe with the package thing on windows with Code::Blocks which seems to run fine when playing a bit HttT, except for that I can't change the language but I don't care.
It seems that I can start SciTE's remote debugger correctly, although there seems to be a bug in extman.lua in the function path_of which returns path+filename instead of the path only. After starting wesnoth then, I'm getting lua errors in wesnoth about lua files which aren't found, partly they are somewhere in my lua installation. I didn't do anything about the "canonicalize filenames" bug, though...
Anyway, at least I've compiled wesnoth now.^^

[silene]

It seems that I can start SciTE's remote debugger correctly, although there seems to be a bug in extman.lua in the function path_of which returns path+filename instead of the path only.

Possibly it is just a filename issue. Try using a Windows-style path for your ext.lua.startup.script property, that is, use backslashes.

After starting wesnoth then, I'm getting lua errors in wesnoth about lua files which aren't found, partly they are somewhere in my lua installation.

I'm guessing it doesn't find some packages. Add new entries to package_path in the remdebug initialization lines of your Luascripts (;-separated, with % placeholder for the package name).

I didn't do anything about the "canonicalize filenames" bug, though...

You will know you are hitting this bug, when you set some breakpoints yet Wesnoth doesn't stop on them. For instance, you set a breakpoint in a file named dir1/dir2/name.lua but Wesnoth has named the file dir1/./dir2/name.lua. While equivalent, these names are not equal, and the breakpoint will be ignored. The one-liner change in remdebug/engine.lua works around this issue.

[Anonymissimus]

Possibly it is just a filename issue. Try using a Windows-style path for your ext.lua.startup.script property, that is, use backslashes.

It did some good; SciTE's remote debugger seems to run fine now without changes in extman.lua.

I'm guessing it doesn't find some packages. Add new entries to package_path in the remdebug initialization lines of your Luascripts (;-separated, with % placeholder for the package name).

I'm getting errors like this

Spoiler:

Code: Select all

20100522 15:58:12 error scripting/lua: E:/Programme/Lua/5.1/lua/socket.lua:13: module 'socket.core' not found:
	no field package.preload['socket.core']
	no file '.\socket\core.lua'
	no file 'E:\Programme\wesnoth-1.8.1\lua\socket\ '
	no file 'E:\Programme\wesnoth-1.8.1\lua\socket\core\init.lua'
	no file 'E:\Programme\wesnoth-1.8.1\socket\core.lua'
	no file 'E:\Programme\wesnoth-1.8.1\socket\core\init.lua'
	no file 'E:\Programme\Lua\5.1\lua\socket\core.luac'
	no file 'E:/Programme/Lua/5.1/SciTE/scite-debug/remDebug/socket\core.lua'
	no file 'E:/Programme/Lua/5.1/lua/socket\core.lua'
	no file '.\socket\core.dll'
	no file '.\socket\core51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\socket\core.dll'
	no file 'E:\Programme\wesnoth-1.8.1\socket\core51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\socket\core.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\socket\core51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\loadall.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\loadall.dll'
	no file '.\socket.dll'
	no file '.\socket51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\socket.dll'
	no file 'E:\Programme\wesnoth-1.8.1\socket51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\socket.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\socket51.dll'
	no file 'E:\Programme\wesnoth-1.8.1\loadall.dll'
	no file 'E:\Programme\wesnoth-1.8.1\clibs\loadall.dll'
stack traceback:
	[C]: in function 'require'
	E:/Programme/Lua/5.1/lua/socket.lua:13: in main chunk
	[C]: in function 'require'
	...a/5.1/SciTE/scite-debug/remDebug/remdebug\engine.lua:6: in main chunk
	[C]: in function 'require'
	...data/add-ons/Dwarven_Kingdom/utils/lua/functions.lua:4: in main chunk
	[C]: in function 'dofile'
	[string "..."]:3: in main chunk

Previously to that, it looked for socket.lua which I resolved by adding a path to package.path, but there's nowhere a file named core.lua. Where do you have these files in your lua installation, or didn't you get the request for them ?
I used LuaForWindows_v5.1.4-37.exe to install lua and SciTE.

[silene]

Previously to that, it looked for socket.lua which I resolved by adding a path to package.path, but there's nowhere a file named core.lua. Where do you have these files in your lua installation, or didn't you get the request for them ?

Presumably, it's not core.lua but core.dll. It comes from the socket library: http://w3.impa.br/~diego/software/luasocket/.

And just in case you later get complaints about Lua not finding lfs.dll, this file comes from the filesystem library: http://keplerproject.github.com/luafilesystem/.

[Anonymissimus]

Have you made the remote debugging issue more convenient, silene (fixed the bug and (maybe) added the --unsafe option) ?
Is it in 1.8 or in which version will it be, then ?
If so, I'll take on the next trial to get it to work then. Small engine changes are no longer an issue, however.

[silene]

Have you made the remote debugging issue more convenient, silene (fixed the bug and (maybe) added the --unsafe option) ?
Is it in 1.8 or in which version will it be, then ?

I didn't fix the bug. It is still there, but I wasn't experiencing it anymore. So I considered it was not worth trying to fix it; especially since it can be worked around by modifying a single line in a text file.

I have implemented the unsafe mode however. It's an internal command in 1.8.2:

Code: Select all

:debug
:unsafe_lua
:lua wesnoth.dofile "lua/debug.lua"

And the debug.lua file that I have put in my data/lua/ directory contains

Code: Select all

package.path = package.path .. ";/home/guest/src/scite-debug/remDebug/?.lua"
require "remdebug.engine"
remdebug.engine.start()
dummy = nil -- dummy statement for avoiding the first break to possibly occur outside a named file, which would break remdebug

[Jovycos]

Where is the screenshot from? (I mean the scenario)

[silene]

Where is the screenshot from? (I mean the scenario)

It's the test scenario. You can run it by passing "-t" as a command-line option to the wesnoth executable.

[Anonymissimus]

I have a small lua file A.lua which requires B.lua
I can successfully remote debug B from within SciTE after starting A from from a console. A symbolizes the wesnoth engine in my naive view. It also contains those key lines

package.path = package.path .. ";E:/Programme/Lua/5.1/SciTE/scite-debug/remDebug/?.lua"
require"remdebug.engine"
remdebug.engine.start()

No other packages than those from the lua for windows remDebug install directory are needed (or no packages are not found, so to say). The only option I set in scite properties is

debug.target=:remote.lua

Only when trying to run A from within wesnoth I keep getting error messages about packages not found - but since in my naive understanding neither wesnoth nor Scite should need them, I think that at least I don't need to install additional stuff.
Isn't the lua-native require function disabled or overwritten by wesnoth ?

EDIT
I tried adding various stuff to package.path, also from the libraries you gave me the links to...no matter what I do I'm running into errors and am frustrated enough to give up again...
If anyone else got it to work let me know please.

Why are there always on open source download pages tons of files for download without any information whether I need them all or only some and if yes which ones ??

[silene]

Isn't the lua-native require function disabled or overwritten by wesnoth ?

The native "require" function starts as disabled. It gets enabled as soon as you type ":unsafe_lua". If you get actual error messages about packages not being found, I guess it means it was properly enabled.

No idea about your issue though. It's strange that all the packages are found when A is run outside Wesnoth, but not when it is run inside. The first thing I would try is to print the content of the package settings (package.path and package.cpath) in order to check that they have the same value in both cases.

[Anonymissimus]

I've added all paths that were seemingly missing to those two variables and
GOT IT WORKING!

I'm guessing I should write some guide now. This feature will be extraordinary useful for all add-on authors, especially when I look at Exasperation's lua codes or so.
After all it doesn't need that much (I'm surprised).

The require function seems to work differently btw.

So lua wml files can be debugged directly now, and wml files can be debugged indirectly, though there's still the need to write some lua code for it, which may make it faster to do mainscreen-f5-reload.

EDIT
Well, it doesn't seem to work always well...I managed to crash wesnoth and SciTE several times now, and it seems that somehow some wml tables are turned into tables - that is, wesnoth omits an error with debugger, but none without. Nevertheless, a superb thing.

[Anonymissimus]

-I see that many wml action functionalities are currently being exported from the C++ engine to the external lua files - just curious: why ? (it's not that I have an opinion about it...)

-You didn't yet disable the math.random() function (and add a replacement to helper), or is it wired to set_variable's implementation ?

-I suggest adding the get_locations functionality to helper which you've posted somewhen earlier in this thread, about:

Spoiler:

Code: Select all

function get_locations(filter)
  local cfg = filter.__literal
  local var_name = "LUA_store_location"
  cfg.variable = var_name
  wesnoth.fire("store_locations", cfg)
  local t = helper.get_variable_array var_name
  wesnoth.clear_variable var_name
  return t
end

I need it often, actually.