wmllint tutorial

[galicae]

Hello everybody
wmllint is one of the best tools that come with wesnoth. It's a no-brainer to edit your campaign with this, instead of re-writing again manually in order to make it compatible with the new version. Now, not all of us (including me) have any kind of experience with Python, or Lua or C++ or any other programming language, so I think it would really be of service to all UMC authors (real and would-be) to post a tutorial about it (what it is, how you use it, why it must ignore some strings in campaign and scenario files etc) on the forum or even in the site.
ideas?

[Elvish_Hunter]

I posted the things below in Fossil thread, so I'm quoting myself here hoping to be useful:

This is how I run wmllint:
- first, open Windows Command Prompt (on Linux, open a text shell)
- then, type the path to your python.exe file (mine is C:\Python26\pyton.exe; not needed on Linux) and a space
- then, the path to wmllint (something like "C:\Programs\Battle for Wesnoth 1.8\data\tools\wmllint"; if your path has spaces, place this between quotes, like I did) and a space
- if needed put here some parameters (the most important is --dryrun, that runs wmllint showing only the output on screen and not changing the files)
- then, the path to your Wesnoth core folder ("C:\Programs\Battle for Wesnoth 1.8\data\core\") and a space
- then, the path to your add-on ("C:\Programs\Battle for Wesnoth 1.8\userdata\data\add-ons\Fossil\")
- if you need a log file, add (without any space) >wmllint.log
- you can press Enter and enjoy! Wmllint will do the corrections, and inform you with messages on the command prompt about the changes that you'll need to do by hand.
In the same way, you can run also wmlscope. I hope that this will be useful.

Wmllint is automatically installed when you install Wesnoth You should download Python from http://www.python.org. To include parameters, you just type them. The final command line is: C:\Python26\pyton.exe "C:\Programs\Battle for Wesnoth 1.8\data\tools\wmllint" --dryrun "C:\Programs\Battle for Wesnoth 1.8\data\core\" "C:\Programs\Battle for Wesnoth 1.8\userdata\data\add-ons\Fossil\">wmllint.log (if you want that wmllint changes the files, just remove --dryrun) Ah, the good old MS-DOS 6.22/Windows 3.11 times... (just joking, I never used them )

I saved that string in a text file and, when I want to run wmllint, I just copypaste it, change the name of the addon folder and that's it. Or I drag and drop the folders (or the files) on the command prompt window and Windows types it for me. It's also possible to run it as a batch file, theoretically - but I never made a batch file.

[itota]

Hello everybody
wmllint is one of the best tools that come with wesnoth. It's a no-brainer to edit your campaign with this, instead of re-writing again manually in order to make it compatible with the new version. Now, not all of us (including me) have any kind of experience with Python, or Lua or C++ or any other programming language, so I think it would really be of service to all UMC authors (real and would-be) to post a tutorial about it (what it is, how you use it, why it must ignore some strings in campaign and scenario files etc) on the forum or even in the site.
ideas?

yes i think that it helps a lot.
and i hope the site (or this page) includes the download links to old versions of wmllint and its dependency so that everyone can convert ancient scenarios made b4 1.6version of wesnoth without downloading whole packages of wesnoth.

[Max]

just had a look at the changelog - seems only code for releases earlier than 1.4 has been removed. my guess would be that the current version should work fine for >1.4.

edit: there's an official note within the code itself:

Code: Select all

# Note: Lift logic for pre-1.4 versions has been removed; if you need
# it, check out a copy of wmllint from the 1.4 stable branch and use
# that to lift before running this one.

[Elvish_Hunter]

and i hope the site (or this page) includes the download links to old versions of wmllint and its dependency so that everyone can convert ancient scenarios made b4 1.6version of wesnoth without downloading whole packages of wesnoth.

Download links
For wmllint 1.4: http://svn.gna.org/viewcvs/wesnoth/bran ... ls/wmllint
For wmllint 1.6: http://svn.gna.org/viewcvs/wesnoth/bran ... ls/wmllint

[Dixie]

Hmm, interesting... So basically, it debugs the code you write and finds the errors/possibly corrects them, or it just spots te stuff that has changed with each passing version?

Also, if anybody knows, would it be the same procedure for mac, but use Terminal instead of Windows Command Prompt?

[Anonymissimus]

my guess would be that the current version should work fine for >1.4.

yes (it's >= 1.4)
I've used 1.4 wmllint to convert old maps recently.

So basically, it debugs the code you write and finds the errors/possibly corrects them, or it just spots te stuff that has changed with each passing version?

both
It's a convert-to-recent-standards and a sanity tool at the same time. With --dryrun it's only a sanity checker...

Also, if anybody knows, would it be the same procedure for mac,

probably not exactly

but use Terminal instead of Windows Command Prompt?

yes
I can tell for windows<->Linux that it works in principle in the same way, while windows is a lot more difficult.

Note that wmllint's spellcheck (you notice it only when installing python enchant) and wmlscope are bugged in 1.8; both problems are only fixed in the 1.9 branch. The developers working on the python tools are mostly not using windows themselves, that's why there tend to be win-only bugs which don't get spotted and then people wonder why they can't get it to work...
I've created a patch for my private use that summarizes the neccessary fixes for both bugs, in case you find out what to do with it. You may apply those changes also manually.

Spoiler:

Code: Select all

Index: data/tools/wesnoth/wmltools.py
===================================================================
--- data/tools/wesnoth/wmltools.py	(revision 44128)
+++ data/tools/wesnoth/wmltools.py	(working copy)
@@ -13,6 +13,10 @@
 resource_extensions = map_extensions + image_extensions + sound_extensions
 image_reference = r"[A-Za-z0-9{}.][A-Za-z0-9_/+{}.-]*\.(png|jpg)(?=(~.*)?)"
 
+def is_root(dirname):
+    "Is the specified path the filesysten root?"
+    return dirname == os.sep or (os.sep == '\\' and dirname.endswith(':\\'))
+
 def pop_to_top(whoami):
     "Pop upward to the top-level directory."
     upwards = os.getcwd().split(os.sep)
@@ -306,8 +310,11 @@
     def mark_matching_resources(self, pattern, fn, n):
         "Mark all definitions matching a specified pattern with a reference."
         pattern = pattern.replace("+", r"\+")
+        pattern = os.sep + pattern + "$"
+        if os.sep == "\\":
+            pattern = pattern.replace("\\", "\\\\")
         try:
-            pattern = re.compile(os.sep + pattern + "$")
+            pattern = re.compile(pattern)
         except sre_constants.error:
             print >>sys.stderr, "wmlscope: confused by %s" % pattern
             return None
@@ -615,6 +622,8 @@
                         # Catches maps that look like macro names.
                         if (name.endswith(".map") or name.endswith(".mask")) and name[0] == '{':
                             name = name[1:]
+                        if os.sep == "\\":
+                            name = name.replace("/", "\\")
                         key = None
                         # If name is already in our resource list, it's easy.
                         if name in self.fileref and self.visible_from(name, fn, n):
Index: data/tools/wmllint
===================================================================
--- data/tools/wmllint	(revision 44128)
+++ data/tools/wmllint	(working copy)
@@ -1928,7 +1928,7 @@
     # and for all directories above it.
     up = fn
     while True:
-        if not up or up == os.sep:
+        if not up or is_root(up):
             break
         else:
             local_spellings += declared_spellings.get(up,[])

[itota]

and i hope the site (or this page) includes the download links to old versions of wmllint and its dependency so that everyone can convert ancient scenarios made b4 1.6version of wesnoth without downloading whole packages of wesnoth.

Download links
For wmllint 1.4: http://svn.gna.org/viewcvs/wesnoth/bran ... ls/wmllint
For wmllint 1.6: http://svn.gna.org/viewcvs/wesnoth/bran ... ls/wmllint

thank u Elvish_Hunter. i have tested the wmllint 1.4 with 1.8 dependencies(i.e. wmltools and so on in 1.8 packages), and have confirmed 1.2 map is correctly updated to 1.4 format.

[Anonymissimus]

Actually there was already some sort of tutorial that already covered most of what's needed.
http://wiki.wesnoth.org/Maintenance_tools
@Elvish_Hunter, feel free to improve my additional description...

[Elvish_Hunter]

@Elvish_Hunter, feel free to improve my additional description...

Anonymissimus, you did a good job . The only thing that I want to add here is that the bugged wmlscope is fixed in trunk, r44124:

loonycyborg says the other bit of the bug is fixed at r44124.

To use (at least partially) these bugfixes in 1.8 is needed to replace the wmltools.py with the one from trunk (that I did in my copy when the first part of the bug was fixed).

[Anonymissimus]

To use (at least partially) these bugfixes in 1.8 is needed to replace the wmltools.py with the one from trunk (that I did in my copy when the first part of the bug was fixed).

It won't work for long, since e.g. a function gets renamed, leading to a broken reference. Later on in 1.9 you'll need to apply the fixes manually. I begged esr to apply the fixes to 1.8 too yet he didn't.
And I don't know what loony is referring to with the other bit, I'm happy with these fixes. On the other hand, the sense of that check for mismatched references is also rather mysterious to me...

EDIT
You could keep copies from the current wmllint and wmltools.py on your harddrive and use them when downloading 1.8.4+ ...

[galicae]

So would it just be safer to do on Linux?
Anyway, thanks guys. I can see my old campaign in the 1.8 addon list!!

[Anonymissimus]

So would it just be safer to do on Linux?

Safer in terms of security ? Why should it ? (Despite that Linux is more secure in general.)
wmllint doesn't do everything, there's this thread about import syntax changes, you also need to consult it.

btw if someone on Linux finds a mistake in the description point it out; I wrote it partly from memory.

[galicae]

no, I meant safer in terms of efficiency, because of the win-specific bugs reported before. Thanks, though, I'll try it ASAP

[galicae]

omg, how long is it supposed to take?
I installed the enchant library to allow spellchecking, and after I entered the lines and pressed enter the machine went on to growl for half an hour or so. I had to terminate it manually, because I wanted to sleep.
i'll try again, though