Python Toolchain - Moving forward.

Discussion among members of the development team.

Moderator: Forum Moderators

User avatar
Elvish_Hunter
Posts: 1575
Joined: September 4th, 2009, 2:39 pm
Location: Lintanir Forest...

Re: Python Toolchain - Moving forward.

Post by Elvish_Hunter »

Tetha wrote:In fact, why'd you even need to run wmllint manually? We could go ahead and beef up GUI.pyw with inotify or equivalent tools to just run wmllint if you save files, so you get immediate feedback about the state of your code.
Although this may be technically possible, there are good reasons to not use pynotify. To quote from that page:
as a user, you ought to be suspicious of any programs you use that monitor filesystem activity
Xudo wrote:
jstitch wrote:a Wesnoth IDE for WML developers :D
There is one. Though, it is actually a Eclipse plug-in. http://forums.wesnoth.org/viewtopic.php?f=21&t=30880
Yes, but its interface to run the Python tools leaves very much to be desired, to say the least.
Pentarctagon wrote:
Problem of distribution of <anything> can be easily solved by creating web service, which will do the job.
Until the internet goes out :P
+1 to Pentarctagon.
Not only "until the internet goes out", but "until the website running the tools goes out" as well. I mean, publishing the call for help was enough to put wesnoth.org down, although only for a few minutes...
UnwiseOwl wrote:If the internet goes out, I'd like my next version of Wesnoth on CD posted to my address, please.
OK, if you're available to pay envelope, CD and postage we can talk :P :lol:
chaverma wrote:There is a tradeoff in your argument not mentioned: that keeping the status quo hinders the tools' progress on the fronts of correctness, usability and robustness. All of these contribute to that undesirable end state of "too much work, forget it."
Keeping the "status quo" means "not updating nor improving anything". And right now, I already merged some pull requests and did some pushes on my own.
chaverma wrote:There's no reason we can't store an artifact from the separate repo in the main repo in a predictable place, whose interface can be as simple as the double-click. It can yield the same usability standard you've established. Or using git subtrees as aquileia suggests.
OK, I must admit that I didn't know git subtrees. I wonder if there's some configuration that can be done, that will automatic pull commits from a repo to another, if we go for that route.
Anonymissimus wrote:Couldn't they call the wesnoth executable as a subprocess to use the C++ preprocessor!?
Yes, they can. Spawning a subprocess with Python is quite easy (it's one of the things that GUI.pyw does, combined with threading to not freeze the GUI). That needs to write some platform specific code to address the different executable names.
Anonymissimus wrote:wmllint should operate on the --preprocess output.
I never used that command line switch, but... won't that destroy all comments, including wmllint directives? :?
Anonymissimus wrote:As for trackplacer being broken...I used to be able to use it on various Linux systems, but not on windowses. T... IIRC the problem seemed to be upstream (pygtk or gtk), not trackplacer itself.
As for the first problem, I remember looking at the code a few times and noticing that there was some problem in detecting the BfW directory. The code that I use in GUI.pyw is very different, especially because I tried fooling it in every possible way that I could possibly think of (starting it out of the BfW data/tools directory, starting it through a symlink, ...). As for PyGTK, that's a lost cause: trackplacer currently uses PyGTK2, which will never be available on Python 3. Instead, for Python 3, there is the PyGObject module, that from a small test that I did is still heavily bugged and quite confusing to install. There's a reason if I decided that it should be moved to Tkinter, after all (actually, there's more than one...).
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
User avatar
Celtic_Minstrel
Developer
Posts: 2166
Joined: August 3rd, 2012, 11:26 pm
Location: Canada
Contact:

Re: Python Toolchain - Moving forward.

Post by Celtic_Minstrel »

Elvish_Hunter wrote:
Anonymissimus wrote:wmllint should operate on the --preprocess output.
I never used that command line switch, but... won't that destroy all comments, including wmllint directives? :?
Other special "comments" are already handled by the parser, so you could probably make it aware of "#wmlint". Or possibly better still, you could just throw in a new preprocessor directive (#pragma or whatever you want) which would be used to replace all wml* directives. (So, instead of "#wmlint: recognize chiefdruid" you might use something like "#pragma lint recognize chiefdruid"... though the recognize directive might also become unnecessary if preprocessor macros are being expanded.)

By the way, if these tools are to be rewritten as the opening post suggests, it might be worth taking a look at wmlxgettext as well. It's currently Perl, though.
Author of The Black Cross of Aleron campaign and Default++ era.
Former maintainer of Steelhive.
User avatar
GunChleoc
Translator
Posts: 506
Joined: September 28th, 2012, 7:35 am
Contact:

Re: Python Toolchain - Moving forward.

Post by GunChleoc »

Celtic_Minstrel wrote:By the way, if these tools are to be rewritten as the opening post suggests, it might be worth taking a look at wmlxgettext as well. It's currently Perl, though.
We are currently working on improving out xgettext for Lua script at Widelands - it supports simple ngettext and pgettext calls as well. The code is GPL v2, so you should be able to reuse anything that is usable to you:

https://bazaar.launchpad.net/~widelands ... gettext.py
Melch
Posts: 1
Joined: September 2nd, 2015, 3:09 am

Re: Python Toolchain - Moving forward.

Post by Melch »

Hello,

I'm ready to help you guys out. Most of my work in python has been either doing data analysis/modeling or mathematical research. I'll start getting familiar with the codebase during the next couple of days. I'll keep an eye out in #wesnoth-dev as well. I am a perl developer as well, so if you want me to look at something else using that crazy language, I can.

Melch
User avatar
Elvish_Hunter
Posts: 1575
Joined: September 4th, 2009, 2:39 pm
Location: Lintanir Forest...

Re: Python Toolchain - Moving forward.

Post by Elvish_Hunter »

GunChleoc wrote:We are currently working on improving out xgettext for Lua script at Widelands - it supports simple ngettext and pgettext calls as well.
Slight off-topic:
Melch wrote:Most of my work in python has been either doing data analysis/modeling or mathematical research. I'll start getting familiar with the codebase during the next couple of days. I'll keep an eye out in #wesnoth-dev as well. I am a perl developer as well, so if you want me to look at something else using that crazy language, I can.
As CelticMinstrel said, we have a script called wmlxgettext, which is written in Perl and whose purpose is to extract translatable strings from WML and Lua files.
From today's IRC logs:

Code: Select all

20150902 03:29:08< shadowm> Rewriting wmlxgettext in Python was not worth the effort.
However, the Python wmlxgettext had serious troubles, in the sense that it was built to work more or less like wmllint (relying on WmlIterator...), while the Perl version does its job almost exclusively with regexps - and using two different technologies invariably leads to two different results :| . In fact, I remember that the Python version was removed first because its output wasn't guaranteed to be the same as the Perl version, and second because of performance issues (slower than the Perl one).
I'm of the opinion that, given the improvements to Unicode support in Python 3 (handling Unicode strings is four times faster than in Python 2 - from the official documentation), converting it as a drop-in replacement (same API, same output, avoid importing crazy stuff...) is worth the effort.
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
User avatar
GunChleoc
Translator
Posts: 506
Joined: September 28th, 2012, 7:35 am
Contact:

Re: Python Toolchain - Moving forward.

Post by GunChleoc »

Elvish_Hunter wrote:
GunChleoc wrote:We are currently working on improving out xgettext for Lua script at Widelands - it supports simple ngettext and pgettext calls as well.
Slight off-topic:
Thanks, I'll pass this information on :)

ETA: the dev who implemented this said that the Scanner code is public and in production world-wide. In case it ever should get dropped, we can simply copy the code over.
Post Reply