Why WML?

[JavaBean]

I don't understand the motivation behind using WML. It looks like XML, except 1) it allows preprocessor macros, and 2) it is less consistent.

Is there a reason Wesnoth uses WML, other than tradition? XML would be a more standards-based approach, and there are many excellent tools for editing and viewing XML files, but not many for WML files (examples include Notepad and emacs).

I think it would become easier to write and modify WML-ish add-ons if they were written in XML instead. Additionally, I think having a DTD/schema available will help make finding problems easier (by using a validating parser), and that all preprocessor-like syntax can be replaced by XML-based concepts. For example: #ifdef can easily be replaced by <if></if>; the

Code: Select all

[tag]
name1=val1
name2=val2
...
[/tag]

idiom can be replaced with element attributes; macro definitions such as {DEFAULT_MUSIC_PLAYLIST} can be replaced by XML external entity references, and so on. I haven't looked through all the inconsistencies, but it seems like most of the changes would be straightforward.

I would like to know if the developers/community share my feeling that XML would be better than WML. If so, I would be interested in helping create tools that will help this conversion. Of primary concern are:
1) A tool to convert WML to XML. This would be absolutely necessary to get any kind of XML movement going. It would also mean that users who prefer the WML syntax can still use it. This could be used either once, before contributing new content, or at Wesnoth launch time, to parse old WML files. Writing this will take some hand-coded C++ (perl?), and a little human interaction, to decide where to place external entities, and so on.
2) A tool to convert XML to WML. This is not strictly necessary, but would be very useful. This way, if the XML parsing code takes longer to write than the conversion code, XML can be converted into WML and then read in the old way. This would probably be the easiest thing to write, because XSLTs are tailor-made for this.
3) A C++ module to process the XML. Probably bundling libxml with Wesnoth will solve a lot of this problem. I haven't looked at the details, but hopefully the code for reading WML is modular enough that it should be fairly straightforward to get it working with XML instead.

So, what do you think? Is XML a good idea? Is anyone else interested?

Note also that this is my first nontrivial post here. If this should instead be posted as a bug report/RFE, or if I've done something wrong in this post, please let me know.

[JW]

<if>
XML>WML
<then>
yes
<else>
no
</if>

[Noyga]

Because it would eat a lot of time for a minimal, unclear benefit.
Btw things like #ifdef are necessary to avoid pollution from an unrelated content.
Even if two campaigns use the same unit names, one shouldn't overwrite the other one.

[JavaBean]

I don't understand your point about #ifdefs, possibly because I haven't looked closely at the WML engine. But it seems to me that if the <if> construct had the same semantics as #ifdef currently does, it wouldn't be a problem. The parser could read <if>, check the condition, and then ignore all children if the condition is not met.
Regarding your objection to the time required: do you mean run time, or the time required to switch over from WML? I understand that it may be judged useless because of the amount of work that has gone into WML already, but I am interested in finding out whether the idea, given infinite resources, would be desirable. That is, if there is a benefit to WML over XML other than its established codebase, I would like to find out about it.
I agree that the potential benefits are not huge, but the large number of tools that can understand XML could be beneficial, and it would shorten the learning curve for addon content, and make it easier for other programs to interoperate with Wesnoth.
If the change to XML is opposed from an ideology standpoint rather than a logistical one, I will stop worrying about it. But I am a fan of XML and standardization, so if logistics are the primary concern I will think about it some more.

[appleide]

I don't understand your point about #ifdefs, possibly because I haven't looked closely at the WML engine. But it seems to me that if the <if> construct had the same semantics as #ifdef currently does, it wouldn't be a problem. The parser could read <if>, check the condition, and then ignore all children if the condition is not met.
Regarding your objection to the time required: do you mean run time, or the time required to switch over from WML? I understand that it may be judged useless because of the amount of work that has gone into WML already, but I am interested in finding out whether the idea, given infinite resources, would be desirable. That is, if there is a benefit to WML over XML other than its established codebase, I would like to find out about it.
I agree that the potential benefits are not huge, but the large number of tools that can understand XML could be beneficial, and it would shorten the learning curve for addon content, and make it easier for other programs to interoperate with Wesnoth.
If the change to XML is opposed from an ideology standpoint rather than a logistical one, I will stop worrying about it. But I am a fan of XML and standardization, so if logistics are the primary concern I will think about it some more.

Let's not put Wesnoth in a project almost as fruitless as the TC project that would take so much time... At least the TC project caused graphics to be redone which actually have a great benefit, imho.

[Leo]

Pro:
1. Cause XML is standard, parsers for XML is aviable for all major platforms as libraries. So you can use this libs as you use SDL. So doing this actually reduces amount of code which you must support.
2. XML files can be read by many programs. Ex: you can show XML files in browser with CSS'es, so units.wesnoth.org script will be another thing simpler to mantain.
3. XML files can be validated, so you can just reject to load invalid XML file.
Contra: This requires a lot of work, i think.

[Viliam]

WML is similar to XML, but with a few important exceptions:

1) It accepts simple expressions as values. For example when there is a string value, it can be hardcoded string, localized string, or a concatenation of hardcoded and localized strings. And you can use variables in some tags.

2) The WML macros cannot be replaced with XML entities... as far as I know XML entities do not have parameters. The macros would have to be translated e.g. by new tags.

So the change from WML to XML would not be straightforward, some things would have to be changed. But I think it is possible.


The correct approach would be, in my opinion, to design your own Wesnoth DTD, and make a "compiler" from XML to WML. Then try to re-write the default campaigns to that XML -- as a proof of concept. If you succeed with this, and if the XML version will be easier to read, other campaign authors may convert to your system. Then you do not have to care whether Wesnoth officially supports XML or not, you can write in XML anyway.

The XML language could be even stronger than current WML. For example, in WML there is no event like "every night, do this...". However you could implement such event in XML. And the "compiler" would just look at the tags to see which turns are in night, and replace the event with "in turns 5,6,11,12,17,18, do this...". The "compiler" does not have to be straightforward.


I imagine something like this:

Code: Select all

<campaign
  id="heir_throne"
  define="CAMPAIGN_HEIR_TO_THE_THRONE"
  textdomain="wesnoth-httt"
>

  <campaigninfo
    rank="10"
    icon="konrad-commander.png"
    image="portraits/Heir_To_The_Throne/konrad.png"
    first_scenario="The_Elves_Besieged"
  >
    <name><string l10n="yes">Heir to the Throne</string></name>
    <description><string l10n="yes">Fight to gain back the throne of Wesnoth, of which you are the legitimate heir.</string></description>
    <difficulties>
      <difficulty define="EASY" icon="elvish-fighter.png">
        <name><string l10n="yes">Fighter</string></name>
        <description><string l10n="yes">(easiest)</string></description>
      </difficulty>
      <difficulty define="NORMAL" icon="elvish-hero.png">
        <name><string l10n="yes">Hero</string></name>
      </difficulty>
      <difficulty define="HARD" icon="elvish-champion.png">
        <name><string l10n="yes">Champion</string></name>
        <description><string l10n="yes">(hardest)</string></description>
      </difficulty>
    </difficulties>
  </campaigninfo>

  <xi:include href="scenarios/Heir_To_The_Throne/Elves_Besieged.xml"/>
  <!-- ... -->

</campaign>

[torangan]

It's very easy: when starting Wesnoth Sirp decided that he didn't like XML and invented WML. The chance that WML will ever be replaced with XML is close to zero even if you provide the patch yourself. Having a custom language where we can define the rules is just more flexible.

[Noyga]

Also the preprocessor directive are evaluated before the cache is made, so only when necessary (ie the WML code has changed), while the WML is evaluated everytime.
When playing a campaign you usally don't have the other campaigns in the cache, they are not necessary (there is one cache by campaign...).

[Darth Fool]

To answer the question that is the subject of this thread by title, although not by the content of the first post: WML was originally a very simple language that served one purpose, to define the configurations for the game. As such XML would have been way overkill. As time has gone by and the game has been made more flexible, various features have been added. These features have grown organically and not as the result of a grand design. As a result it lacks some of the elegance that a purposefully designed file structure like XML has, but it also avoids some of the bloat as well. It has some syntax that is very convenient for the specific tasks at hand that probably would be a pain to do in XML, while other idiosyncracies of WML make it so that the WML for certain tasks requires tortuous convolutions. For any file/language specification this will always be the case to some degree, the exceptions being when it is always tortuous to do things (see intercal).

In my mind there is not a lot to be gained by converting to an XML format, although I am not strictly opposed to it. While some might argue that using a standard format means we can reuse existing code, I would respond that the WML parser is also existing code and the basics of it have not changed in a long time. All recent changes in WML are really changes in core functionality of the game and how it interacts with the data in WML, not in the WML parser itself. As for having lots of XML editors, well, I have yet to see an XML editor that is really superior to just using a text editor for the type of files that would be needed for wesnoth. Not only that, but everyone has access to a text editor. Not everyone has access to an XML editor, and if you are editing the XML in a text editor, than there is no benefit.

So, if you really are interested in using XML instead of WML, I would change the order that you pursue things. Do #2 first. As you said it is the easiest to do, and it will enable you to try and create some simple WML with your favorite XML editor and see if it really is easier than using a text editor on WML. After that you can go back and do #1 and run it on some existing WML. Then run #2 on the resulting files and see if you get back what you started with. This will undoubtedly reveal a lot of corner cases in your translation XML<->WML that you didn't anticipate or know about. Only once you have succesfully got #2 and #1 working on a variety of campaigns that include a wide selection of WML features would I consider pursuing step 3. If you have done #2 and #1 properly, you should understand WML well enough to attempt it, or decide that it is not worth the effort.

As with all potential contributions, you will not get a "green light" from the development team before you have a working example. You should really only do this if it is something that will give you pleasure, regardless of whether or not it is accepted into the mainline. I will say that such a radical change would not only require a working patch, but also a lot of convincing that it is a real major step forward.

[Jym]

For any file/language specification this will always be the case to some degree, the exceptions being when it is always tortuous to do things (see intercal).

We need a befunge-like syntax for WML !

[Ethan]

I don't expect official devs to do it and I won't want them to do it. (That's true that it would need work, and maybe a lot without a lot of benefits.)

About why xml-editor may be better, there is the candy synthax highlighting, but also the fact that the creation can be guided/tested by usind dtd or xschema but that is true that not everybody has an xml editor... However, such editor may become the first (very basic) WML(xml_based) editor.

I may think to do it when I would have learn enough WML.

And yes the new WML(xml_based) parser would be a xml parser used with specific xpath variables...
The prepocessor would be basically xslt, I don't mean user will have to write xslt, but that if we define specific preprocessor tag, it would be pretty easy to create a xslt XML->XML.

[torangan]

If you want to write XSLT templates for WML <-> XML just go ahead and do it. 1.2 WML should be stable enough to allow such a feature. Then you can use a XML editor to work on Wesnoth. Just don't expect Wesnoth to be ported to XML.

[Viliam]

One of the problems is that despite so many people (including me) proposed to replace WML by XML, no one has yet written a complete specification how such XML should look (or at least specification containing work with variables and localized strings). This makes it impossible to decide whether such XML would truly be simpler to edit than WML, or just some parts would be simpler, other ones more difficult, and no overall improvement.

Please note that by requesting switch from WML to XML you request abandoning a complex system which works (sometimes not perfectly, but it evolves), without having any idea how the new system should work. Such switch would be a complete madness. If there would ever happen such switch (which many developers oppose, anyway), having the (DTD) specification of XML would be absolutely necessary. If no one is able to produce it, that's one more reason to not switch.

XML files are nice for storing data; which is by the way where WML files are even nicer. WML starts looking terrible when it is used for writing algorithms. For an example how XML looks in such situation, look at some non-trivial XSLT examples... not very nice, too. Other languages (such as Python, Lua, etc) are much nicer for writing expressions and algorithms.


Therefore,
If anyone proposes "use XML" without posting a complete specification with examples (or at least complete enough to show work with expressions and localized strings), such idea provides nothing useable, and belongs to FPI list.


Many situations where WML is painful to use, can be repaired without abandoning the language. For example it would be nice to support evaluating expressions, such as:

Code: Select all

[set_variable]
  name= x
  value= 2 * $y + 7
[/set_variable]

Implementing the example above in XML would not be easier...