Call for Code Samples / WML Unit Tests
Moderator: Forum Moderators
Forum rules
- Please use [code] BBCode tags in your posts for embedding WML snippets.
- To keep your code readable so that others can easily help you, make sure to indent it following our conventions.
Call for Code Samples / WML Unit Tests
I'm writing to announce a development feature that I hope will help to solve two problems.
---
So, if you have an idea for a test, whether or not it's based on your add-on, please consider submitting it to us. We will continue to write tests ourselves, but to get the full benefit we would like to get some from UMC as well (particularly, some tests for the lua api would be nice to have). You can write here, or you are welcome to make a github pull request with some tests. If you aren't sure how to format it all properly, we can help you with that.
Problem #1: UMC Code Samples / Backwards Compatibility.
Problem #2: Automated testing.
What this means for you:
What exactly is a unit test?
How to run unit tests yourself
So, if you have an idea for a test, whether or not it's based on your add-on, please consider submitting it to us. We will continue to write tests ourselves, but to get the full benefit we would like to get some from UMC as well (particularly, some tests for the lua api would be nice to have). You can write here, or you are welcome to make a github pull request with some tests. If you aren't sure how to format it all properly, we can help you with that.
-
- Inactive Developer
- Posts: 2461
- Joined: August 15th, 2008, 8:46 pm
- Location: Germany
Re: Call for Code Samples / WML Unit Tests
I was just browsing old fixed bug reports of mine for suitable wml snippets, things that would have been detected by such a unit test. (This one feels appropriate http://gna.org/bugs/index.php?20734) Maybe that's a general strategy to get more tests ?
Unfortunately though, it seems to me that most bugs I reported wouldn't have been found by such a test.
Unfortunately though, it seems to me that most bugs I reported wouldn't have been found by such a test.
projects (BfW 1.12):
A Simple Campaign: campaign draft for wml starters • Plan Your Advancements: mp mod
The Earth's Gut: sp campaign • Settlers of Wesnoth: mp scenario • Wesnoth Lua Pack: lua tags and utils
updated to 1.8 and handed over: A Gryphon's Tale: sp campaign
A Simple Campaign: campaign draft for wml starters • Plan Your Advancements: mp mod
The Earth's Gut: sp campaign • Settlers of Wesnoth: mp scenario • Wesnoth Lua Pack: lua tags and utils
updated to 1.8 and handed over: A Gryphon's Tale: sp campaign
Re: Call for Code Samples / WML Unit Tests
Yeah I guess its good practice to write a test when you fix a bug, when you can.
Another possible strategy is to mine the wiki code examples. Then at least if the test breaks it either signals a bug, or it's a reminder to update the wiki.
I remember other issues discussed on the forums that might make good tests, for instance, someone was asking about how first-strike interacts with berzerk, and presumably everyone loaded up a test scenario and watched the video slowly to try to make sense of it. A different way is, write a test, see which way makes it pass, then commit that forever -- if it ever changes then we'll know. Besides this, even if the test isn't targeted at a specific bug, it may still catch bugs it wasn't intended for.
Another possible strategy is to mine the wiki code examples. Then at least if the test breaks it either signals a bug, or it's a reminder to update the wiki.
I remember other issues discussed on the forums that might make good tests, for instance, someone was asking about how first-strike interacts with berzerk, and presumably everyone loaded up a test scenario and watched the video slowly to try to make sense of it. A different way is, write a test, see which way makes it pass, then commit that forever -- if it ever changes then we'll know. Besides this, even if the test isn't targeted at a specific bug, it may still catch bugs it wasn't intended for.
Re: Call for Code Samples / WML Unit Tests
If there is not currently a test to check if 2 different events both name=starts are fired in the order they were defined, I could try write one for it
Just a shame multiplayer safety is out of unit test range, was what more interested me
Just a shame multiplayer safety is out of unit test range, was what more interested me
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II
World Conquest II
Re: Call for Code Samples / WML Unit Tests
Im not 100% sure, if this is something that can be done with a unit test, I will just mention it. While in Wesnoth version 1.11.13+- the game checks the unit and code for spaces and not allowed characters, it does not check the images. Maybe a unit test could not only check all files and the image folder as well.
The future belongs to those, who believe in the beauty of their dreams.
Developer of: Trapped, Five Fates, Strange Legacy, Epical, UR Epic Era
Dungeonmasters of Wesnoth, Wild Peasants vs Devouring Corpses, Dwarf Dwarfson Dwarvenminer
Developer of: Trapped, Five Fates, Strange Legacy, Epical, UR Epic Era
Dungeonmasters of Wesnoth, Wild Peasants vs Devouring Corpses, Dwarf Dwarfson Dwarvenminer
Re: Call for Code Samples / WML Unit Tests
No, we don't have a test like that right now. That would be a good testtekelili wrote:If there is not currently a test to check if 2 different events both name=starts are fired in the order they were defined, I could try write one for it
Yeah so I did think about this.tekelili wrote:Just a shame multiplayer safety is out of unit test range, was what more interested me
In the majority of cases replay-safety and mp-safety should be the same -- the info that, for example, the server sends to an mp observer, is more or less the same as what wesnoth will load from a replay.
Spoiler:
Spoiler:
Hmm this is interesting, I wonder if this is an oversight on our part.Heindal wrote:Im not 100% sure, if this is something that can be done with a unit test, I will just mention it. While in Wesnoth version 1.11.13+- the game checks the unit and code for spaces and not allowed characters, it does not check the images. Maybe a unit test could not only check all files and the image folder as well.
I searched for the commits that prohibited spaces and such in filenames, I think it's these two:
https://github.com/wesnoth/wesnoth/comm ... 53ac5686ee
https://github.com/wesnoth/wesnoth/comm ... 1386167918
My feeling is that the game should complain for images just the same as the other kinds of files.
But anyways, yeah I think in principle we should be able to test the code that loads images in a unit test.
Spoiler:
-
- Inactive Developer
- Posts: 2461
- Joined: August 15th, 2008, 8:46 pm
- Location: Germany
Re: Call for Code Samples / WML Unit Tests
This makes up for an excellent test. It is something that could easily break, is important for wml authors and hard to test manually.iceiceice wrote:No, we don't have a test like that right now. That would be a good testtekelili wrote:If there is not currently a test to check if 2 different events both name=starts are fired in the order they were defined, I could try write one for it
It should be generalized - "If we have 2 or more events of the same type, do they fire in the order which appears in the scenario file ?" I'm sure it can be macroized to cover all event types or at least all important ones.
projects (BfW 1.12):
A Simple Campaign: campaign draft for wml starters • Plan Your Advancements: mp mod
The Earth's Gut: sp campaign • Settlers of Wesnoth: mp scenario • Wesnoth Lua Pack: lua tags and utils
updated to 1.8 and handed over: A Gryphon's Tale: sp campaign
A Simple Campaign: campaign draft for wml starters • Plan Your Advancements: mp mod
The Earth's Gut: sp campaign • Settlers of Wesnoth: mp scenario • Wesnoth Lua Pack: lua tags and utils
updated to 1.8 and handed over: A Gryphon's Tale: sp campaign
- Elvish_Hunter
- Posts: 1576
- Joined: September 4th, 2009, 2:39 pm
- Location: Lintanir Forest...
Re: Call for Code Samples / WML Unit Tests
As we discussed elsewhere, here there is my proposed test.
In a scenario, I need to iterate backwards a WML array, because I need to delete items from it. Sure, it's bad practice, but so far my attempts to do it in the right way (iterate the array, then copy only the elements that pass the check in a new array, then replace the old array with the new array) have gone badly, maybe just because I didn't yet have the right idea. So I developed two helper macros, REV_FOREACH and PREVIOUS, that are specular to FOREACH and NEXT:I'm using this code in the First Contact scenario in TSoG, starting from line 1132 onwards. I'm attaching it, so you can see its purpose.
In a scenario, I need to iterate backwards a WML array, because I need to delete items from it. Sure, it's bad practice, but so far my attempts to do it in the right way (iterate the array, then copy only the elements that pass the check in a new array, then replace the old array with the new array) have gone badly, maybe just because I didn't yet have the right idea. So I developed two helper macros, REV_FOREACH and PREVIOUS, that are specular to FOREACH and NEXT:
Code: Select all
# wmlindent: start ignoring
#define REV_FOREACH ARRAY_VALUE VAR
# last index of an array is always length-1
{VARIABLE {VAR} "$(${ARRAY_VALUE}.length)-1"}
[while]
[variable]
name={VAR}
greater_than_equal_to=0
[/variable]
[do]
#enddef
#define PREVIOUS VAR
[set_variable]
name={VAR}
sub=1
[/set_variable]
[/do]
[/while]
{CLEAR_VARIABLE {VAR}}
#enddef
# wmlindent: stop ignoring
- Attachments
-
- 05_First_Contact.cfg
- (39.74 KiB) Downloaded 686 times
Current maintainer of these add-ons, all on 1.16:
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
The Sojournings of Grog, Children of Dragons, A Rough Life, Wesnoth Lua Pack, The White Troll (co-author)
Re: Call for Code Samples / WML Unit Tests
I have not forgot this, but it is in the waiting room for some months until I manage to get a campaign working in 1.11tekelili wrote:If there is not currently a test to check if 2 different events both name=starts are fired in the order they were defined, I could try write one for it
however I employ sometime in think how could be the best way to do and what cases should be tested. I discarded [era] events as it looks not having expected behavior currently. There is other issue I had with introducing an event inside a unit that I have no clue if is expected behavior. As code was complicated, here is a brief description:
I had a recruit event that made basically 2 things: Attach a magic item and modify unit xp unstoring to cause leveling if neccessary. In a particular case, one magic item introduced a name=postadvance event into unit to change halo when levelin. My issue was that posadvanced event was not fired in that recruit event after have copyed it into unit. However I had what I wanted attaching magic item in prerecruit and then causing the advance in recruit event. So the event inside unit behavior was: "I dont exist until current event ends"
Is this expected or a bug?
(this was running BfW 1.10.6 in a pc with Windows)
-----EDIT-----
here is my issue resumed in code: I introduced this in hamlets scenario for my test (tested in BfW 1.10.6)
Code: Select all
[variables]
[event]
id=darkness_aura
name=post advance
first_time_only=no
{VARIABLE unit.halo halo/darkens-aura.png}
[unstore_unit]
variable=unit
[/unstore_unit]
[/event]
[/variables]
[event]
name=recruit
first_time_only=no
[set_variables]
mode=append
name=unit.event
[insert_tag]
name=value
variable=event
[/insert_tag]
[/set_variables]
{VARIABLE_OP unit.experience add 50}
[unstore_unit]
variable=unit
fire_event=yes
[/unstore_unit]
[/event]
To have desired behavior I used this code
Code: Select all
[event]
name=prerecruit
first_time_only=no
[set_variables]
mode=append
name=unit.event
[insert_tag]
name=value
variable=event
[/insert_tag]
[/set_variables]
[unstore_unit]
variable=unit
fire_event=yes
[/unstore_unit]
[/event]
[event]
name=recruit
first_time_only=no
[store_unit]
variable=unit
[filter]
x,y=$unit.x,$unit.y
[/filter]
[/store_unit]
{VARIABLE_OP unit.experience add 50}
[unstore_unit]
variable=unit
fire_event=yes
[/unstore_unit]
[/event]
Code: Select all
[variables]
[event]
id=darkness_aura
name=post advance
first_time_only=no
{VARIABLE unit.halo halo/darkens-aura.png}
[unstore_unit]
variable=unit
[/unstore_unit]
[/event]
[/variables]
[event]
name=recruit
first_time_only=no
[store_unit]
variable=unit
[filter]
x,y=$unit.x,$unit.y
[/filter]
[/store_unit]
[set_variables]
mode=append
name=unit.event
[insert_tag]
name=value
variable=event
[/insert_tag]
[/set_variables]
{VARIABLE_OP unit.experience add 50}
[unstore_unit]
variable=unit
fire_event=yes
[/unstore_unit]
[/event]
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II
World Conquest II
Re: Call for Code Samples / WML Unit Tests
tekelili:
I'm not sure exactly what behavior was intended, but looking at your code I think it's not a bug and it's what I would expect. (Someone else may disagree though.)
Here's my understanding:
For every event name, there is a list. When an event is declared, it gets added to the list for that name. When an event name is fired, all of the events fire in the order that they were declared.
However, when we fire an event, if it generates more events, those new events don't go on the list until we are done firing. The mere act of adding events to the list cannot cause events to fire, so this guarantees that the process will stop.
I guess you could imagine that it would be otherwise, and that events should be added to the list immediately but then it would be easy to cause infinite loops.
Edit:
I did some code inspection,
the code that adds the new events to the queue is here I think:
https://github.com/wesnoth/wesnoth/blob ... p.cpp#L137
When this guy is created, it swaps out the event queue,
https://github.com/wesnoth/wesnoth/blob ... p.cpp#L505
and restores it in its destructor. However, I don't understand why there's the check for "!done()" there, it sounds like its for exception handling, but I think it should be putting the newly registered events in the queue regardless...
Anyways I think it's pretty clear that the intended behavior is, "don't think about any new events that got registered until after we are done firing all events." So in your case, if you register the post_advance event at the same time as you unstore the unit, it won't fire the first time, only at subsequent advances.
I'm not sure exactly what behavior was intended, but looking at your code I think it's not a bug and it's what I would expect. (Someone else may disagree though.)
Here's my understanding:
For every event name, there is a list. When an event is declared, it gets added to the list for that name. When an event name is fired, all of the events fire in the order that they were declared.
However, when we fire an event, if it generates more events, those new events don't go on the list until we are done firing. The mere act of adding events to the list cannot cause events to fire, so this guarantees that the process will stop.
I guess you could imagine that it would be otherwise, and that events should be added to the list immediately but then it would be easy to cause infinite loops.
Edit:
I did some code inspection,
the code that adds the new events to the queue is here I think:
https://github.com/wesnoth/wesnoth/blob ... p.cpp#L137
When this guy is created, it swaps out the event queue,
https://github.com/wesnoth/wesnoth/blob ... p.cpp#L505
and restores it in its destructor. However, I don't understand why there's the check for "!done()" there, it sounds like its for exception handling, but I think it should be putting the newly registered events in the queue regardless...
Anyways I think it's pretty clear that the intended behavior is, "don't think about any new events that got registered until after we are done firing all events." So in your case, if you register the post_advance event at the same time as you unstore the unit, it won't fire the first time, only at subsequent advances.
Re: Call for Code Samples / WML Unit Tests
@iceiceice: If it is expected behavior, then there is no a bug, is fine
But I think this should be explained in wiki at [event]. Before WML I had only experience with primitive coding languages and I asumed that [events] were kinda subroutines... and is not intuitive be unable to create a subroutine in memory
But I think this should be explained in wiki at [event]. Before WML I had only experience with primitive coding languages and I asumed that [events] were kinda subroutines... and is not intuitive be unable to create a subroutine in memory
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II
World Conquest II
Re: Call for Code Samples / WML Unit Tests
tekelili:
Actually when I wrote I didn't completely understand the code, so I went back and ran some actual tests:
Here's the tests I wrote. All of them pass except for the 4th:
So actually the thing I was saying is not accurate, I don't have any idea why your example wasn't working.
(In particular, your example is roughly like test #2, so I would think the event handler in your case would fire if that test is passing.)
It might be a bug, more info to come hopefully...
(Note: Tests run in version 1.13.0 (but I don't think any of the event handler code has changed))
EDIT:
Okay, here are the subsequent tests that I wrote. All of these pass. I think that there might just be something funky going on with your insert_tag syntax?
Note: These tests have now been committed to the project.
Actually when I wrote I didn't completely understand the code, so I went back and ran some actual tests:
Here's the tests I wrote. All of them pass except for the 4th:
Code: Select all
{GENERIC_UNIT_TEST "events_in_events_1" (
[event]
name=test
{VARIABLE pass_test 1}
[/event]
[event]
name=start
{VARIABLE pass_test 0}
[fire_event]
name=test
[/fire_event]
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_2" (
[event]
name=start
{VARIABLE pass_test 0}
[event]
name=test
{VARIABLE pass_test 1}
[/event]
[fire_event]
name=test
[/fire_event]
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_3" (
[event]
name=start
[event]
name=test
{VARIABLE pass_test 1}
[/event]
[/event]
[event]
name=start
{VARIABLE pass_test 0}
[fire_event]
name=test
[/fire_event]
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_4" (
[event]
name=start
{VARIABLE pass_test 0}
[fire_event]
name=test
[/fire_event]
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
[event]
name=start
[event]
name=test
{VARIABLE pass_test 1}
[/event]
[/event]
)}
(In particular, your example is roughly like test #2, so I would think the event handler in your case would fire if that test is passing.)
It might be a bug, more info to come hopefully...
(Note: Tests run in version 1.13.0 (but I don't think any of the event handler code has changed))
EDIT:
Okay, here are the subsequent tests that I wrote. All of these pass. I think that there might just be something funky going on with your insert_tag syntax?
Code: Select all
{GENERIC_UNIT_TEST "events_in_events_5" (
[event]
name=start
{VARIABLE pass_test 0}
{UNIT 1 "Orcish Grunt" 1 1 ()}
[store_unit]
[filter]
x=1
y=1
[/filter]
variable=my_unit
kill=yes
[/store_unit]
[event]
name=post_advance
{VARIABLE pass_test 1}
[/event]
{VARIABLE_OP my_unit.experience add 50}
[unstore_unit]
variable=my_unit
fire_event=yes
[/unstore_unit]
[/event]
[event]
name=start
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_6" (
[event]
name=start
{VARIABLE pass_test 0}
{UNIT 1 "Orcish Grunt" 1 1 ()}
[store_unit]
[filter]
x=1
y=1
[/filter]
variable=my_unit
kill=yes
[/store_unit]
[set_variables]
name=ev0
[value]
name=post_advance
{VARIABLE pass_test 1}
[/value]
[/set_variables]
[insert_tag]
name=event
variable=ev0
[/insert_tag]
[fire_event]
name=test
[/fire_event]
{VARIABLE_OP my_unit.experience add 50}
[unstore_unit]
variable=my_unit
fire_event=yes
[/unstore_unit]
[/event]
[event]
name=start
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_7" (
[event]
name=start
{VARIABLE pass_test 0}
{UNIT 1 "Orcish Grunt" 1 1 ()}
[store_unit]
[filter]
x=1
y=1
[/filter]
variable=my_unit
kill=yes
[/store_unit]
[set_variables]
name=ev0
[value]
name=post_advance
{VARIABLE pass_test 1}
[/value]
[/set_variables]
[event]
name=test
[insert_tag]
name=event
variable=ev0
[/insert_tag]
[/event]
[fire_event]
name=test
[/fire_event]
{VARIABLE_OP my_unit.experience add 50}
[unstore_unit]
variable=my_unit
fire_event=yes
[/unstore_unit]
[/event]
[event]
name=start
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
{GENERIC_UNIT_TEST "events_in_events_8" (
[event]
name=start
{VARIABLE pass_test 0}
{UNIT 1 "Orcish Grunt" 1 1 ([variables]
[my_event]
name=post_advance
{VARIABLE pass_test 1}
[/my_event]
[/variables])}
[store_unit]
[filter]
x=1
y=1
[/filter]
variable=my_unit
kill=yes
[/store_unit]
[insert_tag]
name=event
variable=my_unit.variables.my_event
[/insert_tag]
{VARIABLE_OP my_unit.experience add 50}
[unstore_unit]
variable=my_unit
fire_event=yes
[/unstore_unit]
[/event]
[event]
name=start
{RETURN ({VARIABLE_CONDITIONAL pass_test equals 1})}
[/event]
)}
Re: Call for Code Samples / WML Unit Tests
My syntax is really funky... but it was for a good reasoniceiceice wrote:Okay, here are the subsequent tests that I wrote. All of these pass. I think that there might just be something funky going on with your insert_tag syntax?
In my scenario it had totally sense, as I had lot of complicated stuff in my magic items and I had some delayed variable substitution not working until I employed it (I think problem was with feeding event)
Btw, sorry if I am wrong... but it looked to me that you are writing the post_advance event in the scenario instead inside the unit?
Edit: It is off topic, but just for clarification that was only syntax I found to have this working
Spoiler:
Last edited by tekelili on June 21st, 2014, 9:30 pm, edited 3 times in total.
Be aware English is not my first language and I could have explained bad myself using wrong or just invented words.
World Conquest II
World Conquest II
Re: Call for Code Samples / WML Unit Tests
Just wanted to say: this is a good idea. The previous way was to add an example to the scenario-test.cfg. I've added some tests there myself that would be good candidates for inclusion.
- "Start Dynamic Events!" menu item, tests recursive insert_tag behavior including infinite insert recursion prevention
- [filter_radius] test, captures connected villages
- "Start Dynamic Events!" menu item, tests recursive insert_tag behavior including infinite insert recursion prevention
- [filter_radius] test, captures connected villages
http://www.wesnoth.org/wiki/User:Sapient... "Looks like your skills saved us again. Uh, well at least, they saved Soarin's apple pie."
Re: Call for Code Samples / WML Unit Tests
Yeah so I worked slowly to the hard case, I only put the event in the unit in test 8 above. In that case it still works though.tekelili wrote: Btw, sorry if I am wrong... but it looked to me that you are writing the post_advance event in the scenario instead inside the unit?
I will write again after I look at this feeding code...