Make Python Scripts useable from WML

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderators: Forum Moderators, Developers

Forum rules
Before posting a new idea, you must read the following:
Post Reply
arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Make Python Scripts useable from WML

Post by arielbys » May 13th, 2008, 3:45 pm

Okay, i want to create some campaign with a Random Map generator with an algorithm that is (obliviously) hard to translate to WML. Can wesnoth support using Python scripts in a scenario:
For example:

Notes:
In my version, it works as addition to Set Variable. If you use entry method, then you put the script in <STP script PTS>, when STP stands for STart Python and PTS is STP backwards then put the entry method and parameters in ? (This wouldn't work if you use <SPT or
PTS> in the file). Otherwise, just write the script.
(i remember that if you use both closing brackets and space you can't use Macros, so i didn't used). Arguments use format translation. Code translates variables like format, but a ? is needed before the $ (like {VARIABLE num python ?$v1|+2}).
Example 1 (a part of what i would use in the scenario):
[set_variable]
name=map
python=<STP {@War_of_Iaar\scripts\mapgen.py} PTS>?genmap(96, 48, 'Uu', 'Xu', ',', 'border_size=1
usage=map')
[/set_variable]
[set_variable]
name=map
python=<STP {@War_of_Iaar\scripts\mapgen.py} PTS>?convert('Uu','Uh',8,5)
[/set_variable]
[set_variable]
name=map
python=<STP {@War_of_Iaar\scripts\mapgen.py} PTS>?convert('Uu','Uu^Uf',5,12)
[/set_variable]
This does some complex things you will know @ campaign's 1st release (as beta).
Example 2:
{VARIABLE_OP num python ?$num|*2}
This multipies the value if num by 2.

Okay
Last edited by arielbys on May 13th, 2008, 6:18 pm, edited 1 time in total.

User avatar
allefant
Units Database Administrator
Posts: 516
Joined: May 6th, 2005, 3:04 pm

Re: Make Python Scripts useable from WML

Post by allefant » May 13th, 2008, 4:36 pm

Can wesnoth support using Python scripts in a scenario
It does not currently support it. But I think it's a wanted feature.

arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Re: Make Python Scripts useable from WML

Post by arielbys » May 13th, 2008, 6:13 pm

That looks good for you?

User avatar
irrevenant
Moderator Emeritus
Posts: 3692
Joined: August 15th, 2005, 7:57 am
Location: I'm all around you.

Re: Make Python Scripts useable from WML

Post by irrevenant » May 14th, 2008, 1:13 pm

I vaguely recall there were some security weaknesses in supporting python scripts.

Can anyone elaborate?

Thanks.

arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Re: Make Python Scripts useable from WML

Post by arielbys » May 14th, 2008, 1:38 pm

irrevenant wrote:I vaguely recall there were some security weaknesses in supporting python scripts.
You CAN USE Python AI, in which you can do the same bad things (like, downloading Trojans) as Python scripts.


Also, the Python scripts would be made alot more useful if you add the ability to execute variables as WML

User avatar
allefant
Units Database Administrator
Posts: 516
Joined: May 6th, 2005, 3:04 pm

Re: Make Python Scripts useable from WML

Post by allefant » May 14th, 2008, 2:26 pm

irrevenant wrote:I vaguely recall there were some security weaknesses in supporting python scripts.

Can anyone elaborate?

Thanks.
Yes, we do limit python scripts by removing possibly unsafe constructs from the accessible built-ins (like file access), and by only allowing a few white-listed modules. Also, any Python code is only executed after going over the code and checking for possible circumventions of the above (e.g. any variable names starting with _ are not allowed, but it would be required to change internals who are named like that). The same would of course be done for scripts called from WML, if we want them distributable over the campaign server.

arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Re: Make Python Scripts useable from WML

Post by arielbys » May 14th, 2008, 6:32 pm

allefant wrote:
irrevenant wrote:I vaguely recall there were some security weaknesses in supporting python scripts.

Can anyone elaborate?

Thanks.
Yes, we do limit python scripts by removing possibly unsafe constructs from the accessible built-ins (like file access), and by only allowing a few white-listed modules. Also, any Python code is only executed after going over the code and checking for possible circumventions of the above (e.g. any variable names starting with _ are not allowed, but it would be required to change internals who are named like that). The same would of course be done for scripts called from WML, if we want them distributable over the campaign server.
Okay, that's the best way to do this. Executing Variables as WML is good or has a security problem? (it would improve Python scripts a lot because you cloud run scripts as WML). Also, i would want the ability to save the game to an variable as XML Document (like Save Game but for easier for Python to read, so you can create things dependent of Game State).

User avatar
Noyga
Inactive Developer
Posts: 1790
Joined: September 26th, 2005, 5:56 pm
Location: France

Re: Make Python Scripts useable from WML

Post by Noyga » May 15th, 2008, 5:01 pm

I think the problem with this feature is compatibility over the network, which is a bad thing.
Currently there is no problem with the python version since the python AI code is only executed on the host of the AI, for the other side of the network it behaves as a normal player. So it is the same result for everyone.
With this feature enabled and used, you'll also need to have python enabled (currently python is an optional feature enabled by default) with the same version on all side to be sure everything will be always compatible :
- without python the python part will not work, so it will produce Out of Sync error
- with a different python version, the python part might not work or produce different result, so it might sometimes also produce out of sync errors. AFAIK the windows build use python 2.4 and most modern linux distros use python 2.5
"Ooh, man, my mage had a 30% chance to miss, but he still managed to hit! Awesome!" ;) -- xtifr

arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Re: Make Python Scripts useable from WML

Post by arielbys » May 17th, 2008, 8:27 am

Noyga wrote:I think the problem with this feature is compatibility over the network, which is a bad thing.
Currently there is no problem with the python version since the python AI code is only executed on the host of the AI, for the other side of the network it behaves as a normal player. So it is the same result for everyone.
With this feature enabled and used, you'll also need to have python enabled (currently python is an optional feature enabled by default) with the same version on all side to be sure everything will be always compatible :
- without python the python part will not work, so it will produce Out of Sync error
- with a different python version, the python part might not work or produce different result, so it might sometimes also produce out of sync errors. AFAIK the windows build use python 2.4 and most modern linux distros use python 2.5
You can just say "Needs Python 2.5" in the Campaign\Scenario selection window, or say it an MP-Unsafe code (like random was).

User avatar
irrevenant
Moderator Emeritus
Posts: 3692
Joined: August 15th, 2005, 7:57 am
Location: I'm all around you.

Re: Make Python Scripts useable from WML

Post by irrevenant » May 17th, 2008, 9:01 am

arielbys wrote:You can just say "Needs Python 2.5" in the Campaign\Scenario selection window, or say it an MP-Unsafe code (like random was).
Requiring installation of additional software severly lowers the appeal. The more hoops the user has to jump through, the less likely they are to do so.

User avatar
allefant
Units Database Administrator
Posts: 516
Joined: May 6th, 2005, 3:04 pm

Re: Make Python Scripts useable from WML

Post by allefant » May 17th, 2008, 2:19 pm

Requiring installation of additional software severly lowers the appeal. The more hoops the user has to jump through, the less likelier they are to do so.
Since the Windows binaries ship with Python currently (as far as I know) - probably for a large percentage of users it simply would work. Just needs someone to write the scripting interface. I guess for a random level generator, mostly an API to modify terrain would be needed.

User avatar
Noyga
Inactive Developer
Posts: 1790
Joined: September 26th, 2005, 5:56 pm
Location: France

Re: Make Python Scripts useable from WML

Post by Noyga » May 17th, 2008, 5:08 pm

arielbys wrote:You can just say "Needs Python 2.5" in the Campaign\Scenario selection window, or say it an MP-Unsafe code (like random was).
The problem is the average player have no idea of his Python setup.
allefant wrote:
Requiring installation of additional software severly lowers the appeal. The more hoops the user has to jump through, the less likelier they are to do so.
Since the Windows binaries ship with Python currently (as far as I know) - probably for a large percentage of users it simply would work. Just needs someone to write the scripting interface. I guess for a random level generator, mostly an API to modify terrain would be needed.
The other problem is incompatibility with python major version.
For example lines like this :

Code: Select all

x = true_value if condition else false_value
Use a new v2.5 feature, this won't work for clients with v2.4
AFAIK both Python v2.4 (Windows binaries) and Python v2.5 (Most linux distros) setups are widespead for wesnoth players.

Incompatibity is not an issue if it say one one side of the network like for AI, but if the python script needs to be sent over the network, it starts to be really messy...
"Ooh, man, my mage had a 30% chance to miss, but he still managed to hit! Awesome!" ;) -- xtifr

AI
Developer
Posts: 2394
Joined: January 31st, 2008, 8:38 pm

Re: Make Python Scripts useable from WML

Post by AI » May 17th, 2008, 9:33 pm

Scenario writers already need to avoid certain things in WML to prevent OOS, having to do something similar to avoid OOS in python sounds acceptable to me.

In other words, it would be the scenario writer's responsibility to write his script in a portable way.

A more important issue would be fact that python is optional.

User avatar
allefant
Units Database Administrator
Posts: 516
Joined: May 6th, 2005, 3:04 pm

Re: Make Python Scripts useable from WML

Post by allefant » May 18th, 2008, 12:28 pm

Noyga wrote: The problem is the average player have no idea of his Python setup.
Yes, and they should not - that's the reason to ship Python with the Windows version of Wesnoth. Whether users have Python (or another version of Python) on their system will not matter, it never will be used. Things are different in Linux of course (and no idea about OSX).
Noyga wrote: The other problem is incompatibility with python major version.
For example lines like this :

Code: Select all

x = true_value if condition else false_value
Use a new v2.5 feature, this won't work for clients with v2.4
AFAIK both Python v2.4 (Windows binaries) and Python v2.5 (Most linux distros) setups are widespead for wesnoth players.

Incompatibity is not an issue if it say one one side of the network like for AI, but if the python script needs to be sent over the network, it starts to be really messy...
True. Probably should say somewhere that no Python 2.5 statements should be used. In fact, since safe.py was written for Python 2.4 and whitelists the allowed bytecodes, chances are 2.5 constructs won't work anyway :)

arielbys
Posts: 51
Joined: May 6th, 2008, 2:57 pm

Re: Make Python Scripts useable from WML

Post by arielbys » May 20th, 2008, 4:51 am

allefant wrote:
Noyga wrote: The problem is the average player have no idea of his Python setup.
Yes, and they should not - that's the reason to ship Python with the Windows version of Wesnoth. Whether users have Python (or another version of Python) on their system will not matter, it never will be used. Things are different in Linux of course (and no idea about OSX).
Noyga wrote: The other problem is incompatibility with python major version.
For example lines like this :

Code: Select all

x = true_value if condition else false_value
Use a new v2.5 feature, this won't work for clients with v2.4
AFAIK both Python v2.4 (Windows binaries) and Python v2.5 (Most linux distros) setups are widespead for wesnoth players.

Incompatibity is not an issue if it say one one side of the network like for AI, but if the python script needs to be sent over the network, it starts to be really messy...
True. Probably should say somewhere that no Python 2.5 statements should be used. In fact, since safe.py was written for Python 2.4 and whitelists the allowed bytecodes, chances are 2.5 constructs won't work anyway :)
So we got no problem in this area

Post Reply