On Beta Testing
Moderator: Forum Moderators
On Beta Testing
A recent thread on the Wesnoth mailing list deals with testing of pre-release versions. I am one of those people who likes to use "bleeding edge" code and find/report bugs. At the same time, I'm more apt to do it when it's not a super technical ordeal to stay up to date. Two projects stick out as very user friendly for staying at the forefront of the codebase:
Blender, which has a buildbot http://builder.blender.org/ as well as user made builds on graphicall.org
And Krita, which has a fantastic debugging/reporting system- basically if it crashes the appropriate code is saved, the user tells what made it happen to the best of their ability, and it goes to the tracker. The Ubuntu ppa from dmitri keeps it up to date and the fixes rolling in.
Both of these are relatively small downloads, 100MB or less.
One of the points made on the ML is that the devs should test more, but at the same time devs may subconsiously avoid certain things that a user would break in 2 minutes. User testing is invaluable because users do whacky things that devs wouldn't think of. I know there is git if one wants to stay on top of things, but the initial download is massive (2GB?) and command line operations can be daunting. I wonder if there could be a nice way to keep users up to date with the latest code but keep it to a civilian level.
One idea is to create weekly packages of only the actual source code text files, omitting the need to re-download the images and music. If one could just drop that into the user's existing wesnoth folder (if it weer pre-built for OSX or Windows), or source code folder and rebuild, to me that's a viable option. Heck, even a zip of the modified files, eg. .cpp files on a weekly basis would be helpful, and save big downloads.
It'd be great to get input from UMC devs, players, and coders on this subject. Maybe it's not needed, Wesnoth is pretty stable as it stands, but big growth spurts do need lots more testing and it seems to be in that stage atm.
Blender, which has a buildbot http://builder.blender.org/ as well as user made builds on graphicall.org
And Krita, which has a fantastic debugging/reporting system- basically if it crashes the appropriate code is saved, the user tells what made it happen to the best of their ability, and it goes to the tracker. The Ubuntu ppa from dmitri keeps it up to date and the fixes rolling in.
Both of these are relatively small downloads, 100MB or less.
One of the points made on the ML is that the devs should test more, but at the same time devs may subconsiously avoid certain things that a user would break in 2 minutes. User testing is invaluable because users do whacky things that devs wouldn't think of. I know there is git if one wants to stay on top of things, but the initial download is massive (2GB?) and command line operations can be daunting. I wonder if there could be a nice way to keep users up to date with the latest code but keep it to a civilian level.
One idea is to create weekly packages of only the actual source code text files, omitting the need to re-download the images and music. If one could just drop that into the user's existing wesnoth folder (if it weer pre-built for OSX or Windows), or source code folder and rebuild, to me that's a viable option. Heck, even a zip of the modified files, eg. .cpp files on a weekly basis would be helpful, and save big downloads.
It'd be great to get input from UMC devs, players, and coders on this subject. Maybe it's not needed, Wesnoth is pretty stable as it stands, but big growth spurts do need lots more testing and it seems to be in that stage atm.
Re: On Beta Testing
idk whether it help you, but at least for windows you can download a new wesnoth.exe from jenkins:
http://baldras.wesnoth.org:8080/view/Al ... ebian7-32/
http://baldras.wesnoth.org:8080/view/Al ... ebian7-32/
(just copy it into the folder where a wesnoth.exe from a downloaded wesnoth windows build is.)
http://baldras.wesnoth.org:8080/view/Al ... ebian7-32/
http://baldras.wesnoth.org:8080/view/Al ... ebian7-32/
(just copy it into the folder where a wesnoth.exe from a downloaded wesnoth windows build is.)
Scenario with Robots SP scenario (1.11/1.12), allows you to build your units with components, PYR No preperation turn 1.12 mp-mod that allows you to select your units immideately after the game begins.
Re: On Beta Testing
The thing is, this won't really work because we sometimes also have to modify files in /data/, like campaign wml, or the core lua that implements many of the wml tags. So it's not like you can just drop /src/ folder into your repo and expect it to work.Crow_T wrote: One idea is to create weekly packages of only the actual source code text files, omitting the need to re-download the images and music. If one could just drop that into the user's existing wesnoth folder (if it weer pre-built for OSX or Windows), or source code folder and rebuild, to me that's a viable option. Heck, even a zip of the modified files, eg. .cpp files on a weekly basis would be helpful, and save big downloads.
What you really need to update your version correctly is the "diff". In principle if you absolutely must avoid git, you can download .patch files (collections of diffs) for each commit that you don't have yet, and use a standard tool to apply them in order.
But that is quite complicated, and it is exactly what git is designed to do very well at the push of a button
Maybe we should put like, just an icon at the root of the repo called "download updated master" that will just execute the commands to update git? Specifically I'm thinking of the block of commands written here: http://wiki.wesnoth.org/Git_for_Wesnoth ... e#AddendumCrow_T wrote: I wonder if there could be a nice way to keep users up to date with the latest code but keep it to a civilian level.
I would think that if we instructed users that were so inclined to run "git clone" to set up the repo off of github to install, and had a script that they could click on from the file browser that saidGit crash course wrote: get a clean slate by syncing with upstreamif this errors, try again afterCode: Select all
git pull upstream master git fetch upstream git checkout -B master upstream/master git push --force origin master
Code: Select all
git reset --hard HEAD
Code: Select all
git reset --hard HEAD
git fetch origin
git checkout -B master origin/master
git checkout -f master
Also I'm not sure how that can be done in a totally cross platform way / if there are cross platform icons? Maybe this could be made into a python script or something? (most users have python right?)
We could have an icon and script for 1.12 also.
Is that sufficiently civilian friendly, or is writing "git clone" already too much? When you download a .zip archive from github, does that contain a git repo, or just a bunch of files without the .git ones?
Re: On Beta Testing
Thanks for the answers- I actually fixed a bug in (the skip AI moves bug) in my 1.11.15 source by copying the code from the wesnoth git website and pasting it into the proper .cpp then building again- not the safest or best way for sure.
How to go about doing it, perhaps git is simply the best way for this particular project. Is there a way to avoid the large download of master and only stick within a particular branch?
When one compiles, what are the actual wesnoth application files? The executable and Data folder?
How to go about doing it, perhaps git is simply the best way for this particular project. Is there a way to avoid the large download of master and only stick within a particular branch?
When one compiles, what are the actual wesnoth application files? The executable and Data folder?
- Pentarctagon
- Project Manager
- Posts: 5565
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: On Beta Testing
A couple problems with the source code distribution method:
The project files for different OSs are not always updated at the same time. For example, over about the last year I've had to search src/ and manually add missing files to the code::blocks project multiple times. One time I ended up PMing shadowm when they weren't fixed even after a couple weeks.
Wesnoth doesn't actually build on the most recent version of code::blocks even with the most up-to-date source code due to a weird problem with it trying to compile animated.i.
The project files for different OSs are not always updated at the same time. For example, over about the last year I've had to search src/ and manually add missing files to the code::blocks project multiple times. One time I ended up PMing shadowm when they weren't fixed even after a couple weeks.
Wesnoth doesn't actually build on the most recent version of code::blocks even with the most up-to-date source code due to a weird problem with it trying to compile animated.i.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: On Beta Testing
Hmm, so recently I think we've been much better about project files. It seems like code blocks, code blocks-scons, and VC are typically updated right after by gfgtdf and I whenever we add a new file, also I think the new files related to SDL got added to project files right away. Most recently, when someone forgot to update the code blocks project files, SigurdFD actually submitted pull requests with the fix like, within the hour or so. So those ones seem pretty well maintained atm.
XCode is harder because the project files are quite complex and hard to edit as text files, that pretty much has to be updated by someone with XCode, but again mattsc usually takes care of that on a reasonable basis.
I'm not so sure about the "codelite" project files, they seem to be generally a mess and I'm not even really sure if anyone is using them. I can tell from looking at them that they are not alphabetized and many files are missing, so I stopped updating that one a while ago since it's probably a lost cause. That's the only one like that afaik though.
Pentarctagon: I'm not sure about the most recent version of codeblocks, that sounds strange...
But anyways if you aren't developing patches why do you even need an IDE? Why not just use scons / cmake? IMO IDE's are totally unnecessary anyways, I don't use one even for development. All it does is encumber you with the need to update project files, and provide you with gui that is better provided anyways by your file browser / text editor / command line. Our scons script is extremely good anyways so there's really no value added by the IDE. scons with ccache enabled seems to outperform them all by a wide margin. </rant>
Seriously even if you like IDE's its really hard to argue that you need one just as an end user.
Edit: Maybe the script file just needs to be:
Also a windows .bat equivalent.
XCode is harder because the project files are quite complex and hard to edit as text files, that pretty much has to be updated by someone with XCode, but again mattsc usually takes care of that on a reasonable basis.
I'm not so sure about the "codelite" project files, they seem to be generally a mess and I'm not even really sure if anyone is using them. I can tell from looking at them that they are not alphabetized and many files are missing, so I stopped updating that one a while ago since it's probably a lost cause. That's the only one like that afaik though.
Pentarctagon: I'm not sure about the most recent version of codeblocks, that sounds strange...
But anyways if you aren't developing patches why do you even need an IDE? Why not just use scons / cmake? IMO IDE's are totally unnecessary anyways, I don't use one even for development. All it does is encumber you with the need to update project files, and provide you with gui that is better provided anyways by your file browser / text editor / command line. Our scons script is extremely good anyways so there's really no value added by the IDE. scons with ccache enabled seems to outperform them all by a wide margin. </rant>
Seriously even if you like IDE's its really hard to argue that you need one just as an end user.
Edit: Maybe the script file just needs to be:
Spoiler:
- Pentarctagon
- Project Manager
- Posts: 5565
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: On Beta Testing
Mostly because the README I found with step by step instructions for compiling used code::blocks and I've never used scons or cmake before. Also code::blocks has a simple option for using multiple processors to compile.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: On Beta Testing
It was hard to set up initially for me (mattsc had to walk me through it), but with changes he makes there's a fairly simple process to step through to handle the changes, and it's really easy tool with which to build.iceiceice wrote:XCode is harder because the project files are quite complex and hard to edit as text files, that pretty much has to be updated by someone with XCode, but again mattsc usually takes care of that on a reasonable basis.
Mainline Maintainer: AOI, DM, NR, TB and THoT.
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
UMC Maintainer: Forward They Cried, A Few Logs, A Few More Logs, Start of the War, and Battle Against Time
Re: On Beta Testing
I mean, I get that everyone has their own preferences and solves problems the way that makes sense to them.
But how can it get easier than scons? Sure I mentioned all those other options you can use but that's just to make it slightly faster.
If you can navigate to the wesnoth directory, and manage to type "scons", you are going to get a wesnoth executable, one way or the other, assuming you have python installed, and the necessary wesnoth dependencies.
What exactly does the IDE help you to do? Scons will tell you if libraries or dependencies that wesnoth needs are missing, I somewhat doubt if the IDE is as helpful with that.
But how can it get easier than scons? Sure I mentioned all those other options you can use but that's just to make it slightly faster.
If you can navigate to the wesnoth directory, and manage to type "scons", you are going to get a wesnoth executable, one way or the other, assuming you have python installed, and the necessary wesnoth dependencies.
What exactly does the IDE help you to do? Scons will tell you if libraries or dependencies that wesnoth needs are missing, I somewhat doubt if the IDE is as helpful with that.
- Pentarctagon
- Project Manager
- Posts: 5565
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: On Beta Testing
I think I'll try scons now just to compare how fast it can compile compared to code::blocks, though I'll be really surprised if it gets much faster than the 3:20 code::blocks does.
Also, the README has a link to all the libraries, etc that are needed so I've never even had to worry about that.
Also, the README has a link to all the libraries, etc that are needed so I've never even had to worry about that.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: On Beta Testing
What’s the point of theiceiceice wrote:Git crash course wrote: get a clean slate by syncing with upstreamCode: Select all
git pull upstream master git fetch upstream git checkout -B master upstream/master git push --force origin master
git pull upstream master
command? If the user is on branch master
, any changes that that command makes will be overwritten by the subsequent git checkout -B master upstream/master
command. If the user is not on branch master
, I don’t see why pulling upstream
’s master
branch into whatever the current local branch is would be desirable there.Re: On Beta Testing
8680: I agree, that looks buggy
I'm not really sure where the "addendum" came from, maybe we need to review that section.
Pentarctagon: 3:20 is pretty good, that's as good as I will generally get even with a warmed up ccache. With no ccache, in my experience gcc is about 30-40 minutes, and clang is about 20 minutes. With a hot ccache its 3-5 minutes, unless you change key headers. But the question here is ease rather than speed, right? As you point out, sometimes the code::blocks project file isn't totally up to date, and if you get the wrong code::blocks version you can't build. How many windows do you have to click through to set up and build? With scons, all I do is install, and then I'm ready to type "scons" in the project root directory. So it seems a bit easier in that regard.
Edit: By the way, I have discovered that scons can do parallel builds as well, with "-j 2" splitting jobs among two cores, etc.
http://www.scons.org/doc/2.0.1/HTML/sco ... c2076.html
Didn't know this before, thanks for the tip. I guess I will found out if it actually makes things any faster...
I'm not really sure where the "addendum" came from, maybe we need to review that section.
Pentarctagon: 3:20 is pretty good, that's as good as I will generally get even with a warmed up ccache. With no ccache, in my experience gcc is about 30-40 minutes, and clang is about 20 minutes. With a hot ccache its 3-5 minutes, unless you change key headers. But the question here is ease rather than speed, right? As you point out, sometimes the code::blocks project file isn't totally up to date, and if you get the wrong code::blocks version you can't build. How many windows do you have to click through to set up and build? With scons, all I do is install, and then I'm ready to type "scons" in the project root directory. So it seems a bit easier in that regard.
Edit: By the way, I have discovered that scons can do parallel builds as well, with "-j 2" splitting jobs among two cores, etc.
http://www.scons.org/doc/2.0.1/HTML/sco ... c2076.html
Didn't know this before, thanks for the tip. I guess I will found out if it actually makes things any faster...
- Pentarctagon
- Project Manager
- Posts: 5565
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: On Beta Testing
So... I installed python and scons (which apparently doesn't support x64 python), then went to the Wesnoth directory and typed "scons", and all it did was print "scons" in an endless loop.iceiceice wrote:But how can it get easier than scons? Sure I mentioned all those other options you can use but that's just to make it slightly faster.
If you can navigate to the wesnoth directory, and manage to type "scons", you are going to get a wesnoth executable, one way or the other, assuming you have python installed, and the necessary wesnoth dependencies.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code
Re: On Beta Testing
Hmm, you might have an old version of scons or something? I never had an issue like that.
- Pentarctagon
- Project Manager
- Posts: 5565
- Joined: March 22nd, 2009, 10:50 pm
- Location: Earth (occasionally)
Re: On Beta Testing
I downloaded 2.3.1, which is on the front page of their site as the most recent bugfix, and python 2.7.7.
99 little bugs in the code, 99 little bugs
take one down, patch it around
-2,147,483,648 little bugs in the code
take one down, patch it around
-2,147,483,648 little bugs in the code