Rating system for user-made campaigns

Brainstorm ideas of possible additions to the game. Read this before posting!

Moderator: Forum Moderators

Forum rules
Before posting a new idea, you must read the following:
jg
Posts: 244
Joined: September 12th, 2005, 7:17 am

Rating system for user-made campaigns

Post by jg »

I would like to propose the idea of an rating system for the dowload section. This would give people a choice to see (from quite a lot of options) which has been good in other peoples opinion. There are of course pros and contras.

Pros:
-one would know what campaign/faction has been done properly
-would make people work harder on factions/campaigns to get a good rating
-if more and more campaigns/factions come into the download section it is hard to pick the "right" one (so rating might give a direction as to which one would be good)

Contras:
-it might make people more relucant towards making their own campigns/factions
-might be difficult to implement
(-people's feelings might get hurt)

jg
User avatar
turin
Lord of the East
Posts: 11662
Joined: January 11th, 2004, 7:17 pm
Location: Texas
Contact:

Post by turin »

This might be a good idea. I agree some sort of way to sort campaigns would be a very good idea. I'm inclined, however, to support a more subjective system - have two campaign servers, one open, one closed. To upload content to the closed server, it must be approved by someone (probably a dev). The requirements would be:
* No bugs
* Good plot (subjective, I know)
* Interesting gameplay

Of course, the objection that will immediately be brought up is that this suggestion is too elitist, supporting an already too-elitist community of devs. Well, too bad. I still think it's the best solution.
For I am Turin Turambar - Master of Doom, by doom mastered. On permanent Wesbreak. Will not respond to private messages. Sorry!
And I hate stupid people.
The World of Orbivm
Boucman
Inactive Developer
Posts: 2119
Joined: March 31st, 2004, 1:04 pm

Post by Boucman »

I'll add to turin's list

* close enough to being finished

however the basic idea (open and closed server) sounds good to me....
Fight key loggers: write some perl using vim
jg
Posts: 244
Joined: September 12th, 2005, 7:17 am

Post by jg »

Have to admit, it is also a good(if not a better) idea. But my original still stands! :P

jg
User avatar
Eleazar
Retired Terrain Art Director
Posts: 2481
Joined: July 16th, 2004, 1:47 am
Location: US Midwest
Contact:

Post by Eleazar »

Not a new idea, but one i'd like to see.
The A GUI has already been dummied by friem.
The idea has been discussed here.

I don't see a reason to have 2 servers, but i do think it would be advantageous if the player could sort by Dev assigned metadata. Perhaps a symbol in the list to indicate that a dev has tried it and found it to be playable, and not totally lame.

Additionally I'd like a prompt to come up when a player completes a campaign, asking for the player to rate it, but the player shouldn't have to suffer through a lame campaign to rate it. There should be a way to rate or re-rate a campaign at any time.
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
User avatar
zookeeper
WML Wizard
Posts: 9742
Joined: September 11th, 2004, 10:40 pm
Location: Finland

Post by zookeeper »

IMHO it would be sufficient to allow authorized people (devs, other trusted folks) to collectively rate campaigns (some kind of voting or average rating system). How this should be done is another matter...a web interface would most likely be too complicated, direct access to the campaign server for dozens of people doesn't sound like a good idea, etc.

Of course I don't think we need ratings as such, simple tags like "complete", "broken", "incomplete" would probably do. You can usually tell if a campaign sucks or is good in 10 seconds, and there aren't a lot of complete and working campaigns.
Boucman
Inactive Developer
Posts: 2119
Joined: March 31st, 2004, 1:04 pm

Post by Boucman »

could stats.wesnoth.org be used to somehow help rating
(just throwing ideas around)
Fight key loggers: write some perl using vim
User avatar
Eleazar
Retired Terrain Art Director
Posts: 2481
Joined: July 16th, 2004, 1:47 am
Location: US Midwest
Contact:

Post by Eleazar »

Boucman wrote:could stats.wesnoth.org be used to somehow help rating
(just throwing ideas around)
Hmm, a something like a measurement of how many minutes it's been played divided by the number of days that stat has been recorded, might provide a pretty objective measurement of "popularity" Newly released campaigns would probably jump to the top, but that should be OK, they get a little bit of free publicity at the beginning, but if the campaing rots, people will stop playing it, and it will go down. This sort of measurment would benefit long campaigns, assuming that the campaign can keep people playing.
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
User avatar
zookeeper
WML Wizard
Posts: 9742
Joined: September 11th, 2004, 10:40 pm
Location: Finland

Post by zookeeper »

Eleazar wrote:Hmm, a something like a measurement of how many minutes it's been played divided by the number of days that stat has been recorded, might provide a pretty objective measurement of "popularity" Newly released campaigns would probably jump to the top, but that should be OK, they get a little bit of free publicity at the beginning, but if the campaing rots, people will stop playing it, and it will go down. This sort of measurment would benefit long campaigns, assuming that the campaign can keep people playing.
Doesn't sound very good to me, because as you said longer campaigns would benefit a lot. And it would be most likely impossible to calibrate so that even short campaigns could get a near-perfect rating. And still, it would only be a popularity measurement, which isn't interesting. The quality is what people would benefit from knowing.

It was interesting to note, when the 1.2 add-on server was wiped and recreated, that the order at which stuff was uploaded to the server is pretty much the same order you get today by sorting by downloads - the first ones to get added are still at the top, even though they probably aren't nearly the best the server has to offer. People look at the downloads, and most often download the ones at the top. If there was a quality-based ranking, this might change. If there was a popularity-based ranking, I don't think much would change. The ones at the top would still get played most. Besides, it's not really possible (and for quite a while still) to measure how often a MP map, era or basically anything other than a campaign is played, so this could only be applied to campaigns.
User avatar
Casual User
Posts: 475
Joined: March 11th, 2005, 5:05 pm

Post by Casual User »

Personally, I think an amendment is required.

The Dev-acceptance requirement is not a good idea (IMO). Posting unfinished campaigns on the server for feedback and/or help has been great for a lot of campaigns that are now considered good. This idea would all but make that impossible.

The rating system is a good idea, but with an amendment: The campaign writer should be allowed to specify in the pbl file if he wants his campaign evaluated.

That way, an unfinished campaign can be put up on the server without accumulating bad marks, and once the writer feels confident about what he's made, he can put it up for evaluation.

Also, that way, players would be informed right from the start how confident the writer is in his own campaign.
User avatar
Eleazar
Retired Terrain Art Director
Posts: 2481
Joined: July 16th, 2004, 1:47 am
Location: US Midwest
Contact:

Post by Eleazar »

The fundimental issue is that campaigns on the server vary from broken bad ideas to well polished works. Version numbering as it's used has little connection to the playability of a campaign. It's great if a player wants to playtest somebody's rough WIP, but he should know that what he's getting into when he downloads a campaign. Nobody wins if players are tricked into becoming playtesters.

One of my complaints about version numbering is that the "number" can actually start with a letter.

I'd like to see more than one field replace "downloads" as a way to pre-evaluate a campaign. There's no reason to limit ourselves to only one change. Downloads really isn't that helpful, since it's heavily slanted in favor of alphabetially early and old campaigns.

• A few status lables from "broken" to "complete" as zookeeper suggested is the simplest and probably most useful. These should be assigned by devs.

• A user rating would be great, perhaps tied into the same ID used for the stats.

• I think a Popularity rating drawn from the stats would be pretty interesting.

• Tabs dividing the different types of content "Campaigns", "Eras", "Maps", and "Other" would make it much easier to find what you want.


Another way to help players find content they would enjoy is to provied a way for players to write reviews, perhaps with a boilerplate that that gives blanks for the player to rate different aspects, like "story" "strategy" "graphics" etc.

Casual User wrote:Personally, I think an amendment is required.
Personally i don't think you've really read what's been written in this thread.
No one has proposed that "bad marks" be given out, or that unfinished campaigns be disallowed from our servers.
Feel free to PM me if you start a new terrain oriented thread. It's easy for me to miss them among all the other art threads.
-> What i might be working on
Attempting Lucidity
jg
Posts: 244
Joined: September 12th, 2005, 7:17 am

Post by jg »

Eleazar wrote:The idea has been discussed here.
Sorry, didn't do a search... Though I did look into the FPI.

I see people have a lot of good ideas:

-putting the server into folders
-having a approval-sign by a Dev/approval board
-having seperate servers
-reviews.

I will also add another (random) proposal, that one should have an sign/folder with the completed campaigns/eras/factions. This (IMO) should be done either by the authors themselves or a approval-board. This would at least give some clearance already.

jg
User avatar
Casual User
Posts: 475
Joined: March 11th, 2005, 5:05 pm

Post by Casual User »

Eleazar wrote:Personally i don't think you've really read what's been written in this thread.
Yes I have.
Eleazar wrote:No one has proposed that "bad marks" be given out
Yes they have.

From the moment there is a system of popular voting about the campaigns on the server, and all campaigns are up for evaluation, campaigns which are posted earlier will accumulate low evaluations while they are still unfinished, which in turn will make them get downloaded less in the future, lowering their chances of them ever getting a better mean for the evaluations.

Essentially, this would punish those that post campaigns for early feedback.

Hence my proposal that the writer be allowed to specify whether or not he wants his campaign to be evaluated.
Eleazar wrote:or that unfinished campaigns be disallowed from our servers.
Yes, I misread Turin's proposal. I had thought he meant all campaigns would have to be dev-approved to be on the server.
PingPangQui
Posts: 267
Joined: July 18th, 2006, 11:52 am

Post by PingPangQui »

Casual User wrote:From the moment there is a system of popular voting about the campaigns on the server, and all campaigns are up for evaluation, campaigns which are posted earlier will accumulate low evaluations while they are still unfinished, which in turn will make them get downloaded less in the future, lowering their chances of them ever getting a better mean for the evaluations.

Essentially, this would punish those that post campaigns for early feedback.
There wouldn't be a real problem about this if you indicate that your campaign isn't finished yet, is basicly finished but in a testing stage or finished and tested. You could give reasonable version numbers, which should be below 1.0 unless it is really finished.

Example: if I see a campaign with vers. number 0.2... than I know it is not finished, thus if one likes what the author has done so far then they will certainly download it again when a new version is published.

It should be clear that an unfinished campaign must be judged differently than a finished one. However, in case such a voting system is going to be implemented it might be better to allow voting only for campaigns that are indicated "finished" (however that is done: version number, an option choosed by the author, etc.) since one can read from time to time unqualified coments for programs with version number 0.x.y like this: "Program is not good because lots of bugs, feature don't work, annoying ..."
The Clan Antagonist.

"Larry the Cow was a bit frustrated at the current state of Linux distributions (...) until he tried Gentoo Linux" - Free Software for free people.
jg
Posts: 244
Joined: September 12th, 2005, 7:17 am

Post by jg »

So should I make a voting option, for say 10 days, and then which one was voted the most, is what we suggest the Developers to implement?

jg
Post Reply