Download count being an implicit ranking system

[Dugi]

I see several problems with Gambit's idea.
1) Many people download MP Factions, mappacks and other things instead of campaigns before learning what should they do. Do you think that they would actually look at the feedback threads?
2) Learning something from a feedback thread might be uneasy. The feedback thread of my campaign has 1900 posts by now, for example, while the first pages are full of comments from times long past, where the campaign was something much different. There is no way to learn what is actually happening without reading 300 posts backwards.
3) This is a fact. The descriptions of many campaigns contain URLs of their feedback threads. I believe it is globally ignored, because for example campaign Antar, son of Rheor has in total 10000 downloads on 1.10 and 1.11, its description contains the URL of its feedback thread, and its feedback thread has 230 posts. Mine has 24000 downloads (resulting from more numerous updates), its description does not contain any URLs, but its feedback thread has 1900 posts.
4) Many add-ons are developed by various unofficial forums, and they have their feedback threads there. Allowing to link these would allow spamming, and not doing so would disadvantage these add-ons. These unofficial forums are frequently in other languages, so the links would be generally useless here.

Another problem I have spotted right now. Many people have uploaded their add-ons to the 1.8 server without realising that it will cause severe compatibility problems, for example with harm_unit and new terrains. I believe that 1.8 is still played a lot on Android, judging from the 500 downloads that an ancient, specially ported version of my campaign got there in the last 3 months. Problems of this kind will not be detected by neither of the ways suggested here.

[Iris]

What if there was a new pbl field? "forum_thread=". The forum thread could show up as a URL in the add-on description. This would allow users to see if an add-on has no place to leave feedback (probably a bad add-on). If the add-on does have a thread, they can go there to see other user's feedback on the add-on, and leave their own. It would also let them see how actively developed the add-on is.

[...]
* It lets them quickly spot inactive add-ons.
[...]

I’ll just say that one could easily assume that AtS’s development (one of my add-ons) is inactive because there aren’t posts all the time and I don’t have much to do if nobody is reporting bugs, pointing out balance issues, asking questions, etcetera.

1) Many people download MP Factions, mappacks and other things instead of campaigns before learning what should they do.

I added a help section dedicated to add-ons for 1.11.0 precisely for this reason, as there wasn’t any in-game text on the matter before. I am sorry, but if people start downloading things without checking documentation that’s literally one click away from the add-ons listing, then they absolutely deserve to feel that their time has been wasted.

Screenshot for reference:

3) This is a fact. The descriptions of many campaigns contain URLs of their feedback threads. I believe it is globally ignored, [...]

I would say that’s just a consequence of the lack of clickable links. I have referred to the technical aspects behind that in this section before, I believe.

Another problem I have spotted right now. Many people have uploaded their add-ons to the 1.8 server without realising that it will cause severe compatibility problems, for example with harm_unit and new terrains.

People cannot upload add-ons to the wrong instance of the server by accident, unless they are using the wrong version of Wesnoth to perform the upload, which would raise an amount of questions.

[Dugi]

I added a help section dedicated to add-ons for 1.11.0 precisely for this reason, as there wasn’t any in-game text on the matter before. I am sorry, but if people start downloading things without checking documentation that’s literally one click away from the add-ons listing, then they absolutely deserve to feel that their time has been wasted.

I believe that most people expect that all add-ons are campaigns, and don't think that it might be something else. There should be at least a big notice in red, bold letters telling to understand that not all add-ons are campaigns. When I was a newbie to it, I downloaded a mappack and looked for a new campaign on the list.

People cannot upload add-ons to the wrong instance of the server by accident, unless they are using the wrong version of Wesnoth to perform the upload, which would raise an amount of questions.

It can be uploaded there by writing add-ons.wesnoth.org:15001 instead of add-ons.wesnoth.org:15002 (or add-ons.wesnoth.org) into the address of the add-on server. Many people seem to know that the 1500X numbers will let them log in to other servers, and they might not understand what are these servers. And because their add-ons aren't there, they feel an urge to add them there. Or they have previously browsed the 1.8 server and forgotten that they don't have the right address there when uploading. ChaosRider has just uploaded like 10 add-ons there, and I am quite sure that they contain more regular bugs than 1.8/1.10 incompatibilities (he even told that how he is writing add-ons: edit, upload, ask why it doesn't work). And people as incompetent as ChaosRider are very likely to misunderstand it despite being around for long.

[Iris]

I believe that most people expect that all add-ons are campaigns, and don't think that it might be something else.

Right, because there obviously isn’t a Multiplayer menu hinting at more possibilities. Then again, see below.

There should be at least a big notice in red, bold letters telling to understand that not all add-ons are campaigns. When I was a newbie to it, I downloaded a mappack and looked for a new campaign on the list.

I am absolutely certain that 1.11.0 did not even exist back then. Also, I would save big warning signs in red for potentially destructive operations. Assuming that all or the majority of users are inherently stupid is how some software vendors get me to avoid their products. Really, with a very visible Help button and a very visible and varied Type column in the add-ons list (for which not all add-ons are labeled ‘Campaign’), shouldn’t the majority of users make some kind of connection?

[Stuff about alternate add-ons server instances] [...] And people as incompetent as ChaosRider are very likely to misunderstand it despite being around for long.

Okay, I am glad that we are definitely not going back to pointing at singular people and their flaws and assuming that they are extremely common. Or that they would heed any warnings that could be implemented. The question is, where are people getting the port numbers from? There isn’t a built-in list of add-ons server instances, so perhaps they are reading some documentation on our website? In that case, there is not much that could be done to improve their reading skills short of brain transplants.

[Dugi]

I am absolutely certain that 1.11.0 did not even exist back then...

It was on 1.6. But I haven't seen any changes in its visibility from then, except for some extra buttons. The traditional rule says that help is read only if the reader is in a big trouble, or if it appears while loading. Maybe the warning might not be in red, but a standard text right under the filter telling that not all add-ons are campaign with an exclamation mark would be suitable. The numbers are still telling that the most downloaded add-on on 1.11 is an era that is not played and contains no special features. Or maybe the types of add-ons might be coloured, campaigns green, eras red, resources dark grey, mp mappacks blue, mp campaigns purple, etc.

I don't understand neither how can people upload add-ons to the 1.8 server without thinking about it, I am just making hypotheses. Add-ons with 1.10's ghasts as add-on pictures on the server clearly aren't made for 1.8. Porting my campaign that was almost entirely made for 1.10 to work on 1.8 took me about a day, and I knew what should I do because I used to code on 1.8 too, and porting 10 add-ons would take a lot of time.

[Midnight_Carnival]

Hey guys, I found this great site which has details on how Wesnoth add-ons are going, which ones are popular and also some honest good reviews of them, it also includes the option to post your own reviews, learn more about the game and even make suggestions for incomplete senarios.

here is the link:

www.wesnoth.org/forum

[Iris]

Hey guys, I found this great site which has details on how Wesnoth add-ons are going, which ones are popular and also some honest good reviews of them, it also includes the option to post your own reviews, learn more about the game and even make suggestions for incomplete senarios.

here is the link:

http://www.wesnoth.org/forum

Because, obviously, every Wesnoth player speaks English and wants to register to post in these forums. And there’s obviously a dedicated section in this forums for posting reviews and informing users of what add-ons are popular/trending.

Yeah, no.

[H-Hour]

It seems that there are two problems being discussed. 1) How to tell the difference between good and bad add-ons. 2) How to prevent new users from wasting their time on broken add-ons. A simple solution to the second problem might be to offer a button to report broken add-ons, as shown in the attached mockup. Then the number of broken reports could be displayed in the add-on information and users can make their own judgement.

This is similar in some ways to the feedback system Wordpress uses for its plugins, where users can report whether a plugin works with a specific version. In that context I have found it effective at saving me time trying out bad plugins.

[GunChleoc]

I think a tagging system would help people filter out the type of add-on they want to play. Clickable links to an addon's homepage/forum thread is also a good idea IMO, and a link to the UMC Guide on top. OpenTTD has clickable links in their NewGRF descriptions, maybe you can get some inspiration as to how to code from there.

Some random ideas for ranking systems - feel free to shoot them down

As automated ranking systems, maybe the number of users who repeatedly loaded a UMC might be an idea? E.g. if an era is a top download, but nobody actually plays it, it would show this way. Campaigns is a bit more difficult, because their length varies so much, and they would need their own ranking system. Maybe the author could specify the number of completed scenarios/WIP scenarios and if there are more scenarios planned. As an automated ranking, maybe the number of users who load the 3rd scenario?

[fabi]

May I ask why they should not be in competition?

It's the same reason there is no persistent stats or leaderboard in multiplayer. Competition fosters hostility.

There is negative and positive competition, google finds several interesting articles about the difference.
In general you do not want to avoid positive competition.
Our current situation fosters negative competition more
since the only known visible counter for success can be manipulated and is self increasing.

Because the nature of UMC design is a competing one,
trying to deny competitive behavior is just outright silly.

Instead one should try to encourage positive competition and dis-encourage negative one.

People will take it way too seriously, get obsessed with being in "first place" and start getting angry at each other. It is not healthy for the community.

Indeed, sounds somehow familiar.
Like the dugi/franz relationship.
Thus we already have negative competition.

But you made an excellent point about wasting users' time downloading bad add-ons.

Yes, I guess we can at least all agree that the current situation is not tolerable and action must be taken.

There must be a solution that doesn't involve ranking add-ons ...

I guess that implies that every ranking system is encouraging negative competition.
I doubt that.

... or creating an undue new burden for both programmers and moderators.
I think I have an idea.
...

I proposed exactly the same some time ago.
The ideas aren't bad, but I can't see how they solve the problem.

[Dugi]

Our current situation fosters negative competition more

Exactly. Things that should make an add-on ranked better, ordered:
1. Amount of joy it brings
2. Number of good and unusual elements, increasing the variability of the game
3. Didactic value for coders
This is what actually makes add-ons better:
1. Upload time (especially visible on 1.10)
2. Number of uploads
3. Author's description
4. Amount of joy it brings (how many people recommended it to others)
5. Spamming a notice about it in signatures
...
I think that it is time to remove the downloads count and possibly replace it by reviews or rankings or both, regardless of FPI #32. Maybe it would cause bad kinds of behaviour, but less than the downloads count, that is so prone to cheating that it is almost impossible not to cheat (delete add-on before each upload? that would be masochistic).

Instead one should try to encourage positive competition and dis-encourage negative one.

I strongly agree. Competition is what fuelled evolution and created us. Competitiveness is what socialism lacked compared to capitalism and caused its downfall. Cheating didn't work in these competitions, because any means to win worked and served their purpose. In the small world of wesnoth's add-ons, the competition must be controlled, and used to motivate people to work better.
Enough philosophy.

Like the dugi/franz relationship.
Thus we already have negative competition.

I do not hate him for being the first. I hate him for cheating, regardless of the good and bad. Despite being caught cheating already twice. I believe that my campaign is very good, judging from huge amounts of positive feedback, so I naturally want people to choose to play it, because I think it is good for them. This does not make me cheat like him of course. I strongly doubt franz even believes that his era is good, because he knows that nobody plays it, that people left some unofficial forums when he tried to force them to play a ladder using his era, that his feedback topic is almost empty, and that it is just a mashup from various other eras. He just wants to be the first, no matter that it will manipulate people to download his useless add-on. I remember him posting around joyfully that his era has the greatest number of downloads, without even saying thanks to other players (as if they had helped him somehow when he got there by downloading his era himself). If the first add-on was IftU or any other campaign that isn't severely incomplete, broken or obviously low-quality, or just something from somebody like ChaosRider who creates endless bugs and struggles to fix them, uploading happily whenever he finds a solution, I would accept it, but if it is obvious that in this case, severe cheating is involved, I can do nothing else than consider franz a narcissistic, ill-manered, reckless brat, and hate him for it.
___

I think that almost everybody is persuaded that some kind or reform will have to be done, so maybe we could discuss rather what should be done, not whether there are other ways. Adamant14, Dunno, me and some other people tried to make some kind of experimental review/ranking system here, I wonder what do you think of it.

Sorry if I tried to moderate again, but because I started this thread, I have a feeling that I am responsible of the flow of the discussion.

[fabi]

I think that it is time to remove the downloads count and possibly replace it by reviews or rankings or both, regardless of FPI #32.

I do not agree about removing the download count.
If you start removing the downloads count, people will look for other signs of quality.
Mine, after the download count, is the size of the download.
Most good stuff is at least some MB big, but there are exceptions.
Very huge addons are mostly only compilations, again there are exceptions.

Thus I use a combination of the flawed download count and the flawed mb size to find quality.
The result is still broken but better than if it was only based on size.

Maybe it would cause bad kinds of behaviour, but less than the downloads count, that is so prone to cheating that it is almost impossible not to cheat (delete add-on before each upload? that would be masochistic).

Yes, making the addon server more useful is certainly worth some effort and disadvantages.

Edit:

One change that might reduce the clutter a little and avoid wrong downloads:
There could be a item in the campaigns menu "Download New Campaign".
The same could be in the era menu and so on.

[Insinuator]

There is negative and positive competition, google finds several interesting articles about the difference.
In general you do not want to avoid positive competition. Our current situation fosters negative competition more since the only known visible counter for success can be manipulated and is self increasing.

Instead one should try to encourage positive competition and dis-encourage negative one.

Heh, "dis-encourage", I like that. :p

You bring up a valid point. There is good and bad competition. I am inclined to agree with Gambit that competition usually fosters hostility. But why is this? You don't address that point. Metric based ranking is the negative sort. It gives people a seemingly objective number and says: "This is good. You must get this to be good. If you do not get this, you are bad." It is simplistic and contrary to the lovable caretakers of Wesnoth.

So, what is good competition? Again, you do not say. A Reviewing system is good competition. This is an obviously subjective narrative that does not give a single metric for success but evaluates on all sorts. Whether good or bad, it shows a level of interest from other humans and fosters both constructive criticism and positive reinforcement.

There must be a solution that doesn't involve ranking add-ons ...

I guess that implies that every ranking system is encouraging negative competition. I doubt that.

You doubt that? Yet you fail to provide even one example... There is a solution that doesn't involve ranking add-ons. It's called reviewing them and it is superior for the reasons I mention above.

I like Gambit's idea of linking to forum threads, but Dugi brings out several good points against it. Such threads do not always stay "on-track" and are continually developing as the add-on does. Garnering useful information is possible, but it is a chore, not the sort of precursory examination that would benefit someone looking for a good add-on.

Additionally, Shadowmaster highlights another issue: only forum members make those reviews. Limiting again who makes the reviews. Now, just speaking from my own experience, I've been on the forums for nine years and maybe left two or three reviews. I'd be much more inclined to review something immediately after completion instead of navigating to a website and going through the rigamarole of doing it on the forum.

[fabi]

You bring up a valid point. There is good and bad competition. I am inclined to agree with Gambit that competition usually fosters hostility. But why is this? You don't address that point.

I do not want to address this issue because I do not agree with the thesis.
If you think that competition usually fosters hostility you have to prove it.

Metric based ranking is the negative sort. It gives people a seemingly objective number and says: "This is good. You must get this to be good. If you do not get this, you are bad."

Most people get used to metric systems quite well.
Everybody knows that the movie with the highest rating at some portal won't be his best liked movie.

It is simplistic and contrary to the lovable caretakers of Wesnoth.

Aren't those "lovable caretakers of Wesnoth" all in love with the KISS principle?

So, what is good competition? Again, you do not say.

And I won't.
All this talking about competition and elite does not fit in my world picture.
We need a working addon server, I do not see a reason to get in detail with the "no elite" and "no competition" thing.
It is just silly.

A Reviewing system is good competition. This is an obviously subjective narrative that does not give a single metric for success but evaluates on all sorts. Whether good or bad, it shows a level of interest from other humans and fosters both constructive criticism and positive reinforcement.

Again, I do not care about competition.
The question is if a reviewing system can solve the problem.
And I do not see how it would.

While I can see that a rating system would solve the problem.

I doubt that.

You doubt that? Yet you fail to provide even one example...

An example for what?
A working rating system?

There is a solution that doesn't involve ranking add-ons. It's called reviewing them and it is superior for the reasons I mention above.

Sorry, I fail to see them.

[Adamant14]

I know, this is already mentioned by Dugi.
There exists a wiki page that contains Player Reviews.
Currently there are just a few entry's, but mind, the page is brand new.

If everyone who has posted here in this thread will add just one single review, then the page will be nearly complete.

I am sure the site can, and will be helpful for all UMC players here, someday.
It would be cool to read a few statements here about the Reviews-wiki page.
1. What do you think about it.
2. Do you think it can be helpful for the issue we are discuss about?
3. Will you add a review?