Download count being an implicit ranking system

[Dugi]

I know that I have mentioned something like this some time ago, but this is not the same. I have noticed this problem some time ago and it remains a thorn in my side.

If somebody wants to try a new add-on out, he tries to separate the good ones from the bad ones somehow. So, he looks for some kind of ranking. Authors' descriptions are frequently too objective or too subjective, and the quality of the add-on cannot be learned from it, just what kind of thing it might be. So, the natural choice is to look at the download count. I can tell that on 1.11, the add-on with the most downloads gets the most downloads, regardless of the type. This causes that some kind of implicit ranking system exists. The download count, however, strongly depends on the number of uploads. Uploads cannot be visible on the download client, so an add-on with 100 updates seem to be super-awesome, recommended by everyone, although it usually means that like 70% of the downloads were mere updates. This gives an advantage to add-ons that are work in progress, and a disadvantage to complete or finished add-ons, that are the ones most people want to download.

Another problem is abuse, people like franz will just keep abusing it. First, it was coloured add-on name. Then, it was downloading his own add-on to become the 1st on the list. Later, it was uploading even 3 times per day. Now, he watches when add-ons behind his add-on approach him, and then he uploads. Because it is my add-on that is second after his (because it has like 50 uploads, but what can I do, I am working on it for long), I see him uploading always several days after that I upload (seriously, I wasn't updating for a month, he wasn't neither, then I uploaded, and he uploaded three days later, and this happens repeatedly). Seriously, there is no rational way why his add-on should have so many downloads otherwise, the only comments on his feedback topic are 2 comments from usual forum visitors, while the following ones have dozens of pages there. Never seen anybody using that era. He also numbers the versions in the 1.4.9, 1.5.0, 1.5.1 way, making a major version from each tenth version always.

Not all superuploaders are abusing it, because sometimes you just realise you've made a mistake in the last version and need to update quickly, and also some people like ChaosRider upload a lot, without malice aforethought I believe.

For short, the current system is unprotected from abuse, contains a ranking system that developers consider unfair, and gives an advantage to unfinished add-ons (that might lower the reputation of all add-ons).

Things that might help:
-removing the downloads count totally, it only ranks add-ons, nothing else
-making the client send an information to the server if it is an update or a new download; this would not prevent abuse of the I download my add-on style, but make it harder at least
-removing the 'update all add-ons' button, adding some checkboxes to make it possible to update all desired add-ons, to prevent people from automatically uploading all add-ons they don't play anymore or just don't feel like removing, consuming the server bandwidth uselessly
-some kind of versioning, only files changed from the last time will be uploaded, saving server bandwidth and preventing updates from increasing the download count
-adding some reviews, and while just score is subjective, reviews can be objective and can tell you if the add-on does what you think
-adding a Ubuntu Software Centre style score rating, even if it is an FPI, it is still better than the download count

[fabi]

Why don't you make it a poll?

Something like:

Remove rating/review system from the FPI list?

[Dugi]

Not quire sure if you are trolling or not, but I suppose you aren't. My point wasn't to add a ranking system, it was just one of the possible solutions to the problem I described. I would personally prefer the possibilities above, like simply removing the download count. And how can I place a poll somewhere? I don't think that normal users can do it.

[doofus-01]

I would personally prefer the possibilities above, like simply removing the download count.

I agree that the download count is a sort of ranking system, I don't see what other purpose it serves. And it's not a very good ranking system.

And how can I place a poll somewhere? I don't think that normal users can do it.

If I start a new post, the poll options are at the very bottom of the page. I don't think I have any special forum powers.

[Insinuator]

Another problem is abuse, people like franz will just keep abusing it. First, it was coloured add-on name. Then, it was downloading his own add-on to become the 1st on the list. Later, it was uploading even 3 times per day. Now, he watches when add-ons behind his add-on approach him, and then he uploads. Because it is my add-on that is second after his (because it has like 50 uploads, but what can I do, I am working on it for long), I see him uploading always several days after that I upload (seriously, I wasn't updating for a month, he wasn't neither, then I uploaded, and he uploaded three days later, and this happens repeatedly). Seriously, there is no rational way why his add-on should have so many downloads otherwise, the only comments on his feedback topic are 2 comments from usual forum visitors, while the following ones have dozens of pages there. Never seen anybody using that era. He also numbers the versions in the 1.4.9, 1.5.0, 1.5.1 way, making a major version from each tenth version always.

This seems like a personal issue...

I would personally prefer the possibilities above, like simply removing the download count.

I agree that the download count is a sort of ranking system, I don't see what other purpose it serves. And it's not a very good ranking system.

Here is the real crux of the matter. Download count IS a ranking system whether we want to admit it or not, and while it is not a great one (due to reasons Dugi mentioned), I think it is preferable to none. At it's best, it indicates the popularity of an add-on, and when something is popular, it is often enjoyable. Now, unfortunately, this also makes it harder for new good material to surface and so people may miss out on some quality work.

But with so many add-ons now available, just scrolling through them all is tedious and turns people off. Remember too, the popularity system can help to keep lame crud down at the bottom. So until the developers decide to reconsider their stance on a real rating/review system, this is the best we got and losing it would cause more harm than good.

[Pentarctagon]

I can add a poll if its wanted.

[Iris]

Another problem is abuse, people like franz will just keep abusing it. First, it was coloured add-on name. Then, it was downloading his own add-on to become the 1st on the list. Later, it was uploading even 3 times per day. Now, he watches when add-ons behind his add-on approach him, and then he uploads. Because it is my add-on that is second after his (because it has like 50 uploads, but what can I do, I am working on it for long), I see him uploading always several days after that I upload (seriously, I wasn't updating for a month, he wasn't neither, then I uploaded, and he uploaded three days later, and this happens repeatedly). Seriously, there is no rational way why his add-on should have so many downloads otherwise, the only comments on his feedback topic are 2 comments from usual forum visitors, while the following ones have dozens of pages there. Never seen anybody using that era. He also numbers the versions in the 1.4.9, 1.5.0, 1.5.1 way, making a major version from each tenth version always.

Firstly, any conflicts you may have with some specific person do not really belong in Ideas (or any other public forum section, for that matter). You are perfectly aware of this.

Secondly, there is no reason someone shouldn’t be allowed to download his or her own add-on multiple times for debugging or testing purposes. Determining whether the downloads are taking place for the sole purpose of bumping the download count is not an easy task because the add-ons server is not currently equipped with a mind-reading utility library on account of the abysmal lack of progress in that field of research. Perhaps later, when the add-ons server software (campaignd) is replaced with a cleaner and more extensible solution we will be able to tweak the download count system to ignore multiple downloads from the same IP over the course of some period of time (although that would impact statistics from users behind NAT routers or other such networks where all clients share a single IP). Perhaps you people could even submit patches to add new statistics features then!

Finally, many people are under the misconception that version numbers follow a decimal-like progression (x.y.9, x.z.0 with z = y + 1, as you mentioned). This is in no small part helped by well-known software rarely reaching minor version numbers of 10 or more. This is definitely not something that proves anything about anyone.

If I start a new post, the poll options are at the very bottom of the page. I don't think I have any special forum powers.

You are an Art Contributor, so you actually do have the poll creation privilege (Forum Regulars, Developers, and Forum Moderators do as well) for historical reasons I am not free to disclose at this particular time.

[Gambit]

Ironically, this thread is pretty much the perfect example of why there should never be an add-on rating system.

[Sapient]

Yeah, my idea was to only allow forum regulars be allowed to cast a single vote on each add-on, to build a somewhat tamper-proof rating system. But after reading Gambit's post, I am inclined to agree that the amount of drama and potential harassment of those individuals would not be worth it.

[doofus-01]

Yeah, my idea was to only allow forum regulars be allowed to cast a single vote on each add-on, to build a somewhat tamper-proof rating system. But after reading Gambit's post, I am inclined to agree that the amount of drama and potential harassment of those individuals would not be worth it.

Funny, my response to his post was: there's already an add-on rating system - it's called the download counter. If rating systems are so bad, stop displaying the download count.

There used to be a voluntary statistics-gathering feature (which I think angered some people, I'll edit to provide a link if I can find it), and I think it tagged each player or installation or something so you could follow their progress through the add-on. If there was a universally visible database that was like that, presented in some way that the casual user could make sense of it, that might be an alternative to a "rating system". No, I don't know how to implement this, but this is Ideas Forum, and a person can dream.

EDIT:Not sure this is the best reference for what I was talking about, but it should at least refresh memories: http://forum.wesnoth.org/viewtopic.php? ... 73&start=0

EDIT2: "stats.wesnoth.org", it used to be called, apparently. It seems defunct now.

You are an Art Contributor, so you actually do have the poll creation privilege (Forum Regulars, Developers, and Forum Moderators do as well) for historical reasons I am not free to disclose at this particular time.

Oh. That didn't occur to me.

[Insinuator]

Yeah, my idea was to only allow forum regulars be allowed to cast a single vote on each add-on, to build a somewhat tamper-proof rating system. But after reading Gambit's post, I am inclined to agree that the amount of drama and potential harassment of those individuals would not be worth it.

Potential harrassment? I appreciate your tender concern for such people, but don't you think they should decide whether it would be worth the "potential harrassment"? I mean, no one would be forcing them to rate stuff, would they? Not that I believe such an elitist system is a good way to rate add-ons. Instead it would become just a breeding ground for the sort of higher criticisms that only that elite group appreciate. Not always representative of the Wesnoth group in general. And, as you infer, would build resentment among the proletariat.

Since this idea has been tabled, I think there should be a clear distinction between a rating system and a review system:

1.A rating system attempts to impose an objective metric onto the system to standardize all inputs (add-ons in our case). Like a five star system, a 1-10 scale, etc...

2. A review system is an entirely subjective method where users describe their impressions via written submission. We already use this somewhat via the forum, but in an awkward form that is almost useless to new users.

A rating system is problematic and difficult to implement, even though it is easier to understand and far clearer. Because of that, I think a review system attached to the add-on window would be a less troublesome thing to implement. After completion of the campaign, every user would be given the option to leave a review. That won't get to rate it in any objective way, but they will get to leave their impression. If these were character limited, it would not be hard to scroll through and get a general impression of the campaign from others.

Ironically, this thread is pretty much the perfect example of why there should never be an add-on rating system.

Why? Because people disagree? Because sometimes we're intolerant or stubborn? That's ridiculous. Open discussion fosters ideas. That's why we have this sub-forum topic. In fact, having a reviewing system could lead to even better designed campaigns because common pitfalls can more easily be brought to light and thus avoided by subsequent authors.

Consider this recent thread: http://forums.wesnoth.org/viewtopic.php?f=6&t=38868 The opening poster has obviously played many add-ons, and his insight could help others. IF THEY SEE IT. If it just gets buried in the hundreds of forum pages, no one will benefit from what he noted. Honest criticism and open praise help mold quality campaigns. The best designers welcome such feedback.

[Sapient]

Potential harrassment? I appreciate your tender concern for such people, but don't you think they should decide whether it would be worth the "potential harrassment"? I mean, no one would be forcing them to rate stuff, would they?

You are underestimating the lengths to which some authors will go to boost the ratings for their content. But I don't blame you because I didn't believe it at first myself, either.

Not that I believe such an elitist system is a good way to rate add-ons. Instead it would become just a breeding ground for the sort of higher criticisms that only that elite group appreciate. Not always representative of the Wesnoth group in general. And, as you infer, would build resentment among the proletariat.

Since when are forum regulars the "elite"? They are simply members of the community who have proven themselves to behave in a consistently friendly and well-behaved (or tolerable yet amusing) manner. If that qualifies as "elite" then I must have missed a vocabulary lesson. Either that or you are exaggerating for dramatic effect and, in my opinion, damaging your credibility in the process. If we went ahead with the idea, of course, it'd make sense to be more diligent about adding to the official list these valued members.

[ancestral]

Dugi makes some suggestions at the end of his post that I think merit some discussion. It would be nice if some of the commenters in this thread would address these ideas instead of getting caught up in opinion-based remarks.

Things that might help:

1. Removing the downloads count totally, it only ranks add-ons, nothing else
2. Making the client send an information to the server if it is an update or a new download; this would not prevent abuse of the I download my add-on style, but make it harder at least
3. Removing the 'update all add-ons' button, adding some checkboxes to make it possible to update all desired add-ons, to prevent people from automatically uploading all add-ons they don't play anymore or just don't feel like removing, consuming the server bandwidth uselessly
4. Some kind of versioning, only files changed from the last time will be uploaded, saving server bandwidth and preventing updates from increasing the download count
5. Adding some reviews, and while just score is subjective, reviews can be objective and can tell you if the add-on does what you think
6. Adding a Ubuntu Software Centre style score rating, even if it is an FPI, it is still better than the download count

My 2¢:

1. Removing download counts would be a fair way to address the issue. I have trouble finding a better solution to make download counts fair. I’d be fine with it myself.
2. Actually, that would be nice. It can’t be perfect, but the more I think about it, I think you’re right, it doesn’t make sense to include updates in the download count.
3. Is this a Feature Request? Sure would be nice to have.
4. Boy, that would be great, if say, add-ons were hosted on a server with git. It would probably require a complete overhaul, but that would be pretty awesome, if extremely unlikely to happen.
5. The biggest challenge is finding people to monitor and manage the reviews, in a manner devs, mods, and regular players and contributors are going to be on board with. We’ve talked about this before, and I just don’t think the number of people to manage this is worth it. That being said, I wouldn’t mind seeing a wiki detailing the add-ons, screenshots, feature lists, explaining what they are. (But again, who does this?)
6. I’m not familiar with USC, and it’s not to say ratings might not be better than download counts, but I’m just not convinced. I do like Starcraft II’s ranking by popularity — the best way to rate an add-on is how frequently it’s being played. The only thing is it’s practically impossible to account for single-player add-ons, and games which use or combine multiple add-ons.

I’ll add one more thing: Wesnoth add-ons aren’t a popularity contest. If you want to have the #1 add-on, you’re playing the wrong game. No one is making money by having more downloads. It’s human to feel wanted, to get that feeling that people appreciate the time you’ve put into your work. But the bad news is, you’ll find a high, meaningless number isn’t going to give you this satisfaction in the long run.

[Dugi]

Remember too, the popularity system can help to keep lame crud down at the bottom.

No. It keeps the new stuff at the bottom, regardless of the quality. Just order them by popularity and see what is on the top. On 1.10, on the top is an MP faction. I never heard of anybody playing it in MP. Shortly after it, there is an unfinished campaign that does not even work without downloading a resource pack manually. See here how many downloads do the add-ons that have a lot of uploads have.

I can add a poll if its wanted.

Yeah, would be suitable. Please include a note asking people to read through the discussion before voting. And the options might be just yes or no, change the downloads count somehow or let it be.

Firstly, any conflicts you may have with some specific person do not really belong in Ideas (or any other public forum section, for that matter). You are perfectly aware of this.

You are right that I do have personal conflicts with that guy, but this is what I loathe him for and why I want to prevent misbehaviour of this kind. I wanted to use him as an example of an abuser, but I think I got too sentimental.

Secondly, there is no reason someone shouldn’t be allowed to download his or her own add-on multiple times for debugging or testing purposes.

There is no reason why it should add to the download count anyway.

Ironically, this thread is pretty much the perfect example of why there should never be an add-on rating system.

But the download count is an add-on ranking system, and a pretty bad one.

Since when are forum regulars the "elite"?

Well, they can post poll topics, while I can't... They ARE some kind of elite.
↑I am not trying to persuade anybody to give me that title, I know that I occasionally break some rules.

But again, who does this?

There would be volunteers. For example, Adamant14 has built a nice Guide to UMC campaigns on the wiki.

Wesnoth add-ons aren’t a popularity contest.

It isn't, but the people who see it so will misguide others to download their (possibly crappy) add-ons. That is why I used that franz guy and his mashup era as an example. It could have been one it wasn't possible and mandatory to cheat. If I have a lot of downloads, I know that it is a fake number, because I uploaded a lot.

[H-Hour]

For what it's worth, I like seeing the download count of my own addons. I don't game the system so it gives me a reasonable idea of how many people download them (which is very very few). It's still nice to know.

It seems like the main concern of the OP is users who don't research what they're downloading. Perhaps there could be some addition to the _server.pbl file which would allow an addon author to add a link to the forum thread about the addon? If it was visible when looking at more info about an addon, maybe it would encourage users to follow the link and see for themselves what the user response has been.