Add-ons client/server support for add-on ratings

[Dugi]

What if an addon is incomplete? There should be an option to disable ratings so incomplete or WIP content is not rated badly.

Do you think people seek to play unfinished add-ons? The rating is intended to let you find good, complete and not too bugged add-ons, not to be forced to be an alpha-tester, to play campaigns that end in the middle of story and can't be played without debug mode. Unfinished add-ons should be rated low, you can always remove them and put them back to reset it if you think that you've finished and it does not suffer from the problems it previously had.

I agree that 0.5 ratings should be allowed, but your text box allows for any decimal rating. The simplest solution would be a slider with steps from 0.5 to 5 or 10, depending.

Slider isn't a bad idea. Does somebody disagree with the option of using a slider there?

Will this user interface somehow handle "outdated" votes, or multiple voting ?

Multiple voting should be disallowed or limited, because there might be people who decide that an add-on is underrated and will give it 10/10 ten times. But allowing the players to vote once per some period of time might not be counter-productive, because add-ons change, people change, and people progress through campaigns (cool beginning, the rest is bad or unexpectedly missing), it should be possible to vote again, (maybe overwriting the previous vote). Keeping track of all votes might not be feasible, because the amount of data stored would hoard quickly, so old votes might be forgotten (they might or might not affect the total average). You see that I have not decided here yet.

Cause they are authors of own addons created by them self, and they might be voted only if such a author allow for that (in server.pbl file could be option to write it like a "allow_voting=yes" (or no), still if this file wont have such line it should be written as "no"). Still vote rating is made on addition, you can make your own voting on forum (then you wont need author accepting).

You still haven't provided any reason why should somebody heed the demands to disallow votes. And why would authors want not to see their work rated when it helps the public, unless they are aware that their add-ons are awful and they want to hide it.
Yes, authors can vote for their own work if they avoid the protection against it, but because it's logged, they would probably raise suspicion if repeated too much.

[Pentarctagon]

What if an addon is incomplete? There should be an option to disable ratings so incomplete or WIP content is not rated badly.

Do you think people seek to play unfinished add-ons? The rating is intended to let you find good, complete and not too bugged add-ons, not to be forced to be an alpha-tester, to play campaigns that end in the middle of story and can't be played without debug mode. Unfinished add-ons should be rated low, you can always remove them and put them back to reset it if you think that you've finished and it does not suffer from the problems it previously had.

Encouraging people to delete and re-upload in order to clear their rating just seems like a bad idea in general. WIP/incomplete add-ons are exactly that: incomplete. They are not put on the add-on server so they can get rated on the same scale as a completed add-on, they are put on there primarily so that other people can give meaningful feedback and report bugs. To have them be rated on the same scale as finished add-ons is rather misleading as well as harmful to gathering feedback since fewer people would be willing to try it.

[iceiceice]

Dugi:

I also think that add-on makers should be able to opt out of vote. Making add-ons is fairly personal, sometimes the maker wants to distribute something while limiting the kind of criticism it can receive. For example this is one reason we have the art development forum.

I don't see that it limits the information a prospective player gets, as they will see that the maker disabled votes. I don't see any reason why we shouldn't allow designers to disable votes at their discretion.

[Dugi]

The intended purpose of that raking was to allow players to choose something nice to play. They don't want to help add-on developers by downloading incomplete add-ons, they want to download the complete ones that are also good. The problem that happened many times was that a player downloaded an add-on, realised it was incomplete very early, downloaded another one, realised it was something he didn't know to use, downloaded another one, realised that it was bad, and left all add-ons be. This is not what we want. People shouldn't be encouraged to download incomplete add-ons.

They should be required to help add-on developers only if they are willing to. If they want, it usually means that they have enough skill to be able to learn that some add-ons have low rating because they are not complete. Maybe the incomplete add-ons could be hidden by default, but with a visible option to make them visible (identified by some PblWML key or version bellow 1.0.0, and marked on the list of add-ons somehow). In that case, it could be written in the rating window that we shouldn't be very strict because it is still being improved. Or maybe not rated at all, making the players read the reviews written for it.

I also don't understand what is the purpose of playing add-ons that are severely incomplete, a four scenario project with little interesting functionality (for example Story of Dacyn) needs more work from the author more significantly than some feedback. If the author wants something from the players, he should do his homework. 10 scenario campaign with full storyline, that will be later expanded with more characters, some scenarios in between (their events told just in narrator speech before), maybe with some additional functionality is better than a bit of story where plot events make no sense yet, even if some units and such are done well (unless these auxiliary functions are very significant). The story is the base of a campaign, it can be enriched later with extra scenarios, units or functionality; this way it is a playable campaign from the start, and it is getting better. Functionality first, balance and bugs later, worse balance and stability isn't as bad as most scenarios missing. Or doing it like Shadowm did it in After the Storm, publishing it first with a playable and fully operational episode, adding more episodes later.

[Pentarctagon]

The intended purpose of that rating was to allow players to choose something nice to play. They don't want to help add-on developers by downloading incomplete add-ons, they want to download the complete ones that are also good. The problem that happened many times was that a player downloaded an add-on, realized it was incomplete very early, downloaded another one, realized it was something he didn't know to use, downloaded another one, realized that it was bad, and left all add-ons be. This is not what we want. People shouldn't be encouraged to download incomplete add-ons.

Which is, if anything, a good reason to disable ratings for incomplete add-ons: It gives a very easy and visible way to distinguish between add-ons that are just plain bad and ones that are incomplete, as well as another possible category for filtering results.

[Dugi]

I think that I have already written that in the post above. Incomplete add-ons might be unrated, but the incomplete ones and not the ones whose author decided that he does not wish to be rated. Without rating, they'd end up at the end of the list or be filtered out by default (but that option would be somewhere clearly visible).

[Great_Mage_Atari]

This is really cool. I hope there are more implements like this added to the online aspect of BfW in the future.

[Iris]

@Dugi:

Out of curiosity, have you seen this devs ML thread, and in particular this post?

[pauline]

@ Dugi: "... allow players to choose something nice to play. They don't want to help add-on developers."
And that coming from you ? Please, don´t generalize or underrate the players:
There are who DO wish to "give something in return" but can only contribute to BfW by "test-playing" … qualified or not.
And IMNSHO, finding/logging bugs in the work of others is a (all too human) motivation that should certainly not be underestimated !
(I´m even under the impression: the better a WIP the harder to get feedback ...)

@ Dugi: "incomplete add-ons could be hidden by default"
Good idea, too ! Or "separate add-on servers" as Ivanovic suggested in that ML thread just mentioned by shadowm.
"... probably not good because the experimental server will get almost no visits" … Well, see above

@ Dugi: "player downloaded an add-on, realised it was incomplete very early, downloaded another … downloaded another one ..."
Hmmm ... the add-on description should help to avoid such "misinterpretation", no ? But maybe it´s not enough for some ...

@ Pentarctagon: "a good reason to disable ratings for incomplete add-ons"
I agree: Bad ranking might ideally stimulate an author "to do his homework",
but can dismay designers who care about meaningful feedback. I can see that now, and it would be a pity !
Contributors should get all the attention they deserve.

pauline (vanilla user)

[Dugi]

@shadowm
I think I have seen that.

@pauline
I was trying to say that the primary intention of players is to play, not to help develop stuff. There are people who like helping by playing stuff, but I doubt they are a majority. I did not mean to offend you or something, sorry.

Some add-ons' descriptions are not mentioning that the add-ons are incomplete, see for example Antar, son of Rheor - nice campaign till you come to the middle of the story and realise that there are no more scenarios. Bad add-ons' descriptions aren't telling that the add-ons are bad (keep also in mind that bad add-on means something different for every individual).

[Iris]

@shadowm
I think I have seen that.

And what do you think of my proposal, then?

[iceiceice]

Not to sidetrack the discussion from shadowm's proposal, but I want to also say something about the text box.

(also making ratings like 7.5 impossible).

Dugi, it might be a worthwhile trade if many other things are also impossible. Terrible things can happen when the user can type anything they want:

http://xkcd.com/327/

And before you say, "that's fine I'm not using SQL", see the following comment from the wesnoth source code itself, in a feature as innocuous as chat messages:

Code: Select all

		// We've had a joker who send an invalid utf-8 message to crash clients
		// so now catch the exception and ignore the message.

https://github.com/wesnoth/wesnoth/blob ... .cpp#L1062

Make sure you think carefully what are the bad things that can happen, it would be a shame if someone found a way to destroy all the reviews or something. Come to think of it, I hope that the add-on server code is sanitizing its inputs in a sane way.

[Dugi]

@shadowm
Okay, deeper analysis:
Like/dislike - if an add-on has no particular flaw (no matter how mediocre it is otherwise) it will get only likes, and the like/dislike ratio will tell next to nothing
Broken/Inappropriate - can be reported on the forum, it would get mentioned in a review if an add-on was broken or inappropriate
Star based rating - currently working on this, its main advantage is that the players will be able to sort it by something like quality to quickly find the popular add-ons (this is fabi's idea, not mine)
Add-on server separation - as mentioned, experimental server will get almost no visits, furthermore this already exists, I am for example using the trunk server for the purpose of publishing stuff for beta testing, and very few people use this possibility
flag for good add-ons - this is a flamebait like white phosphorus; there would have to be a quality threshold, and everybody just bellow the threshold would be angry and hate the rating team
outdated votes getting less influential - I agree and intend to implement this
more separate ratings - this would have to be viewed in the add-on's description anyway, and I think that implementing an automated system for reviews would be more accurate than this

@iceiceice
The data is read as a string, then transformed into a number and further processed as a number. If something goes wrong (weird number, no number), the number is ignored and it is considered that no voting occurred. The risk that somebody breaks it by sending something wrong is the same as somebody breaking it by uploading an add-on with a weird pbl file and lower than somebody breaking it using a modified client. Anyway, I think that there was an agreement that a slider would be better.

[pauline]

Like/dislike – "ratio will tell next to nothing" … Guess that´s true.
Broken/Inappropriate – "can be reported on the forum / in a review" …
Why not "flagging" them on the server to satisfy players who don´t speak English or don´t like reading ?
Star based rating – "currently working on this" … May you do well ! "this is fabi's idea" … A touch of genius ! (Don´t bother, ghost: You´re welcome !)
Add-on server separation - "using the trunk server … very few people use this possibility"
Is THAT what Ivanovic meant with the "experimental server " ???
flag for good add-ons – "everybody … would be angry and hate the rating team"
Personally, I like the idea of an "expert-rating-team", but as it turned out on the "Players’ Reviews":
creating such a team seems to be the main obstacle …
outdated votes – "I intend to implement this" … All the best in your work !
more separate ratings – "break down the rating into different aspects"
My opinion, for what it is worth: The efforts of management and contributors to provide a prime BfW are admirable and highly appreciable. However, sometimes I think the average users are abashedly overestimated by the quality-conscious peers …
With all due respect: I hope that this rating-project won´t be disputed "into bits" because opinions on the details differ
on a matter, which has already been discussed … five ? eight ? years ago if I'm not mistaken. Sorry, slightly off-topic.

@ Dugi: "primary intention of players is to play, not to help develop stuff"
No offense taken ! It was just a giggle. I figure you have little reason to complain a lack of feedback ...
The fact that almost 70 % of "over-critical" members don´t ever give feedback on the forums (or once at most)
seems to proof you right. An automated system for review/server supported add-on rating might change this dramatically.

pauline (vanilla user)

[Ravana]

For problems with unfinished addons these ratings could expire over time so they always get closer to current state.
If author has option to disable rating just show them after rated ones, I dont see problem there.