How difficult did you find the scenario? (1-10)

General feedback and discussion of the game.

Moderator: Forum Moderators

Post Reply
User avatar
Maiklas3000
Posts: 532
Joined: June 23rd, 2010, 10:43 am

How difficult did you find the scenario? (1-10)

Post by Maiklas3000 »

In the campaign scenario reviews, usually people rate the difficulty of the scenario on a scale from 1 to 10. I sometimes see reviewers say something like, "It was a 2/10. I only had to save-reload twice to beat it." In my universe, if you had to save-reload, it wasn't a "2". You are free to rate it how you want, but you are only going to confuse the scenario author with that kind of rating.

For what it's worth, here is how I rate the difficulty of scenarios:

10 - Impossible to win, or only likely with considerable luck and/or multiple save-reloads.
9.5 - Held me up for a long time - I thought it was impossible, but finally beat it, maybe with a save-reload.
9 - Very hard, probably lost a few times, may have required me to change strategy.
8 - Hard, may have lost once or twice, forced me to concentrate.
7 - Challenging, may have lost once, need to think to beat it.
6 - Mildly challenging, beat it the first time, but I see how I could have lost.
5 - Pretty easy, some strategy or tactics involved for victory.
4 - Easy, very little strategy, just need sound tactics to avoid defeat.
3 - Very easy, but defeat is possible if you are careless.
2 - Child's play, but you could lose if you actually tried.
1 - Story, literally impossible to lose.
User avatar
taptap
Posts: 980
Joined: October 6th, 2011, 5:42 pm

Re: How difficult did you find the scenario? (1-10)

Post by taptap »

Wholeheartedly agree. However, when some campaign authors (for UMC at least) are under the impression it is ok to balance with a "modest amount of saveloading", it is hard to urge players to be consistent.
I am a Saurian Skirmisher: I'm a real pest, especially at night.
User avatar
beetlenaut
Developer
Posts: 2813
Joined: December 8th, 2007, 3:21 am
Location: Washington State
Contact:

Re: How difficult did you find the scenario? (1-10)

Post by beetlenaut »

It might be a good idea to sticky this list or something similar in the campaign feedback threads.
Campaigns: Dead Water,
The Founding of Borstep,
Secrets of the Ancients,
and WML Guide
kiss
Posts: 124
Joined: July 9th, 2014, 5:50 pm

Re: How difficult did you find the scenario? (1-10)

Post by kiss »

I think this is a real problem for all of us!
And when I say that, I take into account all what you are working on, I mean you, the whole dev team. But I stay focused on difficulty.
I agree with most of what Maiklas3000 says but it doesn't make "all" the job like that ... at least for me. It is far most difficult than that.

Now, I'm in my desperate lone hole ... and you need some explanations, I'm sure that without that it wasn't fair. Read that. Now I can explain myself.

I don't know how but it should be neutral, else it's a self judgement and it's wrong. For example I'll never say 2, never!
A short way is to link all games with a server and grab all needed.

The easiest other way I can imagine is a kind of matrix : replay, reload, time spent, loss, kill, ... I don't know what values you need.
And extend that to all other informations ... scenario, fun ... staying simple as possible.

I know it's difficult.
kiss
Posts: 124
Joined: July 9th, 2014, 5:50 pm

Re: How difficult did you find the scenario? (1-10)

Post by kiss »

Since my last post I've stayed stuck on this rating system question.
To understand well, I've searched for Maiklas3000's reviews and discover some high level ones (expert's campaigns played on hard mode). I've don't read too much to avoid self spoiling!
Nearly all give the starting gold amount with other few parameters: the number of turn to finish, units lost and killed, ...
Looking at the replay file to know if there is a way to get that easily, I've found all those statistics in the file. Then I hope you don't really need us to provide all those informations when the replay file is attached.
Saying that, perhaps here is the official ladder some asked for?
But you can't have : time spent, save/reload/restart, study of the map, starting strategy and it's modifications, ...
Maiklas3000 wrote:In the campaign scenario reviews, usually people rate the difficulty of the scenario on a scale from 1 to 10. I sometimes see reviewers say something like, "It was a 2/10. I only had to save-reload twice to beat it." In my universe, if you had to save-reload, it wasn't a "2". You are free to rate it how you want, but you are only going to confuse the scenario author with that kind of rating.
For me the link between how many save/reload and the difficulty scale from 5 to 10 is obvious. Then why don't ask for : how many time do you save/reload (and perhaps why)? and add one or two other questions to cover the lower part of the scale.

Let me try with an example. I give you that:
0 time played before, no help (walkthrough, forum, ...), beginner mode, 0 restart, 0 reload, 100 starting gold, time spent to study the map 90 mins, time spent to chose a starting strategy 15 mins, number of turns 10/16, game's time 15 mins, lost/killed/leveled units 0/2/0.

You should agree that from the player side it is straight forward to answer, from the dev team and author side it should be easy to judge and classify as well, won't you?

Here is my example, I answer 1.
But, yes, I see how I could have lost so it should be 6. And I always think and concentrate, so, an 8?

An other little annoying point is that the first post (the one with the list) isn't the same for all scenario. I can understand that for campaigns that are still under development, I can even understand a modification of the list when some points are settled or arise. But for main campaigns, shouldn't we have the same one?
Thus, the first post can be an explanation of what, why, when do a review with a link toward a blue print. Or better, more than one blue print because playing a scenario for the first time is always special, play on easy or hard isn't really the same, ...

I feel a little stupid to insist on this, but, I've spent so much time on it and any way to make the process easier will be welcome. To give you an idea, in time, it is something like 2/5 playing, analysing and reading, 3/5 trying to point out something interesting in a review. 3/5 is reducing as I discover less than in my early hours and I simplify.
Velensk
Multiplayer Contributor
Posts: 4002
Joined: January 24th, 2007, 12:56 am

Re: How difficult did you find the scenario? (1-10)

Post by Velensk »

I would suggest that you simply view what you're doing as a favor you're doing for the developers than a log of your play or anything that requires a strict format. They're trying to get feedback, they want it to be useful.

So, telling them that every scenario has a '1' level of difficulty doesn't really help them much. The point of the exercise isn't to come up with some absolute rating for how difficult a scenario is but to tell them how much trouble you had with it. In this, it doesn't matter if you're an expert or a newbie because all they care about is what the experience was like for you. If you rate every scenario a 1 but have trouble with some (requires multiple tries or change of tactics) and no trouble with others (beat it on first try without too much strain) then that kind of removes the point of giving a number at all and you might as well just describe it.

All of the other things are there to give them extra info on related things. For example, you might find Clearwater Port to be impossible unless you leave on the first ship; does this mean that the scenario is too hard? Maybe, maybe not. What other information would help? Well, it'd help to know if you still have your loyal heavy infantry and white mages in your army and other units that make a solid defense. Nobody comes out of the last scenario with much gold so that doesn't matter too much. Then there is the matter of how skilled a player you are but as few are good judges of that we instead take a sampling of many people and trust that many of the people are relatively new and that others are experienced. Also, you can frequently figure this out by other clues.

Then there are other things unrelated to balance that are good to know. No matter how good/bad the player or hard/easy the scenario, each scenario is an experience and the developer is trying to craft that experience to be enjoyable (or to fit some other agenda). As long as your feedback helps explain what your experience was like it can be useful. Giving numbers can help and can be easy to analyze on mass but only if the numbers have some kind of meaning. Numbers get meaning from being diverse from, and relative to, each other. So slapping down that each scenario had a difficulty of 1 an enjoyment of 7 or whatever tells a developer nothing about the individual challenge and impression each scenario was to you.
"There are two kinds of old men in the world. The kind who didn't go to war and who say that they should have lived fast died young and left a handsome corpse and the old men who did go to war and who say that there is no such thing as a handsome corpse."
kiss
Posts: 124
Joined: July 9th, 2014, 5:50 pm

Re: How difficult did you find the scenario? (1-10)

Post by kiss »

I understand much more than what you write, should I record kiss007!
Thank you for that, really. But I'm afraid I'll still need more help, I'm only 17.

I hope I understand well your comments. I'll drop the list, I don't feel comfortable with it anyway, and build mine.

Will you agree to comment on my main thread for my coming few reviews?
Of course anyone else is welcome.
LienRag
Posts: 127
Joined: September 24th, 2018, 4:03 pm

Re: How difficult did you find the scenario? (1-10)

Post by LienRag »

Maiklas3000 wrote: June 13th, 2014, 4:11 pm In the campaign scenario reviews, usually people rate the difficulty of the scenario on a scale from 1 to 10. I sometimes see reviewers say something like, "It was a 2/10. I only had to save-reload twice to beat it." In my universe, if you had to save-reload, it wasn't a "2". You are free to rate it how you want, but you are only going to confuse the scenario author with that kind of rating.

For what it's worth, here is how I rate the difficulty of scenarios:

10 - Impossible to win, or only likely with considerable luck and/or multiple save-reloads.
9.5 - Held me up for a long time - I thought it was impossible, but finally beat it, maybe with a save-reload.
9 - Very hard, probably lost a few times, may have required me to change strategy.
8 - Hard, may have lost once or twice, forced me to concentrate.
7 - Challenging, may have lost once, need to think to beat it.
6 - Mildly challenging, beat it the first time, but I see how I could have lost.
5 - Pretty easy, some strategy or tactics involved for victory.
4 - Easy, very little strategy, just need sound tactics to avoid defeat.
3 - Very easy, but defeat is possible if you are careless.
2 - Child's play, but you could lose if you actually tried.
1 - Story, literally impossible to lose.
I played Wesnoth since version 1.14 (not continuously, though) and rated a few scenarios, but I only discover this very useful post today.
It really should be stickied somewhere easy to find, probably in the "feedback" forum (or linked to by each first post of each "feedback" thread).

One minor quibble, though : the ladder shouldn't start with "story"; if "story" scenarios need to be ranked, they should be ranked 0 or NaN.
There also should be a little more emphasis about "needing to think really hard and to try different strategies"¹.

So, may I propose this revised ranking system or does it run contrary to the idea of having one general² agreed-on reference for ranking scenario difficulty ?


10 - Impossible to win, or only likely with considerable luck and/or multiple save-reloads.
9 - Held me up for a long time - I thought it was impossible, but finally beat it, maybe with a save-reload.
8 - Very hard, probably lost a few times, have required me to change strategy or at least to display my best tactic and strategic skills.
7 - Hard, may have lost once or twice, forced me to seriously concentrate.
6 - Challenging, may have lost once, need to think to beat it.
5 - Mildly challenging, beat it the first time, but I see how I could have lost.
4 - Pretty easy, some strategy involved for victory.
3 - Easy, very little strategy, just need sound tactics to avoid defeat.
2 - Very easy, but defeat is possible if you are careless.
1 - Child's play, but you could lose if you actually tried.
0 - Story, literally impossible to lose.

(yes, I also changed a little bit the 5 values : tactics are already weighted in 4, so the difference should be the need for strategy)

Note that this objective ranking system (either the original one or the revised I suggest) would many times not have resulted in the same notes in the scenarios that I ranked - a proof either of its necessity, or of the fact that I'm a fool (your choice).


¹ Or is this point just me involuntarily admitting to playing most of the time without much thinking ?
² Though subjective : like Velensk wrote, the point is for developers to get a glimpse of what the players' experiences were with the scenario/campaign, so if a beginner rates "10" a quite easy scenario because he actually couldn't find a way to beat it, then it's fine, the system works as intended. But it's still better than the developer not knowing whether "7" meant "quite easy" or "nearly impossible" if there is no standard reference.
User avatar
lhybrideur
Posts: 355
Joined: July 9th, 2019, 1:46 pm

Re: How difficult did you find the scenario? (1-10)

Post by lhybrideur »

Only bumping into this threat because of the new reply, but wouldn't the rating depend on the level of the player? A good player might rate a level differently than a bad player.
LienRag
Posts: 127
Joined: September 24th, 2018, 4:03 pm

Re: How difficult did you find the scenario? (1-10)

Post by LienRag »

As I wrote, yes it would, but that's (mostly) how it is intended : it's an objective rating of the subjective difficulty.
A beginner-level player "having to think hard and muster his best skills" to beat a scenario will not have the same abilities that a high-level player will by "mustering his best skills and thinking hard", so yes a beginner may rate a scenario 9 while a good player (on the same difficulty) will rate it 5, but both opinions are useful as feedback for the developers, it helps them understanding the variety of experiences.
Starg4z3r
Posts: 5
Joined: September 2nd, 2022, 9:37 pm

Re: How difficult did you find the scenario? (1-10)

Post by Starg4z3r »

Wesnoth is a strategy game, strategy should always be required. Any scenario that does not require it should be very low.
It should also be relevant how reliant the winning conditions are on the heritage units, aka: "can you beat it with just tier 1 rookies?" and "what type of units do you need to inherit to win it?"

So I'd propose something on the line of:

10 - Impossible to win, or only likely with considerable luck and/or multiple save-reloads.
9 - Very Hard, requires excellent tactics, extra gold, luck and some very specific units inherited from previous games. You lost ten times already to this.
8 - Hard, requires excellent tactics and tier 3 units inherited from previous games. Could have lost a couple times even with the right setup.
7 - Challenging, requires good tactics and evolved units from the previous games, beating it with basic soldiers would require a lot of luck.
6 - Fair, may have lost once, need to think to beat it, properly exploiting weaknesses is a requirement.
5 - Mildly challenging, only tier 1 units required, beat it the first time but I see how I could have lost.
4 - Easy, some strategy involved for victory, you must build the right units.
3 - Very easy, gold is easy to get, you can win just by building scores of random units.
2 - Child's play, no tactics required, you'd win this by sending archers against woses the entire game.
1 - Story difficulty, literally impossible to lose.

I must say I'm not satisfied with this list at the moment since it doesn't properly take into consideration that a campaign is supposed to have high level units (at a certain point) from your previous games, and some scenarios could be pretty easy once you have, like, two fire mages. I need to think a bit more about this, but I only wanted to clarify my point for the now.
Only bumping into this threat because of the new reply, but wouldn't the rating depend on the level of the player? A good player might rate a level differently than a bad player.
Yes, but it's all averaged out in the end. Expert players give feedback, newbies give feedback, the devs can tally it all and make an average to get an idea of the perceived difficulty. I tried to add some reference though, just to offer an idea of what a player should expect (in my opinion of course).
Post Reply