Suggestion on improving CW4's "Colonial Space" rating system

[Grabz]

This suggestion aims to improve two things:

• Player enjoyment, through being able to consistently find maps better suited for them.
• Mapping satisfaction. Less players playing maps that aren't suited to their skill level means ratings are based more on quality, and less on difficulty.

Part 1
One of the problems with Colonial Space that I find is that it's the ordered container of every map, easy or hard. This means so it can be difficult for a player to adjust the maps they want to play to their preferred skill level, or just whatever they are feeling like playing at the time.

As players generally prefer playing easier maps, this creates another issue where players will rate more difficult maps lower, because some players have a tendency to launch every map with the expectancy that they are good enough at the game to complete it.

I suggest that CW4's "Colonial Space" is fragmented into four difficulty categories: Easy, Normal, Hard, Expert. This is basically a glorified filter setting, where one of these four settings is always selected. Here's an example idea:


How will the filter know which maps are easy and which ones are hard? Clear rate.

This is an extremely difficult map. With 417 downloads and only 9 scores, it has a clear rate of 417/9=~0.02


This is an easy map. One in four players have beaten it, which from my observation is about the average ratio for maps which can be beaten by most people.


Clear rate can be used to determine the difficulty of a map. Now, I admit this setup is not 100% perfect, but no system driven by data is perfect - my goal is to make it better and more fun overall.

Example difficulty brackets (these are to be adjusted):

• Easy - 0.2 clear rate and higher
• Normal - 0.13 - 0.2 clear rate
• Hard - 0.5 - 0.13 clear rate
• Expert - 0.5 clear rate and lower

Why not just add clear rate, and let players pick their own clear rate in the filters, rather than force players to only see one of four difficulties at the time? Simplicity. I'm sure most players don't use filters to their strengths in the first place because of their overwhelming nature. I believe categorizing all maps into difficulties will make it so newcomers who choose "Easy" will feel more welcome playing our maps. In CW3, for many new players, Colonial Space is this big scary behemoth of player stuff where everything is difficult. We mapmakers also frequently underestimate just how good we are at the game, and what is easy to us might be hard to someone who'd just gone through the story and hungers for more.

Naturally, map clear rates will change over time. This is fine - this is only a filter setting, and maps will just simply swap places as more data is gathered. Maps with not enough data (say, less than 50 downloads) are not categorized, and are pending categorization, to eliminate high variance. There could be another category for Pending maps, where players can play the absolutely newest maps similar to browsing the first page of CW3's Colonial Space, but this setting would not be the default one so a player would have to consciously click the button to browse these maps.

Part 2
Players must have finished the map to be able to rate.

Ratings should be an indicative of the map's quality and enjoyment through play. If you can't finish a map, you don't have the skill level for it - that doesn't mean the map is bad, but many players feel the need to rate the map lower if it's beyond their capabilities. This leads to more difficult CW3 maps being less interesting to produce overall, as even if you make The Greatest Map Imaginable, you are guaranteed not to have many people see them after your map drops to the second page of the newest section, because people won't rate the map high for being good, they will rate it low for being unable to complete it.

Inspiration
This suggestion is inspired by Super Mario Maker, a game for the Wii U.

In SMM, you only play user made Mario levels. Each level is put in one of five categories: Easy, Medium, Hard, Expert, Super Expert, based on how many people have beaten it compared to how many retries the level has accumulated. When a new level is released, it is inserted into one of the five categories depending on how well the author did at his own level, and then is quickly moved into the correct category from having other players attempt it, if the original author's performance was too variant from average.

Notes
I believe this system has a good shot at tackling several problems.

New players get to join us in "Colonial Space" without most maps being unbeatable for them. Now, I can tell if a map is hard by doing this clear rate calculation to myself, but to a new player, it won't be immediately obvious which maps they should be picking. This way, a new player can stay on Easy and play maps they can beat, and then slowly move up in difficulty as they get better. For veterans, picking Easy would be nice if they want to relax.

This solution also makes it so players looking for difficult maps only will have an easier method of doing so. In CW3, the only solution is to look through all the maps, and pay attention to maps with low ratings that look like they're high quality and have a visibly low clear rate.

In my short although I think pretty decent mapmaking career, I've seen other mapmakers get frustrated at ratings. I've been frustrated by ratings. I know that "we shouldn't just take it to heart", and I agree - but when there's room for improvement, why not make the system better, yeah?

Players also frequently have the urge to "beat everything". I get that myself - it's fun for a completionist to complete an entire category. Maps in CW3 come out very often, and CW4 has a chance of getting even more popular than that, meaning that CW4's "Colonial Space" might get flooded really fast. Dividing maps into categories would help contain that, as well as give players smaller goals to strive for. For example, beating the entire Colonial Space in CW3 is an obviously impossible task (or at least if Alter Old starts making maps :) ), but beating the entire Easy category of CW4's "Colonial Space" would be a lot more possible, making it more fun to progress through user made levels for those of us completionists out there.

[cornucanis]

I like the idea of organizing the maps into categories based on difficulty. As you say, it would make it much easier for players in search of a challenge to find what they're looking for. Mandatory category division may ruffle some feathers, though, since many people like to be able to browse everything at once.

I doubt the difficulty categories alone would be enough to have any impact on ratings. Of course, you've addressed that with part 2 of your suggestion. I feel that requiring a player to complete a map before rating could lead to some controversial situations, though.

For example, let's say I were to create a map that can only be beaten by performing a ludicrously specific set of steps. Then I tell a few friends how to beat it knowing they will also rate the map 10. Now my friends and I have the only 4 scores posted on the map, so we were the only ones allowed to rate it. But it also happened to get more than 50 downloads, so now it's officially categorized and listed as the highest rated map in Colonial Space (or CW4 equivalent.) It may be categorized under "Expert" but people will probably still be justifiably upset that it's only listed as the top rated map due to such circumstances.

As a fan of challenging maps, I personally really like the idea and would love to see it implemented in some form. I wanted to play Devil's Advocate for a moment, though, and point out some of the considerations that must be made when implementing such a system.

[Grabz]

For example, let's say I were to create a map that can only be beaten by performing a ludicrously specific set of steps. Then I tell a few friends how to beat it knowing they will also rate the map 10.

Yeah, this would allow a map like this to briefly take up a top spot in ratings. However:

• This has been done before, and is prohibited by the rules, which will lead to your map being deleted. Players are not allowed to create solutions that involve effectively using a "password" through scripts, regardless of whether you did it because you didn't feel like beating your map, or your map is unbeatable without it.
• A map like that will gain some infamy. We can read the scripts, we can figure it out, beat your map and rate it down, if taking it down is not an option/might take a while.

You raise a valid point, and I'm not sure if there's a better way of tackling it than the old fashioned human moderation. I can't deny that this system would require more moderation as this sort of exploit involving multiple people would take a lot longer to naturally fade than it would in CW3.

Popularity of the game plays a factor, the more people there are playing the more people will figure out your tricks and beat your map.

Part 1 of the suggestion might still work without Part 2, which is the problematic one. Players might not get frustrated and rate down maps when they don't have to see your maps by playing maps from another category. However, I feel like rating maps without beating them is a damned if you do, damned if you don't situation. Both solutions have very pronounced pros and cons to them.

[J]

Another consideration: some maps are 'beaten' quite easily, but take years to actually complete. I'd say the people who beat that map, but ignored the long slog after it should be able to vote.

Anyways, a completion ratio filter would be a nice addition.

[chwooly]

I agree with your idea of a difficulty filter. It is a great idea.

However as previously stated I believe that only allowing those that finish a map to rate it is a bad idea.

You don't have to finish something to know if you like it. This is seen in eating a meal, watching a movie, reading a book. staying in a hotel.

What I do notice is that so far the only advocates for that style of rating system are mapmakers, But the end users need a say,

Maybe instead of having to finish the map a system could be implemented where you have to have played the map for a set amount of time or that your scored is marked by a finished/unfinished tag of some kind so that others will know. Even just having your rating shown and could be filtered might be a way to go so that people can see if what a persons patterns and then base if they want to play a map based on another persons rating sort of along the lines of going to see a movie because a specific critic liked it.

But of course I am sure someone will get butt hurt by my comment as is usually the case.

[GoodMorning]

Not at all - it it, as you point out, necessary to get various opinions on it.

[chwooly]

There is a map I am currently in that I will never finish, I know after restarting 3 times and playing for about 20 mins that it is not 1 I enjoy. I know this even though I have chosen not to finish it, Not because I can't figure out how to beat since it is effectively beat but because I have reached a point where my mind is numbed by all the micromanagement I have to do. And spending anymore time on it is not something I am interested in. 

It has now been running and is at 1.2hours since I have it at 4X and it is getting laggy Another reason I won't finish it.

I didn't rate the map but if I were to it would receive a low score because for me it was not fun to play. Should I be forced to finish a map I get little to no enjoyment from just to rate it?

Those players that like having to pause and like micromanagement will probably love this style of map.

I hope to see more maps that use the mechanics of this map.

Another option might be to have a pull down menu that requires you to check off why you rated the map the way you did.

1) Loved it.
2) too boring.
3) I can't beat it.
4) Too much Lag.
5) Not fun.
6) I hate the map maker.
7) Don't like slogging.
8 ) I am just an evil person and rate every map low.

I do understand that those that invest time in making maps want to see high scores as it is a validation of their effort and rightly so. That is why I think being able to see the who rates a map would be a good idea then you could discount those that you know are rage raters and those that genuinely give maps a fair shake. Either way I think there is going to unfairness in any system chosen.

And at the end of the day I am still going to keep playing Knucklecracker games and enjoy them and the custom maps that others put time and effort into making (for the most part 8)).

Cheers

[Grabz]

I didn't rate the map but if I were to it would receive a low score because for me it was not fun to play. Should I be forced to finish a map I get little to no enjoyment from just to rate it?

Those players that like having to pause and like micromanagement will probably love this style of map.

I don't know what exactly the map is since you didn't mention that. However, if this idea were in the game, this map would probably be in the Expert section, which by definition would only contain maps that require lots of micromanagement, as difficulty in these games usually comes down to getting the best start possible. As such, you would probably never look in the Expert section as you don't like micromanagement. This is the important layer of separation and that's why I mentioned it first, but it seems like you have chosen to ignore it in favor of only commenting about part 2 of my suggestion.

Why should you be able to rate the map if it's out of your league? I think the ratings make more sense as "I completed this map, this is how I felt about playing it". When everybody can rate a map, what you end up with is maps being rated by difficulty based on the average player skill, more than maps being rated by how enjoyable they were, because when Colonial Space is a congregation of literally everything in order, players feel like they should be able to beat everything, no matter how good they actually are.

The point you're missing with what I'm trying to say is that you shouldn't be playing maps that are out of your skill league in the first place. And if micromanagement is not your thing, that's perfectly fine, but understand that being better at micromanagement is raising your skill at playing the game. Of course this is purely personal preference, personally I don't like micromanaging that much either and I just play to relax, but I understand the frustrations of people who want to make/play harder maps but the game just doesn't provide much of an outlet for that if you intend for your maps to be recognized.

I know for a fact that some of the popular mapmakers are perfectly capable of producing and beating difficult maps, but they opt not to do that to avoid getting low ratings/bad reputation, and instead make easier maps to improve ratings and player enjoyment. The way I see it, there's just no place for hard maps in these games, and I think it would be cool if there were. I believe that this way everybody benefits, players who want to play easier maps get categories they can stay in, and know that they are capable to beat every map in the category (completionists happy). Players who want harder maps can switch to expert and get a roster of difficult maps to sink their teeth in and see which ones are the best based on other experts who have beaten them and rated based on how much fun they had.

I have some regret that I posted both ideas under one thread because I hope Part 1 won't be immediately dismissed based on Part 2, as I think it can still work without Part 2, assuming that players don't jump into Expert expecting they can beat the maps, then rate down everything out of frustration. After all, if every map in Expert is rated down, that means every map in Expert is also on a level playing field as far as ratings are concerned, as long as you don't compare between different categories.

[chwooly]

I choose not to name the map since I didn't want to single out 1 map maker.

And somehow you missed the part about me effectively beating this map. Just not choosing to finish it.  This map was not particularly difficult I made a choice not to finish it as it was not a fun map to play. Nor did I want to invest any more time into something I was not enjoying. 

So yes I should be able to rate it.

I play these game for personal enjoyment not to stroke someones ego. I would not mind if the ratings system went away completely. Let people post on the forum if they have an opinion. 

If the map is enjoyable I say so and my ratings reflect that. If it is not enjoyable I am not going to finish it just to rate it.  Then the ratings get skewed the other way.


I am not disagreeing with having difficulty levels, I think that is a great idea. I just think that only letting those who beat a map rate it is not the best way to do ratings and limits the true value of any rating system.

If you never play maps out of your skill set how do you improve said skill set? As to micromanagement, Personally I hate that you can pause the game. Yes I use it at times, But most of the time I play without pausing or restarting and subsequently my scores suffer, I have even recommended that there should be 2 timers 1 for pausing and one for total time.

Bottom line none of this really matters since they are just imaginary ratings from people we will most likely never meet and will not have any impact on our day to day lives. So enjoy your fathers day.

Cheers

[Karsten75]

However, if this idea were in the game, this map would probably be in the Expert section, which by definition would only contain maps that require lots of micromanagement, as difficulty in these games usually comes down to getting the best start possible.

In general, I don't rate maps, but I think your assumption here is a fallacy. Many times I don't finish a map because it's (as an example) a 256x165 map, with no upgrades, 50 emitter5s and is a multi-hour slog. It's not an expert map, it's a map where, after I've killed 5 emitters, I can win, but I don't want to sit around going after the other 45. emitters on the map.

Players should have the right to rate a map like this low.

Insta-win maps, why should I give the mapmaker the satisfaction of a completion score to rate the map as terrible?

[Builder17]

For example, let's say I were to create a map that can only be beaten by performing a ludicrously specific set of steps. Then I tell a few friends how to beat it knowing they will also rate the map 10.

A map like that will gain some infamy. We can read the scripts, we can figure it out, beat your map and rate it down.

Makes me think of vanilla puzzle map that is very hard.

If that requires using N key (Step to step) duration of whole map to be winnable, I hope your friends have enough patience to win it.

In other words, it wouldn't work either. :)
[/quote]

[TDplay]

On terms of rating, I think we need to change the 0-10 rating for a different one. For example (mock-up)

As you can see, there are 4 sections for rating:
Difficulty: Basically, if the map was really difficult, vote "Super Hard" or "Expert". If it was easy, vote "Easy". All the subjective opinions should merge into a general objective idea of its difficulty.
Enjoyability: Kinda like our current voting system - Amazing is like 10, Boring is like 0.
Performance: How well the level ran. Good for saying to the map maker, "you got too many crpls in here boss"
"Slog": This is separate from Enjoyability because of its marmite appeal - some enjoy slog, others despise it.

These would, of course, have their own filter options, so a player can look for, say, amazing expert maps with a short slog. The map dev should also be able to vote in Difficulty, Performance and "Slog", so the map players can make an informed decision when first choosing to stop people saying "too easy - score 0" or "too hard - score 0" or "too much slog - score 0" etc. They would not vote in Enjoyability for obvious reasons.

[creeper destroyer]

On terms of rating, I think we need to change the 0-10 rating for a different one. For example (mock-up)

As you can see, there are 4 sections for rating:
Difficulty: Basically, if the map was really difficult, vote "Super Hard" or "Expert". If it was easy, vote "Easy". All the subjective opinions should merge into a general objective idea of its difficulty.
Enjoyability: Kinda like our current voting system - Amazing is like 10, Boring is like 0.
Performance: How well the level ran. Good for saying to the map maker, "you got too many crpls in here boss"
"Slog": This is separate from Enjoyability because of its marmite appeal - some enjoy slog, others despise it.

These would, of course, have their own filter options, so a player can look for, say, amazing expert maps with a short slog. The map dev should also be able to vote in Difficulty, Performance and "Slog", so the map players can make an informed decision when first choosing to stop people saying "too easy - score 0" or "too hard - score 0" or "too much slog - score 0" etc. They would not vote in Enjoyability for obvious reasons.

I like TD's idea.

Me, I would be doing the easy-to-hard maps. I would try low-end S. hards, anything else i would avoid. there's probably a lot of people who feel the same way. but some people are worse than me, or better than me, or they're gods and ONLY play expert, but everyone's different.

Of course, everyone wants the best they can get, but there's certain people who will or won't like certain things, so they will rate the enjoyability accordingly. this could lower their ratings, but there will be more positives than negatives on a good map.

This is what I need. I don't think i will be able to play CW4's big or complex maps because of my computer, so i will only be able to play the "smooth" ones. people that have bad*** computers might rate the ones that are otherwise laggy smooth, and vice versa, but it should even out. I, personally, need this.

I hate slog, although slog is what make many maps what they are. I can bear it for a bit, but if it gets too long, it may cause me to get bored and lower my rating, or completely quit the map. other people love slog, and will look for maps accordingly.

Even if we get the best rating system possible, there will still be people who hate on the maps, because they hate the map maker, or they're (amazingly) unaware of the rating system, or simply because they hate their life and they want to make everyone miserable. i hate that last kind of people, but what can you do about them?

[Grabz]

I believe in this scenario K.I.S.S. is pretty applicable. Your rating suggestion IMO is vastly overcomplicating things, you create three more tick boxes for the player to worry about and click at the end of each map, you need four filter settings dedicated to rating rather than one, and it would be much less obvious for an average player how to find good maps with filters. I do not trust players that they will be able to answer truthfully to any of those things. Things like a map being a slog or not can usually be figured out based on the average time spent in mission. Performance is something the game is supposed to always do well, unless there's bad scripts involved, but then in the original system the map will just get a bad rating and you won't have to see it anyway.

Personally I find the rating system perfectly fine for what it does. I see it like rating a movie - you finish watching a movie then you rate it based on whatever criteria you have. That's why I find that it should be required for you to beat the map - and I say that from a player perspective, not just a mapmaker one. I believe that if there's a map I really can't beat because the difficulty is not hard but unfair, I should still have to take the time beating it to give it a low rating, because you never know how it might actually turn out, or whether or not you're maybe missing something. To me it's actually a win-win, it's not just about keeping the players with an agenda against you from rating your maps without playing them, but it's also that seeing something through is a fundamental step to leaving a review. However, plenty of people have already shown disdain to how I feel about this which is fine. I'm just doing my best to keep it as simple as possible, with as much potential positive impact as possible - as suggestions should be.

If a map is actually just broken, or laggy beyond belief, it should be reported, not rated. I don't think it's good for the game that maps like this stay in the custom map browser - going back to Part 1 of my suggestion where I mentioned completionist players wanting to beat everything - well you mess it all up if you let a map that's practically unbeatable stay in something like Colonial Space - what eyes don't see, heart doesn't desire.

[Alter Old]

Players must have finished the map to be able to rate.

I would left the assessment available to all, but the 'more' or 'advanced' button displays a separate rating from those who finished map, and from those who did not finish. In this case, there are general information about map and specific (in the light of score table).