Map ratings.

[Karsten75]

This thread got me thinking.

Currently there are only 15 maps rated over 8.5, there is only one map rated over 9 (Arby;s, and I don't expect it to survive the day with that rating). and a whole bunch of maps rated over 8. Interesting where the break is. Unfortunately I can't do the same filtering for bottom-rated maps.

[planetfall]

Has your stance changed?

[Karsten75]

Has your stance changed?

I don't think so. I am not encouraging a vote-off or selection for a specific author, I'm looking at maps overall.

I was highlighting the harshness of map ratings, more than anything. I didn't call out any author with the worst map, and even if I could look at the lowest scoring maps, I'd do it as an aggregate (Eg. in contrast to 15 top-scoring maps, there are x number of poorly rated maps.)

If you see a specific problem in what I did, please PM me.

[planetfall]

No, no. Just a curiosity.

It's possible to sort by ratings and reverse the list for a similar effect, but you have to do some combing yourself to come up with the exact numbers. The fact that you can't filter to only low-scoring maps is probably for the best, as it makes it slightly more difficult to make "worst of" compilations in places the KC community has little to no influence (YT, blogs etc.)

Now, the ability to search for maps that *I've* rated 1 would be helpful for this discussion - but not much else.

Why the ratings are lower than one would expect is an altogether more interesting topic. For anyone who wants to confirm that the ratings are especially harsh, compare the results of this program to the stats of CS:

Code Select Expand


import random

rankings = {
10.0: 0,
9.5: 0,
9.0: 0,
8.5: 0,
8.0: 0,
7.5: 0,
7.0: 0,
6.0: 0,
5.0: 0,
4.0: 0,
3.0: 0,
2.0: 0,
1.0: 0,
}

for i in range(5000):
x = random.triangular(1.0, 10.0)
for j in rankings:
if x >= j:
rankings[j] += 1

for j in sorted(rankings.keys())[::-1]:
print("greater than " + str(j) + ":" + str(rankings[j]))


Of course, this code is making some assumptions. The distribution wouldn't be triangular even if we came up with some hypothetical objective algorithm that evaluates map quality. Experts are few, charlatans many; and it takes much, much longer to make a good map than a bad one. We also know of one bias that the flawed humans rating these maps hold: quite frequently, well-made but difficult maps will be rated 1 by less-skilled players out of frustration. Players who have stuck with the game long enough to play at a high level, on the other hand, are not likely to rate down well-made easier maps out of boredom.

Yet another "problem" is the policy for comments on the forum. "If you can't say something nice, don't say anything at all." If we were allowed to light into every mapmaker who posts a squiggly first-try map where there's digitalis growth area but none of it is connected to an emitter, they would have reason to either get better or go away. I am not saying that we should become a cesspit like the rest of the gamer-sphere (excuse me, I threw up in my mouth just typing that) but that an influx of bad maps are the price we pay for civility - and low ratings manifest what people can't/won't say on the forums (and the opinions of people who don't know the forums exist, or haven't joined because they want a place they can be angry all the time.)

There's also the fact that you can't make a map that appeals to all players. I can barely stand to play PAC maps, but I don't reflexively rate them 1. I imagine that a number of others are less scrupulous in this regard, or even not wholly aware of how their preferences may skew assessments of quality. Are we trying to direct players to the best maps, or just have a popularity contest?

I use PAC as an example because I'm pretty sure V had no idea it would exist, or be so popular, and then be sold on the shelf (so to speak) right next to more conventional maps as though they're the same thing. Make-a-completely-new-game-with-CRPL can be hugely hit-and-miss among the normal-CW3-maybe-some-scripted-enemies crowd. I don't think there will be much agreement to disagree when looking at these disparate genres as just "CW3 maps." It's sort of akin to trying to settle a dispute between a fan of The Name of the Rose and a fan of Transformers by saying "Just recognize that you have different taste in movies." To each of them, the other is fundamentally missing the point of what makes a good film.

Spoiler

The metaphor breaks down because while I can acknowledge that there is a merit to PAC that I simply don't personally enjoy, one of those movie fans is wrong.

[close]

[GoodMorning]

Interesting you should mention this. I thought the distribution looked like the top of this table. The graduations on the lower part aren't much use.
https://xkcd.com/1098/

If you look at Kongregate (as an example) then 4.4/5 is about as good as it gets. The best of CS sits at about 9/10. From what I've seen, there is a fairly reliable rating distribution from any community.

However, the general tone of existing ratings has an effect, because they are given by humans (probably). This means that we look upon well-reviewed things more favourably from the outset, and vice-versa.

Edit: I wonder if, from V's perspective, it would help to list maps in order of rating by default.

[GameGibu]

See, I had this issue most of the time: I put a lot of effort into making a nice-looking map, and putting reasonable amounts of scripting in, and at least until I got whisked away by other games for a year, my maps nearly always went like this in colonial space: