Change to Review System

[aids]

I sort of have a more radical idea of just keeping all the bullshit with the current review system and just adding a completely new system on top of it, called "map of the week", where a map of extremely high quality is showcased and placed just under the map reviewed for that day on the homepage. Reviews would most definitely need to be proofread, and map quality at least somewhat validated by admins or other reviewers. Satisfies people who want really high quality maps reviewed, and satisfies people who want to see a cool new map every day.

Edit: Inspired mentioned something that actually really bothers me: a lot of the maps that are featured seem like they were only featured because the reviewer and author are friends, and that absolutely needs to stop. In years past that was very rarely a problem, and I think it correlates well with the drop in feature quality.

Don't expect any changes to the site. Arachnid left us, and I don't think even M&R will be able to bring him back. That's why we're gonna be stuck with the current system and we'll have to make the most with what we have.

[Scrivener]

There should be a time period after submission during which a map cannot be reviewed. I suggest two months.

There should definitely be a peer review by at least one other person, because incorrect English makes a review less enjoyable to me and probably a lot of others. For example, I think zoasBE has some great ideas, but his inexperience with the language is distracting while reading his reviews.

That's my two cents until I think of more.

[TommyWiseau24]

I still am of the opinion that featured maps are going to get stale if we aim just for "great maps".

The sentiment of this makes me think you're confusing "great maps" with "the same types of maps by good authors". /Every/ map featured should be utterly great to at least the reviewer, I'm tired of this "I want to highlight an innovative but completely underdeveloped mechanic" bullshit. If you like the mechanic but the map is garbage, tell them to work harder and polish it and make a real map out of it before reviewing it. See ManBearPig's latest featured map and compare it to Lord Day's before it; both cool concepts, but obviously a lot more effort went into Lord Day's and it shows.

The latter, sorta what I meant. I want to see maps that are not necessarily great, but still notable; there's lots of flawed maps that are still really quite good, not to mention more creative than the more refined types (hell, imo flaws are sometimes pretty intrinsically interesting).

By creative I mean real standouts; ie. Pheidi's maps are visually creative, but imo they generally fall into the category of "pretty map" without pushing its boundaries too much. I want more true experimentation.

[Pizzles, M.D.]

By creative I mean real standouts; ie. Pheidi's maps are visually creative, but imo they generally fall into the category of "pretty map" without pushing its boundaries too much. I want more true experimentation.

Fair enough but that's why you're a reviewer - not everyone else wants that. Of course, I and I'm sure a lot of others would enjoy incredibly innovative maps, but it's silly to try and set that as the standard for the sake of a few people's opinion.

On another note, an incredibly innovative concept isn't enough to make a map a "real standout", so to speak. One might argue that a flawed map is more interesting because it breaks the norm, but if the flaw is that it is so unrefined that it isn't enjoyable, then the map simply isn't developed enough to be featured. I'm not saying all featured maps need to be flawless, but they should be flawless at /something/. On the subject of concept maps, take a look at MidnightGetaway; his maps are certainly flawed in some aspects, but he is flawless at developing and integrating certain concepts because his maps are still well-rounded and fun instead of awkward and gimmicky.

[Godless]

I'm in the same boat as A_P (whee!). I'm never on the forums, so this is an issue that I think is important.

My thoughts: Proofreaders should be a must, at this point. It's clearly called for - not because people are fuckin' up left and right or anything, just that it should have been part of the system to begin with. Proofreading would be logistically complex within the current system, obviously, however I'm sure it wouldn't be too hard to have an admin take applications from community members to simply proofread reviews.

As for the problem at large: How difficult would it be to have all of the reviewers, instead of submitting their own reviews on their given days, simply review when they have the time and put their reviews in a communal place - to be posted by someone reliable enough not to skip a day (or two days, or a week or whichever). It seems like this would solve the problem of skipped days, while also creating time for both the reviewer and potential proofreaders to reflect on the worthiness of both the review and subject map. The only problem I foresee in this is overexposure of the review prior to it going public. I imagine, though, that that could be sorted out by the proper bureaucratic measures.

Cheers.

[aids]

As for the problem at large: How difficult would it be to have all of the reviewers, instead of submitting their own reviews on their given days, simply review when they have the time and put their reviews in a communal place - to be posted by someone reliable enough not to skip a day (or two days, or a week or whichever). It seems like this would solve the problem of skipped days, while also creating time for both the reviewer and potential proofreaders to reflect on the worthiness of both the review and subject map. The only problem I foresee in this is overexposure of the review prior to it going public. I imagine, though, that that could be sorted out by the proper bureaucratic measures.

Cheers.

That was the old system but it actually broke on NUMA.

[otters~1]

This thread has nothing to do with map choice at all. It has to do with quality of reviews, quality of reviewers, duration of reviews, and basically nothing else. Map choice is an opinion, and we're not going to curb opinions. Sorry. (The one exception is the *possibility* of a time period after submission during which maps cannot be featured -- but from my read through, that doesn't seem to be anywhere near unanimously popular, so don't hold your breath.) Anyone who isn't talking about the things that are up for discussion is basically just wasting space.

Now.

Two things. If you want to become a reviewer, posting here does absolutely nothing. Me, gloomp, Aidiera, whoever -- we're not going to reach out to you, because that's ridiculous. PM me sample reviews, or at the very least PM me something. "I'm interested." Etc.

Second, one thing that's cropped up a lot that I'm liking is mandatory proofreading. This is easy to execute in theory, tough to maintain in practice. We'd basically go by the honor system. The best way I can think of is to post on the feature's comments, immediately after the review goes up, "Proofread by <insert a different reviewer>." I like this and don't see any reason not to start doing it. If you forget to write your review until the last moment, and then there's no one on IRC to read it, submit it and hope for the best, but that shouldn't happen too often.

Anything I'm missing?

[Nexx]

Second, one thing that's cropped up a lot that I'm liking is mandatory proofreading. This is easy to execute in theory, tough to maintain in practice. We'd basically go by the honor system. The best way I can think of is to post on the feature's comments, immediately after the review goes up, "Proofread by <insert a different reviewer>." I like this and don't see any reason not to start doing it. If you forget to write your review until the last moment, and then there's no one on IRC to read it, submit it and hope for the best, but that shouldn't happen too often.

I'm not a reviewer, but this sounds reasonable.

In any case, I just wanted to chime in to say that Sunset's original idea doesn't seem anywhere near good enough to warrant that kind of radical change. I don't know, I see that the point is to try to get people to focus on their reviews a bit more, but making the "review-posting period" a bigger deal so that reviewers "have" to pay attention just seems like it'll end in the same problem as now but amplified.

[Pizzles, M.D.]

Mandatory proofreading is a good idea, but I don't think people will get in the habit of writing their reviews *not* the day of and having time to have them proofread unless there is a one day break between each review. No harm in trying it out though.

Also I think map choice is at least worth talking about, can we open up a discussion somewhere else?

[TommyWiseau24]

I'm against mandatory proof-reading, honestly. I can't see it making reviews generally less mediocre without major overhaul.

[mercuri]

maybe i am stupid, but why a "map of the month" or "of the week" won't work?
reviewing maps of people that vanished years ago isn't really a good thing

[ska]

maybe i am stupid, but why a "map of the month" or "of the week" won't work?

This http://www.nmaps.net/featured effectively displays what you mean. I don't think highlighting one specific map would be worthwhile since most people who frequent numa do so much more than once a month. We have The Dronies for those kinds of things.

reviewing maps of people that vanished years ago isn't really a good thing

This is not a wise argument. Spoiler: Leonardo da Vinci is dead and we still queue up to see the Mona Lisa.
There are many great old maps that fell through the cracks for one reason or another, and featuring said maps might be the only way to give them the attention they deserve. *Prepares to apply to get his old job back*

[Godless]

Ska is blatantly correct on both counts. Though to be honest, I wasn't terribly impressed with the Mona Lisa - it's darker than it looks on a screen, and tiny as hell. So whatever the opposite of penis-envy would be was pretty much my reaction.

@Tommy
In what possible sense could proofreading not be a good thing? Worst-case scenario: nothing about the review changes. Best-case: potentially embarrassing errors are corrected. Seems legit to me.

@Pizzles
I agree. However, if we go with skipping days or queueing reviews in a manner similar to what I suggested, then it shouldn't be a problem. And it seems like a time-gap of some sort is pretty unanimously considered to be a good move.

@▲▪►▪▼◄▲
How did that break NUMA? I didn't know about that. I was thinking less about a change to the site and more about a communal paste-bin mechanism thing that the reviewers could just dump their reviews whenever they were so inclined; to then be posted at the correct time by someone with the means to do so.
----

EDIT:

Mercuri: btw these words are useless because 90% all this will vanishes when n2 come out, peace

:|

[mercuri]

no ska, i expressed myself wrong as always, what i meant was a featured map should be one submitted in the last month.
like, each 5, 15 and 25 of a month you feature a map that has been submitted in the last 2 months, this way you get new mappers to do the best.
you know, when a good mapper is doing something good and he sees a feature from 2009, well...
i got what you mean, and it's right to feature also old maps.

btw these words are useless because 90% all this will vanishes when n2 come out, peace

[Corky Romano]

The last thing we need to do is mollycoddle new members, mercuri. The ones that stay are the ones that choose to on their own, and they are the only ones we acknowledge.

[ska]

no ska, i expressed myself wrong as always, what i meant was a featured map should be one submitted in the last month.
like, each 5, 15 and 25 of a month you feature a map that has been submitted in the last 2 months, this way you get new mappers to do the best.
you know, when a good mapper is doing something good and he sees a feature from 2009, well...
i got what you mean, and it's right to feature also old maps.

btw these words are useless because 90% all this will vanishes when n2 come out, peace

There is no point picking arbitrary dates to feature relatively new maps. If someone thinks a recent map ought to be featured, but forcing certain dates to contain recent maps is frankly pointless. Besides, I don't make maps purely in an attempt to get them featured. If a map is poignant enough, one day, even if it's years from now, will get the feature it deserves.

[mintnut]

Having been an active Reviewer for a long time, I have two words: impractical desires.

No matter how you structure or format the way Featuring works, the overall quality and consistency honestly still depends on the reviewers. A person can claim seven days in a month and only feature on three of them, but still be a good reviewer. Or a person can claim two days and feature on both of them but bell disliked by the community. I thought that if we could get some fresh faces on the Reviewer Squad then maybe we wouldn't be barraged with four consecutive days of Aidiera reviews as we have been in the past, and I got more replies than I expected. But for whatever reason, there seems to be an uproar about n00bs fucking up all the shit, abrupt-like. As an aside, I worked today and was asked to filter the fryers. I did it the way I was taught, but didn't know that there was a different way of doing it in the morning [because it seems that I was taught the midday filtering routine]. My coworkers could have gotten mad at me for giving them extra work, but they didn't. They laughed it off and didn't mind. Because I'm new, and they know that when you're new to something, you shouldn't be expected to be an expert. So what I'm getting at is that y'all need to be more welcoming to the beginner Reviewers. Should reviews only be in English? I dunno, I wrote one in Serbian to be different, and because I don't think that N should be an English-only game, but that's just my opinion. There shouldn't be any shame in having somebody proofread your review either. I had Sunset proof one of my reviews and it's going to be one of the best ones I've ever made. Not to point fingers, but some of you have been quick to insult zoasBE's English, but have done nothing to help him with it. The other thing that I take issue with is how people are once again playing the "quantity vs. quality" card at the same time they're playing the "reviews are meaningless" card. Once again, speaking for myself, but it seems that almost half of the maps I've reviewed went unnoticed. I don't do reviews as a favor to people and I don't review maps I made; sure, maybe not every review of mine is a gem, but to me, the maps are truly exemplary. And I try to review maps I think people will like, and that should be a Reviewer's #1 goal, to satisfy his audience. And the more varied the Reviewers are, the more often that will happen. eganic might be a huge thorn in my side now and then, but I bet he would be a great reviewer because he would review whatever he wanted, and it seems that that speaks for many people in the community. Perhaps I've force-fed my ideals to you guys for too long, but they just make so much sense to me.

Meow.

Can't help but feel that some of this is aimed at me, but forgive me if I'm just being self-important.

Sure we should be willing to forgive people mistakes when they start out, but not forever. Zoas was getting called up on his grammar from day one more or less, which was months ago. People offered to proof-read his reviews pretty quickly, but I dunno what happened to that? This doesn't just apply to Zoas obviously, he's just the most prominent example. I think people reviewing in their own language is a brilliant idea, not least because I'm fairly sure it was my idea. It would go a long way towards alleviating the issues with grammar and spelling.

We really ought to change the tag-line above the reviewers in the "who's who" though.

The other thing that I take issue with is how people are once again playing the "quantity vs. quality" card at the same time they're playing the "reviews are meaningless" card.

I understand your issue, but I never said that reviews are meaningless. I did say "Which leads us to the question of, why bother featuring at all?" That was rhetorical, I was trying to get at the disconnect between the way people approach the featured maps and their function.

[Godless]

I think featuring in native languages is a good idea as well. I'd say it would be prudent, however, to at least attempt an english translation of the review just for pure practicality's sake. English isn't the only language spoken by the community, true, but it is absolutely the most common by far and the review system ought to reflect that. Posting other-language reviews would be fun, but shouldn't come at the expense of the majority of members.

[Leonidas]

About the maps-must-be-old idea, that is pretty good since most people have seen more recent ones, and like someone stated, we need to dig for treasure. However the author of the featured map would probably not even be around to enjoy it... Generally I have featured maps recently that are 2012 or older.

Just brainstorming here.

[TommyWiseau24]

@Tommy
In what possible sense could proofreading not be a good thing? Worst-case scenario: nothing about the review changes. Best-case: potentially embarrassing errors are corrected. Seems legit to me.

It's not a good thing when it is pointless busywork. Good on the reviewers if their spelling is corrected, but the review will still be mediocre.

Not to mention, I've already mentioned why I like zoas's reviews, and proofreading would go against that (just my opinion though)

[mistaken72]

What if reviewers not only requested for features on certain dates but also for proofreading of others' features on other dates?

[Godless]

@Tommy
I don't think that's the case. It seems like spelling errors alone are a great annoyance to some/most people. And I see no reason why a proofreader couldn't also check the content of the review. I'm not seeing your argument.

[aids]

Pre-2009 or so, the reviewer queue was automated. Reviewers would submit a few reviews to the site, and the site would post them either when the reviewer asked or randomly if a date wasn't filled. I don't know the technical details, but somehow it broke.



"If you want quality reviews, hire quality reviewers" is the motto but those are few and far between these days. 20 current Featurers, most of whom take 3 days a month. How the shit is that math supposed to work out...

[zoasBE]

Not to mention, I've already mentioned why I like zoas's reviews, and proofreading would go against that (just my opinion though)

Thank you tommy for liking my reviews, I always try to put some 'spark', humor or at least be creative with them. But I totally understand as script says, they can lose strength by a poor grammar. I promise from now on, always get to IRC for a correct proofread, I just hope people do not bother with that. Thank you.

[Sunset]

"If you want quality reviews, hire quality reviewers" is the motto but those are few and far between these days. 20 current Featurers, most of whom take 3 days a month. How the shit is that math supposed to work out...

I'd go for more days if I could. But I'm almost always only free during Fridays and Saturdays.