The Neditor Nation Discussion Thread

[blackson]

1) You'd see a lot less progress that way. Everyone may be making it look easy right now, but those AGD's will come in handy against the later Gyms, which on average have higher scores than the earlier ones. That's how I sorted them. The AGD's prevent a total roadblock of the competition, which would be likely at some point because of the level of skill that the Leaders have.

I'm half and half here. I thought it was fun when all of my friends were stuck on the same gym leader playing the games, racing to see who would be the first to beat it. Although, I can see frustrated mappers who wouldn't find this being "fun".

2) Reagreed.

*resists temptation*

3) Again, that really can't be helped. The Judge scoring your map is pretty much luck-of-the-draw. You may end up with someone who likes your map, you may not. My point is that the opportunity is there, and besides, you're assuming the worst. What if you're not horribly unlucky with the Judges? Then you'd be doing pretty well. Differences in Judge's opinions can work for you as well as against you. Case in point...

I just don't like the disadvantage. Not really a way to fix that, so I'll stop fueling that one.

...4) You're right, this Judge didn't have as high opinion of Templex's map as the other Judges. He also liked that map a good bit. If you honestly don't think he should've advanced (and apparently the Judge disagrees; again, bound to happen), do you think he can keep finding the low-scoring Judge that likes his map, just by chance, to keep advancing? If he's not good enough, he'll eventually get held up, just as a good mapper will eventually make it through despite hold-ups.

Same as the last one, -the disadvantage.

[TribulatioN]

You're right, this Judge didn't have as high opinion of Templex's map as the other Judges. He also liked that map a good bit. If you honestly don't think he should've advanced (and apparently the Judge disagrees; again, bound to happen), do you think he can keep finding the low-scoring Judge that likes his map, just by chance, to keep advancing? If he's not good enough, he'll eventually get held up, just as a good mapper will eventually make it through despite hold-ups.

That's probably the most important aspect of this competition, because of a way the system works, someone might be stuck on the first gym for quite a while, but then make a comeback as other's will be stuck further in.

The only flaw I see right now that I haven't seen mentioned, is bias in the judges. The way they get to pick who's map the judge, they can always make someone win. Like for instance, it's the beginning of judging for that week, and a judge comes along, and waits till other judges pick maps to judge, then they go and take the one that they want to give a win too.
Unless there is some secret random picking among the judges that I don't know of, then I think that should be looked at.

I'd like to think that the Judges wouldn't do that, seeing as they have no incentive for doing so. From what I can tell, they seem to be taking the earliest-posted challenge not yet scored whenever they chance across the forums. Random assortment would be ideal, but then we might have to wait until the end of a week to assign maps to Judge, which would suck for the Judges on busy weeks like this one. The Judges don't give you any reason to suspect or anticipate corruption, either, so why not cut them a little slack? I'm pretty darn sure they won't choke us with it. ;)

I take your word on this one.
If it's first come first serve, I'm perfectly fine with that. In fact, I was blind not to notice that might be a better option.

[mintnut]

Would it be feasible for the judges not to be able to judge an author for two consecutive gyms? That way, even if there was bias, a judge could only help someone half way through the competition.

x

[Erik-Player]

I had an idea. I'm sorry if this has already been discussed.

I had an idea that one judge could possibly just judge one gym?

[notsteve]

that would work, because so many people challenge the same gym every week

but the idea that the same judge couldn't judge more than like once or maybe one every three submissions is a really good one

[Brainwasher]

I say, "Go, grumps, be a griping grinch." ...(means let complainers complain, everything has flaws, and they will grow out of it ^-_-^)
Anyway, the randoms of picking who ever is earliest seems to be working.

Just like in pokemon games, there are crests and troughs in the gyms. While your map style may be bad for relaxed floorguard maps, you may be good at rockets, or lazers, or chainguns and E (this excludes if you are just plain BAD ;P).

And in the pkmn games, you really did have an advantage over the leaders (AGD bonus); you could raise your levels, but they couldn't. Also, the leg up prevents this from becoming a center for genious mappers only.

[bufar]

You're right, this Judge didn't have as high opinion of Templex's map as the other Judges. He also liked that map a good bit. If you honestly don't think he should've advanced (and apparently the Judge disagrees; again, bound to happen), do you think he can keep finding the low-scoring Judge that likes his map, just by chance, to keep advancing? If he's not good enough, he'll eventually get held up, just as a good mapper will eventually make it through despite hold-ups.

That's probably the most important aspect of this competition, because of a way the system works, someone might be stuck on the first gym for quite a while, but then make a comeback as other's will be stuck further in.

The only flaw I see right now that I haven't seen mentioned, is bias in the judges. The way they get to pick who's map the judge, they can always make someone win. Like for instance, it's the beginning of judging for that week, and a judge comes along, and waits till other judges pick maps to judge, then they go and take the one that they want to give a win too.
Unless there is some secret random picking among the judges that I don't know of, then I think that should be looked at.

I'd like to think that the Judges wouldn't do that, seeing as they have no incentive for doing so. From what I can tell, they seem to be taking the earliest-posted challenge not yet scored whenever they chance across the forums. Random assortment would be ideal, but then we might have to wait until the end of a week to assign maps to Judge, which would suck for the Judges on busy weeks like this one. The Judges don't give you any reason to suspect or anticipate corruption, either, so why not cut them a little slack? I'm pretty darn sure they won't choke us with it. ;)

It wouldn't suck for the Judges if it was done correctly. Monday through Wednesday we submit maps. Thursday the maps are randomly assigned and judging begins. Scores are reported to Pheidippides on Saturday. He updates the spreadsheet and PMs us by Sunday. Repeat.

[lord_day]

I only have one complaint to do with judging so far. I know it's not good sportsmanship to raise a point like this, but I have to.
I've played all the maps that have passed Templex (as far as I know) and almost none of them seemed to be 'relaxed'. Should we just ignore the relaxed aspect of our first maps, or is it part of the judging score criteria? Either way. I think we should get some definition on this!

[otters~1]

I only have one complaint to do with judging so far. I know it's not good sportsmanship to raise a point like this, but I have to.
I've played all the maps that have passed Templex (as far as I know) and almost none of them seemed to be 'relaxed'. Should we just ignore the relaxed aspect of our first maps, or is it part of the judging score criteria? Either way. I think we should get some definition on this!

I agree! I've been trying to judge relaxation a little under the gameplay section, but it simply doesn't fit in very well. It may be too late, but making a map to the requirements should factor into the judging.

[Pheidippides]

I only have one complaint to do with judging so far. I know it's not good sportsmanship to raise a point like this, but I have to.
I've played all the maps that have passed Templex (as far as I know) and almost none of them seemed to be 'relaxed'. Should we just ignore the relaxed aspect of our first maps, or is it part of the judging score criteria? Either way. I think we should get some definition on this!

I agree! I've been trying to judge relaxation a little under the gameplay section, but it simply doesn't fit in very well. It may be too late, but making a map to the requirements should factor into the judging.

You're not the only one. Other Judges have been deducting from Gameplay if a map is too hectic.

[otters~1]

Good, good. I just think it should have been made an official part of the scoring at the beginning. But hey, you can't think of everything, and this really is running very well, assuming we get more judges going soon.

[yappydog]

Just to add my two-pence-worth to the debate, I recall suggesting that Templex's "relax" rule was broken by... Templex! Most of the level's calm, but the initial move is distinctly non-laid-back. As rejecting a Leader's map would, frankly, look silly, a score edit seemed most appropriate. I assumed other judges were using the same technique for challengers.

[Nexx]

And in the pkmn games, you really did have an advantage over the leaders (AGD bonus); you could raise your levels, but they couldn't.

That's a good point.

[isaacx]

And in the pkmn games, you really did have an advantage over the leaders (AGD bonus); you could raise your levels, but they couldn't.

That's a good point.

That is a good point, although this does restrict the advantage

i.e, you can only advance 3 levels above what you have instead of leveling up to lv.52 before battling the first gym (which i do most of the time :P)

[stonedeagle]

And in the pkmn games, you really did have an advantage over the leaders (AGD bonus); you could raise your levels, but they couldn't.

That's a good point.

That is a good point, although this does restrict the advantage

i.e, you can only advance 3 levels above what you have instead of leveling up to lv.52 before battling the first gym (which i do most of the time :P)

I think what we have going now fits that. Like in pokemon: we try right when we get there, when we fail we can make a newer better level, but they can't. They are stuck with what they have. That is alot like leveling, simply getting better at mapping.

EDIT: I would like to see a thread specifically for scrimmages at some point, because those will probably become much more popular as the game goes on.

[Zivilyn]

It wouldn't suck for the Judges if it was done correctly. Monday through Wednesday we submit maps. Thursday the maps are randomly assigned and judging begins. Scores are reported to Pheidippides on Saturday. He updates the spreadsheet and PMs us by Sunday. Repeat.

I did think that was how it was planned. Judging doesn't start at all until the submission window for that week is closed, and then it can be randomly assigned. That throws a few kinks in the current free-for-all mechanism though, so here's my suggestion:

Monday - Wednesday: New challenges can be submitted. Nothing happens in the Gym thread but submissions. During this time, judges PM Pheid with availability for the week, a maximum number of maps they feel they can judge that week, etc.

Thursday: Pheid responds to the judges with PMs telling them which map or maps they are judging after submissions are closed.

Thursday - Saturday/Sunday: Judges do their job, and PM Pheid with the results, which include some feedback about the map, why they decided it was better/not better than the Gym map, by what margin it won/lost, and how much any demos helped, if any were part of the submission.

Sunday: Pheid sends out the results to members. Who judged who's map remains totally anonymous, the Gym topic becomes like ten times cleaner, and challengers get a good idea of their score without being able to complain about judge procedures.

This solves a large number of problems all at once. However, for the kind of optimum judging that blackson has in mind, and quite frankly I agree a better system is needed, each map would have to have three judges assigned to it (still random, of course). They wouldn't know who else was assigned the map, but it'll all be random by Pheid's hand anyway. Keep in mind that numerical values don't really matter in this contest- the judge decides if it's better or worse than the gym map. I don't know what system you're using now, what with initial judging of the Gym maps and then checking to see if the score is higher, but in the end it is really just a yes or no decision on the judge's part. Numbers shouldn't ever even need to enter into it for scoring. The judges do their business, drop a PM to Pheid, and then Pheid has three pass/fail results for a single map. So the result system would be that at least two out of the three judges give a Pass and the map clears the gym. Then the challenger can get an even better idea of their score and how much to improve, when they are told 0/3, or 1/3, or even 3/3 what a badass. It would be harder to point fingers at members who got lucky or have any legitimate complaints for that matter when such a system is used.

Ignoring the extra judge idea, this is not a huge change and it wouldn't hurt to try it out for a week. All it really is is keeping to the schedule a little more strictly in order to achieve random judge distribution. Hell, if by Thursday there aren't too many challenges and the judges PM you with enough availability you could even try out the two out of three system.

[Pheidippides]

You present a lot of good ideas, Zivilyn. Although, I don't like the pass/fail idea. It's a lot easier to decide something like that when you break it down into smaller categories than an overall yes/no. So I don't think we'd implement the pass/fail, but if the whole process becomes streamlined enough we may be able to have multiple Judges.

I also have an idea to prevent things from backing up by waiting until Thursday. It also solves the problem that arises when a Judge is available early in the week but isn't later. I propose that in this system, I should assign maps after every day of submissions. For example, all the early Judges are assigned maps from Monday, others are assigned maps from Tuesday, and others are assigned maps from Wednesday. The assignments would go out based on availability so that the Judges get the maps at the best time for them.

Basically, here's what I see happening. We'll play out this week as we've done the other weeks. Then, we take a two-week break for the holidays, during which we can completely retool the Gym challenge process. That way, there's not as much pressure to rush a solution before the start of a new week. We have some good ideas floating around here, and we can use the break to discuss them and pick the best new model for challenges.

[Zivilyn]

So long as results aren't given until Sunday and judges aren't posting in the gym topic, it shouldn't matter when the judging actually takes place. Having a solid window, on the other hand, would allow early submitters to make last minute changes before judging starts. Like, lets say I submit a map on Monday, first thing, and then on Wednesday morning I notice something is wrong with it or I made one since and I feel it's better. It's not too late, since judging hasn't started, so I just edit my post. Small example, and people should be sure before they submit, blah blah blah, but it's a reason to have a solid window. There are others.

The pass/fail thing isn't so much an idea as it is a statement of fact. It is a pass/fail system, you either beat the gym or you don't, pass fail. I know that judging isn't as simple as that and there are different categories that are each considered on their own, but in the end, the map as a whole is either a pass or a fail.

Like I said, I don't know what numerical system you have in place for judging criteria, but from the challenger's point of view, it really is just whichever judge picked up your map deciding if it's better or worse than the Gym. So however you want to word it, the three results come in, each one individually would be a win or lose to the challenger. And rather than taking the average of the three scores, you do the 2/3 thing for a win. That prevents a situation where two judges were in general agreement that yeah, it was better by however much, and the third thinks it was total crap and gave it a flat zero. The map should pass, since the majority agreed, but just taking an average of the scores at that point could easily put it under. Using a numerical system to decide the final result makes having a panel kind of useless when one individual judge still has the power to make the map fail.

Since we're just brainstorming at this point, could you tell us how the current judging procedure works? And how the Gyms were originally judged and how the comparison occurs. Then I could give some actual examples of the 3-panel idea working with the current judging system, and explain my ideas a little better.

[otters~1]

I agree with notkitt from a few pages ago--the judges should have been handpicked. Either judges who have been around a while, or those who have consistently made quality maps. Too late now though.

Anyway, as to the current argument, I think the system is working just fine. Those who get stuck on a map, well, deal with it. In the end, if you make a map that is truly great, that map will get recognized.

Also, I think the AGD bonus is completely necessary, because, frankly, some of the gym maps should not be beaten. They're just too good.

[Zivilyn]

I agree with notkitt from a few pages ago--the judges should have been handpicked. Either judges who have been around a while, or those who have consistently made quality maps. Too late now though.

There is no way hand-picking of judges could have occured. What if who you hand-pick from this judge-worthy tier of members wants to be a challenger? Would you have sent out invites to be a judge in your competition, instead of asking for volunteers with some experience like Pheid did? You don't need to be great at making maps to know a great map when you see one. In fact I'd say the less time you've been tied to this community the better you probably are at judging them from a totally egalitarian standpoint.

if you make a map that is truly great, that map will get recognized.

hahahahaha

And while we're on that topic, here's another idea, but this one is pretty radical. Anonymous map submission. Challengers send their map data to Pheid, he sends the judges their maps for the week, no title or author or description other than the gym it is challenging, they judge, and send back a score. Nothing becomes public until after the map is scored and you are made aware of your win/loss. Then you can submit it to numa and post it in the thread or what have you. Or Pheid can just post in the challenge thread at the end of the week with who challenged what and who won and who lost. Seriously though, challenger anonimity is the way to go if you're looking for total fairness from the judges.

[isaacx]

I challenge someone to a scrimmage.

PM me with the other details if you choose to accept

My rule is (if you accept) you're map can only use 2 kinds of enemy's (mines don't count)

Already have challenge

[spudzalot]

I challenge someone to a scrimmage.

PM me with the other details if you choose to accept

My rule is (if you accept) you're map can only use 2 kinds of enemy's (mines don't count)

Are scrimmages supposed to be here...?

[Hoohah2x2sday]

i thought scrimmages went in the challenge thread

but i could very well be wrong

[Erik-Player]

I challenge someone to a scrimmage.

PM me with the other details if you choose to accept

My rule is (if you accept) you're map can only use 2 kinds of enemy's (mines don't count)

Are scrimmages supposed to be here...?

You work things out via pm. You should read the rules about it.

[Nexx]

Anonymous map submission. Challengers send their map data to Pheid, he sends the judges their maps for the week, no title or author or description other than the gym it is challenging, they judge, and send back a score. Nothing becomes public until after the map is scored and you are made aware of your win/loss. Then you can submit it to numa and post it in the thread or what have you. Or Pheid can just post in the challenge thread at the end of the week with who challenged what and who won and who lost. Seriously though, challenger anonimity is the way to go if you're looking for total fairness from the judges.

That could work. You needn't have people hold off putting their maps up on NUMA though. Just trust the judges not to go snooping.

Also, a scrimmage thread would be nice, for those who don't want to scrimmage a particular person. They would just post in the thread giving their rule and someone else agrees (via post or PM) from there on out. I suppose it might get a bit messy though. Perhaps have a policy that only people looking for scrimmages can post, and if someone's responding to a post, they just send a PM. How's that sound?