Demo Data: TWO WARNINGS!!!

[999_Springs]

I'll try to keep this post short. I have been doing a lot of experimenting with demo data recently, and I have discovered that people can edit demos by cutting frames off the end of them which makes demos a few frames faster and look legitimate but actually are not. This could have disastrous consequences for all competitions in which submissions are done by posting demo data, and I want to highlight this problem.

The problem is that: suppose you take a demo and use demo editing techniques to take the last few frames off. Now you load it along with the map data in the box in userlevels, and play it. Now the demo plays as usual, and when the end-point is reached, something notable happens. The demo continues to be played as if the ninja were to do nothing. Since this is only for the last few frames, the ninja almost always reaches the exit, regardless of what happens. So the run looks legitimate, even though frames have been taken off. The problem? When the demo end-point is reached, THE CLOCK STOPS while the demo continues playing. So the time reached at the end when the ninja reaches the exit is a few milliseconds higher than what it should be.

Let's illustrate the potential problems with an example. I will use the fbf'd demo posted by bhz in the "Post a run!" for 121027, simply because it makes the number-crunching easier. This is his demo:

32:202116108|12632256|202116108|12632256|3084

If you load this into the box in userlevels, you will find that the ninja reaches the exit with a time of 89.200. This is consistent with the number of frames in the demo.

Now suppose that this level were being used for a contest where people submit their demo data. Suppose someone submits this demo which has been trimmed by FOUR whole frames:

28:202116108|12632256|202116108|12632256

(I know that it's fbf so can't be used for a contest, but that's beside the point here.) If you load this and watch it, it does look like the ninja reaches the exit. The ninja does reach the exit, BUT only because after the demo end-point it does nothing for the next few frames, and its upward velocity is enough to get it to reach the exit. However, THE CLOCK STOPS before the ninja reaches the exit, so the time appears to be 89.300, which is also consistent with the frame count, so it looks legitimate. So it does seem that someone managed to do a 28-frame demo on this level, which is impossible - it is just that demo editing has been used to chop frames off.

Now suppose we stick some extra frames onto the end.

35:202116108|12632256|202116108|12632256|0

Here the ninja reaches the exit in 33 frames, the last 5 of which were spent doing nothing. The timer, however, continues for the full 35 frames to 89.125; but we see that at the top left, the frame counter says "35f 33f", indicating that this was actually a 33-frame demo. This is a way of checking whether a demo has had frames cut off, like the one above.

The point of all this is that I am warning anyone whose contest involves posting demo data about the dangers of demo editing to achieve "better" times by stopping the clock. You will never know whether a demo has had such treatment done to it just by looking at it. In particular this may affect very greatly competitions such as golfkid's "The Time Is Right" where a single frame means everything. It would be muhc better to, for example, ask that a link be posted instead of demo data to prevent this problem. Of course, at the moment people trust each other here not to do these things, but I am highlighting a potential problem just in case someone decides to do this.

Wow... what a loooooooooooooooooooooooooong post!

[Kablamo]

People can check the data by recording it while they are playing it, then the map data will increase to the actual frames and the person can be found out for cheating

[Ad]

A) Everyone knows this.

B) Watch the demos in NED, it adds more frames on to the end until you actually complete the map. Just check the frame count before and after.

[blackson]

I found out another trick. Lets say you have a difficult map. You can beat it part by part, but not all at once. Press 1, get past the first part, and stop everything. Go back to debug mode, copy the demo data, and press 1, 3, Q. Your demo will start. Where you stopped, you can continue, and it will add it to the original demo. If you mess up, just paste the demo data you copied, and then press 1, 3, Q again. If you make it past another difficult obstacle, go to debug mode, and copy the demo data. Rinse and repeat until you've finished the map.

[TheRealOne]

I found out another trick. Lets say you have a difficult map. You can beat it part by part, but not all at once. Press 1, get past the first part, and stop everything. Go back to debug mode, copy the demo data, and press 1, 3, Q. Your demo will start. Where you stopped, you can continue, and it will add it to the original demo. If you mess up, just paste the demo data you copied, and then press 1, 3, Q again. If you make it past another difficult obstacle, go to debug mode, and copy the demo data. Rinse and repeat until you've finished the map.

ok well its not that easy, because I did that on my stupidly hard map (too lazy to look up the map id) but yes you can string together demo data to make a run, but its takes alot of work to get n to line up on the exact pixel that he left off on even with playing the demo data and continuing.

EDIT: the map id is the first link if you scroll over my NUMA id on the forums under my avatar.

[PsychoSnail]

I found out another trick. Lets say you have a difficult map. You can beat it part by part, but not all at once. Press 1, get past the first part, and stop everything. Go back to debug mode, copy the demo data, and press 1, 3, Q. Your demo will start. Where you stopped, you can continue, and it will add it to the original demo. If you mess up, just paste the demo data you copied, and then press 1, 3, Q again. If you make it past another difficult obstacle, go to debug mode, and copy the demo data. Rinse and repeat until you've finished the map.

Wow, I tried doing something like that before, but I think I did something wrong, because it didn't work. This does, though!

[Nexx]

So to sum this thread up (adding in my own info):

When watching demos in userlevels, 2 framecounts are shown: the purported framecount (left), and the real framecount (right). So that's nice and all, but unfortunately there's a glitch: if the demo continues past the purported framecount (exit not reached), the real framecount will disappear (at least with NReality v6c2). Therefore, you can still know that a demo has undergone framecutting if there's only 1 framecount by the time the exit is reached.

To confirm this (and to find the real framecount), you can open up the map in NEd. Load up the demo, but before you watch it, press 1 to begin recording. NEd will re-record the demo while showing it to you. Because NEd's demo record function (when playing a map) only stops when the exit is reached, the framecount will be accurate.

Did I get everything?

[999_Springs]

I have discovered another potential problem with demo data: underclocking. I don't think that there is any way to prevent a level from being played in underclock mode and the demo data taken from the userlevels box in underclock mode and submitted to a highscoring/speedrunning competition. Of course, this isn't too much of a problem as the underclock highscores in NReality are almost blank indicating that few people make use of it, but on some levels like those requiring one or two corner jumps it would be a huge advantage for the player to use underclock mode to get a faster time. Yet I don't think it is possible to detect whether submitted demo data has been underclocked (unless it's really obviously fbf'd).

Again, one of the contests most badly affected is golfkid's "The Time Is Right" because (say) if the ninja gets next to the exit with 0.3 seconds too much time, the player can count off remaining frames in his head (or on his fingers) and get the exact time.

[Nexx]

I have discovered another potential problem with demo data: underclocking.

Yes, underclocking is like the weaker-sauce version of FBFing: it helps you do better than you would do normally and it's undetectable if done well.

But in any case, this doesn't change much. FBF has been around since at least N v1.4, so everyone's aware that if you want to do a serious scoring contest, you need to use NReality submission. If, on the other hand, it's not that big of a contest, then the use of NReality can hurt participation, and in any case there's less incentive for people to cheat on a smaller contest.