Page 1 of 2

The Great Highscore Archive

Posted: 2015.07.25 (17:25)
by zapkt
Hi,

As you may know, I've been working for a while on a project to automatically download N 1.4 scores, and display various kind of information about them.
Now I think the project is mature (and pretty) enough to deserve its own thread.

What does it do?
- Display new highscores in "real time" (with a one-hour granularity), so that you don't have to run NHigh yourself to see new scores
- Display various stats on a player
- Display history of highscores for a level on a chart
- Display past highscores for a level, with link to demo data (even if this demo is not reachable in the game anymore!)
- Display various stats

Link : http://ploudseeker.com:5000/
Link to automatically downloaded .hs files: http://ploudseeker.com/files/n/ (downloaded every hour for the moment, but that's going to change to "every day" very soon) kept one per day

Note that it's quite slow because it's running on a home-hosted raspberry pi, and 10 years of highscores history makes quite a huge quantity of data to dig into. If you can host the project on a more powerful (and stable) server, please do so!

Source code (feel free to contribute!)

Re: The Great Highscore Archive

Posted: 2015.07.25 (17:50)
by ska
No, I didn't know you were doing this. I can't wait to delve into this when I am sober enough to comprehend all of the work that you have no doubt put into this.

Re: The Great Highscore Archive

Posted: 2015.07.31 (18:57)
by EddyMataGallos
Woah this is awesome! Most people must be on holidays, because this deserves more attention. I love the new "Show all scores" functionality, great way to visualize the graph by text means.

I've noticed that the automatically downloaded scores aren't being cleaned from some cheaters i.e. naem, L3X, etc scores are appearing. I doubt I'm dexterous enough to contribute, but since I enjoy it I might give it a probably poorly optimized try.

Re: The Great Highscore Archive

Posted: 2015.07.31 (19:31)
by zapkt
EddyMataGallos wrote:I've noticed that the automatically downloaded scores aren't being cleaned from some cheaters i.e. naem, L3X, etc scores are appearing. I doubt I'm dexterous enough to contribute, but since I enjoy it I might give it a probably poorly optimized try.
I made a script to remove all cheater scores from the database without breaking all scores, so I can remove them easily.

For the moment, the list contains:
SEKular_proKRAStinator JoE_DiZZle 00danny1212 Haxxer a_m_i_r_s kryX-orange eRaSeR_ Vuteur _________________ [several names containing hew values] jason_k alllan managans KOKOZUDO JC239 banan049 igotbored:( haxYOscoreboard luanplayer ninjump Cuppy33 JoaoGCNunes haxyoscoreboard dnawrkshp Marcao PolakJamByc sry4trbleIMdone Yvsk getpucht

If you have a complete list of people to remove from the scoreboard, I'm interested.

I removed hackers because they're pointless. As for cheater... I don't know. Their scores had some importance at their time, so completely removing them from history does not completely makes sense to me. What do you think?

Re: The Great Highscore Archive

Posted: 2015.07.31 (20:29)
by EddyMataGallos
Oh yeah you have a point there, they are somehow relevant. How about finding a way to grey out or mark cheated scores so they are noticeable, or maybe a toggle button?

Regarding hackers, I compiled a list which I believe to be complete, or at least includes all hackers that appear in the pre-2011 cleansing campaign NHigh files that we have in our collection, and surely all the post-2011 (and hence current boards) new hackers. Take the list:

Code: Select all

Cheaters:

ANGERFIST
kryX-orange
L3X
crappitrash
Bonzai
naem
Goo
Sp33dy
pokemaniac1342
ACEOFSPADEWINS
Vegeta
BAS3
cheese_god
fuckingyourmom
fuckingcrappitrash
fuckingyourdad
VotedStraw61372

Hackers:

sry4trbleIMdone
haxYOscoreboard
haxyoscoreboard
igotbored:(
Cuppy33
alllan
ninjump
dnawrkshp
PolakJamByc
Yvsk
Marcao
luanplayer
Samijo192
Opelotus
sonysk84life
RacerD
adn888
firegods
hugo
babu141
Hassan_Shahriar_Ayon
Aaron_Gray_2013
calmo_11
Monkey0058
Shaun_D
llama_plop
zxcvbnm714
MechMykl
TESTICULAR_RUPTURE
FLAILING_TITS
Banned
1h8n3rdz
poo9293
Froaking
DD32123
Galli
Adeher
mwr718
moonjodjon
GodOfTheRedMoon
fightingguyz
benoit213
chernix911
Qwertyemo9furby
Alexis_lazootin
Quarta-O
starwolf
ELEPHANT_SHIT
mynameisjoshb
SWT_EZEY
L0LDPARTY
masterkert
DemogorgonXI
SSN
magicman256
kitten_rapist
Stephen_Chung
the_below_are_gay
DrewF
grn_trtle
wotan
golem93
Denis_Gruzdev
DAREAPER33333
XxKuchikiRukiaxX
alexmipt
kadu8080
1nsane89
Pinecone2
Redbart
lucky1304261992
roflmao_lol_xD
WatchᅠtheᅠReplayᅠHaveᅠFun
Kaizza
Blue_tetris_Sucks_
patrickdodd
bugfinder
tennismanclay
mousekamikadze
Alex_Rex
Pedevade
Cellux
minime12358
JC239
Vessey
QuEsTiOn?MaRk
Drewby
paulgummerson
hibernatingbear1
Arne_the_great
1337
()()
(oo)
(_")(_")
KOKOZUDO
banan049
Adameleryk
ninja,_pro
Sakigate
XxUrsusxX
14
EdK
NekoKitty^^
managans
TheZapper
frederik65
ITHINKUSUK
bethany111
torogadude
muriel
Bradsta4
theprocessofbelief
KealingProdigy
all-destruction
staticshot
mchcopl
razorrebel
Kidheart
gaaraofdafunk19
comosesupone
ninjaninjaninja
TweekDash
dracon316
JoaoGCNunes
Lex_Lazootin
windcentry
Spider_Jerusalem
Winnar
Below_Dawn
a_m_i_r_s
Vuteur
_________________
jason_k
eRaSeR_
blueheat45
I haven't included the unprintable names ones. Also, I've included all cheaters, because in fact this is my ignore list which I use to elaborate the rankings, but I think I've put them all at the beginning so you can choose what to do with them.

You haven't had problems with the old currently deleted ones as you already took clean files I presume, however I'm giving you the full list for two reasons: I haven't checked which ones are old and which ones are current hackers, and for extra security, as its not absurd to think we could include more old files if we get them, for extra data precision.

Also, has getpucht hacked/cheated any level?

Re: The Great Highscore Archive

Posted: 2015.07.31 (23:37)
by shomman
EddyMataGallos wrote:Also, has getpucht hacked/cheated any level?
On v2, he admitted to cheating chimneys, and I personally believe he is just an outright cheater, but he only admitted to chimneys.

Re: The Great Highscore Archive

Posted: 2015.08.01 (08:02)
by Innuendonewb
shomman wrote:
EddyMataGallos wrote:Also, has getpucht hacked/cheated any level?
On v2, he admitted to cheating chimneys, and I personally believe he is just an outright cheater, but he only admitted to chimneys.
Also he hacked 00-2 and ep 00,

His ep 00 score even shows up on http://n.infunity.com/ep_score.php?name=getpucht

Re: The Great Highscore Archive

Posted: 2015.08.01 (13:09)
by shomman
Innuendonewb wrote:
shomman wrote:
EddyMataGallos wrote:Also, has getpucht hacked/cheated any level?
On v2, he admitted to cheating chimneys, and I personally believe he is just an outright cheater, but he only admitted to chimneys.
Also he hacked 00-2 and ep 00,

His ep 00 score even shows up on http://n.infunity.com/ep_score.php?name=getpucht
Wow, that's damning. Certainly he hacked on nreality. In my opinion he doesn't play legitimately anywhere nor ever has.

Re: The Great Highscore Archive

Posted: 2015.08.01 (14:07)
by EddyMataGallos
Oh yeah I just saw that one, that's dissapointing.

He didn't hack NReality though, as you can see the score doesn't appear as legit.

Re: The Great Highscore Archive

Posted: 2015.08.01 (20:06)
by zapkt
EddyMataGallos wrote:Oh yeah you have a point there, they are somehow relevant. How about finding a way to grey out or mark cheated scores so they are noticeable, or maybe a toggle button?
The thing is, I have to find a solution which does not add significant extra calculation, since the application is already slow... I can add a client-side function to grey out or hide cheaters from a customizable list, but they would still be counted in various stats, 0th wouldn't be given to legit persons etc.

Re: The Great Highscore Archive

Posted: 2015.09.13 (14:12)
by Unreality
EddyMataGallos wrote:Oh yeah I just saw that one, that's dissapointing.

He didn't hack NReality though, as you can see the score doesn't appear as legit.
haha I was scared for a second. Yep NReality server doesn't treat it as legit score.

Re: The Great Highscore Archive

Posted: 2015.12.02 (20:19)
by EddyMataGallos
There's a little bug on the way this works: http://ploudseeker.com:5000/player?pseu ... Straw61372
First 0ths: April 25, 2015 (01-3, 103.625), (01-3, 103.875)

Re: The Great Highscore Archive

Posted: 2016.07.14 (14:56)
by zapkt
EddyMataGallos wrote:There's a little bug on the way this works: http://ploudseeker.com:5000/player?pseu ... Straw61372
First 0ths: April 25, 2015 (01-3, 103.625), (01-3, 103.875)
Huh, I probably have 2 files from that day and the 0th got improved between those 2 recordings
Will fix someday

Re: The Great Highscore Archive

Posted: 2016.10.16 (11:45)
by zapkt
Hey,

I was wondering how much the game was dying, so I plotted the number of new scores over time. See the stats page. (I only plot one figure per year, per-month is more annoying to do)

Re: The Great Highscore Archive

Posted: 2016.10.17 (12:56)
by ska
Thanks. This graph is both interesting and telling. It's amazing to think that, just two years ago, the community were getting—on average, anyway—a new 0th each day. Am I interpreting this correctly?

Re: The Great Highscore Archive

Posted: 2016.10.17 (13:18)
by zapkt
Hm, yea, but you must remember that we only have one recording per day. As a result, if there is a huge battle on a level and 10 different persons take the 0th successively (or 2 persons taking the same 0th several times), it will be counted only once.

Re: The Great Highscore Archive

Posted: 2016.10.18 (15:38)
by EddyMataGallos
ska wrote:Thanks. This graph is both interesting and telling. It's amazing to think that, just two years ago, the community were getting—on average, anyway—a new 0th each day. Am I interpreting this correctly?
Also the "community" was essentially a couple players, which is why it died so quick after something like N++ release.

Btw, it's a much stronger statement, it means 388 0ths out of a possible 600 changed their previous owner, it's impressive that over half of 0ths used to change every year, or close. The number of exchanges was probably much higher.
zapkt wrote:Hm, yea, but you must remember that we only have one recording per day. As a result, if there is a huge battle on a level and 10 different persons take the 0th successively (or 2 persons taking the same 0th several times), it will be counted only once.
Might I ask, how come some years have over 600 new 0ths? Does that include all scores that beat the previous year's 0th score? Because, if it's what I explained above, 600 should be the max.

Re: The Great Highscore Archive

Posted: 2016.10.18 (18:28)
by MiBeM
EddyMataGallos wrote:Might I ask, how come some years have over 600 new 0ths? Does that include all scores that beat the previous year's 0th score? Because, if it's what I explained above, 600 should be the max.
One recording a day != one recording per level - the same level could have been traded back and forth over multiple days.

Re: The Great Highscore Archive

Posted: 2016.10.18 (18:44)
by EddyMataGallos
Oh damn, true, silly me.

Re: The Great Highscore Archive

Posted: 2016.10.18 (19:44)
by zapkt
MiBeM wrote:
EddyMataGallos wrote:Might I ask, how come some years have over 600 new 0ths? Does that include all scores that beat the previous year's 0th score? Because, if it's what I explained above, 600 should be the max.
One recording a day != one recording per level - the same level could have been traded back and forth over multiple days.
I could calculate that as well, though

Re: The Great Highscore Archive

Posted: 2016.10.19 (09:13)
by zapkt
zapkt wrote:
MiBeM wrote:
EddyMataGallos wrote:Might I ask, how come some years have over 600 new 0ths? Does that include all scores that beat the previous year's 0th score? Because, if it's what I explained above, 600 should be the max.
One recording a day != one recording per level - the same level could have been traded back and forth over multiple days.
I could calculate that as well, though
Done! The result is still impressive... And this time, does not depend on the number of recordings we have.
So, in 2014, almost half the levels had their 0th improved...

By the way, if you have other ideas for stats to had to this page, let me know and I'll see whether it's doable.

Re: The Great Highscore Archive

Posted: 2016.10.20 (05:51)
by ska
zapkt wrote: So, in 2014, almost half the levels had their 0th improved....
That is a shocking statistic to me. 2014 turned out to be our community's swan song.

Re: The Great Highscore Archive

Posted: 2018.04.23 (15:05)
by zapkt
Just leaving a backup of downloaded HS files as of April 2018.
I might upgrade the server to a more recent machine soon, which would may the web application way more usable. But as I guess hardly anyone use it as of now, I'm not sure that's worth the trouble.

Edit: Duh wrong file, see later posts

Re: The Great Highscore Archive

Posted: 2018.04.23 (21:42)
by jasdanu
zapkt wrote:Just leaving a backup of downloaded HS files as of April 2018.
I might upgrade the server to a more recent machine soon, which would may the web application way more usable. But as I guess hardly anyone use it as of now, I'm not sure that's worth the trouble.
Is that of all HS??

Re: The Great Highscore Archive

Posted: 2018.04.24 (08:29)
by zapkt
All the one I started collecting automatically in 2015. The rest can be found in this thread: http://forum.droni.es/viewtopic.php?f=20&t=10211