N 2.0 data format

[sidke]

IF YOU WANT TO GET MAP DATA FOR MAPS ON THE SERVER, GO TO http://droni.es/getdata.php?l=http://droni.es/getdata.php?l=1086

This is for any potential scripters who want to integrate with the new format... and this is not fun, but here we go:

There are two types of integers you'll need to pull out of it, single bytes and shorts. And wonky endianness

Byte:
AB = A + B * 16

Short:
ABCD = (A + B * 16) * 256 + (C + D * 16)


There are no more separators for easily managing data anymore. No '|'s or '^'s or ','s or '!'s

EDIT: I FORGOT ABOUT THE FIRST FOUR BYTES YOU NEED TO READ FOUR BYTES OFF THE TOP OF THE DATA FOR SOME REASON

Data starts off similarly as 1.4, the first portion is tile data.
713 tiles, each represented by a byte in hex format. So 1426 characters of tile data total [EDIT: AFTER THE FIRST EIGHT].

Now comes the fun part... objects.

The codes for objects are:
0: player
1: mine
2: gold
3: exit
4: regular door
5: locked door
6: trap door
7: launchpad
8: oneway
9: chaingun
10: laser
11: zap
12: chaser
13: floorguard
14: bounceblock
15: rocket
16: gauss
17: thwump

The objects are stored in this order, and you need to go through each of them for possible stored data.
Right after the tile data ends, we need to loop through every 18 of the possible objects starting with the player at 0.
Read a short (four characters). This is how many objects of that type of objects are coming up.
The tricky part now is that there are a different number of bytes of data based on what the objects is, and here is that information:

2 bytes: player, bounceblock, mine, rocket, turret
3 bytes: regular door, launchpad, oneway, thwump
4 bytes: chaingun, laser, zap, chaser, exit
5 bytes: locked door, trap door

Read that many bytes for whatever object type you're currently on, then repeat from grabbing the short.



Let's run through an example:
"0010e3e20020a3232423002024a2a3a2000000000000000000000000000000000000000000000000000000000000"

Start our type at 0, the player
Read a short,
0010 (1). That's how many players we have

A player has two bytes, so read two bytes,
e3, e2 (62, 46)

No more players, so up our type to 1, mines, and read a short for how many mines we have,
0020 (2). Two mines, read two for the first mine
a3, 23 (58, 50)
Read two for the second
24, 23 (66, 50)

No more mines, we're at type 2, for gold, so read a short for gold
0020 (2), read two golds
24, a2 (66, 42)
a3, a2 (58, 42)

You'll notice the rest of the data is 00, so that means we have no more objects. But you will use this logic for other objects if present.


Actually using this data? Have fun with that. As far as I know, the first two bytes of information are positions. The extra bytes for the other objects are other information, some of it is other objects as well. The last two bytes for exits, locked doors, and trap doors are the switch/key positions. Third bytes are probably position in the tile. I haven't actually gone any further than just figuring out how the data is stored.

Hopefully this helps further scripters to write tools for the new format, but I'm really hoping they will redo the format to be friendly for third partiers.



Also if you want to get your map data from the server, get your level id and go to http://bucket.thewayoftheninja.org/[level_id].txt
The data's just base64 encoded and zlib compressed, so undecode and decompress it for your map data from the server.


I made a crappy notation to describe the data format

Code: Select all

<bytes:variable>, [number of elements:element]

Code: Select all

map data = <4:who knows>[713:<1:tile>][18:<2:object count>[object count:[2-5:<1:data>]]]

Code: Select all

server response = <2:title length><title length:title>%map data%

[Leonidas]

Very interesting. Hopefully someone will utilize this in the near future.

[otters~1]

Very interesting. Hopefully someone will utilize this in the near future.

"it's gonna be me" -justin timberlake

[Nexx]

Thanks, sidke! Valuable information.

[Meta Ing]

First off, nice work. You've saved me a lot of time and effort trying to explain this.

Second, yes, why are there eight 0s at the beginning of the data. WHY?

Let's see what I can contribute to this...

Tile data no longer loads column by column from top to bottom, but rather row by row from left to right. (Have fun converting, guys!) Each tile consists of two characters in reverse hex, ranging from (in unreversed hex) 00 to 21:

Code: Select all

00 = 00 - D tile (empty)
10 = 01 - E tile (filled)
20 = 02 - 5 tile, W
30 = 03 - 5 tile, S
40 = 04 - 5 tile, A
50 = 05 - 5 tile, Q
60 = 06 - 1 tile, S
70 = 07 - 1 tile, A
80 = 08 - 1 tile, Q
90 = 09 - 1 tile, W
a0 = 0a - 8 tile, S
b0 = 0b - 8 tile, A
c0 = 0c - 8 tile, Q
d0 = 0d - 8 tile, W
e0 = 0e - 4 tile, S
f0 = 0f - 4 tile, A
01 = 10 - 4 tile, Q
11 = 11 - 4 tile, W
21 = 12 - 3 tile, S
31 = 13 - 3 tile, A
41 = 14 - 3 tile, Q
51 = 15 - 3 tile, W
61 = 16 - 7 tile, S
71 = 17 - 7 tile, A
81 = 18 - 7 tile, Q
91 = 19 - 7 tile, W
a1 = 1a - 2 tile, S
b1 = 1b - 2 tile, A
c1 = 1c - 2 tile, Q
d1 = 1d - 2 tile, W
e1 = 1e - 7 tile, S
f1 = 1f - 7 tile, A
02 = 20 - 7 tile, Q
12 = 21 - 7 tile, W

Object data can be grouped, I think, into five types: Type I - Ninja, mine, gold, floorguard, bounceblock, rocket, gauss; Type II - Drones; Type III - Locked and Trap doors; Type IV - Launchpads, oneways, thwumps, and normal doors; and Type V - The exit.

All types begin the same way: [amount][x1][y1] -- where [amount] is a set of four characters grouped into two sets of reversed hex values (noticing a pattern here?), [x1] and [y1] are also reverse hex values and represent the x and y coordinates for the main part of an object; those coordinates can be converted to v1.4 coordinates by reversing the value, converting the number from hex to decimal, and then multiplying the result by 6. For example... if an object has coordinates e3e2:

Code: Select all

e3 e2 -->  
(reverse these values to get) -->
3e 2e --> 
(magically convert hex to decimal, I don't know the math) --> 
32 46 --> 
(multiply both values by 6 and add a comma between for a final convereted v1.4 value of...)  -->
192,276

Now, let's elaborate on types... after [amount][x1][y1]...

Code: Select all

Type I: ...[x2][y2][x3][y3] etc...
Type II: ...[direction 1][path 1][x2][y2][direction 2][path 2] etc...
Type III: ...[direction 1][switch x1][switch y1][door x2][door y2][direction 2][switch x2][switch y2] etc...
Type IV: ...[direction 1][x2][y2][direction 2] etc...
Type V: ...[switch x1][switch y1][exit x2][exit y2][switch x2][switch y2] etc...

Direction is a set of 2 (probably reverse hex) characters ranging from (in unreversed hex) 00 to 07, and goes clockwise starting facing right: (that may be worded badly but here)

Code: Select all

00 = 00 -> right
10 = 01 -> down right
20 = 02 -> down
30 = 03 -> left down
40 = 04 -> left
50 = 05 -> up left
60 = 06 -> up
70 = 07 -> up right

Of course, you'll only get 10, 30, 50, and 70 for launchpads; Drones, oneways, and thwumps will have 00, 02, 04, or 06; and normal, locked, and trap doors will (for some reason) only have 00 or 02.

As for pathing (which only applies to drones), the values are once again a set of 2 (probably reverse hex) characters ranging from (in unreversed hex) 00 to 03:

Code: Select all

00 = 00 -> Clockwise Surface-following (C pathing)
10 = 01 -> Counterclockwise Suface-following (Z Pathing)
20 = 02 -> Clockwise Dumb (E pathing)
30 = 03 -> Counterclockwise Dumb (Q pathing)

That's about all I can say at this time (it's pretty late and I need to sleep); hopefully it's clear enough to understand. Good luck to anyone who wants to create a converter.

Side note - The format for coordinates is why z snap is no longer possible (afaik).

[sidke]

just to let you know, the reversed hex is just little endian
https://en.wikipedia.org/wiki/Endianness

the nibbles are little, but the bytes are big

10 = 1
01 = 16
1000 = 256
0100 = 4096

[Pembie]

Wow I wish I had of read this a little earlier, this was a pain in the ass to work to work out. In any case, I updated the level image generator to work with V2. It's a PHP script that as far as I can tell is bug free, however it has problems running N-art's converted from 1.4. I think this is due to the loss of free-snapping, or that too many objects screws with it a little bit.

After reading this I may update it to work a bit better, as I pretty much hacked it together.

Online version: http://mapswap.wickedosity.net/image/

[aids]

and normal, locked, and trap doors will (for some reason) only have 00 or 02.

Horizonal and vertical or vice versa. It's one of the things I like about this version, doors align not to tiles but to the grid itself.

[Pembie]

Updated my script. Now it works properly with the endians. Feel free to use the script for whatever, or make it better. The way I made it draw objects is based off the column position, rather than pixels. So I converted the 2 byte object positions into the column number it corresponded to, making the top left corner (at the edge) 0,0.

Updated this too:
http://mapswap.wickedosity.net/image/

[Paradox]

Here's a .exe that will convert tiles, ninjas, gold & mines… maybe some other objects… you'll have to fix the drones & doors manually but it's a start… i'll make its own thread when it's finished

Mediafire

[sidke]

Here's a .exe that will convert tiles, ninjas, gold & mines… maybe some other objects… you'll have to fix the drones & doors manually but it's a start… i'll make its own thread when it's finished

Mediafire

.exe bad
release source so people can see what they're running :C

[Pembie]

I was bored...

http://mapswap.wickedosity.net/image/

Now does animations!

I was going to work on a convertor, but seeing as someone else is I won't bother. What else is there to do?

[The Black Lion]

ow lot of interresting stuffs.
just saying but... do you know there's already a convertor in the new editor itself? ([page down] to acces data)

[Meta Ing]

That only converts 1.4 data to 2.0 data. We're working on converting 2.0 to 1.4.

[The Black Lion]

ow okay then :)
though i don't really see the direct utility of a such convertor?

[aids]

ow okay then :)
though i don't really see the direct utility of a such convertor?

2.0 uses a wacky format, and being able to understand it should be at least possible for new users. Plus it helps if you wanted to post a map to NUMA or such.

[Pembie]

So I was bored tonight and made a converter. Unfortunately I only know PHP well, or I could do it in MATLAB (I should really learn something else).

I'm pretty sure everything works, doors may be a bit screwy as 1.4 has a lot of data for doors, where as V2 is very simple. And I really don't want to work out the logic for determining if a door needs to be in the previous or next tile space..
Let me know if something doesn't work.

I'll upload the code once I have cleaned it up a bit.

http://localhost/mapswap/convert/

[sidke]

http://localhost/mapswap/convert/

this is so adorable, lv and i had a laugh

[Pembie]

That's a little dumb..

Try again http://wickedosity.net/mapswap/convert/

Has the source code on the page. The only thing I have come across is that floor guard snapping has changed, so if any are placed on half tiles in V2, it won't work in V1.4.

Let me know if there are any other problems