[Discussion] EE Standardization -- let's all get on the same page

lrussell · Before February 2015

Hexagon wrote:

tikenalpha wrote:
What I did for EEPainter was the same thing as the "init" message you receive:
"block ID + param
x,y;x,y;x,y;..."
It's in clear text, easy to read an modify instead of opening the soft again.
It's nice when the world doesn't have many different blocks, and tends to grow a bit when many portals/coin doors& are used. But it won't grow too big anyway, and I don't think the space use will be consequent enough to bother. Except if you work on a potato.
I didn't think about standardizing the init messages (as the world is just copied directly from the database then serialized). Your method looks like it could be very human-readable. Also, since the init messages are in a different format than the database, it might be interesting to consider how to merge the two formats.
However, with all formats they need to be forward-compatible and can't operate on a fixed schema (to an extent). Do you mind if you could post the syntax for encoding portals and coin doors? Thanks!

Considering each contain the same world data, just in a different format, a new converter could be made to put it into the same format as the database-pulled worlds.

TiKen · Before February 2015

Hexagon wrote:

I didn't think about standardizing the init messages (as the world is just copied directly from the database then serialized). Your method looks like it could be very human-readable. Also, since the init messages are in a different format than the database, it might be interesting to consider how to merge the two formats.

I didn't know how to save the map in the first time and crudely copied the init message. I however added the database extraction method after but kept the saving format. The special blocks are saved as following:

25;25;normal 0;242;portal 1.png;0;2;33 2,2; 4,2 0;43;b43.png;42 2,4 0;385;dec153.png;text01 2,6 0;374;world portal.png;target01 2,8

The first line describe the kind of world (25x25 normal gravity) then comes the block list:

layer;ID;sprite_name;[param1];[param2];[param3];... x,y;x,y;...

The parameters are set in the order the bot as to send them to facilitate reading. And because it was the first idea I had XD

Last edited by tikenalpha (Jan 2 2015 4:37:07 am)

lrussell · Before February 2015

Looking at the beta version of the JSON schema, I don't like the 7-zip requirement, because it makes creating a serializer that works in Mono rather difficult. 7-zip has a hefty CPU requirement for both compressing and decompressing, and the only 7-zip method I see for Mono is to call p7zip from the shell (which wouldn't retain Windows compatibility). My current Protobuf+Gzip format runs well even on a Raspberry Pi, when I used Json along with the same Gzip compression it was constantly at full CPU.

By the way, under large scale testing I found that my current format runs into some errors (thanks to corrupt EE data). I have fixed them as well as added some explanation comments to the WorldFormat file.

http://raspberry.lrussell.ga/~lrussell/ … Example.7z (hosted by said Raspberry Pi)
(yes I used 7zip because it's good for compression, not speed or cpu usage)

Using the current format in play for the same level,
Normal-level 7-zip Compressed Json: 62.6 KB (64,120 bytes)
Gzip Compressed Protobuf: 60.2 KB (61,682 bytes)

Protobuf+Gzip has a better compression ratio with considerably faster speed and lower resource usage.

Hexagon · Before February 2015

lrussell wrote:

[...]I don't like the 7-zip requirement, because it makes creating a serializer that works in Mono rather difficult. 7-zip has a hefty CPU requirement for both compressing and decompressing[...]

7-Zip is quite heavy on the CPU (but it does give a really good compression ratio at times). Since disk space is becoming cheaper and cheaper, and CPU is not (as much), I do agree with using Gzip instead of the hefty 7-Zip.

However, with protobuf, while it is very compressible, and very quick to write and read from, a fixed schema has to be developed in order to write the data (to my knowledge). This means that if something is added to the EE database, the fixed schema won't be able to write it to the file, making an update for the level-savers necessary. With the JSON encoder, it's simply a database dump and so it can incorporate everything.

EDIT: protobuf, with the use of a dictionary, is able to store the "unknown" values. In this respect, since protobuf is able to store these values, it currently holds an advantage over JSON.

Last edited by Hexagon (Jan 2 2015 10:01:20 pm)

Official Everybody Edits Forums

#26 Before February 2015

Re: [Discussion] EE Standardization -- let's all get on the same page

#27 Before February 2015

Re: [Discussion] EE Standardization -- let's all get on the same page

#28 Before February 2015

Re: [Discussion] EE Standardization -- let's all get on the same page

#29 Before February 2015

Re: [Discussion] EE Standardization -- let's all get on the same page

Board footer