Storing special blocks in world data

Tomahawk · 2016-09-22 16:38:19

While writing the world data documentation for Thanel's protocol, I've noticed that the majority of special blocks could be stored in chunks by using extra byte arrays for their additional parameters.

For instance, all purple switches could be stored together like this:

[0] int BlockID
[1] int Layer
[2] ByteArray X
[3] ByteArray Y
[4] ByteArray SwitchID

This works for all blocks which do not contain strings (currently signs, text and world portals) and so would massively shorten the size of the "init" packet for large worlds with many special blocks in them.

~ For those who don't know, the current method is to have a separate chunk [ID, layer, X, Y, parameters] for every special block with different values for its parameters - i.e. only purple switches with the same Switch ID are stored in the same chunk. This means you have a chunk for every unique switch ID, for every uniquely occurring coindoor value, coingate value, deathdoor and deathgate value, every block rotation, portal ID, target ID, music block ID, decoration morph... a huge amount of information gets repeated, making it all very inefficient to store.

I understand that the extra arrays would have to be serialized in order to store values of up to 999, but as the X and Y arrays are serialised this way the overall change wouldn't be huge.

Is there a reason why the current method is used, and could it be improved for UnityEE?

den3107 · 2016-09-22 22:12:51

Simple: You'd still hit the exact same problem, only then the integers and strings would be slightly smaller by making them into byte arrays.
Aside from that (as you said) it can also cause some uncertainties: "Where the f will my string end end does my int start?!", "Does this block even have an int?!", "Is the int secretly concealed as a string?!" or "What is love?!"
Obviously it would get VERY confusing to make every non-string-using block use a byte array, while all the string-using ones, which could potentially also contain other parameters like morph state, simply use extra parameters! It would make serializing even less logical for people who've never seen it, and generally add unnecessary code.

TL;DR;
It just makes things very confusing since you're mixing data types together in byte arrays and you'd still need to somehow split the different parameters of the special blocks, which still brings you to sending either multiple blocks, or pairing parameters with their x-y partners (which will result in duplication).

hummerz5 · 2016-09-22 22:28:26

I'm not sure what to make of it. I imagine it'd save space, but what if we just did away with all the DatabaseObject stuff and went for one giant byte array? Wouldn't that be fun?

idk how you'd serialize it, though.

Tomahawk · 2016-09-22 23:33:54

@Den: You're exaggerating, and not making a huge amount of sense.

However, you raise some issues with my suggestion. Storing a portal this way would mean you'd have serialised arrays for X, Y, PortalID and TargetID, but not necessarily a serialised RotationID array. It could well be serialised to only store the ID in the last 2 bits of the second byte, but that's 14 wasted bits. If it was serialised, the RotationID arrays for rotatable blocks would also have to be serialised for consistency. If neither were serialised, you'd have to specifically check for the portal block ID in order to know which arrays to deserialise and which to just read from.

It would break the current version of InitParse, which goes backwards collecting args until it finds the Y array.

...

I didn't consider the deserialising of this method's world data enough. It would work fine for the older deserialising method, which checks the block ID then skips forward the correct amount (and breaks whenever a new special block is added), but would require a rewrite of the new method of checking data types. The rewrite would work until a block was added which contained both a string and a byte array in its extra params - there'd be no way after that to avoid having to check the block ID.

Locked cuz discussing premature and ill-fated ideas is a waste of time.

...Because we really gotta time-manage in order to read and reply to all the active threads in this busy af forum. =3 Inb4 downsizing demods, because EE can't afford to pay you guys anymore (hah).

hummerz5

Official Everybody Edits Forums

#1 2016-09-22 16:38:19

Storing special blocks in world data

#2 2016-09-22 22:12:51

Re: Storing special blocks in world data

#3 2016-09-22 22:28:26

Re: Storing special blocks in world data

#4 2016-09-22 23:33:54, last edited by Tomahawk (2016-09-22 23:43:04)

Re: Storing special blocks in world data

Board footer