[Request] Archive of Worlds as They Were at a Prior Point in Time

iPwner · 2017-05-06 20:39:37

Not recently, that wouldn't help... I mean like early 2016, 2015, or 2014, or prior. Does someone out there have any "archive" of world data as they were in the past (1 year+)? Also, if there isn't any, is there a way to get the data of a world from prior saves? Thanks.

capasha · 2017-05-06 20:47:14

iPwner wrote:

Not recently, that wouldn't help... I mean like early 2016, 2015, or 2014, or prior. Does someone out there have any "archive" of world data as they were in the past (1 year+)? Also, if there isn't any, is there a way to get the data of a world from prior saves? Thanks.

Atilla have backups.

XxAtillaxX · 2017-05-07 01:36:07

I've stored every world in Everybody Edits in 2015-2016, and have been storing snapshots of every world since.
You can access these snapshots of your worlds via the World Service room in the lobby, which is always online.

I'd highly recommend following the sign, leading to the repository where the source code as well as synchronization tool executable is located.
The synchronization tool is designed such that it doesn't require updates to handle new blocks, and shouldn't need to for the foreseeable future.

Vinyl Melody

iPwner · 2017-05-08 00:25:00

capasha wrote:

helpful

atilla wrote:

helpful

wow thas helpful thaymks ;D The only thing is that the world I want to access is of jeffreyshi1345's, he copied over a snowy level of mine to his profile, but deleted it in 2016 and replaced it with a boost afk for some reason... It'd be cool to get snapshots of other people's worlds too. So theoretically one would put a level ID onto a tool, and it would provide all available previous snapshots of the level? And this wouldn't infringe on the "ownership" law, as it's essentially just an archive.

Norwee

XxAtillaxX · 2017-05-17 18:30:18

Yikes. I've recently become aware of a few issues relating to exports in the World Service bot-assisted room.
Unfortunately, it's not entirely clear what the primary cause is, given the relatively smooth functionality for months prior.

I'd just like to inform anyone reading this topic that the service will be temporarily offline during maintenance.
It's not entirely a huge concern however, given the archival process will persist in the background in the meantime.

I've been looking forward to allocating some amount of time towards the archive, towards a web-based preview and synchronization client.
I'm unable to conceptualize any potential downsides to publicly releasing the entirety of the data-set, so if anyone is interested I may do so.

The archive currently contains over a million worlds (...yes, really) spanning several years.
It's of course stored in a non-binary format (JSON) so it's fairly easy to create statistics or perform analytical tidbits on it.

In addition, if anyone is willing to store copies of the archive (I'm thinking seeding a torrent.) - it would be most helpful.
I'm currently storing the archives on a up-to 20 gigabit hosting service, and I should be capable of harnessing it as a seed.

The current archive totals around ~25 GB uncompressed.

Jesse

LukeM · 2017-05-17 19:38:55

If you could convert the files into some other format which takes up less space, you could use things like google drive as a backup
Im sure that there are loads of ways to compress world data, and I might look into some soon, if that 25GB could be decreased to something more managable like 1GB (probably fairly easy) then it would probably be easier to upload / download / transfer backups (and probably cheaper too)

den3107 · 2017-05-17 22:32:03

I think currently it would be best to per default store it as compressed as possible, meaning it'd be a bit file.
Then when people'd like to run analytics they'd have to decompress it with a supplied too.

The tool should be able to decompress single worlds, a list of worlds or all the worlds, since each batch could have their own use.

LukeM · 2017-05-17 23:42:09

den3107 wrote:

The tool should be able to decompress single worlds, a list of worlds or all the worlds, since each batch could have their own use.

You would only need to have it able to decompress one world, then you could just run it for each to decompress multiple (its not a good idea to have multiple worlds in one file)

XxAtillaxX · 2017-05-18 08:22:50

The compressed archive is significantly less, below 5 GB.

To clarify, the format won't be changed as it is a direct conversion between Player.IO database objects and JSON which ensures replicability without introducing any additional ****.

LukeM · 2017-05-18 08:50:36

I wasn't thinking of changing the format you save them in, just having some 'backups' format where you could store them somewhere else (for free) as well just incase

XxAtillaxX · 2017-05-18 10:20:26

Well, in the past I've participated in large file sharing repositories with BT Sync which has since been bought off.

If there were a viable alternative to this I'd be interested in developing an architecture on top which would effectively accomplish a permanent persistent archive rather than relying on a single point.

I do not wish to create compressed volumes as it takes far too much time to enumerate and compress a million files 5 KB on average.

Official Everybody Edits Forums

#1 2017-05-06 20:39:37, last edited by iPwner (2017-05-06 20:40:38)

[Request] Archive of Worlds as They Were at a Prior Point in Time

#2 2017-05-06 20:47:14

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#3 2017-05-07 01:36:07, last edited by XxAtillaxX (2017-05-07 01:37:35)

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#4 2017-05-08 00:25:00, last edited by iPwner (2017-05-08 00:30:14)

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#5 2017-05-17 18:30:18

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#6 2017-05-17 19:38:55, last edited by LukeM (2017-05-17 19:39:16)

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#7 2017-05-17 22:32:03

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#8 2017-05-17 23:42:09

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#9 2017-05-18 08:22:50, last edited by XxAtillaxX (2017-05-18 08:25:16)

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#10 2017-05-18 08:50:36

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

#11 2017-05-18 10:20:26

Re: [Request] Archive of Worlds as They Were at a Prior Point in Time

Board footer