Official Everybody Edits Forums

Do you think I could just leave this part blank and it'd be okay? We're just going to replace the whole thing with a header image anyway, right?

You are not logged in.

#1 2020-12-31 05:32:26, last edited by LukeM (2022-02-03 00:17:52)

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

All of them - ArchivEE

logo.png

This is an archive of every single EE world containing content from the point at which the game was shut down, as well as a large collection of metadata to help you search for the worlds you want.

Download: ArchivEE (moderately large download, ~1GB)
Source: https://gitlab.com/LukeM212/ArchivEE/
ArchivEE C# library: https://www.nuget.org/packages/ArchivEE/

The download includes the ArchivEE database (ArchivEE.sqlite3) and a tool to search through it with versions for Windows (ArchivEE.exe), Linux (ArchivEE_linux), and MacOS (ArchivEE_macos). The ArchivEE library is available from NuGet and provides EFCore bindings for the database for if you want to access it programmatically. This comes along with a new version of the EELVL library which supports EELVL bundles and content-only EELVL serialisation (nuget, source).

How to use the search tool:
- When you open the tool, you will be prompted to start a 'query'. This is how you search for data in the database. The options you can pick will be listed, and to select one, enter its number and press enter.
- 'Start new query' starts a new query from scratch.
    - 'All' selects all entries. You will usually want to follow this with a 'Filter'.
    - 'By ID' asks you for a list of IDs, and returns the entries with those IDs.
    - 'By name' asks you for a list of names, and returns the entries with those names.
    - 'Custom SQL' lets you enter some custom SQL. This should be an SQLite "SELECT table.*" statement where the table name is the singular of the type of data, with underscores to separate worlds, e.g. Completed campaign worlds -> completed_campaign_world.
- 'Filter' filters the data you currently have selected depending on certain properties.
    - The options for what to filter by depend on what type of data you currently have selected.
    - Often it will prompt you for a range of values. These are inclusive, and if you want an open-ended range such as x >= 10, you should select 'none' for the end you don't need.
- 'Related data' selects another type of data related to the data you currently have selected.
    - This also depends on what type of data you have selected.
    - This selects every piece of data related to any entry currently selected in the way you select. For example, selecting 'Friends' when you have players selected will return all friends of any of the players in the selection.
- 'Continue earlier query' allows you to select an earlier query to continue.
    - This selects an earlier query to be continued.
- 'Combine with earlier query' lets you combine the currently selected query with a previous query of the same type.
    - 'X and Y' selects only the entries that were in both queries.
    - 'X or Y' selects any entry that was in either query.
- 'View result' lists the result of a query.
- 'Count resulting entries' counts how many results there were.
- 'Export worlds individually' exports each of the worlds in a query as .eelvl files.
    - This will prompt you for a directory to export all the eelvl files to.
    - It will export each world you have selected to this directory as an .eelvl file with the file name "world name - owner - world ID".
- 'Export worlds as bundle' exports the worlds in a query as an .eelvls bundle.
    - This will prompt you to select a 'default world'. This is the world you first join when you open the bundle in EEO.
    - This will also ask whether you want to 'remap' world portals. EEO uses integer world IDs based on the index in the world archive, e.g. the default world will have ID 0. This goes through and replaces all world portals pointing to a world in the archive with it's archive ID so that the world portal will work.

Example 1: Exporting all worlds that you own.

  1. Start a new query

  2. We want to load all worlds owned by a player, so first select 'Player' to find the right player

  3. Enter the player you want to find (your username) then press enter twice to mark the end of the list

  4. To check that you've selected the right player, you can run 'View result'. This should list some information about you

  5. We then want to find all worlds owned by that player, so select 'Related Data' then 'Worlds'

  6. We're finished with the query, so select 'Export worlds individually' and enter a folder to export to (e.g. 'output')

Example 2: Exporting all worlds that you have liked or favorited.

  1. As before, start by selecting your player

  2. To find all liked worlds, select 'Related data' then 'Likes'

  3. To find all favorited worlds, select the earlier player query by selecting 'Continue earlier query' then 'Players by name', then select 'Related data' then 'Favorites'

  4. To combine these, select 'Combine with earlier query' then 'Worlds liked by player' then 'X or Y'

  5. Export the worlds as before

Example 3: Exporting all worlds created by any of your friends that have more than 250 plays.

  1. As before, start by selecting your player

  2. To find all your friends, select 'Related data' then 'Friends'

  3. To find all worlds owned by those players, select 'Related Data' then 'Worlds'

  4. To select only worlds with more than 250 plays, select 'Filter' then 'Play count', then enter '250' for the minimum, and 'none' for the maximum

  5. If you want to make sure that you have written this query correctly, select 'View result'

  6. Export the worlds as before

Example 4: Creating an .eelvls bundle of all worlds reachable through world portals from a certain world.

  1. Select the starting world in some way, for example by using 'By ID'

  2. To find all worlds reachable through world portals, select 'Related data' then 'World portal graph'

  3. This doesn't include the original world itself if there isn't a world portal loop to get back to it, so select 'Combine with earlier query' then your original world query then 'X or Y'

  4. To export these as a .eelvls file, select 'Export worlds as bundle', enter the name of the bundle file you want to create (e.g. 'output.eelvls'), select your original world as the 'Default world' (the world you start in when you open the .eelvls file), then opt to remap world portals so that they're recognised by EEO.

If you are familiar with SQL and want to run your own queries, for example to get statistics from the raw database, I would recommend installing https://sqlitebrowser.org/. This is a tool which allows you to view and run queries on an SQLite database. If you want to run these queries programmatically, I would recommend using C# with the EFCore and ArchivEE libraries, though any SQLite library should work.

Original post

Offline

#2 2020-12-31 05:33:07

NoNK
Member
Joined: 2019-07-13
Posts: 922

Re: All of them - ArchivEE

can we get a

TSON TO EELVL CONVERTER

Offline

Wooted by:

#3 2020-12-31 06:04:31

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

LukeM wrote:

others, I believe including N1KF and SirJosh, either had copies of or were working on getting copies of all worlds

To be clear, I've never made copies of all the worlds. I have (outdated) lists of world IDs from Capasha, and I had a JavaScript program to merge Atilla's EEWorlds into larger JSON files so my computer can actually handle sorting the data. So ultimately the data's the same as what Atilla had a while back. I don't remember if I finished the process, but I at least made it around halfway. Also, I think I downloaded a few thousand worlds once before EELVL files were a thing, but that's nowhere close to having a complete collection.

Offline

Wooted by:

#4 2020-12-31 12:03:37

Weirdoverse
Member
From: A Really Really Really
Joined: 2015-02-20
Posts: 1,044
Website

Re: All of them - ArchivEE

Will the archive also include worlds from older world backups? For example I've had tons of worlds rebuilt in same 200x200 and 400x200 worlds in <2014 and i think other players might've done the same


A signature is a small piece of text that is attached to your posts. In it, you can enter just about anything you like. Perhaps you would like to enter your favourite quote or your star sign. It's up to you! In your signature you can use BBCode if it is allowed in this particular forum. You can see the features that are allowed/enabled listed below whenever you edit your signature.

Offline

#5 2020-12-31 12:20:26

soniiiety
Member
From: peaceful dojo
Joined: 2016-02-10
Posts: 1,752

Re: All of them - ArchivEE

apparently my acc is not in the archieve i asked on discord


de543c4c-ed76-4170-896f-8fa2a5183225_1_784x184.jpg?ex=667f757e&is=667e23fe&hm=b6817d138e71c14a2bdbf73bbe7213a5d2d1b7300d78449d1f0fe240e603ab04&

Offline

#6 2021-01-02 02:28:36

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

Re: All of them - ArchivEE

Progress on my version of the archive:

I've decided to put all the data I have into a database (using SQLite), which I'm currently in the process of doing, then once that's done I'll write a program to allow people to search through the archive and export any matching worlds as EELVLs.

I'm hoping this will end up small enough that it could be put up for download somewhere so anyone can just download the whole thing, but if not I might either provide it to other people to host on a website, or work on improving the format it's stored in to get the overall size down.

Weirdoverse wrote:

Will the archive also include worlds from older world backups? For example I've had tons of worlds rebuilt in same 200x200 and 400x200 worlds in <2014 and i think other players might've done the same

My one, no, but I think Atilla has an older one which has multiple versions of worlds.

soniiiety wrote:

apparently my acc is not in the archieve i asked on discord

It's possible that it's not in one of the other archives, but my one at least contains all (currently existing) accounts.

Offline

Wooted by: (2)

#7 2021-01-08 03:02:23

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

Re: All of them - ArchivEE

Almost done with this now, I've managed to put everything in a database, and I've got a tool which lets you search through it using various criteria.

One thing I'm including is a list of all staff members so that you could search for something like "all worlds created by pre-2015 staff members" or whatever, but I'm missing a few of the dates, I'd be grateful if anyone could fill in any of the ones I haven't been able to find if you know them or know where to find them:

CJMAEDER developer start date
CYCLONE developer start date
KASLAI developer start date
KASLAI developer end date
KERFF developer start date
KERFF developer end date
ULTRABASS 2nd moderator end date
MINISAURUS original graphics designer start date
NOU moderator start date - If we don't get this one I'll just put down 2015/4/17 (his first forum announcement)
RPGMASTER2000 developer start date
RPGMASTER2000 developer end date
SHOWPATH developer start date
THESOURCE85 moderator end date
TIRALMO start date
TOBY developer start date
XJEEX developer start date
XJEEX developer end date

Idk if Satanya will allow jewel transfers, but if she does I'll give 40 jewels for each of these dates you can fill in! (Exact dates please, and ideally let me know where you got the information from so I can double check it)

Once these are filled in I think it should be pretty much ready to use, and it's looking like while it'll be moderately large (~1GB), it should be small enough to host somewhere so people can download the whole thing //forums.everybodyedits.com/img/smilies/smile

Offline

Wooted by: (2)

#8 2021-01-08 05:18:54

Grilyon2
Member
From: Chile
Joined: 2015-02-15
Posts: 374

Re: All of them - ArchivEE

https://forums.everybodyedits.com/viewt … p?id=44554 Minisaurus graphics designer start date, 2018-11-22

Offline

Wooted by:

#9 2021-01-08 05:36:47

Kentiya
Member
From: United States
Joined: 2015-02-16
Posts: 1,112
Website

Re: All of them - ArchivEE

LukeM wrote:

One thing I'm including is a list of all staff members so that you could search for something like "all worlds created by pre-2015 staff members" or whatever, but I'm missing a few of the dates, I'd be grateful if anyone could fill in any of the ones I haven't been able to find if you know them or know where to find them:

CJMAEDER developer start date
CYCLONE developer start date
KASLAI developer start date
KASLAI developer end date
KERFF developer start date
KERFF developer end date
ULTRABASS 2nd moderator end date
MINISAURUS original graphics designer start date
NOU moderator start date - If we don't get this one I'll just put down 2015/4/17 (his first forum announcement)
RPGMASTER2000 developer start date
RPGMASTER2000 developer end date
SHOWPATH developer start date
THESOURCE85 moderator end date
TIRALMO start date
TOBY developer start date
XJEEX developer start date
XJEEX developer end date

Minisaurus joined 2018/11/21
Tiralmo joined 2018/10/10

Everything else was before Xenonetix's staff discord.


atikyne-cartoony-120.png
Kentiya / Atikyne — EE & EEU lead artist 2018-2020

Offline

Wooted by: (4)

#10 2021-01-12 17:16:08

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

Re: All of them - ArchivEE

This is done!

Check the original post for the details for how to download and use the archive, and let me know if anyone has any problems, particularly either without .NET installed or on Linux or MacOS (I haven't been able to fully test these).

It's a console app because I couldn't be bothered to design 50+ different menus for each of the options, but I'm hoping I made it easy enough for people to use //forums.everybodyedits.com/img/smilies/smile

Offline

#11 2021-01-13 03:24:08

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

Excellent. Now we shall make a website listing individual worlds and we will be one step closer.

Offline

#12 2021-01-14 01:04:44

soniiiety
Member
From: peaceful dojo
Joined: 2016-02-10
Posts: 1,752

Re: All of them - ArchivEE

N1KF wrote:

Excellent. Now we shall make a website listing individual worlds and we will be one step closer.

exactly


de543c4c-ed76-4170-896f-8fa2a5183225_1_784x184.jpg?ex=667f757e&is=667e23fe&hm=b6817d138e71c14a2bdbf73bbe7213a5d2d1b7300d78449d1f0fe240e603ab04&

Offline

#13 2021-01-17 19:29:36

capasha
Member
Joined: 2015-02-21
Posts: 4,066

Re: All of them - ArchivEE

Thanks for this awesome tool and database. I like it so much.

Offline

Wooted by: (3)

#14 2021-01-18 05:24:13, last edited by N1KF (2021-01-18 10:38:47)

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

Is more niche data like items owned by a player, potions enabled in a world, or woot counts being archived somewhere? These could be really helpful to figure out when a world was popular or when a player was active. iirc this technically isn't all the worlds, as there are like a million worlds but most of them are the default empty or home worlds. Are these also being preserved somewhere? I don't know if they have actual world data (placed blocks and such) but at least their metadata was there.

I probably sound like a worrywart but I'm worried that some weird parts of history could get lost, like an empty world that has thousands of plays or something weird like that, or some oversight resulting in some legitimate data being left behind. And of course I'd hope these bonus archives would be public where feasible.

Bug report

edit: cut out a few I figured out.

Bug report 2

Offline

Wooted by: (2)

#15 2021-01-18 13:08:11

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

Re: All of them - ArchivEE

N1KF wrote:

Is more niche data like items owned by a player, potions enabled in a world, or woot counts being archived somewhere? These could be really helpful to figure out when a world was popular or when a player was active. iirc this technically isn't all the worlds, as there are like a million worlds but most of them are the default empty or home worlds. Are these also being preserved somewhere? I don't know if they have actual world data (placed blocks and such) but at least their metadata was there.

I probably sound like a worrywart but I'm worried that some weird parts of history could get lost, like an empty world that has thousands of plays or something weird like that, or some oversight resulting in some legitimate data being left behind. And of course I'd hope these bonus archives would be public where feasible.

Bug report

edit: cut out a few I figured out.

Bug report 2

I was thinking that data like owned items was private as I thought that was what was breaking ee.yonom.org, but Capasha reminded me that that problem was actually something else, so I might try and scrape all the item data at some point if I have time. As for the rest, I have it all downloaded but chose not to include niche things like that in the database to keep things relatively simple, I could probably add it to the database but just not include it in the search tool though.

I do have a copy of every single world, but I filtered out the worlds that hadn't been modified as otherwise the archive database was several times as large. I guess this might mean there are some stats missing like the total number of plays of all of a player's worlds, so I could probably find some other way to solve this problem at some point. Maybe I could split up the worlds database into separate metadata and contents tables so duplicate world contents are only stored once or something.

As for the bugs, thanks! I'm planning on collecting a list of bugs and things to improve then releasing a version 2 at some point when I have time with those fixed //forums.everybodyedits.com/img/smilies/smile

Offline

Wooted by: (2)

#16 2021-01-18 22:02:28

John
Member
Joined: 2019-01-11
Posts: 2,009

Re: All of them - ArchivEE

N1KF wrote:

These could be really helpful to figure out when a world was popular or when a player was active

The IDs can tell you this, although I forgot when Chris's birthday is.


PW?scale=2

Offline

#17 2021-01-18 22:11:45

LukeM
Member
From: England
Joined: 2016-06-03
Posts: 3,009
Website

Re: All of them - ArchivEE

John wrote:
N1KF wrote:

These could be really helpful to figure out when a world was popular or when a player was active

The IDs can tell you this, although I forgot when Chris's birthday is.

I've included the creation dates where they were valid, but I didn't realise that at some point quite a few years ago the dates changed for some reason, so a fairly large chunk of them are incorrect. I'll be fixing them in the next version.

Offline

Wooted by: (2)

#18 2021-01-18 23:42:36

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

John wrote:
N1KF wrote:

These could be really helpful to figure out when a world was popular or when a player was active

The IDs can tell you this, although I forgot when Chris's birthday is.

They tell you when it was created, but not when players played it. There must be many worlds that remained empty for months or years, then became active later. Having access to woots, for example, would tell you generally how many players were playing it when woots were removed.

On a related note, it would also be possible to estimate an earliest possible "last saved" date by checking the most recent blocks or world settings used, but that would take analyzing the worlddata and is probably beyond the scope of ArchivEE.

LukeM wrote:

As for the bugs, thanks! I'm planning on collecting a list of bugs and things to improve then releasing a version 2 at some point when I have time with those fixed //forums.everybodyedits.com/img/smilies/smile

Alright, I'll dump some more bug reports:

  • entering an invalid file address (blank in my case) crashes the program.

  • Using invalid Custom SQL crashes it as well.

  • Exporting an empty query of worlds as individuals gives the normal response of "Exporting..." and "Done!" though this might be more of an oversight

  • Exporting an empty query of worlds as a bundle makes it list "Default world", and prompts you to select a number from 1 to 0. No response seems to cancel this prompt.

Offline

Wooted by:

#19 2021-02-21 16:13:27

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

Is this still being updated?

Offline

#20 2021-02-21 17:38:51

Different55
Forum Admin
Joined: 2015-02-07
Posts: 16,575

Re: All of them - ArchivEE

Updated in what way?


"Sometimes failing a leap of faith is better than inching forward"
- ShinsukeIto

Offline

#21 2021-02-21 17:47:33

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

Updated in the way LukeM said he planned to update it

Offline

#22 2021-03-13 05:57:28

EnanoTLL
Member
From: 50°36.25′S 165°58.38′E
Joined: 2015-02-26
Posts: 65
Website

Re: All of them - ArchivEE

im not sure if this is a bug with the program of the database but for some reason piano tiles with a negative ID (pretty much anything below C3) becomes 0. this kinda (completely) ruined many music worlds and i was wondering if there was a fix for this


for years i have dreamed of making everything big and soft so thats what i will do

fine.png

FLUFFY【300% マシュマロ〜】エンャノ(っゝω・)っ〜

Offline

Wooted by:

#23 2021-03-26 17:29:51

JME
Member
Joined: 2015-08-26
Posts: 15

Re: All of them - ArchivEE

I want to play my old worlds but windows says the program could be dangerous //forums.everybodyedits.com/img/smilies/sad


JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ

Offline

#24 2021-03-26 20:14:43

N1KF
Wiki Mod
From: ဪဪဪဪဪ From: ဪဪဪဪဪ From: ဪဪဪဪဪ
Joined: 2015-02-15
Posts: 11,113
Website

Re: All of them - ArchivEE

JME wrote:

I want to play my old worlds but windows says the program could be dangerous //forums.everybodyedits.com/img/smilies/sad

What's your username? If you don't mind somebody seeing the names of all your worlds in a list, it should be easy for somebody to get the worlds and send a link for you to download. I wouldn't mind volunteering. If you want to do the work yourself without running the possibly "dangerous" program, it's a bit more complicated:

The program is open-source, so if you know C# well enough you could go through the code and check it for anything malicious. The download has a big sqlite3 file with all the worlds in it, so if you want you can also get a program like DB Browser to browse through lists. The database also has the raw world data which you can export, but it doesn't seem to line up with EELVL files so you'd probably to do weird stuff to get it to work.

Offline

Wooted by:

#25 2021-03-28 08:16:40

JME
Member
Joined: 2015-08-26
Posts: 15

Re: All of them - ArchivEE

N1KF wrote:
JME wrote:

I want to play my old worlds but windows says the program could be dangerous //forums.everybodyedits.com/img/smilies/sad

What's your username? If you don't mind somebody seeing the names of all your worlds in a list, it should be easy for somebody to get the worlds and send a link for you to download. I wouldn't mind volunteering. If you want to do the work yourself without running the possibly "dangerous" program, it's a bit more complicated:

The program is open-source, so if you know C# well enough you could go through the code and check it for anything malicious. The download has a big sqlite3 file with all the worlds in it, so if you want you can also get a program like DB Browser to browse through lists. The database also has the raw world data which you can export, but it doesn't seem to line up with EELVL files so you'd probably to do weird stuff to get it to work.

Thank you! I would be glad if someone could search up my old worlds. I used multiple accounts to play the game: JME, asdfghjklo, JJis, JJJJJJJE. Asdfghjklo got permabanned so I don't know if those worlds could be seen.


JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ JJ

Offline

N1KF1730014267799468

Board footer

Powered by FluxBB

[ Started around 1732374006.3441 - Generated in 0.277 seconds, 12 queries executed - Memory usage: 1.88 MiB (Peak: 2.22 MiB) ]