[Question] Application to search and browse EE Worlds

Hexagon · 2015-04-12 20:55:08

Is there any website/application that allows one to browse/search ee worlds, other than the lobby (including ones that are offline?) This would include searching things like titles, world previews, dominant colors (if you forget which room it was but remember the color), plays etc...

If there isn't one, I'll think about making one.

ewoke · 2015-04-12 20:57:41

i never saw something like this before, would love to see you make one

Hexagon · 2015-04-12 20:59:33

ewoke wrote:

i never saw something like this before, would love to see you make one

I just remembered now that Atilla made WorldSearch, which had essentially the same functionality however it has been deleted (to my knowledge.) If anyone can find it, that'd be great!

ewoke · 2015-04-12 21:07:46

Hexagon wrote:

ewoke wrote:
i never saw something like this before, would love to see you make one
I just remembered now that Atilla made WorldSearch, which had essentially the same functionality however it has been deleted (to my knowledge.) If anyone can find it, that'd be great!

i dont think worldsearch had the option to search on colour or plays

:EDIT: asked atilla, he no longer has it running.

lrussell · 2015-04-12 21:44:24

Gah, I suppose I could give it a shot. I have a fairly good method of reaping the game for levels.

Xfrogman43 · 2015-04-13 02:31:49

Colour? What do you mean by that?

DarkDragon4900 · 2015-04-13 08:54:50

Xfrogman43 wrote:

Colour? What do you mean by that?

e.g. Orange plastic blocks in 200 lava minigames.

Hexagon · 2015-04-13 13:09:44

Xfrogman43 wrote:

Colour? What do you mean by that?

If the level is mainly composed of a certain shade of a color (like 200 red minigames) the color would be red. However, if a world has many different kinds of colors, it would have to detect this and not have any color at all, because a smush of all of the colors together (might be like a brown color) doesn't make sense.

DarkDragon4900 · 2015-04-13 21:57:47

Something like this, you mean?

BlockColor domBlock;
struct BlockColor {int bid; int Amount;} ///"Amount" -> Amount of blocks found in a level with the color specified.
List<BlockColor> colors = new List<BlockColor>();
int temp = 0;
foreach (BlockColor c in colors)
 if (c.Amount>temp && c.Amount>=(worldWidth*worldHeight/2)) {temp = c.Amount; domBlock = c;} // Only set it once it's dominating over half of the level.

Hexagon · 2015-04-14 10:37:37

DarkDragon4900 wrote:

Something like this, you mean?

BlockColor domBlock;
struct BlockColor {int bid; int Amount;} ///"Amount" -> Amount of blocks found in a level with the color specified.
List<BlockColor> colors = new List<BlockColor>();
int temp = 0;
foreach (BlockColor c in colors)
 if (c.Amount>temp && c.Amount>=(worldWidth*worldHeight/2)) {temp = c.Amount; domBlock = c;} // Only set it once it's dominating over half of the level.

Yeah this would basically be it. Just changing my idea slightly, it could be cool to have multiple colors if the level has multiple dominating colors (like three colors), but might be complicated.

Thanks DarkDragon!

lrussell · 2015-04-17 05:45:37

        /*
         * To effectively find as many levels as possible, we need to find
         * as many users as possible. To achieve this, all users currently 
         * online will be listed. The levels of each of these users will 
         * then also be listed.
         * 
         * While that would be fairly effective, there is more we can do.
         * We can list all the levels currently open in the game as well.
         * Since a level can be open without its owner present, we can then
         * use the list of levels for each online user and single out the
         * levels whose owners aren't online.
         * 
         * With this method, you can then find the owners of the abandoned
         * levels and perhaps find a user that hasn't been online. All that
         * would be left then is to list every level that the user owns.
         * 
         * To index a level, we would first check if it already exists in the 
         * database. If it doesn't exist, it is indexed. Otherwise, we check
         * if it is open in the lobby since that is the only way any of the
         * information could have changed. If it is open, we re-index it to
         * be sure that the information is current.
         */

This is my design plan so far, any changes or improvements would be welcome. This is the most effective method I could possibly come up with and I'm preparing to implement it. An optimal delay between checking for new users and levels is still being determined, would 30 seconds be effective?

Hexagon · 2015-04-17 12:08:21

I think that's a pretty good plan, lrussell. I think this would be good if the architecture was similar to a producer/consumer pattern (bots find new info, place it in a queue then have another app which processes it and checks it.) I just have a few modifications:

- build a component that lists all worlds for a username, and lists the owner for a world
- I have a large list of world ids (about 10,000) that I could give to you to start you off plus Atilla gave me some too. I also have a long list of users (maybe 40,000+). It'll take me a while to figure out how to extract them, because I have a bit of a compression mess.
- place a bot in the tutorial world (and another introductory/featured world), that is interested in "add" messages (a good way to "snatch" new usernames).
- if the level is already in the database, and is past a certain age (not sure how old) it would be re-indexed, assuming that you're grabbing the WorldData. Getting live init data would be awesome, but you'd have to have a bot that would peek every so often.
- if you wanted to be really efficient with getting WorldData (if you want to) you could make an algorithm that would calculate the frequency of how quickly the world updates, and crawl it depending on that.

The delay of 30 seconds looks good, however, there is something secret about the lobby. If you scan it very quickly (every second) you can find virtually all registered users starting from today (assuming that they click "Go to Lobby" at any point) because the lobby data has a list of users that are in the lobby, and are not in a world. From there you can get a list of user ids, and get the worlds too.

You could open up something that people just drop in usernames into a form (like google forms) and then have a bot that just goes through and add them to the data base if they don't exist.

lrussell · 2015-04-17 18:33:01

Hexagon wrote:

I think that's a pretty good plan, lrussell. I think this would be good if the architecture was similar to a producer/consumer pattern (bots find new info, place it in a queue then have another app which processes it and checks it.) I just have a few modifications:
- build a component that lists all worlds for a username, and lists the owner for a world
- I have a large list of world ids (about 10,000) that I could give to you to start you off plus Atilla gave me some too. I also have a long list of users (maybe 40,000+). It'll take me a while to figure out how to extract them, because I have a bit of a compression mess.
- place a bot in the tutorial world (and another introductory/featured world), that is interested in "add" messages (a good way to "snatch" new usernames).
- if the level is already in the database, and is past a certain age (not sure how old) it would be re-indexed, assuming that you're grabbing the WorldData. Getting live init data would be awesome, but you'd have to have a bot that would peek every so often.
- if you wanted to be really efficient with getting WorldData (if you want to) you could make an algorithm that would calculate the frequency of how quickly the world updates, and crawl it depending on that.
The delay of 30 seconds looks good, however, there is something secret about the lobby. If you scan it very quickly (every second) you can find virtually all registered users starting from today (assuming that they click "Go to Lobby" at any point) because the lobby data has a list of users that are in the lobby, and are not in a world. From there you can get a list of user ids, and get the worlds too.
You could open up something that people just drop in usernames into a form (like google forms) and then have a bot that just goes through and add them to the data base if they don't exist.

A lot of those components you mentioned are un-needed, because I do get the list of every user online even if they're in a level or not. Essentially I'll be able to find users by having them just login to the game or someone else joining one of their levels. I don't plan on capturing level data, just the metadata. Scanning it every second seems very excessive, I don't believe that my scanning process would even complete in that time.

Hexagon · 2015-04-19 21:26:28

lrussell wrote:

A lot of those components you mentioned are un-needed, because I do get the list of every user online even if they're in a level or not. Essentially I'll be able to find users by having them just login to the game or someone else joining one of their levels. I don't plan on capturing level data, just the metadata. Scanning it every second seems very excessive, I don't believe that my scanning process would even complete in that time.

If you do this, I would really like this feature to be added: http://stackoverflow.com/questions/7630 … -your-site

lrussell · 2015-04-20 00:33:58

Hexagon wrote:

lrussell wrote:
A lot of those components you mentioned are un-needed, because I do get the list of every user online even if they're in a level or not. Essentially I'll be able to find users by having them just login to the game or someone else joining one of their levels. I don't plan on capturing level data, just the metadata. Scanning it every second seems very excessive, I don't believe that my scanning process would even complete in that time.

If you do this, I would really like this feature to be added: http://stackoverflow.com/questions/7630 … -your-site

I'm working on building up a first-run index right now and then I'll move on to completing the software to scan for changes. I managed to get some help so I should have every level in the game indexed.

Official Everybody Edits Forums

#1 2015-04-12 20:55:08

[Question] Application to search and browse EE Worlds

#2 2015-04-12 20:57:41

Re: [Question] Application to search and browse EE Worlds

#3 2015-04-12 20:59:33

Re: [Question] Application to search and browse EE Worlds

#4 2015-04-12 21:07:46, last edited by ewoke (2015-04-12 21:10:26)

Re: [Question] Application to search and browse EE Worlds

#5 2015-04-12 21:44:24

Re: [Question] Application to search and browse EE Worlds

#6 2015-04-13 02:31:49

Re: [Question] Application to search and browse EE Worlds

#7 2015-04-13 08:54:50

Re: [Question] Application to search and browse EE Worlds

#8 2015-04-13 13:09:44

Re: [Question] Application to search and browse EE Worlds

#9 2015-04-13 21:57:47, last edited by DarkDragon4900 (2015-04-13 21:58:24)

Re: [Question] Application to search and browse EE Worlds

#10 2015-04-14 10:37:37, last edited by Hexagon (2015-04-14 10:37:50)

Re: [Question] Application to search and browse EE Worlds

#11 2015-04-17 05:45:37

Re: [Question] Application to search and browse EE Worlds

#12 2015-04-17 12:08:21

Re: [Question] Application to search and browse EE Worlds

#13 2015-04-17 18:33:01

Re: [Question] Application to search and browse EE Worlds

#14 2015-04-19 21:26:28

Re: [Question] Application to search and browse EE Worlds

#15 2015-04-20 00:33:58

Re: [Question] Application to search and browse EE Worlds

Board footer