[September 21 2015] Statistics from EE Analytics

Xfrogman43 · 2015-04-29 00:55:35

Well the bigger level looked pretty cool. I guess the '200 minis' people make or possibly shift or something like that.

Hexagon · 2015-04-29 01:10:11

Here's shift:

(It says something about mini games, ignore that. Unfortunately if I zoom in I think it's just purple everything.)

Here's the 200 lava minigames:

The lava minigames looks really cool. I might post a zoomed in picture of it.

N1KF · 2015-04-29 01:14:46

Hexagon wrote:

*snip*

Snipped thirty-one seconds after being posted? Seems mysterious.

Creature · 2015-04-29 01:19:08

N1KF wrote:

Hexagon wrote:
*snip*
Snipped thirty-one seconds after being posted? Seems mysterious.

He posted this:

Hexagon wrote:

Here's shift (~75000):
I'll fetch 200 lava minigames in a sec

Hexagon · 2015-04-29 01:25:34

Yeah it was the wrong level, so I quickly snipped it (I forgot that I could have deleted my post.) The level that I posted was "cursed tag" I think.

Nou · 2015-04-29 08:19:26

Just a purple box I think the Shift image is upside down btw

Hexagon · 2015-07-11 21:59:44

Just a quick update:

Data is still being collected, and it's getting slightly overwhelming. While it's only 70GB compressed, it's 700GB uncompressed. In order to move the data around into a proper database, or to do searches it has to be decompressed first. Is there anyone that would be willing to "backup" a piece of this via a torrent? The data would be encrypted with an extremely strong password, so that it would only be accessible to me (because of privacy issues.) Alternatively, if I get permission from one of the staff members, I can encrypt it and store it on archive.org.

I really like your ideas about what to graph, because it really gives a new insight on these things. Is there anything else that you'd like to see graphed? I'll do my best to try to parse the data.

Different55 · 2015-07-11 22:07:23

How are you storing this? It might be worth your while now to change your formatting and store things more efficiently now so you can save terabytes in the future.

Hexagon · 2015-07-11 23:21:05

Different55 wrote:

How are you storing this?

Each world recording has a folder. The folder title is year_month_day_[eight alpha characters]. The filename for the file in the folder is the world id.
That file contains:
- date_started (current date, accurate to the second encoded in json)
- timestamp, accurate to 10th of a millisecond, and the event payload encoded in json. Every event that is sent to the client is recorded.

The folders are moved into another folder at the end of the month, and that folder's name takes the template of year-month. From there, that folder is compressed using gzip, then uploaded to a limited but secure cloud storage service after being stored on an external harddrive.

Json is used because the event payload is dynamic and can vary (i.e does not have fixed fields.) A solution would be to use msgpack or a derivative, but this would require that each event is documented into the code (requiring that I update the app whenever something is added to EE.)

Json pros:
- human readable
- flexible for new events

Json cons:
- too verbose (large file size, however ~10X reduction when using gzip)
- takes a really long time to search for things (because of large file size.) Gzip complicates things because it has to be decompressed first.

Msgpack pros:
- very fast to read and write
- very small file size

Msgpack cons:
- not very flexible with new events (have to update app whenever there's a new event)
- cannot compile/search for events using linux's grep command (have to use the msgpack library)
- difficult to see data corruption (json you can just open it up in a standard editor and see the contents)

Finally, another service pings playerio every second and records the current time and the ping time into a separate file. This file is not rotated, and not really large.

AmdS · 2015-07-12 00:30:19

How do you know all this information? I think is impossible to see messages, because are passed. So, weird.

Trung · 2015-07-14 17:42:57

So... how about which block people placed most ? ... and top 10 block placed most ? ...

Hexagon · 2015-09-21 13:47:22

Trung wrote:

So... how about which block people placed most ? ... and top 10 block placed most ? ...

Sorry for the late reply!

Block data from August 1, 2015 at 00:00 GMT+0 to August 31st at 23:59:59 GMT+0

+-----------+----------+-----------------+
| Frequency | Block ID | Percentage Used |
+-----------+----------+-----------------+
|  13189396 |        0 |           18.50 |
|   2659611 |       16 |            3.73 |
|   2377230 |        9 |            3.34 |
|   2133801 |       29 |            2.99 |
|   2116071 |      100 |            2.97 |
|  1998243  |        2 |            2.80 |
|   1342514 |       44 |            1.88 |
|   1323938 |       10 |            1.86 |
|   1239660 |       70 |            1.74 |
|   1107728 |       11 |            1.55 |
+-----------+----------+-----------------+

41,782,682 other blocks used 58.63% of the time.

Sample size of 71,270,874 blocks.

Note: I'll double check this data later to make sure that they've been calculated correctly

Xfrogman43 · 2015-09-21 23:47:16

Hexagon wrote:

(data)

Should add block names so people know what blocks are which, instead of having to look them up, because people are lazy.
0 = erase
16 = brown brick
9 = gray basic
29 = gray metal
100 = gold coin
2 = up arrow
44 = fully black block
10 = blue basic
70 = red mineral
11 = purple basic

some woman, Hexagon, BEE

SirJosh3917 · 2015-09-22 00:02:30

Hexagon wrote:

368 ninja*
138 ninjasupeatsninja

(people abbreviate my name too often)
I'd like to thank my friends, my family, my college, my high school, my middle school, my grade school, the people who randomly talk to me, and myself.

Evilbunny · 2015-09-22 00:44:30

ninjasupeatsninja wrote:

Hexagon wrote:
368 ninja*
138 ninjasupeatsninja
(people abbreviate my name too often)
I'd like to thank my friends, my family, my college, my high school, my middle school, my grade school, the people who randomly talk to me, and myself.

or..ya know.. people talk about red ninja, regular ninja, and ninjas are cool to talk about in general.
Don't get a swelled head.

hummerz5, Krosis, SirJosh3917, some woman

SirJosh3917 · 2015-09-22 02:19:26

Evilbunny wrote:

ninjasupeatsninja wrote:
Hexagon wrote:
368 ninja*
138 ninjasupeatsninja
(people abbreviate my name too often)
I'd like to thank my friends, my family, my college, my high school, my middle school, my grade school, the people who randomly talk to me, and myself.
or..ya know.. people talk about red ninja, regular ninja, and ninjas are cool to talk about in general.
Don't get a swelled head.

divide by 3 or 2
(368 / 3) + 128 = 250

Xfrogman43 · 2015-09-22 03:38:24

ninjasupeatsninja wrote:

Evilbunny wrote:
ninjasupeatsninja wrote:
Hexagon wrote:
368 ninja*
138 ninjasupeatsninja
(people abbreviate my name too often)
I'd like to thank my friends, my family, my college, my high school, my middle school, my grade school, the people who randomly talk to me, and myself.
or..ya know.. people talk about red ninja, regular ninja, and ninjas are cool to talk about in general.
Don't get a swelled head.
divide by 3 or 2
(368 / 3) + 128 = 250

Congrats... You can do math.
2+2 = fishbole

Krosis, some woman, Pingohits

SirJosh3917 · 2015-09-22 03:40:25

Xfrogman43 wrote:

[snip]
Congrats... You can do math.
2+2 = fishbole

^i never thought of that. infact you're right.

also, the point was that even assuming 2/3 of the ee chat's messages are directed towards the actual player ninja, or just a word in general, about 250 chat messages were me.

hummerz5 · 2015-09-22 13:37:03

Well, we cannot assume any distribution of "ninja" without having Hexagon's data and looking through each instance.

We could probably filter out "[color ] ninja" pretty easily. Otherwise you could be 1/100th of the set, or 9/8ths. (ha fractions)

SirJosh3917

BEE · 2015-09-23 11:47:20

There are also a lot of usernames that contain the word Ninja that users could be shortening it to to refer to their friends.

Hexagon · 2015-09-23 12:13:29

From August 1, 2015 to August 31, 2015 (approximate):

- "ninja" (case insensitive) was mentioned 4363 times, including parts of words (such as "ninja7" for example)
- out of those times, "ninjasupeatsninja" was mentioned 460 times

- out of the times "ninja" was mentioned, 2716 times "ninja" was mentioned, excluding the words "orange", "red", "purple", "green", "pink", "yellow", "smiley", "bot", "grey", "gray", "black"

We'll see if we can get some more precise statistics to help clarify things.

Stagecrew, some woman, Pingohits, Krosis

Official Everybody Edits Forums

#51 2015-04-29 00:55:35

Re: [September 21 2015] Statistics from EE Analytics

#52 2015-04-29 01:10:11, last edited by Hexagon (2015-04-29 01:27:52)

Re: [September 21 2015] Statistics from EE Analytics

#53 2015-04-29 01:14:46

Re: [September 21 2015] Statistics from EE Analytics

#54 2015-04-29 01:19:08

Re: [September 21 2015] Statistics from EE Analytics

#55 2015-04-29 01:25:34, last edited by Hexagon (2015-04-29 01:31:52)

Re: [September 21 2015] Statistics from EE Analytics

#56 2015-04-29 08:19:26

Re: [September 21 2015] Statistics from EE Analytics

#57 2015-07-11 21:59:44

Re: [September 21 2015] Statistics from EE Analytics

#58 2015-07-11 22:07:23

Re: [September 21 2015] Statistics from EE Analytics

#59 2015-07-11 23:21:05

Re: [September 21 2015] Statistics from EE Analytics

#60 2015-07-12 00:30:19

Re: [September 21 2015] Statistics from EE Analytics

#61 2015-07-14 17:42:57

Re: [September 21 2015] Statistics from EE Analytics

#62 2015-09-21 13:47:22, last edited by Hexagon (2015-09-21 14:28:54)

Re: [September 21 2015] Statistics from EE Analytics

#63 2015-09-21 23:47:16

Re: [September 21 2015] Statistics from EE Analytics

#64 2015-09-22 00:02:30

Re: [September 21 2015] Statistics from EE Analytics

#65 2015-09-22 00:44:30

Re: [September 21 2015] Statistics from EE Analytics

#66 2015-09-22 02:19:26

Re: [September 21 2015] Statistics from EE Analytics

#67 2015-09-22 03:38:24

Re: [September 21 2015] Statistics from EE Analytics

#68 2015-09-22 03:40:25

Re: [September 21 2015] Statistics from EE Analytics

#69 2015-09-22 13:37:03

Re: [September 21 2015] Statistics from EE Analytics

#70 2015-09-23 11:47:20

Re: [September 21 2015] Statistics from EE Analytics

#71 2015-09-23 12:13:29, last edited by Hexagon (2015-09-23 12:17:44)

Re: [September 21 2015] Statistics from EE Analytics

Board footer