[RELEASE] Scarlet - EE & EEU Minimap Viewer

SirJosh3917 · 2018-06-03 00:28:42

Scarlet

A modern, fast, EE & EEU Minimap API

Source Code | API Reference

RlYHrU3pKjbO Zvrv7TY3bBkI?scale=2 Xd4AzWRkWzSy?scale=2 PW5m6ANVwRcEI PWzvK4_8KVb0I

Scarlet is the API that powers minimaps on the forums. It can provide both EE and EEU minimaps of worlds.

Features

JSON endpoint exposing a list of colors for EE & EEU
Fast, custom PNG serializer

Todo Checklist:

Don't show minimaps for worlds that have minimap disabled
Don't show minimaps for worlds that have visible false

Support development by supporting me on Patreon

▼Older Post

hummerz5, drunkbnu, MartenM, TaskManager, RailMat, Vinyl Melody, lrussell, Loganyoshi, coinage, 2b55b5g, John, Zumza, Piotrek, Edilights, Blackmask, Gabriellfs, Victoria, Snowester

hummerz5 · 2018-06-03 01:14:28

Cool!

ninjasupeatsninja wrote:

Does it respect worlds with MinimapEnabled set to false?

does it respect worlds with /visible false?

Xfrogman43, RailMat, IanJanes

icepegasus · 2018-06-03 05:54:51

hummerz5 wrote:

does it respect worlds with /visible false?

I don't think that's an issue. People with visible false don't want other players joining their world. They don't care about minimap (that's how I see it)

SirJosh3917 · 2018-06-03 06:51:18

Let's settle it with a poll.

Straw poll/

It ends on June 9th 12:00 AM UTC time.

If anybody knows of any good strawpoll alternatives, let me know.

The API will, for now, be built to respect both 'MinimapEnabled' and 'Visible' until the poll results come in.

LukeM · 2018-06-03 09:07:28

I guess visible false could sometimes be used to keep worlds a secret until they are released, so might be best not to show the minimap if it's not visible.

TaskManager, hummerz5, Grilyon2

capasha · 2018-06-03 13:00:55

Looks good. Maybe you should try to make something similar in JavaScript.
LukeM know how to code in JavaScript maybe he can help you with databases?

LukeM · 2018-06-03 13:47:23

capasha wrote:

LukeM know how to code in JavaScript maybe he can help you with databases

No idea how I've managed to completely avoid such a huge part of online programming but I've never actually used a database XD

Edit: Also what do you mean by make it using javascript? Make something you can run client side to generate the minimap? (This is a serverside program so it shouldn't matter what language it is written in, you can make a request from pretty much any language)

hummerz5

capasha · 2018-06-03 14:07:21

LukeM wrote:

capasha wrote:
LukeM know how to code in JavaScript maybe he can help you with databases
No idea how I've managed to completely avoid such a huge part of online programming but I've never actually used a database XD
Edit: Also what do you mean by make it using javascript? Make something you can run client side to generate the minimap? (This is a serverside program so it shouldn't matter what language it is written in, you can make a request from pretty much any language)

Hasn't you made bot's before in JavaScript? Shouldn't be that hard to use database objects that way too if you are good at JavaScript.
Atilla have had so many issues to run as example mono programs. So I guess it would happen the same for ninja.

SirJosh3917 · 2018-06-03 14:08:51

capasha wrote:

Looks good. Maybe you should try to make something similar in JavaScript.

Already have. You can use jsparse to feed it some "init" data, or just load from the BigDB, deserialize that, and make a minimap as well.
Not sure how that went unnoticed

capasha wrote:

LukeM know how to code in JavaScript maybe he can help you with databases?

All I need to do is parse the BigDB, and atilla's World library does that just fine. It'd just need to be "ported" to javascript.

LukeM

SirJosh3917 · 2018-06-03 16:15:13

Open sourced the inner MinimapViewer API.

This is what the official API will use internally to get and fetch minimaps.
I rewrote the first MinimapViewer and open sourced it in case anybody wants to make their own custom alternatives.
Please review the source code and suggest improvements
The code in the github repo will go into effect on June 9th. Expect that by June 10th UTC time, the official API will use these improvements

LukeM, Gosha

SirJosh3917 · 2018-06-12 11:14:32

The MinimapViewer API is now running the updated version of the code.

New link shorteners: minimap.sirjosh3917.com and mm.sirjosh3917.com
If you like the MinimapViewer API, please woot the suggestion topic for it

The official API is running off of this code right here
There are a few new features:

Feature Note:
You can append a ?format=x to the end of a /metadata or /worlddata request to get it in a different format. Supports json, xml, csv, defaults to a custom (awful) format, plaintext.

Old links to /world/PW01 as images will still work, though it's recommended to update them.

SirJosh3917 · 2018-06-12 18:09:30

MinimapViewer Updated & Fixed

New (left) v.s. Old (right)

Minimaps now handle transparency and background colors properly.
See the commit for more details.

The API will be back up in under 20 minutes

SirJosh3917 · 2018-06-12 20:07:37

New Update

More detailed error messages
See the Settings.CS file for detailed information on what errors look like

{"Error": "The world does not exist in the BigDB (or the world was unable to be loaded). Please check that you spelled the World ID correctly."}
{"Error": "World disrespects the guidelines:\r\nThe world's MinimapEnabled setting (False) doesn't match the desired MinimapEnabled (True)"}

Update will be applied in under 20 minutes, please wait for the API to restart.

SirJosh3917 · 2018-08-28 05:19:35

Expected Downtime

There will be downtime for an unspecified period of time, perhaps a long one.
I will be moving from NancyFX to ASP.NET Kestrel ( asynchronous request performance ) and on .NET Core ( new DotNetCore binaries from PlayerIO ), and this will require a major rewrite to do things properly.

SirJosh3917 · 2018-09-03 06:56:56

Laziness

To be entirely honest, I don't feel like re-coding the MinimapViewer API despite it badly needing one.
However, it has beem moved to ASP NET with Kestrel using fancy asynchronous stuff (true async, not the bullcrud nancy does) and it's mostly a hackjob of how it's up and running (caching was removed, a few other things), but at least it's up and running.

SirJosh3917 · 2018-11-24 03:18:21

Revitalization Update

New Feature: the ?scale query parameter
The ?scale parameter will scale an image 1x, 2x, or 3x it's original size.
Example: PWlAgeabQ_b0I?scale=1 PWlAgeabQ_b0I?scale=2 PWlAgeabQ_b0I?scale=3

In other news...
The API has been revitalized - it's now using ASP.NET Core properly, along with some other improvements down in the caching center which should help speed up the experience for you.

New Internal API changes!

We use ImageSharp now, which should bring some performance improvements over the old Bitmap way of doing things
A rewrite of the MinimapViewer API, which should help cleanup the old mess that it was
We use caching more than ever, and it should help with performance a lot!

As for the cache times:

A given World is cached for 3 hours

SirJosh3917 · 2019-04-29 23:57:31

Hosting Updates

Artificial Limits Increased: ?scale can now go to 4x
Example: PWlAgeabQ_b0I?scale=1 PWlAgeabQ_b0I?scale=2 PWlAgeabQ_b0I?scale=3 PWlAgeabQ_b0I?scale=4

In other news...
The hosting box has been moved to an Azure VM, providing blazing fast performance and internet speeds, as long as high costs ;-;
I'm not sure if I'll stay here but for the time being it's fast.

New Internal API changes!

LazyCache is used for caching now.
Scale incresed to 4x
Rewrite of the hosting server, but not of the minimap viewer API

As for the cache times:

A given World request is cached for 6 hours, but will be removed from the cache if not accessed for 30 minutes
A given World image is cached until it's not accessed for 5 minutes

Loganyoshi, PTU

SirJosh3917 · 2019-06-06 23:23:21

BETA Forum Release!

HUGE thanks to SmittyW for helping out create the design of the world info box and error images!

That's right, minimaps finally have come to the forums!
Please see the official announcement for information on how to use the service.

Release Information

Because this is a BETA release, expect there to be some mild bugs. If you find anything, report about it! Want something added? Suggest it!

API Modifications

+ Additional /username/<simpleid> endpoint
+ Additional /world/update endpoint
* Enhancement of /world/metadata/<worldid> endpoint
- Removal of the /world/worlddata/<worldid> endpoint

/username/<simpleid> - An endpoint to return only the username and simpleid of a user based on their simpleid.
/world/update - Endpoint for users. Wipes the cache of a world.
/world/metadata/<worldid> - Additional UserName key, which can be null. If it's not null, loading the /username/<simpleid> endpoint with the worlds Owner simpleid will set it to the username of the owner.

Caching

The MinimapVIewer API is notorious for caching heavily, and in this latest release, there are layers upon layers of caching.
In order to clear the cache of a world, head on over to http://api.sirjosh3917.com/world/update. Type in your World ID, click submit, press the checkmark on the recaptcha, press submit, and you'll be redirected to an image of your world. All new modifications should be shown!

Minimap requests:

1. You make a request for a minimap
2. Is it in memory? if so, return it.
3. If not, is it on disk? if so, add the file to memory, queue up a request to update the minimap, return the image.
4. If it's not on disk, request a world, create a minimap from it.

When a minimap is requested, it needs to fetch the world. A request for a world is very similar:

1. Request for a world is made.
2. Is it in memory? if so, return it.
3. If not, is it on disk? if so, add the file to memory, queue up a request to update the minimap, return the image.
4. If it's not on disk, download it from PlayerIO

Thank you for your time!

SirJosh3917 · 2020-03-16 08:06:09

Scarlet - Revamped EE & EEU Minimap Viewer API

Release Information

This release, EEU minimaps are officially supported and available, along with some new endpoints to assimilate the EE & EEU endpoints. The entire API got an entire rewrite, along with a new shiny name - say hello to Scarlet!

This release has three primary themes:

EEU support
New endpoints
Rewritten & Faster

EEU Support
With EEU minimaps being fully supported and official, the only item on the checkbox is to finally introduce the eeuworld BBCode tag to the forums. In the meantime, please use the following URL for EEU minimaps:

https://mmu.sirjosh3917.com/<WorldId>
vqqG31PYbREv?scale=2

New Endpoints
The endpoints are completely revamped, and make sense now! Check out https://api.sirjosh3917.com/ for the truth on the endpoints at any time.
To highlight one of the important endpoint modifications, the list of colors that Scarlet uses, publicly accessible through an endpoint! Check out the colors for EE or EEU!

Rewritten & Faster
Scarlet is a completely new minimap generator for the forums, written from the ground up. It's completely open source and available on Github.

If you like what you see, consider supporting me on Patreon.

API Modifications

+ https://api.sirjosh3917.com/ now provides basic information on routes. Please refer to it as the primary source for routing!
* Updating/refreshing a world no longer requires a captcha solve.
- Deprecation of /bigdb, /username, and /world endpoints. Please refrain from using them. They will be removed.

Technical Modifications

Scarlet comes with some new technical changes as to how caching works.

Caching Every world download & image generation is now cached for a flat 6 hours. There is no in memory caching.
PNG Serialization is entirely custom built, from the ground up. This will allow for immensely faster PNG creation. In fact, it takes a mere 4 seconds to create a 8000 by 8000 image of PWJesse!

Todo List

Introduce the eeuworld BBCode tag
Don't show minimaps for EE worlds that have visible set to false
Don't show minimaps for EE worlds that have MinimapEnabled set to false

Tomahawk

peace · 2020-03-16 11:27:43

oh this is a thing is this tha api you made for the ofurms to make EEworld tag work and oyu just update dti to work better/add EEU suport? nice goign bro

Official Everybody Edits Forums

#1 2018-06-03 00:28:42, last edited by SirJosh3917 (2020-03-16 07:43:10)