[Proposed Feature] viewer for application/json files

[jedsundwall]

We should provide a preview for all .json files. The default setting should be to create a human-readable representation of the json data. There should be a tab or menu option to display a pretty-printed raw representation of the data. If the .json file advertises itself as STAC and is valid, it should display a STAC-specific preview such as that shown in #200.

[severo]

generic json

If you want, I can work on a simple viewer for generic JSON, something like what hyperparam provides, but a bit simpler, only for JSON, and with light/dark themes adapted to source.coop.

Possibly, we should first check the size of the file, and download it only below some threshold (there is no easy way to stream a generic JSON, you have to download it, then parse and show). Some JSON files are very big, ie for AI, see: https://huggingface.co/datasets/facebook/meta-active-reading/tree/main.

jsonl

Big JSON files are generally formatted as JSONL (linebreak separated JSON that can be streamed), not JSON, and have the .jsonl extension (there is no MIME TYPE yet, see wardi/jsonlines#19). Arethere JSONL files on source.coop and do we want to view them? how?

stac

If the .json file advertises itself as STAC

what is the way to advertise itself as STAC?

geojson

It makes me think that we might also want a viewer for GeoJSON (if the Content Type field is application/geo+json). As a side-note: I just found https://github.com/radiantearth/stac-spec/blob/master/best-practices.md#common-media-types-in-stac, I link it as it gives a good overview of the MIME types we might want to support in source.coop.

[jedsundwall]

Wow! I'd never seen that best practices doc. It includes a lot of useful thinking on this, and I agree that its list of MIME types is with considering.

Yes to a GeoJSON viewer for sure.

I don't know if we have any JSONL files yet, but I imagine we will at some point. At this point, I'd prioritize generic JSON and GeoJSON.

And a JSON file advertises itself as STAC simply by having a stac_version field.

[severo]

simply by having a stac_version field

OK, 'simply', but you have to download it to know that it's a STAC JSON

Is there any specific extension, mime type, or other metadata that we could use in the source.coop database?

[jedsundwall]

Yes, "simply!" 😂

Our database is intentionally unaware of information about individual objects (we're already hosting over 400 million objects and we want to avoid creating a database about all of them). My thinking has been that we would have to load the json file (this only works for files under a certain size) and determine the rendering logic based on what we can see about it.

This is similar to what we'll need to do for our TIFF viewer (#203) in order to handle basic TIFFs and COGs – the difference being that you can stream the head of the TIFF file to get that information.

Perhaps look at what @m-mohr has come up with on #200 for how he handles basic JSON and STAC?

[m-mohr]

Is there any specific extension, mime type, or other metadata that we could use in the source.coop database?

No.

Perhaps look at what @m-mohr has come up with on #200 for how he handles basic JSON and STAC?

My implementation is pretty dumb:

1. Fetch JSON
2. if stac_version is present:
   1. If type is collection or catalog => Show Catalog component
   2. If type if Feature => Show Item component
3. else: show the ObjectTree component (i.e. somewhat render JSON as fallback, this was never meant to be a full-featured JSON Viewer, remember I'm just bundling existing components from another project here.)

Before the else you could potentially also check whether it's a GeoJSON through type being one of the allowed GeoJSON types. GeoJSON also often gets served as application/json with .json extension. And STAC can bei any of GeoJSON or JSON.