Caching (Store board state locally in localStorage)

[EliDoris]

Updated Boot.js to do caching in localStorage. This is basically the implementation that penpa uses, except:

1. I used the encodings already available from the functionality that duplicates the board and the functionality that correlates URL to puzzle. It creates a new URL that incorporates the whole board state then correlates the "search" part of the URL with the "search" part of the base URL.
2. It does not fire on the beforeunload browser event. I noticed that the visibilitychange event was firing at pretty much any time I could imagine wanting to cache, and upon further investigation it seems beforeunload is only really recommended when there is a desire to bring up a dialog and allow the user to cancel their decision to leave

This was tested and works perfectly on Chrome Version 142.0.7444.163 (Official Build) (64-bit). I don't really know what the best way is to go about testing on other browsers, but the code as written should be able to discern if there is no localStorage available and simply not do the caching in that case.

I also tested it for a simple loop up to 75x75, which has a pretty long encoding, and it was fine. Upon inspection, it looks like this fairly extreme case gives me ~67,000 characters in the URL. The base URL is considerably shorter, so this encoding is like <0.2 MB with a high estimate. That means with a localStorage cap of ~5MB for most browsers, there is very unlikely to be a real single puzzle that messes with localStorage. Right now if anything goes wrong with the cache write, all that really happens is that the caching doesn't happen

Notes on implementation:

1. There isn't really a great way to give users an option to cache or not cache, since it seems that all of the persistent options available (e.g., yajilin styling) are encoded directly in URLs. That said, I don't believe penpa gives you this option either and that seems to work fine with almost exactly this implementation

2. Space in localStorage could be very considerably saved with the incorporation of a couple of libraries, but I could not figure out how to do that in ecma v5. md5 would allow the keys to be hashed instead of raw, reducing them to 128 bits instead of a ton of URL characters (which I think are 8-bit encoded). This would be a gigantic saving for each key, and is how penpa does it. lz-string would allow the URLs to be tremendously compressed, since they're VERY compressible (long runs of the same character, many repeated sequences)

[vercel]

@EliDoris is attempting to deploy a commit to the robx's projects Team on Vercel.

A member of the Team first needs to authorize it.

[EliDoris]

Following up on the issue of localStorage becoming full:

I did encounter some people complaining that penpa stops caching for them after a while, requiring localStorage to be deleted. This is pretty unintuitive for most users and is definitely too much to ask of random people, so it would be good to come up with a way to address this.

It seems that all of the different browsers implement different behavior when the localStorage limit is reached, so it would be best to avoid altogether.

A heavy-handed but workable solution would be to just clear the site's localStorage for all puzzles except for the one currently being solved whenever the current puzzle is cached. This would avoid the long-term issue but would mean that if someone is solving multiple puzzles at once (for some reason) then the caching wouldn't work correctly for them

[x-sheep]

Is it possible to add a timestamp to each entry in localStorage, so we can just remove the oldest entries if the storage is full?

[x-sheep]

Could you move this function and store the result in env.js? It's where the list of browser capabilities is currently kept.

[EliDoris]

This has been moved

[EliDoris]

Did an implementation where things are timestamped and if an error occurs where max storage is exceeded it does the following:

1. Pull all keys from localStorage that correspond to puzzles (so excluding UI preferences)
2. Sort those by time
3. Delete puzzle caches oldest-first until storage succeeds or there are no puzzles left (which would happen if there's like a 1000x1000 puzzle that's just not storeable)

My only worry with this is how well this would perform in the practical situation where there's actually hundreds or thousands of puzzles cached when the limit is reached. If the localStorage queries are slow, then doing things this way will be slow each time. With the methodology of just deleting the oldest first, it means that a user who runs into this limit will then run into again nearly every puzzle. If it's fast, this won't be an issue though.

If the above does happen, an alternative would be to just clear puzzles until a fixed amount of memory in localStorage is freed, something like 1MB worth of characters could work to only occasionally trigger memory clearing.

[EliDoris]

I tested this by adding 1,700,000 zeros to each storage item. My browser can handle ~5,100,000 characters total, so this let me store three puzzles before running into the quota exceeded error. I verified that in this case when I went to cache a new puzzle, the least recent one had its item removed (by literally just opening up 4 puzzles and monitoring the behavior upon switching tabs)

[EliDoris]

My only worry with this is how well this would perform in the practical situation where there's actually hundreds or thousands of puzzles cached when the limit is reached. If the localStorage queries are slow, then doing things this way will be slow each time. With the methodology of just deleting the oldest first, it means that a user who runs into this limit will then run into again nearly every puzzle. If it's fast, this won't be an issue though.

Ok I found a thread on stackoverflow that discusses this. https://stackoverflow.com/questions/8074218/speed-cost-of-localstorage

It definitely looks like this will not be noticeable, especially since the action takes place only when navigating away from the window