*ominous drumroll* image/file upload?

[decentral1se]

What is the current cerca party line on image/file upload? 🙃

I would personally support some simple image/file upload functionality.

As a temporary workaround for lack of image upload functionality, comrade crunk has made this. It's great, it works for now.

[cblgh]

instaneous gut response: no

i think having easy to use auxiliary services is an acceptable way to solve this though?

As a temporary workaround for lack of image upload functionality, comrade crunk has made this. It's great, it works for now.

yes exactly this kind of solution, i think this is fantastic :)

long form response for the interested below the line

---

what it boils down from my pov:

1. wanting to strictly limit the amount of complexity, which affects maintenance and how easy it is to contribute
2. images are auxiliary to forum discussions (barring photography forums :) and add a nice flavor but
3. images monotonically affect storage capacity several magnitudes more than text

re 3: the merveilles forum is now 3 years old or something, and the database is still only ~1MB. this is a trivial size! i can commit to its uptime and database growth for a very long time. with images, this changes storage from a non-concern into an actual concern. the problem is compounded by there being no indicator for users of how they are affecting ops concerns—"i'm just uploading a fun image :)". i think mastodon servers are an excellent example of how this can fail wrt storage naiveté and long-term resilience

say you implement resizing of images, and a upload counter to let users know how many images they have uploaded. cool! however that's introduced a significant amount of additional deps and code for something auxiliary and which can be solved in alternative ways, as pmc demonstrated. i also think adding even just the tiniest bit of friction (haugen mentions in her testimony that internal fb research has shown that having more clicks decreases measures of addiction and hate speech) by having an auxiliary service is actually something good.

we've had people using images in posts in the merv forum since the start, ranging from uploading to the masto server to people uploading to their own sites. in some sense this is "externalizing the cost" but not really? i am aware of the images i host on my own website e.g. and for the mastodon instance well, we're all more or less paying for its ongoing expenses. which are currently 250 USD/m and on a personal level, i think this is kind of an insane amount of money to be spending; i don't want my software to empty other people's wallets, i want it to remain useful, resilient, and possible to run and maintain for a long time :)

side note: i had my eyes on a nice go-based single binary tool for providing this functionality in a way that is external to the forum database, but i can't find it now ;_;. edit: i think i found it! https://github.com/mtlynch/picoshare

[cblgh]

i'm such a curmudgeon maintainer lmao, but i promise i have good reasons i don't mean to be cranky foss maintainer X)

[cblgh]

thinking about this a bit more: any auxiliary service will have you either interoping with cerca (mmmm dunno) or linking out to some other place. so, one very low hanging fruit we can pluck to simplify the experience for posters, and open up for other unexpected possibilities, is to add a custom markdown string just below the posting textarea element. wdyt?

[decentral1se]

haha no worries. i must say i enjoyed writing this as i had guessed it would provoke all sorts of wringing of hands and scrunching of eyebrows 😂 your replies did not disappoint 💘

absolutely love the 3 years 1 MB metric. incredible stuff.

i have no answers on this. some reflections however...

i don't think the whole masto storage / cost data point is so relevant here because cerca does not federate.

i'd be curious if you could somehow comb the image links in 3 years on the merveilles forum to gauge what the actual storage would be? ofc this quantity of storage reflects not providing an upload function but still, it is an indication.

the whole aux service thing is at odds with the whole "cerca is all you need" low maintenance thing. 2 vs 1 thing to manage is real.

trying to "bless" some aux service is a bit risky and might lead to more maintenance back and forth depending on whatever that project does in the future.

i have the feeling that probably some further discussion to expand on the limits and look at minimal solutions might offer something good in cerca itself.

i'd love to hear from more folx on this. is there already a merveilles thread? 🙃

[cblgh]

omg what are these long replies hiding inside me

side note

something interesting re the linked file-hosting projects is their scope of ~400-ish commits. one even restarted due to lacking foresight wrt architectural concerns. uploads are just a completely different and difficult problem space :)

one example problem brought by the change in domain being there is less (legally) cursed textual content your machine may host compared to the types of images a problematic user could upload to a machine legally tied to you

another is just - can i live with writing software that can be potentially and directly used, against my will and unbeknownst to me, to spread fucked up shit like CSAM?

end side note

i'd be curious if you could somehow comb the image links in 3 years on the merveilles forum to gauge what the actual storage would be?

a shabby sqlite query gives me 281 images:

   select count(*) from posts where content like "%![%](%)%";
   281

given images at...

• (tiny end) 5kb per piece => +1.4MB
• (low end) 50kB per piece => +14MB
• (mid end) 500kB per piece => +140MB
• (high end) 5MB per piece => +1400MB

a plausible range for sizes could be 150kB-750kB as the middle peak of a normal distribution—some accidentally uploading huge images and others having smol ones ready at hand, but most residing in this mid-range.

with an actual upload facility and affordances i'd guesstimate a magnitude of difference compared to my numbers. especially lacking any storage visibility / cap for users. so, 1.4GB for the mid-end image size given current usage and three years use. factor in multiple naive backups and you start getting into the 10s of GBs given my prior assumptions. a far cry from masto's ridiculous requirements, but also a far cry from cerca's current measly 1MB floppy-level backups X)

the whole aux service thing is at odds with the whole "cerca is all you need" low maintenance thing. 2 vs 1 thing to manage is real.
trying to "bless" some aux service is a bit risky and might lead to more maintenance back and forth depending on whatever that project does in the future.

yes / no? simply linking to an external service, perhaps provided by the same maintainer as the forum, perhaps by someone else in the community, seems low risk. if it breaks it breaks—change/remove the link. it wouldn't affect forum deployment in any way, which is where i see a more significant maintenance risk, personally: interdependent systems that must be fully operational at the same time for the main system to run at all.

but i hear you, running two different systems with different and unrelated development timelines can be a real concern. tho imo if you can have a self-contained binary that keeps on chugging and servicing needs idk, less of a problem from my pov

i have the feeling that probably some further discussion to expand on the limits and look at minimal solutions might offer something good in cerca itself.

ya i think you are probably right on this! the recent collab with @mhfowler on expanding lieu's ascetic limits was quite fruitful and a similar vibe to your request :) the situation for cerca and images is something where it will need to stew for an extended period, doing without email and doing without images were two foundational design choices of my approach to long-term resilience.

thanks for poking and prodding me on this :] gentle nudges like this can be really good

i'd love to hear from more folx on this. is there already a merveilles thread? 🙃

the closest thread that comes to mind

[decentral1se]

epic guestimations, tysm! full on 🍿 energy on this one 😁

i will continue to meditate on this.

[cblgh]

full on 🍿 energy on this one 😁

😂

[brendanhowell]

This is a great thread!

So one thing I have observed with some older web fora that only used external image services is that after a certain number of years, the images start to disappear for many reasons: updates/moves, bit rot, business merger/failure, quotas, etc. For silly jokey stuff it's not so tragic but for more illustrative images that's pretty sad. You are losing knowledge or crucial context.

How many images from the flagship forum are now 404?

But I totally follow the minimalist line of reasoning. I also love text and appreciate situations where the use of graphics is more sparing and considered. In some ways, this problem isn't an issue of the forum software so much as it's a consequence of the smart-phone interface culture that encourages quick and easy, often thoughtless, near constant blasting of snapshots all the time.

Anyway, I know it adds a giant blob of bloat and feature creep, but I also wonder if there is a compromise that would be possible - a way to slow and limit the use of images but still keep the archive intact. Off the top of my head, say scale down all images to max 640x480 dithered black and white PNG - image sizes are in the KB range there? Or don't scale at all, just reject anything big? Only one image per post? Don't allow uploads if your User-Agent is a phone?

Perhaps I'm just at the bargaining stage of image loss acceptance.

[cblgh]

So one thing I have observed with some older web fora that only used external image services is that after a certain number of years, the images start to disappear for many reasons: updates/moves, bit rot, business merger/failure, quotas, etc. For silly jokey stuff it's not so tragic but for more illustrative images that's pretty sad. You are losing knowledge or crucial context.

this agrees with my own observations! following the line of reasoning also prodded another thought: i wonder if the degraded forums we found with those "hotlinks disallowed" thumbnails or images that are just completely missing are examples of survivorship bias. as in, maybe the forums that went bananas on accepting uploads became too heavy to keep afloat after a while and simply faded away entirely, instead of slowly, but visibly and accessibly, decaying. mostly an academic / fun thought and a totally biased viewpoint ofc and anyway, as you say, the culture and context of today is different from what it was around then.

I also wonder if there is a compromise that would be possible - a way to slow and limit the use of images but still keep the archive intact. Off the top of my head, say scale down all images to max 640x480 dithered black and white PNG - image sizes are in the KB range there?

SSB has a pragmatic and strict upload limit of 5MB for binary data uploaded in tandem with and referenced in a single post. since the data is referenced, and not embedded, it can be trimmed from a user's local storage for older posts without also removing the post itself. all the while still keeping the binary data itself around in the wider distributed system ssb comprises

[cblgh]

Perhaps I'm just at the bargaining stage of image loss acceptance.

if i may inject a dose of irony

[decentral1se]

im curious how much of #95 can be achieved with the go stdlib and how this code will read. some prototyping might be good on specific points. we can abandon stuff then if it smells...

in that sense, it might be nice to understand what parts of #95 are critical in terms of non-technical concerns, e.g. safety.

if you have have code/dep smell but you're on a critical feature, then maybe this can't be done.

should we also try to document the "resilience principle" referred to in #94 (comment) (probably best from your brain @cblgh)? if the goalposts are potentially going to move, it's good to have it "in stone" where they were before the moving starts.