lobsters score: 4 · tags: web
I tried the simplest option and
Ctrl+Son Firefox let me save the whole page and also included all the ancillary files (PNG, CSS) in a folder alongside the HTML document. Opening the saved page back up with internet disabled gave me the exact same page. Kudos to the Firefox team, I had figured that it would only store the bare html file and all the CSS and images would be busted.Here on lobste it was somewhat worse as the CSS got screwy and rendered the page as white.
score 1 · canonical
I would be happy to see a project one day that gives me the ability to pitch in compute/hosting for websites for close friends and also for internet strangers if I choose to. Bitrot is really sad to me, and I want the same for me when I’m gone.
Personal backups and centralized ones like Web Archive are at opposite ends of the spectrum, and as far as I can tell, the middle is empty. DNS is a huge hurdle to overcome and I don’t know what a good solution would look like.
score 1 · canonical
The closest thing to the middle is something distributed and content-addressed like IPFS or Iroh’s recent experimentation with pkarr. Though, what you get is persistence of the content when addressed by its hash, so if you only had a human-readable DNS that was aliased to the content-addressed URL, you can still lose the content when the original host goes down.
score 1 · canonical
I was definitely thinking about IPFS, which I know at least one friend had used for this exact purpose. And Iroh seems to be interested in carrying the flag going forward. IIRC Namecoin was one of the players with a goal to replace DNS with something distributed, but they either failed to deliver that or got lost when the crypto hype died down and transferred to AI.
score 1 · canonical
DNS part would probably be something like
<original-domain>.<timespan>.<mirror-domain>.As long as this is semi-cooperative (the author wants the site to be archived, but might not be available when a new mirror gets established) and content-based, I guess some scheme of content signatures could work. Probably with chain-of-custody re-signing by archives every few months (to handle original-key compromises, including computing power advances)
But once you need a single-source-of-truth, like a non-federated wiki, I guess there is the issue that there needs to be continuity in being recognised as that main source of truth for the essence of the website. There might be multiple non-profits offering this, but sudden failure of one would be an issue for websites relying on it as a primary…
score 1 · canonical
I am super interested in a version that’s cooperative for both parties with low stakes to begin with. My blog is an example of something that doesn’t need any compute, its signature can easily be verified, and I’d like it to be accessible easily to anyone after I can’t pay for hosting or domain. I’d be willing to make this easy, like signing each page with my private key, or using a separate name resolution scheme in parallel.
With single-source-of-truth, it gets more complicated for sure. For that to work, cooperation has to be more involved, like chain-of-custody scheme you suggested; I can vet close friends whom I believe can continue running my services in my place. But I’d be happy if we can only solve the easy part today.
score 1 · canonical
Even the easy part needs chain of custody for signatures! Public key signatures produced today will probably not be trusted in twenty years. Single-source-of-truth needs something like chain-of-hand-offs, which is more complicated…
You want semi-cooperative model in the sense of not doing anything extra on handovers between different mirroring services.
The problem is still predicting what setup will get well-organised hand-overs on service shutdown (as one doesn’t want to rely on a single project being long-living, that is Internet Archive model which will hopefully work out but has specific risks). And I think mirroring DreamWidth is a lot of traffic, so you want something more fragmented, which might be hard to check for long-term vitality in advance…
score 1 · canonical
I have to agree that even my easy case doesn’t seem to be as easy as I hoped for, I didn’t consider pubkey security for the far future. So we have to crack the hard case first then, but for high traffic services it should work with federated/distributed infrastructure in theory?
score 1 · canonical
«Both copies are equally valid» is still the easy case…
I am not sure full-hard-case will be ever cracked (I could just as easily imagine a federated setup where there is a sea of claims to be relevant to some «location» however this is defined, and then there are different views, and maybe sometimes you get this https://en.wikipedia.org/wiki/Ring_species situation of the «same» chat has fully disjoint views and smooth transitions between them, depending on which moderators you trust more)
I think that with long term enough thinking at the middle scale what needs to be searched for is some kind of commitment to handovers as a praxis… and I honestly have no clue how to find out whether any small community in this direction already exists!
score 1 · canonical
I also long for an IPFS-like system, that would blend the Web Archive, personal bookmarks, personal websites, and personal backup, in a decentralized/federated offline-compatible fashion (also akin to Secure Scuttlebutt).
I tried to develop this, and kinda still try, but for a number of years now I’m balancing at the edge of burnout and unable to really develop much software in hobby time, lamentably. I tried to build this on IPFS first, since then trying on Iroh (an IPFS successor effort, which now has growing support for both DHT and Scuttlebutt-like mDNS).
Within this project’s vision, I’d like to be able to take a website I browse, and make a bookmark (possibly annotated) to it on my website (and/or a private list), which would also automatically make a WARC and/or a text&images-only (Readability-like) archive of its contents. This would then automatically become shared through p2p with the Internet Archive and a/the DHT, building a (hopefully) growing distributed backup of hand-picked parts of the web. At the same time, it’d become my offline backup and also a collection I can browse whenever I like, also offline.
With mostly the same mechanism, I’d like to be able to backup my photos, documents, PDFs, music, videos, git repos. I’d like to be able to then easily share them with friends, but also to share them with hand-picked backup locations and machines, notably including my phone. I want to be able to also manage the storage and redundancy level of my data, for every item or group of items manually selecting which backup locations store them, or on which ones I delete them (with some grace period protecting from accidental deletion).
If somebody’s interested in working together, somehow, on this, I’m more than happy to be contacted. Please note it’s glacially slow on my end, though, and I can’t promise any reasonable speed of co-working; also I’m rejecting any vibecoding for this (though I won’t stop anyone from trying it this way; but in such case please do not contact me about it).
score 2 · canonical
I save interesting pages as PDF. When the CSS has no proper print setup then I save the Read View in Firefox. Sometimes you get very pleasent PDF that way.
score 1 · canonical
I try to minimise the amount of text I read on websites without saving that to a local file first… (with HTML-to-text and then text goes into a text editor)
score 1 · canonical
That’s exactly what Offpunk is doing for the last 5 years.
You can even use “netcache” directely without using the whole offpunk.
score 1 · canonical
I have some cronjobs pushing things to Archive.org and hope they stay with us.
Otherwise prefer SingleFile over PDF any day.
score 1 · canonical
I started snapshotting pretty much every article/website I find interesting with monolith about a year ago.
If anyone is interested this is the script I use - https://gist.github.com/mtmn/f415c0660ffb9b8a5bf89429af31b8e4
It’s a bit wonky as it relies on wayland and vimium (yanks the active url) but works really well.
score 1 · canonical
Yeah, using WARC files and WARC-related tools and apps is something I’ve been meaning to try out. But, of course have not done it so far.
What I used to do for years was simply print out a webpage to PDF…but I changed my approach in maybe last 6 or 8 months ago, where I simply copy/paste the contents of a webpage into an ODT (open docs text) file - basically the equivalent of a Word doc/.doc/.docx…Why? Because in a word processor doc i can add my own comments, maybe notes, basic annotations, etc. I didn’t like the experience of annotation in PDF tools…but this “archive” method of mine is only worth doing for web pages whose content is simple text, code, maybe some simple imagery…does not work if the content includes videos of course, and some complex tabular data chunks weird sometimes (not always, but sometimes). While my approach is far from perfect, its actually worked out pretty well for me. Usually my goal in archiving stuff is merely to get the main corpus of the content, and if style/css or some interactive feature is missing that’s ok for me the vast majority of the time…Using an open format like ODT helps with longevity (theoretically) to be able to access/open the content far off into the future. I guess, PDF exports do tend to be smaller filesize than word processor docs, which can be a benefit of PDF tools; sometimes PDFs are like half the size of word proc. files…so my approach may not be ideal if storage space is very limited. I suppose also that my approach is very manual, so other methods like PDF exports or maybe WARC tools probably save time. Nevertheless, my “archive (simple) web content into a word processor file” has been working great for me!
Though, I’d still like to someday play with WARC stuff…but who has the time? :-)
score 1 · canonical
This is obviously very different from making my own backup, but I have a habit of archiving pages on the Internet Archive whenever I like them. They have an official browser extension that I use that makes it super easy to archive the page (or view an archived version).
score 3 · canonical
My introduction to online communities was a forum in 2012 named TheIKingsSGC (StarGate Coalition). It was essentially a Doctor Who and MineCraft crossover community centered around a game TheIKing himself was building (well there were several but by that point only the Doctor Who one had serious momentum).
I developed my first art skills on this forum, I met another girl who later became my roommate in college, and I gained many learning experiences with social interaction. I also had my first experience collaborating creatively with other artists and programmers.
Unfortunately the website is long dead. After activity flatlined, TheIKing would very occasionally pay the bill at random times over a few years, but I don’t think it’s ever been online since like 2019 or so. Afaik, he probably still has his personal backups or the content, but I only hold a few HTML/CSS archives of my personal blog posts. Between that and Skype, it’s very sad how ephemeral parts of my childhood turned out to be.
score 1 · canonical