Journal tags: archive

Speaking of serendipity, not long after I wrote about making a static archive of The Session for people to download and share, I came across a piece by Alex Chan about using static websites for tiny archives.

The use-case is slightly different—this is about personal archives, like paperwork, screenshots, and bookmarks. But we both came up with the same process:

I’m deliberately going low-scale, low-tech. There’s no web server, no build system, no dependencies, and no JavaScript frameworks.

And we share the same hope:

Because this system has no moving parts, and it’s just files on a disk, I hope it will last a long time.

You should read the whole thing, where Alex describes all the other approaches they took before settling on plain ol’ HTML files in a folder:

HTML is low maintenance, it’s flexible, and it’s not going anywhere. It’s the foundation of the entire web, and pretty much every modern computer has a web browser that can render HTML pages. These files will be usable for a very long time – probably decades, if not more.

I’m enjoying this approach, so I’m going to keep using it. What I particularly like is that the maintenance burden has been essentially zero – once I set up the initial site structure, I haven’t had to do anything to keep it working.

They also talk about digital preservation:

I’d love to see static websites get more use as a preservation tool.

I concur! And it’s particularly interesting for Alex to be making this observation in the context of working with the Flickr foundation. That’s where they’re experimenting with the concept of a data lifeboat

What should we do when a digital service sinks?

This is something that George spoke about at the final dConstruct in 2022. You can listen to the talk on the dConstruct archive.

When I went up to London for the State of the Browser conference last month, I shared the train journey with Remy.

I always like getting together with Remy. We usually end up discussing sci-fi books we’re reading, commiserating with one another about conference-organising, discussing the minutiae of browser APIs, or talking about the big-picture vision of the World Wide Web.

On this train ride we ended up talking about the march of time and how death comes for us all …and our websites.

Take The Session, for example. It’s been running for two and a half decades in one form or another. I plan to keep it running for many more decades to come. But I’m the weak link in that plan.

If I get hit by a bus tomorrow, The Session will keep running. The hosting is paid up for a while. The domain name is registered for as long as possible. But inevitably things will need to be updated. Even if no new features get added to the site, someone’s got to install updates to keep the underlying software safe and secure.

Remy and I discussed the long-term prospects for widening out the admin work to more people. But we also discussed smaller steps I could take in the meantime.

Like, there’s the actual content of the website. Now, I currently share exports from the database every week in JSON, CSV, and SQLite. That’s good. But you need to be tech nerd to do anything useful with that data.

The more I talked about it with Remy, the more I realised that HTML would be the most useful format for the most people.

There’s a cute acronym in the world of digital preservation: LOCKSS. Lots Of Copies Keep Stuff Safe. If there were multiple copies of The Session’s content out there in the world, then I’d have a nice little insurance policy against some future catastrophe befalling the live site.

With the seed of the idea planted in my head, I waited until I had some time to dive in and see if this was doable.

Fortunately I had plenty of opportunity to do just that on some other train rides. When I was in Spain and France recently, I spent hours and hours on trains. For some reason, I find train journeys very conducive to coding, especially if you don’t need an internet connection.

By the time I was back home, the code was done. Here’s the result:

The Session archive: a static copy of the content on thesession.org.

If you want to grab a copy for yourself, go ahead and download this .zip file. Be warned that it’s quite large! The .zip file is over two gigabytes in size and the unzipped collection of web pages is almost ten gigabytes. I plan to update the content every week or so.

I’ve put a copy up on Netlify and I’m serving it from the subdomain archive.thesession.org if you want to check out the results without downloading the whole thing.

Because this is a collection of static files, there’s no search. But you can use your browser’s “Find in Page” feature to search within the (very long) index pages of each section of the site.

You don’t need to a web server to click around between the pages: they should all work straight from your file system. Double-clicking any HTML file should give a starting point.

I wanted to reduce the dependencies on each page to as close to zero as I could. All the CSS is embedded in the the page. Likewise with most of the JavaScript (you’ll still need an internet connection to get audio playback and dynamic maps). This keeps the individual pages nice and self-contained. That means they can be shared around (as an email attachment, for example).

I’ve shared this project with the community on The Session and people are into it. If nothing else, it could be handy to have an offline copy of the site’s content on your hard drive for those situations when you can’t access the site itself.

This Is My Jam was a lovely website. Created by Hannah and Matt in 2011, it ran until 2015, at which point they had to shut it down. But they made sure to shut it down with care and consideration.

In many ways, This Is My Jam was the antithesis of the prevailing Silicon Valley mindset. Instead of valuing growth and scale above all else, it was deliberately thoughtful. Rather than “maximising engagement”, it asked you to slow down and just share one thing: what piece of music are you really into right now? It was up to you to decide whether “right now” meant this year, this month, this week, or this day.

I used to post songs there sporadically. Here’s a round-up of the twelve songs I posted in 2013. There was always some reason for posting a particular piece of music.

I was reminded of This Is My Jam recently when I logged into Spotify (not something I do that often). As part of the site’s shutdown, you could export all your jams into a Spotify playlist. Here’s mine.

Listening back to these 50 songs all these years later gave me the warm fuzzies.

dConstruct 2022 was great fun. It was also the last ever dConstruct.

If you were there, and you’d like to re-live the magic, the audio from the talks is now available on the dConstruct Archive. Here they are:

• George Oates
• Lauren Beukes
• Daniel Burka
• Sarah Angliss
• Matt Webb
• Seb Lee-Delisle
• Anil Seth

Thanks to some service worker magic, you can select any of those talks for offline listening later.

The audio is also available on Huffduffer on the dConstruct Huffduffer account. Here’s the RSS feed that you can pop into your podcast software of choice.

If you’re more of a visual person, you can watch videos of the slides synced with the audio. They’ve all got captions too (good ones, not just automatically generated).

So have a listen in whichever way you prefer.

Now that I’ve added the audio from the last dConstruct to the dConstruct archive, it feels like the closing scene of Raiders Of The Lost Ark. Roll credits.

Eleven years ago, I made a prediction:

The original URL for this prediction (www.longbets.org/601) will no longer be available in eleven years.

One year later, Matt called me on it and the prediction officially became a bet:

We’re playing for $1000. If I win, that money goes to the Bletchley Park Trust. If Matt wins, it goes to The Internet Archive.

I’m very happy to lose this bet.

When I made the original prediction eleven years ago that a URL on the longbets.org site would no longer be available, I did so in a spirit of mischief—it was a deliberately meta move. But it was also informed by a genuine feeling of pessimism around the longevity of links on the web. While that pessimism was misplaced in this case, it was informed by data.

The lifetime of a URL on the web remains shockingly short. What I think has changed in the intervening years is that people may have become more accustomed to the situation. People used to say “once something is online it’s there forever!”, which infuriated me because the real problem is the exact opposite: if you put something online, you have to put in real effort to keep it online. After all, we don’t really buy domain names; we just rent them. And if you publish on somebody else’s domain, you’re at their mercy: Geocities, MySpace, Facebook, Medium, Twitter.

These days my view towards the longevity of online content has landed somewhere in the middle of the two dangers. There’s a kind of Murphy’s Law around data online: anything that you hope will stick around will probably disappear and anything that you hope will disappear will probably stick around.

One huge change in the last eleven years that I didn’t anticipate is the migration of websites to HTTPS. The original URL of the prediction used HTTP. I’m glad to see that original URL now redirects to a more secure protocol. Just like most of the World Wide Web. I think we can thank Let’s Encrypt for that. But I think we can also thank Edward Snowden. We are no longer as innocent as we were eleven years ago.

I think if I could tell my past self that most of the web would using HTTPS by 2022, my past self would be very surprised …’though not as surprised at discovering that time travel had also apparently been invented.

The Internet Archive has also been a game-changer for digital preservation. While it’s less than ideal that something isn’t reachable at its original URL, knowing that there’s probably a copy of the content at archive.org lessens the sting considerably. I couldn’t be happier that this fine institution is the recipient of the stakes of this bet.

When I post a link, I do it for two reasons.

First of all, it’s me pointing at something and saying “Check this out!”

Secondly, it’s a way for me to stash something away that I might want to return to. I tag all my links so when I need to find one again, I just need to think “Now what would past me have tagged it with?” Then I type the appropriate URL: adactio.com/links/tags/whatever

There are some links that I return to again and again.

Back in 2008, I linked to a document called A Few Notes on The Culture. It’s a copy of a post by Iain M Banks to a newsgroup back in 1994.

Alas, that link is dead. Linkrot, innit?

But in 2013 I linked to the same document on a different domain. That link still works even though I believe it was first published around twenty(!) years ago (view source for some pre-CSS markup nostalgia).

Anyway, A Few Notes On The Culture is a fascinating look at the world-building of Iain M Banks’s Culture novels. He talks about the in-world engineering, education, biology, and belief system of his imagined utopia. The part that sticks in my mind is when he talks about economics:

Let me state here a personal conviction that appears, right now, to be profoundly unfashionable; which is that a planned economy can be more productive - and more morally desirable - than one left to market forces.

The market is a good example of evolution in action; the try-everything-and-see-what-works approach. This might provide a perfectly morally satisfactory resource-management system so long as there was absolutely no question of any sentient creature ever being treated purely as one of those resources. The market, for all its (profoundly inelegant) complexities, remains a crude and essentially blind system, and is — without the sort of drastic amendments liable to cripple the economic efficacy which is its greatest claimed asset — intrinsically incapable of distinguishing between simple non-use of matter resulting from processal superfluity and the acute, prolonged and wide-spread suffering of conscious beings.

It is, arguably, in the elevation of this profoundly mechanistic (and in that sense perversely innocent) system to a position above all other moral, philosophical and political values and considerations that humankind displays most convincingly both its present intellectual immaturity and — through grossly pursued selfishness rather than the applied hatred of others — a kind of synthetic evil.

Those three paragraphs might be the most succinct critique of unfettered capitalism I’ve come across. The invisible hand as a paperclip maximiser.

Like I said, it’s a fascinating document. In fact I realised that I should probably store a copy of it for myself.

I have a section of my site called “extras” where I dump miscellaneous stuff. Most of it is unlinked. It’s mostly for my own benefit. That’s where I’ve put my copy of A Few Notes On The Culture.

Here’s a funny thing …for all the times that I’ve revisited the link, I never knew anything about the site is was hosted on—vavatch.co.uk—so this most recent time, I did a bit of clicking around. Clearly it’s the personal website of a sci-fi-loving college student from the early 2000s. But what came as a revelation to me was that the site belonged to …Adrian Hon!

I’m impressed that he kept his old website up even after moving over to the domain mssv.net, founding Six To Start, and writing A History Of The Future In 100 Objects. That’s a great snackable book, by the way. Well worth a read.

My last long-distance trip before we were all grounded by The Situation was to San Francisco at the end of 2019. I attended Indie Web Camp while I was there, which gave me the opportunity to add a little something to my website: an “on this day” page.

I’m glad I did. While it’s probably of little interest to anyone else, I enjoy scrolling back to see how the same date unfolded over the years.

’Sfunny, when I look back at older journal entries they’re often written out of frustration, usually when something in the dev world is bugging me. But when I look back at all the links I’ve bookmarked the vibe is much more enthusiastic, like I’m excitedly pointing at something and saying “Check this out!” I feel like sentiment analyses of those two sections of my site would yield two different results.

But when I scroll down through my “on this day” page, it also feels like descending deeper into the dark waters of linkrot. For each year back in time, the probability of a link still working decreases until there’s nothing but decay.

Sadly this is nothing new. I’ve been lamenting the state of digital preservation for years now. More recently Jonathan Zittrain penned an article in The Atlantic on the topic:

Too much has been lost already. The glue that holds humanity’s knowledge together is coming undone.

In one sense, linkrot is the price we pay for the web’s particular system of hypertext. We don’t have two-way linking, which means there’s no centralised repository of links which would be prohibitively complex to maintain. So when you want to link to something on the web, you just do it. An a element with an href attribute. That’s it. You don’t need to check with the owner of the resource you’re linking to. You don’t need to check with anyone. You have complete freedom to link to any URL you want to.

But it’s that same simple system that makes the act of linking a gamble. If the URL you’ve linked to goes away, you’ll have no way of knowing.

As I scroll down my “on this day” page, I come across more and more dead links that have been snapped off from the fabric of the web.

If I stop and think about it, it can get quite dispiriting. Why bother making hyperlinks at all? It’s only a matter of time until those links break.

And yet I still keep linking. I still keep pointing to things and saying “Check this out!” even though I know that over a long enough timescale, there’s little chance that the link will hold.

In a sense, every hyperlink on the World Wide Web is little act of hope. Even though I know that when I link to something, it probably won’t last, I still harbour that hope.

If hyperlinks are built on hope, and the web is made of hyperlinks, then in a way, the World Wide Web is quite literally made out of hope.

I like that.

Do you have a favourite non-personal photograph?

By non-personal, I mean one that isn’t directly related to your life; photographs of family members, friends, travel (remember travel?).

Even discounting those photographs, there’s still a vast pool of candidates. There are all the amazing pictures taken by photojournalists like Lee Miller. There’s all the awe-inspiring wildlife photography out there. Then there are the kind of posters that end up on bedroom walls, like Robert Doisneau’s The Kiss.

One of my favourite photographs of all time has music as its subject matter. No, not Johnny Cash flipping the bird, although I believe this picture to be just as rock’n’roll.

This is a photograph of Séamus Ennis and Jean Ritchie. It was probably taken around 1952 or 1953 by Ritchie’s husband, George Pickow, when Jean Ritchie and Alan Lomax were in Ireland to do field recordings.

I love everything about it.

Séamus Ennis looks genuinely larger than life (which, by all accounts, he was). And just look at the length of those fingers! Meanwhile Jean Ritchie is equally indominatable, just as much as part of the story as the musician she’s there to record.

Both of them have expressions that convey how intent they are on their machines—Ennis’s uilleann pipes and Ritchie’s tape recorder. It’s positively steampunk!

What a perfect snapshot of tradition and technology meeting slap bang in the middle of the twentieth century.

Maybe that’s why I love it so much. One single photograph is filled with so much that’s dear to me—traditional Irish music meets long-term archival preservation.

Staying at home triggered a memory for me. I remembered reading a short story many years ago. It was by J.G. Ballard, and it described a man who makes the decision not to leave the house.

Being a J.G. Ballard story, it doesn’t end there. Over the course of the story, the house grows and grows in size, forcing the protaganist into ever-smaller refuges within his own home. It really stuck with me.

I tried tracking it down with some Duck Duck Going. Searching for “j.g. ballard weird short story” doesn’t exactly narrow things down, but eventually I spotted the book that I had read the story in. It was called War Fever. I think I read it back when I was living in Germany, so that would’ve been in the ’90s. I certainly don’t have a copy of the book any more.

But I was able to look up a table of contents and find a title for the story that was stuck in my head. It’s called The Enormous Space.

Alas, I couldn’t find any downloadable versions—War Fever doesn’t seem to be available for the Kindle.

Then I remembered the recent announcement from the Internet Archive that it was opening up the National Emergency Library. The usual limits on “checking out” books online are being waived while physical libraries remain closed.

I found The Complete Stories of J.G. Ballard and borrowed it just long enough to re-read The Enormous Space.

If anything, it’s creepier and weirder than I remembered. But it’s laced with more black comedy than I remembered.

I thought you might like to hear this story, so I made a recording of myself reading The Enormous Space.

I’m in San Francisco to speak at An Event Apart, which kicks off tomorrow. But I arrived a few days early so that I could attend Indie Web Camp SF.

Yesterday was the discussion day. Most of the attendees were seasoned indie web campers, so quite a few of the discussions went deep on some of the building blocks. It was a good opportunity to step back and reappraise technology decisions.

Today is the day for making, tinkering, fiddling, and hacking. I had a few different ideas of what to do, mostly around showing additional context on my blog posts. I could, for instance, show related posts—other blog posts (or links) that have similar tags attached to them.

But I decided that a nice straightforward addition would be to show a kind of “on this day” context. After all, I’ve been writing blog posts here for eighteen years now; chances are that if I write a blog post on any given day, there will be something in the archives from that same day in previous years.

So that’s what I’ve done. I’ll be demoing it shortly here at Indie Web Camp, but you can see it in action now. If you look at the page for this blog post, you should see a section at the end with the heading “Previously on this day”. There you’ll see links to other posts I’ve written on December 8th in years gone by.

It’s quite a mixed bag. There’s a post about when I used to have a webcam from sixteen years ago. There’s a report from the Flash On The Beach conference from thirteen years ago (I wrote that post while I was in Berlin). And five years ago, I was writing about markup patterns for web components.

I don’t know if anyone other than me will find this feature interesting (but as it’s my website, I don’t really care). Personally, I find it fascinating to see how my writing has changed, both in terms of subject matter and tone.

Needless to say, the further back in time you go, the more chance there is that the links in my blog posts will no longer work. That’s a real shame. But then it’s a pleasant surprise when I find something that I linked to that is still online after all this time. And I can take comfort from the fact that if anyone has ever linked to anything I’ve written on my website, then those links still work.

The dConstruct conference is as old as Clearleft itself. We put on the first event back in 2005, the year of our founding. The last dConstruct was in 2015. It had a good run.

I’m really proud of the three years I ran the show—2012, 2013, and 2014—and I have great memories from each event. I’m inordinately pleased that the individual websites are still online after all these years. I’m equally pleased with the dConstruct audio archive that we put online in 2012. Now that the event itself is no longer running, it truly is an archive—a treasury of voices from the past.

I think that these kinds of online archives are eminently suitable for some offline design. So I’ve added a service worker script to the dConstruct archive.

Caching

To start with, there’s the no-brainer: as soon as someone hits the website, pre-cache static assets like CSS, JavaScript, the logo, and icon images. Now subsequent page loads will be quicker—those assets are taken straight from the cache.

But what about the individual pages? For something like Resilient Web Design—another site that won’t be updated—I pre-cache everything. I could do that with the dConstruct archive. All of the pages with all of the images add up to less than two megabytes; the entire site weighs less than a single page on Wired.com or The Verge.

In the end, I decided to go with a cache-as-you-go strategy. Every time a page or an image is fetched from the network, it is immediately put in a cache. The next time that page or image is requested, the file is served from that cache instead of the network.

Here’s the logic for fetch requests:

1. First, look to see if the file is in a cache. If it is, great! Serve that.
2. If the file isn’t in a cache, make a network request and serve the response …but put a copy of a file in the cache.
3. The next time that file is requested, go to step one.

Save for offline

That caching strategy works great for pages, images, and other assets. But there’s one kind of file on the dConstruct archive that’s a bit different: the audio files. They can be fairly big, so I don’t want to cache those unless the user specifically requests it.

If you end up on the page for a particular talk, and your browser supports service workers, you’ll get an additional UI element in the list of options: a toggle to “save offline” (under the hood, it’s a checkbox). If you activate that option, then the audio file gets put into a cache.

Now if you lose your network connection while browsing the site, you’ll get a custom offline page with the option to listen to any audio files you saved for offline listening. You’ll also see this collection of talks on the homepage, regardless of whether you’ve got an internet connection or not.

So if you’ve got a long plane journey ahead of you, have a browse around the dConstruct archive and select some talks for your offline listening pleasure.

Or just enjoy the speediness of browsing the site.