Steve Streza

WebSub and Microformats

As of now my personal website should support WebSub on all pages, posts, and the RSS feed (basically everything linked in the sitemap). If you have the capability to subscribe to pages via WebSub, you should be able to point it at any page on this site and get notified when that page updates. That’s maybe not the most useful on individual pages, but it is useful for the home page, or if you want to be able to subscribe to individual tags or projects.

This should work well with social readers that let you subscribe to websites, at least in concept. To go with that, I’ve updated the microformats across much of the site to make them accessible to those readers. Things appear alright in web-based validators but I’ve been having trouble getting those changes to appear in the readers themselves. Hopefully that will fix itself as caches expire. But it’s one step further down the road of being a good IndieWeb citizen.

Behind the scenes, WebSub support is handled by Webmaster, my custom-built server that integrates with my Gatsby build system. When GitLab finishes building, it sends a webhook to Webmaster that signals that the site has changed, causing Webmaster to fetch the sitemap and scan the ETag of each file in it for changes. If any of those ETags are different from what’s already known, then that page has changed. The WebSub subscribers for that URL are fetched and notified with the changes. No WebSub hub necessary, but down the road I could switch to one pretty easily.

Checking out the IndieWeb Summit remotely and working on my personal website today.

Find me on Mastodon

I created an account on Mastodon.social, you can find me on @stevestreza@mastodon.social. If you’re into that sort of thing, you can follow me over there.

Eventually I hope to create an ActivityPub setup that can publish directly to my own site and the Mastodon network. But until then, this is good enough. Between Twitter’s ongoing moral cowardice and their ongoing hostility towards the developers that made them what they are, I can’t continue siloing my data there.

But Mastodon is so far revealing itself to be much more pleasant than Twitter, and it has some interesting forward-thinking decisions that I’ll talk more about later. In the meantime, go find me on there. You can sign up on the instance I use, Mastodon.social, or you can sign up at any number of other instances, such as those found on instances.social or joinmastodon.org. Even if you sign up on a different instance, just search for my handle @stevestreza@mastodon.social and it should work just fine.

Link

Homebrew Website Club Seattle Notes

Tonight I went to my first Homebrew Website Club at the Wayward Coffeehouse, a real nice coffee shop. I got to meet Doug Beal who put the Seattle club together. It was mostly just the two of us, as well as Margo Vansynghel, a reporter who was interested in what we were up to and how it related to other tech stories about silos like Facebook.

Neither of us really had been to one of these events before, so we weren’t sure what to do. We mostly worked on our own sites, and helped explain some of the IndieWeb principles about things like data ownership and building tools for yourself. I think by the end of it she had sent her first Webmention, a standard I need to learn how to use soon for this site.

I continued working on the syndication system for this site I outlined in the last post. I’ve now got the DynamoDB set up working, so when I commit any post change to the website, it gets built and triggers a Lambda function to scan the entire site for changes, logging those into DynamoDB. This creates a stream of create/update/delete events that other Lambda functions can be triggered by. With tonight’s work, I hooked a new microservice up to that which publishes created posts (but not updated or deleted posts) to a Telegram channel. It works locally with hardcoded keys, so now I just need to make it work when deployed, which should be quick; I just need to figure out how to store credentials properly. Then I can start building out more syndication methods.

Toward the end, we answered some questions about IndieWeb, took our group photo, and went our separate ways. It was a great time, and hopefully we can start building out a proper Seattle club. If you’re interested in taking control of your online data, consider stopping by an event near you, or a virtual one online. They take place every two weeks; the next one will be on August 22.

Setting up Website Notifications with Serverless

The new Gatsby website has been soft launched for a couple weeks now and seems to be working pretty well so far, so I’m now moving on from building out the site to building out a bunch of tools for sending out notifications of new content. My hypothesis here is that if I can make the barrier to entry as low as possible for people to get notified of new posts, they’ll be more inclined to read them, and that means going where people already are. Obviously RSS is a great technology for this, but it never really recovered from Google Reader meeting its far too early demise. People use algorithmic news apps, social networks, chat apps, and more to find and share posts, and so that’s where I need to be.

So far I’m planning on distributing posts to:

  • RSS/Atom/JSON feeds
  • A Twitter account
  • A Telegram channel
  • Medium
  • Flipboard
  • Apple News
  • Email
  • Mastodon

I’m a big believer in making a bunch of little systems that work together rather than one big monolithic system. I chose to build this project using the Serverless framework built on Amazon’s AWS technologies like Lambda. I really didn’t want to manage keeping processes alive and setting up things like cron jobs, so this makes a lot of sense. There are also a few other endpoints I want to implement later for the site like Micropub and Webmentions which would be nice to have managed through Lambda rather than running forever on some VPS somewhere.

Once it’s working, the basic pipeline looks something like this:

  1. I create a post (or make some change the site) and commit it to a Git repo
  2. I push the repo to my GitLab server
  3. When GitLab receives a commit, it kicks off a CI job to build the site and deploy it.
  4. Once deployed, it sends a message to an AWS Lambda function to signal the site has changed.
  5. The function on Lambda downloads a copy of the current website data and a copy of the last seen website data from AWS DynamoDB
  6. New and updated posts are sent to a WebSub server (you might remember this as “PubSubHubbub”) to signal to anyone who subscribes that the page changed. (Which means that, once this is working, the entire site will support WebSub as a server-to-server push mechanism!)
  7. New posts are inserted as rows into a DynamoDB table
  8. A series of AWS Lambda functions will be listening for insert events on that table, one for each service listed above, and they will farm out the API calls to each service

I like this approach because it’ll hopefully mean there is some resilience to normal maintenance things like server restarts and database failures. It fits nicely with a lot of open standards being pushed by the IndieWeb movement. And it’ll be automatically integrated into the build process, so I won’t forget to actually do the crossposting. It’s already well underway with a lot of the infrastructure in place; steps 1-5 and step 7 are all finished. All that’s left is to set up the individual integrations with each service and connect the WebSub plumbing.

Copyright © 2022 Steve Streza.