Homebrew Website Club Seattle Notes

Tonight I went to my first Homebrew Website Club at the Wayward Coffeehouse, a real nice coffee shop. I got to meet Doug Beal who put the Seattle club together. It was mostly just the two of us, as well as Margo Vansynghel, a reporter who was interested in what we were up to and how it related to other tech stories about silos like Facebook.

Neither of us really had been to one of these events before, so we weren’t sure what to do. We mostly worked on our own sites, and helped explain some of the IndieWeb principles about things like data ownership and building tools for yourself. I think by the end of it she had sent her first Webmention, a standard I need to learn how to use soon for this site.

I continued working on the syndication system for this site I outlined in the last post. I’ve now got the DynamoDB set up working, so when I commit any post change to the website, it gets built and triggers a Lambda function to scan the entire site for changes, logging those into DynamoDB. This creates a stream of create/update/delete events that other Lambda functions can be triggered by. With tonight’s work, I hooked a new microservice up to that which publishes created posts (but not updated or deleted posts) to a Telegram channel. It works locally with hardcoded keys, so now I just need to make it work when deployed, which should be quick; I just need to figure out how to store credentials properly. Then I can start building out more syndication methods.

Toward the end, we answered some questions about IndieWeb, took our group photo, and went our separate ways. It was a great time, and hopefully we can start building out a proper Seattle club. If you’re interested in taking control of your online data, consider stopping by an event near you, or a virtual one online. They take place every two weeks; the next one will be on August 22.

The new Gatsby website has been soft launched for a couple weeks now and seems to be working pretty well so far, so I’m now moving on from building out the site to building out a bunch of tools for sending out notifications of new content. My hypothesis here is that if I can make the barrier to entry as low as possible for people to get notified of new posts, they’ll be more inclined to read them, and that means going where people already are. Obviously RSS is a great technology for this, but it never really recovered from Google Reader meeting its far too early demise. People use algorithmic news apps, social networks, chat apps, and more to find and share posts, and so that’s where I need to be.

So far I’m planning on distributing posts to:

  • RSS/Atom/JSON feeds
  • A Twitter account
  • A Telegram channel
  • Medium
  • Flipboard
  • Apple News
  • Email
  • Mastodon

I’m a big believer in making a bunch of little systems that work together rather than one big monolithic system. I chose to build this project using the Serverless framework built on Amazon’s AWS technologies like Lambda. I really didn’t want to manage keeping processes alive and setting up things like cron jobs, so this makes a lot of sense. There are also a few other endpoints I want to implement later for the site like Micropub and Webmentions which would be nice to have managed through Lambda rather than running forever on some VPS somewhere.

Once it’s working, the basic pipeline looks something like this:

  1. I create a post (or make some change the site) and commit it to a Git repo
  2. I push the repo to my GitLab server
  3. When GitLab receives a commit, it kicks off a CI job to build the site and deploy it.
  4. Once deployed, it sends a message to an AWS Lambda function to signal the site has changed.
  5. The function on Lambda downloads a copy of the current website data and a copy of the last seen website data from AWS DynamoDB
  6. New and updated posts are sent to a WebSub server (you might remember this as “PubSubHubbub”) to signal to anyone who subscribes that the page changed. (Which means that, once this is working, the entire site will support WebSub as a server-to-server push mechanism!)
  7. New posts are inserted as rows into a DynamoDB table
  8. A series of AWS Lambda functions will be listening for insert events on that table, one for each service listed above, and they will farm out the API calls to each service

I like this approach because it’ll hopefully mean there is some resilience to normal maintenance things like server restarts and database failures. It fits nicely with a lot of open standards being pushed by the IndieWeb movement. And it’ll be automatically integrated into the build process, so I won’t forget to actually do the crossposting. It’s already well underway with a lot of the infrastructure in place; steps 1-5 and step 7 are all finished. All that’s left is to set up the individual integrations with each service and connect the WebSub plumbing.