I have a new home on the internet. I don’t visit the Twitter home timeline or the Facebook news feed anymore. I don’t open the Instagram app except when I post a photo. I still have accounts there — I just don’t visit those sites anymore. Instead, I have my own new space on the internet where everything I’m interested in is consolidated, and I can read and reply to things from there. But before I go too far into my new online home — an IndieWeb reader — some background.
It used to be the case that when you opened Twitter, you’d see every tweet from everyone you’re following, in order, with the newest at the top.
Over the past few years, Facebook, Twitter, Instagram and many other services have switched to what’s known as an “algorithmic timeline,” meaning posts no longer show up in chronological order. Instead, these services use proprietary algorithms to decide what to show you and when.
You could argue that they’ve done us a favor in one sense. I stopped being able to keep up with my chronological Twitter timeline long ago. But doesn’t it seem wrong that Twitter gets to be the one to decide what to show me? Plenty of people are upset about the new algorithmic timelines, even posting articles like “Instagram is actively ruining my life with its inhumane algorithm” and “14 Ways to Outsmart the Instagram Algorithm.”
We clearly need a way to take back control of what we’re reading online.
You might remember RSS readers did a pretty good job of giving individuals control of what they are subscribed to. However, over the years, Twitter, Instagram and many other social media platforms have shown us that people enjoy reading and sharing short-form content, not just blog posts.
Twitter not only provides an easy way to post content online, it also provides a single place to read what everyone else has posted. More importantly, these sites also enable you to quickly respond to the things you’re following. Whether that’s clicking the heart icon to show your support of a post, or writing a reply to something as you’re reading it.
They are largely stuck in the blogging era, being used to consume blogs and news sites. If we want to have any hope of the open web and independent websites replacing our own use of Twitter and Facebook, we need to be able to have experiences at least as good as we have on those services.
What if you could reply to a blog post in your feed reader, and your reply would show up as a comment on the original post automatically? What if you could click a “heart” in your reader, and the author of the post would see it? What if you had one place to go to follow not just your Twitter friends, but also all of your friends’ blogs, their microblogs, and see the pictures they’re sharing? What if you could have seamless conversations in your reader the way you have seamless conversations on Twitter today?
These are the things myself and the IndieWeb community have been making huge progress on in recent years.
Here’s a screenshot of what my current IndieWeb reader looks like:
My IndieWeb reader looks kind of like a combination of an RSS reader and a Twitter feed. An important difference between this and a traditional RSS reader is that this interface has buttons I can click to reply to posts!
When I click Reply, the IndieWeb reader creates a post on my website, and notifies the person I’m replying to so their site can show it as a comment.
Here’s my reply on my website:
And here’s my reply showing up on the original post:
I’m pretty happy with how my current IndieWeb reader is working right now! I’ve built it as a thin interface on top of a server-side API that handles all the feed fetching. In fact, there are alternative front-ends that work with the same server. When I’m on my phone, I can use a native iPhone app to see all the same content that I see on my computer.
This separation between the reader interface and the server is critical to developing a new generation of readers. This is what lets us have the choice of using multiple different reader apps, all accessing the same data behind the scenes. This separation is documented in a spec called Microsub.
The server side of the reader is software selected by the user. This might be built into their website CMS, or could be a separate service they sign up for. Typically, the server side won’t have much in the way of a user interface, likely just enough interface to subscribe to some feeds, but it doesn’t need to be able to display any of the content itself.
Reader apps can then be built without needing to spend any time dealing with parsing different feed formats or worrying about having enough resources to poll all the feeds people are subscribed to.
Check out my post, Building an IndieWeb Reader, for more details on how all the pieces fit together. If you’re building a reader, check out the Microsub spec to learn how you can participate in this growing ecosystem.
This part of the IndieWeb ecosystem is still in the early stages. I would love to see more development of both the reader apps and also the backend servers! If you use a CMS, consider installing or writing a plugin to add support for Webmention, Microsub and Micropub. If you’re an app developer, this would be a great time to build new Micropub apps to help people post to their websites, or build new Microsub apps with interesting and unique interfaces.
Another fun challenge I’m looking forward to tackling soon is the ability to post and follow private content using our websites. The OAuth 2.0 extension IndieAuth provides us a solid base to work from.
As always, I’m happy to chat about any and all of this. It’s been a lot of fun already to build this all out and see it working! You can find me in the IndieWeb chat via IRC and Slack.
This June, we’re hosting the annual IndieWeb Summit in Portland. IndieWeb Summit is for independent web creators of all kinds — from graphic artists to designers, UX engineers, coders and hackers — and is where we brainstorm and create lots of things like the IndieWeb reader. Head over to 2018.indieweb.org for more information and to register!
Over the last several months, I've been slowly putting the pieces in place to be able to build a solid indieweb reader. Today, I feel like I finally have enough in place to consider this functional enough that I am now using it every day!
One of the major missing pieces of the IndieWeb ecosystem has been having an integrated reading and posting experience that mirrors the ease with which it's possible to post and follow on Twitter and other silo apps.
We've seen a few attempts at indieweb readers over the past few years, but nothing has really taken off or stuck around. Even my own attempts at readers have fallen apart, both the previous iteration of Monocle in 2016, and my fork of selfoss in 2014. My suspicion has always been that we haven't seen many people building out this part of the ecosystem because it turns out there are a whole bunch of different parts to building a reader, many of which have no overlap in skillset: managing the subscription list, polling and fetching feeds, parsing feeds, data storage, rendering posts in a UI, providing inline action buttons to be able to reply and favorite posts, etc.
In April 2017, I started outlining a spec that draws a hard line between these very different parts of building a reader, and called it Microsub (for subscribing), as a complement to Micropub (for publishing). The basic idea is to separate the feed subscriptions from the UI parts of building a reader.
I started working from the ground up on building out the various aspects I knew I would need in order to eventually end up with a fully functional reader.
I based a lot of these design decisions around my previous experience in building a reader, as well as my documentation of how I use IRC to read content across the web.
My goal with this is to use this as my primary online dashboard to follow all kinds of content, as well as being able to interact with the content without leaving the interface.
The main organization of the reader is laid out in "channels". You can also think of these as "folders" if you want. Many feeds (or sources) can be added to a channel, and the posts are all combined into a single timeline view.
Monocle supports displaying a few different types of content in posts. It has native support for notes, articles, photos, multi-photos, videos, audio clips, checkins, favorites, reposts, and replies.
Since I follow my Instagram feed in the reader, I wanted to have a good display for photos, especially when there are many photos attached to a single post. I ended up doing a simple custom layout when there are two or three photos. Four or more they just start tiling as half-size photos. With three photos, the first photo appears larger on the left and the other two are stacked to the right.
I wanted to be able to read full articles in the reader without jumping out to the site, but also didn't want to have to scroll endlessly when I'm just skimming headlines. So if an article has content that is too tall, it gets truncated with a "read more" link to expand it.
If the post has an audio file, such as podcasts, then there is a simple HTML audio player inline!
Each post has a set of buttons to be able to respond to the post. The quickly accessible actions are "favorite", "repost", and "reply". The three dots opens up an expanded menu with some additional options, some of which I have not yet implemented.
Currently it's possible to remove a post from a channel, and to open up a debug view showing the raw data behind the post. Eventually this will expand to include muting the author, blocking the author, or unfollowing the source the post came from.
Clicking the "reply" button drops down a little text box for posting a reply.
There is a character counter just so that I have a sense of how long the post is. Since this is posting the reply back to my website, this interface has no idea what sort of character limits there are, so it's just a simple counter. When I click the "Reply" button, the app makes a Micropub post to my website to create the post there.
My website already has all the logic for adding that to my replies feed and sending webmentions to the post I'm replying to. Since this post I replied to is on Micro.blog, and Micro.blog accepts webmentions, the post showed up there within a few seconds.
The same workflow happens for favoriting and reposting things.
My website also recognizes when the post I'm replying to is a Twitter or GitHub permalink, and will automatically syndicate my reply or favorite appropriately! Since I added my GitHub notifications to a channel, I can actually reply to GitHub issues directly from the interface!
You may have noticed the little blue dots next to the channel names in some previous screenshots. Those indicate how many unread posts are in the channel.
However, some feeds that I follow end up with tons of posts in the channels, so many that the actual number of posts is no longer significant! All I really want to know is whether there is something new or not. To account for this, I can choose whether a channel implements per-item tracking, or just a boolean read/unread indicator, or disables read tracking altogether.
When I'm looking at a timeline, any new posts appear with a yellow glow around them. As the post scrolls out of view, it gets marked as read, and that state is pushed to the server so that other clients will also know it's now read.
I really enjoy not having to manually mark things as read, instead the interface just handles it all for me without any additional interaction.
Since I actually have several different websites I use, I wanted the response buttons to be able to post not just to my website, but also to the other websites I have. For example, my cat Dora, who has her own website is not always the best at using the computer, so sometimes I have to favorite things for her.
I can choose a alternate default account per channel so that the response buttons will actually post to the alternate website. Notice Dora's cat face in the bottom right corner of the screen. This lets me know that interacting with posts in this channel will be posted to this alternate account.
I can even temporarily switch to a different account by clicking on the profile icon and choosing another account.
You may also have noticed the little pen icon in the lower left corner. Clicking that pops up a dialog for writing a new post from the selected account. I chose to keep this interface super simple, providing just a text box and character counter.
If I need to write something more complicated, such as including HTML content, adding a photo, or choosing where the post is syndicated, then I'll just pop over to Quill and write the post there instead.
I mentioned earlier that there were many parts to this, and I haven't talked much about that yet. The most important thing about the architecture of this system is that it is not just a single monolithic app. Instead, there is a server responsible for collecting all the data from the feeds I'm following, and separate apps for displaying them! Since I was documenting everything on the wiki as I was building this out, other people were able to jump in and start writing clients from the beginning!
There are already two other great interfaces that work with the same backend server!
I'm pretty thrilled that already we've been able to have two people jump in and build readers so quickly already, thanks to the hard work of feed fetching being abstracted away by the server!
Now to start getting into the technical bits of how this works. Feel free to skip this section if specs make your eyes glaze over.
I mentioned before that the main separation going on here is splitting off the feed fetching and parsing from rendering the posts.
This accomplishes a few things:
The main idea behind this is the Microsub spec. This is the spec that the Microsub server implements so that clients know how they can talk to it.
Ideally there will eventually be a large ecosystem around the spec, with many clients to choose from, and many servers as well. We'll see some projects build in Microsub support natively, so that they work out of the box, and we'll also see some dedicated feed subscription services support Microsub. The nice thing about using your website identity to tie the pieces together is that you can choose your Microsub server separately from choosing the software that powers your website.
For example, I decided early on when building my website that I didn't want to mix the idea of following feeds into the same software that powers my website. So instead, I wrote an external Microsub server called Aperture, which is responsible for all the feed polling and parsing and storing the posts in channels. Aperture is open source, although I still consider it "in active development", so I am not officially supporting it right now. You are of course welcome to get it running yourself, but be prepared for things to change quickly.
(Aperture actually has two components, Watchtower which is a microservice that polls feeds and delivers them to Aperture itself, and Aperture does the actual feed parsing with the content provided by Watchtower. This allows me to scale out the feed polling separately from the Microsub server.)
Ideally I would love to see some more implementations of Microsub servers, so head over to the spec if that's your thing!
Monocle is the Microsub client I wrote that's featured in the screenshots above. It is also open source. Since it doesn't do any feed parsing itself, it doesn't even have a storage backend! Everything is fetched on the fly with the exception of the channel list and Micropub config which is persisted in the session data.
When I click on a channel to view it, Monocle first makes a Microsub timeline request to Aperture to fetch the channel data, then renders it in the timeline view. This is analogous to the iOS app Indigenous fetching the timeline data from the Microsub server then rendering it on the phone, except Monocle is doing that server-side to generate HTML for the browser.
You might think I'm crazy for having written a PHP app that fetches JSON from an API and then renders --gasp-- static HTML in 2018, but guess what -- it's fast!
Monocle is open source, but I am also hosting a version online that anyone is welcome to use at monocle.p3k.io. Since it doesn't actually store anything itself, I don't expect it to take up any significant resources any time soon! Of course in order to use it, you'll need to have your website pointing to a Microsub server of your choosing. Since that's where all the actual work is done, I am not making my hosted version of Aperture available for general signups right now. You'll need to either get that running on your own server, or build a Microsub server from scratch!
This last section has been a bit of a wall of text, so here is a diagram showing how all the pieces fit together to make this possible!
My website contains the IndieAuth and Micropub bits, but others have chosen to use external services for those as well. I've also chosen to outsource sending and receiving webmentions to external services, whereas other people end up handling those within their own website code as well.
The reader apps all talk to the Microsub server that I've linked to in order to view posts, and when I tap a "like" button or write a reply from the app, they post that to my Micropub server to create a new post.
I use webmention.io to handle my incoming webmentions, and it is configured to send posts to a channel in Aperture using Aperture's Micropub API.
This has been a very long read, so congrats if you've made it this far! Here are some links if you're curious about how you can start building out various pieces of the ecosystem as well!
Open Source Projects
While this is all a good start, and I do actually use this as my primary online home now, there is still a lot more work to do!
As always, I'm happy to chat about any and all of this! It's been a lot of fun already to build this all out and see it working! You can find me in the IndieWeb chat via IRC and Slack, if you send me a Webmention I'll see it in my reader, or find me at an upcoming IndieWeb event!
"Facebook represents a dangerous deviation in media history. Once upon a time, elites proudly viewed themselves as gatekeepers. They could be sycophantic to power and snobbish, but they also felt duty-bound to elevate the standards of society and readers. Executives of Silicon Valley regard gatekeeping as the stodgy enemy of innovation — they see themselves as more neutral, scientific and responsive to the market than the elites they replaced — a perspective that obscures their own power and responsibilities"
You can now subscribe to realtime updates of IndieNews feeds via WebSub! (formerly known as PubSubHubbub)
It was relatively straightforward to add the necessary tags and ping the WebSub hub to make this work. I used Switchboard as the hub. I added the <link> tags and HTTP headers to indicate the hub and self URLs. When a new post is submitted, I then ping the hub, which is just an HTTP POST request with the URL of the feed.
I then went to test the publisher using websub.rocks. That part worked great, websub.rocks was able to subscribe to my feed, and I see it receiving the ping when a new post is added to IndieNews.
Then I went to try subscribing to IndieNews in Woodwind. Woodwind reported that it subscribed successfully, but it appears that it no longer shows me the WebSub statistics that it used to. I wanted to ensure it was able to subscribe to the realtime feed, so I then used websub.rocks to test Woodwind!
As it turns out, Woodwind failed to subscribe to the test feed in websub.rocks, but not because of a bug in Woodwind! It turns out I had a small bug in websub.rocks caused by a previous renaming of some things. It was a quick fix and I'm glad Woodwind was able to point that out!
After adding IndieNews to Woodwind, I then realized that these posts don't look quite right in readers, and we likely need to do some more brainstorming on how they should be marked up and consumed. The challenge is when displaying a bookmarked post, who should appear as the author, and where should the permalink go? To the bookmark or to the post being bookmarked? It's likely the case that you'd want to see both pieces of information, especially since you see both pieces of information on IndieNews.
Let's take a look at a post on IndieNews:
This is a post that Chris (boffosocko.com) bookmarked on his site, and submitted to IndieNews. His goal was to share the Wordpress plugin with the community, so that's the most prominent thing being displayed here. The "WordPress IndieNews" title links to the GitHub repository, and we see "github.com" below reflecting that. At the end of the line, we see "from boffosocko.com" which is a link to his bookmark post. This indicates to the viewer that this was submitted by someone else.
I am reasonably happy with the way this is presented on IndieNews. The question then is how should it appear when being viewed in a reader such as Woodwind? We clearly have more research to do in this area, so I've started by adding this screenshot and a short description to the Bookmark page on the wiki.
"The scariest part: over 90% of my feed’s subscribers use Google Reader. What happens when Google Reader dies?"
In the current world of silos dominating the social media space, people typically have to visit half a dozen websites or apps to see all the latest content from friends. At this point I'm sick of this "multiple inbox" model that I want to find an alternative as fast as possible.
Currently I open my usual social media sites (Instagram, Twitter and Facebook, in that order) and read the home timeline only as a last resort. Below is an attempt to document the primary ways I find things online so that I can consider how I want my IndieWeb reader to present content to me.
I have a persistent streaming search from twitter.com running, looking for specific keywords. These tweets get sent to various IRC channels I am in. The keywords are usually words associated with specific projects.
For example "indieweb, indiewebcamp, indieauth, webmention" all get sent to the public #indiewebcamp IRC channel. A search for "aaronpk, aaronparecki.com, flickstagram, transportini" and a couple other terms (which I don't want to reveal, an important point that will have to be considered in the design of my reader) all go to a private IRC channel. One of the internal Esri channels has a search running for "esripdx, geotrigger, developers.arcgis.com" so we can see when people are talking about our projects.
When a particularly interesting tweet comes in, I may click it to see if anyone has replied to it and read the conversation thread.
Occasionally I come across an article newly posted on Hackernews, or a tweet I think will lead to an interesting conversation, so I leave a browser tab open to the URL and come back to it periodically to see if anything is happening. I usually close these after a couple hours to a day, occasionally up to 2 days.
When I am at a specific event, or in some other temporary situation, I often search a specific hashtag or a keyword to see what other people are saying. After the event, I typically don't care to follow the search anymore.
Along the lines of Tantek's people-focused communications, I often open Twitter or Facebook to a specific person's timeline, such as when I first meet someone, or when I want to get in touch with someone after a while and see what they've been up to. Sometimes this is people I am already following, but sometimes I do not want to follow them publicly or persistently, and just see what they have recently posted instead.
On very rare occasions, I open the Swarm app to look at who is nearby. I would say for the most part this is not terribly useful or interesting information, so I would not prioritize this feature in an interface, as the majority of the time I use Swarm this information is not helpful.
Currently, IRC is my primary "reader." Because I have so many Twitter searches running, I very rarely actually visit twitter.com and look at my stream.
I also have many other bits of information piped into IRC, such as notifications of webmentions of my various websites, debugging information from my website, notifications when someone calls my number on my apartment's call box, debug logs from other websites I run, etc.
In order to replace IRC as a generic notification platform for myself, I'll need a way to easily get arbitrary messages into my reader. Probably my best bet is to create a really simple blog that supports Micropub, so my various systems can quickly create posts on it. Then I can just subscribe to that blog in my reader.
IRC also sends push notifications to my phone for lines that match certain patterns, so I will need a similar mechanism to receive mobile push notifications about certain items that appear in my reader.
There are many existing RSS readers, but most of them follow the same model of reader that has existed since the beginning of RSS readers. I tried extending Selfoss to support Microformats feeds, with mostly success. However I realized that this reader was fundamentally stuck in the RSS reader model of having an "inbox" of unread items, and reading specific feeds individually. I think we can do better than this now.
I've heard many rumblings that the fantastic IndieWeb publishing platform Known will be getting built-in reader functionality soon. I look forward to that for sure, since it will make IndieWeb-enabled readers accessilble to a lot of people.
Another fantastic IndieWeb reader is Woodwind, which is relatively new and Kyle has made great progress on it already! I've been using it already, reading several peoples' blogs from it and using it to post likes on my site. The only reason I'm building my own and not just building on that one is because it's written in Python and I don't know the first thing about making Python apps. If you're looking for a completely functional reader to use right now, I'd definitely recommend it.
You can follow along with the development of Monocle, my IndieWeb reader, here: github.com/aaronpk/Monocle/issues
I am still in the design phase, but taking extensive notes as GitHub issues. Feel free to chime in with suggestions if you wish!
Similar to p3k, I will be writing as much of Monocle as separate libraries that can be re-used by other projects. To give you an idea of what this looks like, here is the list of separate modules I've written or contributed to that p3k uses.
but I suspect the hardest part is the client app for readers, which works in a way analogous to an RSS reader or email client, but would have to support a new format and would be optimized for clean reading and subsequent discovery, rather than the three-pane model which has dominated those apps for the last decade or two