I have a new home on the internet. I don’t visit the Twitter home timeline or the Facebook news feed anymore. I don’t open the Instagram app except when I post a photo. I still have accounts there — I just don’t visit those sites anymore. Instead, I have my own new space on the internet where everything I’m interested in is consolidated, and I can read and reply to things from there. But before I go too far into my new online home — an IndieWeb reader — some background.
It used to be the case that when you opened Twitter, you’d see every tweet from everyone you’re following, in order, with the newest at the top.
Over the past few years, Facebook, Twitter, Instagram and many other services have switched to what’s known as an “algorithmic timeline,” meaning posts no longer show up in chronological order. Instead, these services use proprietary algorithms to decide what to show you and when.
You could argue that they’ve done us a favor in one sense. I stopped being able to keep up with my chronological Twitter timeline long ago. But doesn’t it seem wrong that Twitter gets to be the one to decide what to show me? Plenty of people are upset about the new algorithmic timelines, even posting articles like “Instagram is actively ruining my life with its inhumane algorithm” and “14 Ways to Outsmart the Instagram Algorithm.”
We clearly need a way to take back control of what we’re reading online.
You might remember RSS readers did a pretty good job of giving individuals control of what they are subscribed to. However, over the years, Twitter, Instagram and many other social media platforms have shown us that people enjoy reading and sharing short-form content, not just blog posts.
Twitter not only provides an easy way to post content online, it also provides a single place to read what everyone else has posted. More importantly, these sites also enable you to quickly respond to the things you’re following. Whether that’s clicking the heart icon to show your support of a post, or writing a reply to something as you’re reading it.
They are largely stuck in the blogging era, being used to consume blogs and news sites. If we want to have any hope of the open web and independent websites replacing our own use of Twitter and Facebook, we need to be able to have experiences at least as good as we have on those services.
What if you could reply to a blog post in your feed reader, and your reply would show up as a comment on the original post automatically? What if you could click a “heart” in your reader, and the author of the post would see it? What if you had one place to go to follow not just your Twitter friends, but also all of your friends’ blogs, their microblogs, and see the pictures they’re sharing? What if you could have seamless conversations in your reader the way you have seamless conversations on Twitter today?
These are the things myself and the IndieWeb community have been making huge progress on in recent years.
Here’s a screenshot of what my current IndieWeb reader looks like:
My IndieWeb reader looks kind of like a combination of an RSS reader and a Twitter feed. An important difference between this and a traditional RSS reader is that this interface has buttons I can click to reply to posts!
When I click Reply, the IndieWeb reader creates a post on my website, and notifies the person I’m replying to so their site can show it as a comment.
Here’s my reply on my website:
And here’s my reply showing up on the original post:
I’m pretty happy with how my current IndieWeb reader is working right now! I’ve built it as a thin interface on top of a server-side API that handles all the feed fetching. In fact, there are alternative front-ends that work with the same server. When I’m on my phone, I can use a native iPhone app to see all the same content that I see on my computer.
This separation between the reader interface and the server is critical to developing a new generation of readers. This is what lets us have the choice of using multiple different reader apps, all accessing the same data behind the scenes. This separation is documented in a spec called Microsub.
The server side of the reader is software selected by the user. This might be built into their website CMS, or could be a separate service they sign up for. Typically, the server side won’t have much in the way of a user interface, likely just enough interface to subscribe to some feeds, but it doesn’t need to be able to display any of the content itself.
Reader apps can then be built without needing to spend any time dealing with parsing different feed formats or worrying about having enough resources to poll all the feeds people are subscribed to.
Check out my post, Building an IndieWeb Reader, for more details on how all the pieces fit together. If you’re building a reader, check out the Microsub spec to learn how you can participate in this growing ecosystem.
This part of the IndieWeb ecosystem is still in the early stages. I would love to see more development of both the reader apps and also the backend servers! If you use a CMS, consider installing or writing a plugin to add support for Webmention, Microsub and Micropub. If you’re an app developer, this would be a great time to build new Micropub apps to help people post to their websites, or build new Microsub apps with interesting and unique interfaces.
Another fun challenge I’m looking forward to tackling soon is the ability to post and follow private content using our websites. The OAuth 2.0 extension IndieAuth provides us a solid base to work from.
As always, I’m happy to chat about any and all of this. It’s been a lot of fun already to build this all out and see it working! You can find me in the IndieWeb chat via IRC and Slack.
This June, we’re hosting the annual IndieWeb Summit in Portland. IndieWeb Summit is for independent web creators of all kinds — from graphic artists to designers, UX engineers, coders and hackers — and is where we brainstorm and create lots of things like the IndieWeb reader. Head over to 2018.indieweb.org for more information and to register!
Yep you're right, Monocle should not be sending numeric indexes in the arrays. You can thank PHP's
http_build_query for that. I just pushed up a fix!
Over the last several months, I've been slowly putting the pieces in place to be able to build a solid indieweb reader. Today, I feel like I finally have enough in place to consider this functional enough that I am now using it every day!
One of the major missing pieces of the IndieWeb ecosystem has been having an integrated reading and posting experience that mirrors the ease with which it's possible to post and follow on Twitter and other silo apps.
We've seen a few attempts at indieweb readers over the past few years, but nothing has really taken off or stuck around. Even my own attempts at readers have fallen apart, both the previous iteration of Monocle in 2016, and my fork of selfoss in 2014. My suspicion has always been that we haven't seen many people building out this part of the ecosystem because it turns out there are a whole bunch of different parts to building a reader, many of which have no overlap in skillset: managing the subscription list, polling and fetching feeds, parsing feeds, data storage, rendering posts in a UI, providing inline action buttons to be able to reply and favorite posts, etc.
In April 2017, I started outlining a spec that draws a hard line between these very different parts of building a reader, and called it Microsub (for subscribing), as a complement to Micropub (for publishing). The basic idea is to separate the feed subscriptions from the UI parts of building a reader.
I started working from the ground up on building out the various aspects I knew I would need in order to eventually end up with a fully functional reader.
I based a lot of these design decisions around my previous experience in building a reader, as well as my documentation of how I use IRC to read content across the web.
My goal with this is to use this as my primary online dashboard to follow all kinds of content, as well as being able to interact with the content without leaving the interface.
The main organization of the reader is laid out in "channels". You can also think of these as "folders" if you want. Many feeds (or sources) can be added to a channel, and the posts are all combined into a single timeline view.
Monocle supports displaying a few different types of content in posts. It has native support for notes, articles, photos, multi-photos, videos, audio clips, checkins, favorites, reposts, and replies.
Since I follow my Instagram feed in the reader, I wanted to have a good display for photos, especially when there are many photos attached to a single post. I ended up doing a simple custom layout when there are two or three photos. Four or more they just start tiling as half-size photos. With three photos, the first photo appears larger on the left and the other two are stacked to the right.
I wanted to be able to read full articles in the reader without jumping out to the site, but also didn't want to have to scroll endlessly when I'm just skimming headlines. So if an article has content that is too tall, it gets truncated with a "read more" link to expand it.
If the post has an audio file, such as podcasts, then there is a simple HTML audio player inline!
Each post has a set of buttons to be able to respond to the post. The quickly accessible actions are "favorite", "repost", and "reply". The three dots opens up an expanded menu with some additional options, some of which I have not yet implemented.
Currently it's possible to remove a post from a channel, and to open up a debug view showing the raw data behind the post. Eventually this will expand to include muting the author, blocking the author, or unfollowing the source the post came from.
Clicking the "reply" button drops down a little text box for posting a reply.
There is a character counter just so that I have a sense of how long the post is. Since this is posting the reply back to my website, this interface has no idea what sort of character limits there are, so it's just a simple counter. When I click the "Reply" button, the app makes a Micropub post to my website to create the post there.
My website already has all the logic for adding that to my replies feed and sending webmentions to the post I'm replying to. Since this post I replied to is on Micro.blog, and Micro.blog accepts webmentions, the post showed up there within a few seconds.
The same workflow happens for favoriting and reposting things.
My website also recognizes when the post I'm replying to is a Twitter or GitHub permalink, and will automatically syndicate my reply or favorite appropriately! Since I added my GitHub notifications to a channel, I can actually reply to GitHub issues directly from the interface!
You may have noticed the little blue dots next to the channel names in some previous screenshots. Those indicate how many unread posts are in the channel.
However, some feeds that I follow end up with tons of posts in the channels, so many that the actual number of posts is no longer significant! All I really want to know is whether there is something new or not. To account for this, I can choose whether a channel implements per-item tracking, or just a boolean read/unread indicator, or disables read tracking altogether.
When I'm looking at a timeline, any new posts appear with a yellow glow around them. As the post scrolls out of view, it gets marked as read, and that state is pushed to the server so that other clients will also know it's now read.
I really enjoy not having to manually mark things as read, instead the interface just handles it all for me without any additional interaction.
Since I actually have several different websites I use, I wanted the response buttons to be able to post not just to my website, but also to the other websites I have. For example, my cat Dora, who has her own website is not always the best at using the computer, so sometimes I have to favorite things for her.
I can choose a alternate default account per channel so that the response buttons will actually post to the alternate website. Notice Dora's cat face in the bottom right corner of the screen. This lets me know that interacting with posts in this channel will be posted to this alternate account.
I can even temporarily switch to a different account by clicking on the profile icon and choosing another account.
You may also have noticed the little pen icon in the lower left corner. Clicking that pops up a dialog for writing a new post from the selected account. I chose to keep this interface super simple, providing just a text box and character counter.
If I need to write something more complicated, such as including HTML content, adding a photo, or choosing where the post is syndicated, then I'll just pop over to Quill and write the post there instead.
I mentioned earlier that there were many parts to this, and I haven't talked much about that yet. The most important thing about the architecture of this system is that it is not just a single monolithic app. Instead, there is a server responsible for collecting all the data from the feeds I'm following, and separate apps for displaying them! Since I was documenting everything on the wiki as I was building this out, other people were able to jump in and start writing clients from the beginning!
There are already two other great interfaces that work with the same backend server!
I'm pretty thrilled that already we've been able to have two people jump in and build readers so quickly already, thanks to the hard work of feed fetching being abstracted away by the server!
Now to start getting into the technical bits of how this works. Feel free to skip this section if specs make your eyes glaze over.
I mentioned before that the main separation going on here is splitting off the feed fetching and parsing from rendering the posts.
This accomplishes a few things:
The main idea behind this is the Microsub spec. This is the spec that the Microsub server implements so that clients know how they can talk to it.
Ideally there will eventually be a large ecosystem around the spec, with many clients to choose from, and many servers as well. We'll see some projects build in Microsub support natively, so that they work out of the box, and we'll also see some dedicated feed subscription services support Microsub. The nice thing about using your website identity to tie the pieces together is that you can choose your Microsub server separately from choosing the software that powers your website.
For example, I decided early on when building my website that I didn't want to mix the idea of following feeds into the same software that powers my website. So instead, I wrote an external Microsub server called Aperture, which is responsible for all the feed polling and parsing and storing the posts in channels. Aperture is open source, although I still consider it "in active development", so I am not officially supporting it right now. You are of course welcome to get it running yourself, but be prepared for things to change quickly.
(Aperture actually has two components, Watchtower which is a microservice that polls feeds and delivers them to Aperture itself, and Aperture does the actual feed parsing with the content provided by Watchtower. This allows me to scale out the feed polling separately from the Microsub server.)
Ideally I would love to see some more implementations of Microsub servers, so head over to the spec if that's your thing!
Monocle is the Microsub client I wrote that's featured in the screenshots above. It is also open source. Since it doesn't do any feed parsing itself, it doesn't even have a storage backend! Everything is fetched on the fly with the exception of the channel list and Micropub config which is persisted in the session data.
When I click on a channel to view it, Monocle first makes a Microsub timeline request to Aperture to fetch the channel data, then renders it in the timeline view. This is analogous to the iOS app Indigenous fetching the timeline data from the Microsub server then rendering it on the phone, except Monocle is doing that server-side to generate HTML for the browser.
You might think I'm crazy for having written a PHP app that fetches JSON from an API and then renders --gasp-- static HTML in 2018, but guess what -- it's fast!
Monocle is open source, but I am also hosting a version online that anyone is welcome to use at monocle.p3k.io. Since it doesn't actually store anything itself, I don't expect it to take up any significant resources any time soon! Of course in order to use it, you'll need to have your website pointing to a Microsub server of your choosing. Since that's where all the actual work is done, I am not making my hosted version of Aperture available for general signups right now. You'll need to either get that running on your own server, or build a Microsub server from scratch!
This last section has been a bit of a wall of text, so here is a diagram showing how all the pieces fit together to make this possible!
My website contains the IndieAuth and Micropub bits, but others have chosen to use external services for those as well. I've also chosen to outsource sending and receiving webmentions to external services, whereas other people end up handling those within their own website code as well.
The reader apps all talk to the Microsub server that I've linked to in order to view posts, and when I tap a "like" button or write a reply from the app, they post that to my Micropub server to create a new post.
I use webmention.io to handle my incoming webmentions, and it is configured to send posts to a channel in Aperture using Aperture's Micropub API.
This has been a very long read, so congrats if you've made it this far! Here are some links if you're curious about how you can start building out various pieces of the ecosystem as well!
Open Source Projects
While this is all a good start, and I do actually use this as my primary online home now, there is still a lot more work to do!
As always, I'm happy to chat about any and all of this! It's been a lot of fun already to build this all out and see it working! You can find me in the IndieWeb chat via IRC and Slack, if you send me a Webmention I'll see it in my reader, or find me at an upcoming IndieWeb event!
destinationlist in the session
mp-destinationof the selected account
Monocle+Aperture are coming along nicely!
I just got my GitHub notifications piped into the reader, which is a much nicer experience than reading them via email or trying to track them down on github.com!
I have a few channels set to show just an indicator dot when there are new posts rather than showing the number of new posts, a much calmer experience.
Unread posts show up with a faint yellow glow around them, and they're automatically marked as read when they scroll off the screen.
At this point, I've actually moved all of the feeds I was previously following from IRC into Aperture as a way to force myself to continue putting the finishing touches on it!
When I first built Monocle, I modeled it after the "timeline" or "stream" view now common in social networks, but made it possible to subscribe to h-entry feeds as well. The main UI showed a stream of posts, with the full post contents rendered inline, along with the author info and favorite/repost/reply buttons.
While this sounds good in theory, it turns out that I found myself not actually using it regularly, likely for the same reasons I don't often read my Twitter home timeline. Instead, I continued to primarily follow content using my IRC channels.
Last year, I wrote up details on "why I live in IRC", as a way to capture why that interface is more compelling to me than a stream of posts like Monocle and Twitter.
Since I wasn't using it as my primary reader, I wasn't regularly giving it attention, and several bugs have appeared that have been left unfixed. So for now, I feel it's best to officially shut down Monocle, rather than have it be a partially functional example of an IndieWeb reader.
You can expect to see development of Monocle resume in the future, but it will take on a very different form than what it previously was.
In the current world of silos dominating the social media space, people typically have to visit half a dozen websites or apps to see all the latest content from friends. At this point I'm sick of this "multiple inbox" model that I want to find an alternative as fast as possible.
Currently I open my usual social media sites (Instagram, Twitter and Facebook, in that order) and read the home timeline only as a last resort. Below is an attempt to document the primary ways I find things online so that I can consider how I want my IndieWeb reader to present content to me.
I have a persistent streaming search from twitter.com running, looking for specific keywords. These tweets get sent to various IRC channels I am in. The keywords are usually words associated with specific projects.
For example "indieweb, indiewebcamp, indieauth, webmention" all get sent to the public #indiewebcamp IRC channel. A search for "aaronpk, aaronparecki.com, flickstagram, transportini" and a couple other terms (which I don't want to reveal, an important point that will have to be considered in the design of my reader) all go to a private IRC channel. One of the internal Esri channels has a search running for "esripdx, geotrigger, developers.arcgis.com" so we can see when people are talking about our projects.
When a particularly interesting tweet comes in, I may click it to see if anyone has replied to it and read the conversation thread.
Occasionally I come across an article newly posted on Hackernews, or a tweet I think will lead to an interesting conversation, so I leave a browser tab open to the URL and come back to it periodically to see if anything is happening. I usually close these after a couple hours to a day, occasionally up to 2 days.
When I am at a specific event, or in some other temporary situation, I often search a specific hashtag or a keyword to see what other people are saying. After the event, I typically don't care to follow the search anymore.
Along the lines of Tantek's people-focused communications, I often open Twitter or Facebook to a specific person's timeline, such as when I first meet someone, or when I want to get in touch with someone after a while and see what they've been up to. Sometimes this is people I am already following, but sometimes I do not want to follow them publicly or persistently, and just see what they have recently posted instead.
On very rare occasions, I open the Swarm app to look at who is nearby. I would say for the most part this is not terribly useful or interesting information, so I would not prioritize this feature in an interface, as the majority of the time I use Swarm this information is not helpful.
Currently, IRC is my primary "reader." Because I have so many Twitter searches running, I very rarely actually visit twitter.com and look at my stream.
I also have many other bits of information piped into IRC, such as notifications of webmentions of my various websites, debugging information from my website, notifications when someone calls my number on my apartment's call box, debug logs from other websites I run, etc.
In order to replace IRC as a generic notification platform for myself, I'll need a way to easily get arbitrary messages into my reader. Probably my best bet is to create a really simple blog that supports Micropub, so my various systems can quickly create posts on it. Then I can just subscribe to that blog in my reader.
IRC also sends push notifications to my phone for lines that match certain patterns, so I will need a similar mechanism to receive mobile push notifications about certain items that appear in my reader.
There are many existing RSS readers, but most of them follow the same model of reader that has existed since the beginning of RSS readers. I tried extending Selfoss to support Microformats feeds, with mostly success. However I realized that this reader was fundamentally stuck in the RSS reader model of having an "inbox" of unread items, and reading specific feeds individually. I think we can do better than this now.
I've heard many rumblings that the fantastic IndieWeb publishing platform Known will be getting built-in reader functionality soon. I look forward to that for sure, since it will make IndieWeb-enabled readers accessilble to a lot of people.
Another fantastic IndieWeb reader is Woodwind, which is relatively new and Kyle has made great progress on it already! I've been using it already, reading several peoples' blogs from it and using it to post likes on my site. The only reason I'm building my own and not just building on that one is because it's written in Python and I don't know the first thing about making Python apps. If you're looking for a completely functional reader to use right now, I'd definitely recommend it.
You can follow along with the development of Monocle, my IndieWeb reader, here: github.com/aaronpk/Monocle/issues
I am still in the design phase, but taking extensive notes as GitHub issues. Feel free to chime in with suggestions if you wish!
Similar to p3k, I will be writing as much of Monocle as separate libraries that can be re-used by other projects. To give you an idea of what this looks like, here is the list of separate modules I've written or contributed to that p3k uses.
This post is snapshot of the progress I have made towards this goal in 2014.
I started the year having already been posting articles, notes and replies to my site for several years. When I want to post something to Facebook or Twitter, I first make a post on my site, and syndicate a copy of it to other locations. Similarly when I reply to someone on Twitter, I actually have been writing a reply on my site and copying it to Twitter. This is a key part of being on the IndieWeb.
I still use the Instagram app to post photos because it offers easy-to-use image adjustment tools like tweaking the brightness/contrast and sharpness of images. Whenever I post a photo on Instagram, OwnYourGram posts it immediately to my website as well.
You can use OwnYourGram if your website can create photo posts using Micropub.
In July, I started posting bike rides, runs and car2go trips on my website.
I use the Runkeeper app to log my bike rides and runs, and my website imports the data every 5 minutes. The full GPS data is downloaded and I render my own map for the ride which is what you see on my website.
Until November, the car2go API did not provide access to your rental history. Currently I have been scraping the HTML page that shows my trip history to get the data. As of version 2.1 of the API, it looks like I can get my rental history via official methods, so I will update that soon.
The car2go API returns only the start and end location of the trip, but not the actual route. In order to fill in the data with the route I actually drove, I query my GPS Logs for the start/end range and fill in the post with the route that is returned from the logs. The result is that you see the exact route I drove in the car2go.
For logs prior to 2014, I imported my biking and running history from Runkeeper and Strava.
In September I added support for video posts to p3k, the software that powers my website.
Thanks to browser support for
<video> tags, this was actually surprisingly easy. I host the video files on my server and add a
<video> tag just like I would an
In December I updated OwnYourGram to also post videos, so now all my Instagram videos will end up on my site as well.
In September I added support for bookmarks into p3k, which finally let me move off of Wordpress.
Until then, I had been using a Wordpress custom post type for bookmarks since 2010 when there were rumors that Delicious.com would be shutting down.
Now, bookmarks are a native post type of p3k, and I can create them using a bookmarklet and UI which I built as part of Quill. I imported all my bookmarks from my Wordpress site, and set up redirects from the old URLs to the new p3k URLs.
In October I started publishing everything I eat and drink.
I had already been tracking what I eat and drink since August 2013, but was simply recording it in the "Notes" app on my phone. I imported all the previous logs into my website, and built Teacup which is what I use to post food now.
Teacup has a web interface which also works great as a home screen app, as well as a companion Pebble app that can be used to quickly post entries from a predefined list.
In November I added support for reposts, so that I can repost other peoples' content into my notes stream. In December, I committed to start owning my "likes", along with many other people making similar commitments for the new year.
Along with adding support for likes in p3k, I added a bookmarklet to Quill so that I have a button in my browser I can use to like the current page I'm reading. You can use it to post likes to your site if your site supports Micropub!
At the beginning of November, I adopted a kitty named Dora. Naturally she needed her own website, so I set her up with p3k and got a domain for her, indiewebcat.com.
She posts mostly photos on her site, but occasionally stomps on the keyboard and posts a note.
So what is in store for owning my data in 2015?
I stopped using an RSS reader ages ago, even before Google Reader shut down. I'm not sure exactly why, probably a combination of being overwhelmed by the number of long blog posts that always queued up, coupled with Twitter taking over as a quicker way to keep up with people.
Having an "indie reader" will be a huge step in replacing my use of Twitter and Facebook. It's certainly achievable, as many other people have demonstrated, it just hasn't been a priority of mine until now. Now that I'm owning my likes, my current workflow of needing to visit the page in my browser that I'm liking, or copying the permalink to my posting interface, is getting in the way of liking things as much as I previously did on the various silos. Being able to read content within my own reader will mean I will have one-click likes and inline replies that post directly to my site.
My plan is to re-use a lot of the existing code of p3k which renders external content such as reposts and likes. I will also need to come up with a way to be able to add search feeds from silos, since I follow several hashtags on Twitter.
I haven't yet figured out what style of interface I want to use for my reader. I haven't been super happy with the multiple-column style like Tweetdeck in the past, but also don't want just a single stream of content like Facebook and Twitter. My current reading habits mostly revolve around Twitter searches streaming to several IRC channels, some public and some private. My first step in building a reader is designing the interface and interaction methods.
You can follow along with the development of this reader by following my #monocle tag.
It is unlikely this will make it into my 2015 accomplishments, but I'll list it here anyway. While I have been posting photos on my website since March 2014, I haven't yet replaced my use of Flickr. The photos I post on my site are more things I would have posted to Instagram, a curated select few of the many photos I take.
I use Flickr to post large batches of photos, often organized into sets based on events or topics.
I have previously made a website that can mirror a Flickr account in real time, called Flickr Archiver, so I will at least re-use a lot of the design decisions from that. I haven't yet decided if my photo archive is going to be built into aaronparecki.com or whether it will be a separate "app" the way Quill and Teacup are. I also have too many photos to store on my VPS that hosts my website, so I'll need to have a better solution for actually storing the photos, probably using Amazon S3.
Happy new year, and happy indiewebbing in 2015!