"The other great thing about Microsub, the technology behind Aperture and Monocle, is the separation of “feed collection” from “feed display”. I don’t have to use only Monocle to read my feeds. On my phone, I use an app called Indigenous. I could also use Monocle from a mobile browser. Or I could use Together. Or I could write my own reader interface, if I so chose."
OAuth has become the de facto standard for authorization and authentication on the web. Nearly every company with an API used by third party developers has implemented OAuth to enable people to build apps on top of it.
While OAuth is a great framework for this, the way it has ended up being used is much more centralized and closed than prior efforts like OpenID 1. Every service that spins up an OAuth-enabled API ends up being its own isolated system. For example, if I want to build an app that can read someone's step count from FitBit, I have to first go register as a developer on FitBit's website in order to get API keys to use with their OAuth API.
This works okay for major services like Google, Twitter, Facebook, and even FitBit, but breaks down when you start to consider use cases like having someone's personal WordPress blog be its own OAuth server. If I want to build an app that lets you upload photos to your WordPress site, I'm obviously not going to be able to register for API keys on everyone's own WordPress installations. Enabling third party clients to be built against systems like WordPress or Mastodon opens up a huge possibility for some really interesting things. The trick is always how do these apps authenticate the user or obtain a token they can use to access those APIs.
This post details a few specific challenges with OAuth preventing it from being used by independent websites, as well as the solutions to each.
The first major hurdle to overcome is the need for the developer to register to get API keys for the service. In a world where everyone's own website is its own OAuth server, it's obviously not practical to have an app developer register API keys at each.
In OAuth, client registration gives us a few specific things:
In order to avoid registration, we need a solution for the first three bullet points above.
Client ID: Every application needs a unique identifier. If we're talking about turning every website into an OAuth provider, we need a way to have globally unique identifiers for every OAuth app. It turns out we already have a mechanism for this: URLs! In this Open Web version of OAuth, client IDs can be the application's URL. For web-based apps, this is straightforward, as it's simply the website the app is running on. For native apps, this can be the application's "about" page.
Application name and icon: Since the application's client ID is a URL, we can assume every application has a web page that talks about it, and treat that web page as the place the client defines its own metadata like name and icon. A simple way to accomplish this is with Microformats, so that the application's web page just needs to add a couple classes around the name and icon of the app. This is currently documented and implemented as the h-app microformat.
Redirect URL registration: This one is a bit more subtle. The purpose of redirect URL registration is to prevent an attacker from tricking an authorization server into sending authorization codes to the attacker. This becomes especially important when we aren't using client secrets. The trick is that since client IDs are already URLs, we can shortcut the normal registration process by declaring a rule that redirect URLs have to be on the same domain as the client ID. This way, we can avoid a situation where an application claiming to be good.example.com sets a redirect URL to attacker.example.org and steals the authorization code. The only way to get the authorization code to attacker.example.org would be to set the client ID to that domain as well, which a user would hopefully notice.
There are two different situations to consider with regards to user accounts: authentication and authorization. Authentication is the process of proving the identity of the person signing in. Authorization is how an application obtains permission to do something to someone's account.
When we talk about authentication, we are talking about wanting to allow an unknown user to identify themselves to the site they're logging in to. Common examples of this are using your email address as your identity to sign in to a website. You bring an existing identity (your email address) and then authenticate (usually by clicking a link that was sent to your email). The original version of OpenID was created to solve this problem on the web. People identified themselves with a URL, which they were able to prove they controlled using OpenID. This allows a new user to log in to a site without needing a prior relationship with the site.
When we talk about authorization, the situation is subtly different. In this case, we're talking about a user of a website wanting to give permission to a third-party app to access some part of their account. We're very used to this pattern now, which is the typical OAuth use case of granting an application the ability to access your Google Calendar, or logging in to a third party Twitter app.
Authorization: There isn't really a challenge unique OAuth on the Open Web with regards to authorization. Once the client registration problem is solved, everything else falls into place nicely. It is assumed that users are authorizing an application to access an account they already have, so the application will just end up with an access token that works with their existing account.
Authentication: Where we need to define some new behavior is talking about authentication. In this case, we want users to be able to bring an existing identity and use it to log in to other places. This means we need a way to uniquely identify users across the entire web. We can again use URLs as the solution! Every user is identified by a URL. This can be a short URL like someone's domain name, e.g. https://aaronparecki.com/, or for a site with multiple users, can be a URL that contains a path specifying a particular user on the site, e.g. https://github.com/aaronpk.
With traditional OAuth services, discovery is not needed since the application author knows which OAuth server they're talking to before they start building the app. There is typically a "Sign in with ____" button in the application that begins the authorization process. In the case of using OAuth for authentication, the common pattern is to include buttons for several common "social login" providers such as Facebook, Google, Twitter and LinkedIn. Before the "social login" space essentially consolidated to these four, there were sometimes a dozen of these buttons on an application's login page, which eventually became known as the "NASCAR problem".
In a world where every WordPress or Gitlab site is its own OAuth provider, there obviously can't be a button for each on a login screen. Instead, we need to find out from the user which server to use to authenticate them.
Since we previously stated that every user identifier is a URL, we can ask the user to enter their URL in the sign-in screen, and then fetch that URL and discover their authorization server from there.
Once we've found the user's authorization endpoint, we can start a normal OAuth request and send them to their server to authenticate. When the server redirects back to the application, it will go and verify the authorization code with their authorization endpoint just like normally happens with OAuth.
While knowing any user identity information is technically not part of OAuth, we do need the server to return a user identifier when using OAuth for authentication. In practice, most applications also want at least a unique user identifier in the authorization case as well.
We've previously said that user identifiers are URLs, which solves the global user identity problem, and gives us a mechanism to discover the user's OAuth server. So all we need is a way to return this information to the application after the user has authenticated.
OAuth gives us an easy opportunity to return this to the application: in the access token response when the application sends the authorization code to obtain an access token. The server can at that point return the full user identifier of the user that logged in. As long as the domain name matches the domain that the user entered at the start, the application can consider it successful. This also gives the authorization server the opportunity to canonicalize the user identifier, correcting "http" to "https", or adding a path component to the user's profile URL.
By now, hopefully you're thinking "this sounds great, Aaron, someone should write this up as a OAuth extension!" I'm glad you asked!
Earlier this year, I wrote this all up as an extension to OAuth 2.0, called IndieAuth. IndieAuth encapsulates these small additions needed for OAuth 2.0 to work in the Open Web.
Despite this spec being published in January, it has actually been implemented for several years before that. There are many implementations of this extension on the server side, everything from standalone authorization server projects, to a WordPress plugin, and it's even implemented by a commercial service, Micro.blog. As far as consuming apps, nearly every Micropub app has implemented this for logging users in.
For further details on implementing this extension, there are several guides available depending on whether you're writing a client, a server, or just part of a server.
There are a few existing open source projects you can use to get started if you don't want to write your own!
Webmention is one of the fundamental indieweb building blocks. It enables rich interactions between websites, like posting a comment or favorite on one site from another site. This post will walk you through the simplest way to get started sending webmentions to other sites so that you can use your own site to join the conversations happening on the Indie Web.
So what do you need to walk through this tutorial? We'll use static files and simple command line tools so that you can easily adapt this to any environment or programming language later.
First, we'll create a new HTML file that we'll use to contain the comment to post. At the very minimum, that file will need to contain a link to the post we're replying to.
<!doctype html> <meta charset="utf-8"> <title>Hello World</title> <body> <p>in reply to: <a href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p> <p>Trying out this guide to sending webmentions</p> </body>
Go ahead and copy that HTML and save it into a new file on your web server, for example: https://aaronpk.com/reply.html. Take your new post's URL and paste it into the webmention form at the bottom of this post. After a few seconds, reload this page and you should see your post show up under "Other Mentions"!
That's a great start! But you might be wondering where your comment text is. To make your comment show up better on other peoples' websites, you'll need to add a little bit of HTML markup to tell the site where your comment text is and to add your name and photo.
Let's take the HTML from before and add a couple pieces.
<!doctype html> <meta charset="utf-8"> <title>Hello World</title> <body> <div class="h-entry"> <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p> <p class="e-content">Trying out this guide to sending webmentions</p> </div> </body>
Note the parts added in green. These are Microformats! This tells the site that's receiving your webmention where to find specific parts of your post. We first wrap the whole post in a
<div class="h-entry"> to indicate that this is a post. Then we add a class to the
<a> tag of the post we're replying to, as well as a class to the element that contains our reply text.
Now, take your URL and paste it into the webmention form below again. After a few seconds, reload the page and your reply should look more complete here!
Now we see the text of the reply, and also notice that it moved out of the "Other Mentions" section and shows up along with the rest of the replies!
Of course this web page still looks pretty plain on your own website, but that's up to you to make it look however you like for visitors visiting your website! As long as you leave the
h-entry and other Microformats in your post, you can add additional markup and style the page however you like!
Let's make the comment appear with your name and photo now! To do this, you'll need to add a little section to your web page that indicates who wrote the post.
In Microformats, the author of a post is represented as an
h-card is another type of object like
h-entry, but is intended to represent people or places instead of posts. Below is a simple
h-card that we'll add to the post.
<div class="h-card"> <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40"> <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a> </div>
When we add this
h-card into the post we've written, we need to tell it that this
h-card is the author of the post. To do that, add the class
u-author before the
h-card class like the example below.
<!doctype html> <meta charset="utf-8"> <title>Hello World</title> <body> <div class="h-entry"> <div class="u-author h-card"> <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40"> <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a> </div> <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p> <p class="e-content">Trying out this guide to sending webmentions</p> </div> </body>
Now when you re-send the webmention, the receiver will find your author name, photo and URL and show it in the comment!
Great job! If you've successfully gotten this far, you're now able to comment on things and even RSVP to events using your own website!
One more detail that you'll want to include on your posts is the date that your post was written. This will ensure the receiving website shows the correct timestamp of your post. If you eventually incorporate this into a static site generator or CMS where you show a list of your replies all on one page, then you'll also want to add a permalink to the individual reply in this post. Typically an easy way to solve both is with the markup below.
<a href="https://aaronpk.com/reply.html" class="u-url"> <time class="dt-published" datetime="2018-06-30T17:15:00-0700">July 30, 2018</time> </a>
We can add that to the post below the content.
<!doctype html> <meta charset="utf-8"> <title>Hello World</title> <body> <div class="h-entry"> <div class="u-author h-card"> <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40"> <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a> </div> <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p> <p class="e-content">Trying out this guide to sending webmentions</p> <p> <a href="https://aaronpk.com/reply.html" class="u-url"> <time class="dt-published" datetime="2018-06-30T17:15:00-0700">July 30, 2018</time> </a> </p> </div> </body>
The last piece to the puzzle is having your website send webmentions automatically when a new post is created.
This part will require writing some code in your particular language of choice. You'll start by making an HTTP request to get the contents of the page you're replying to, then looking in the response for the webmention endpoint.
We can simulate this on the command line using curl and grep.
curl -si https://aaronparecki.com/2018/06/30/11/your-first-webmention | grep rel=\"webmention\"
The response will include any HTTP
Link headers or HTML
<link> tags that have a rel value of "webmention".
Link: <https://webmention.io/aaronpk/webmention>; rel="webmention" <link rel="webmention" href="https://webmention.io/aaronpk/webmention">
If you get more than one, the first one wins. You'll need to extract the URL from the tag and then send the webmention there.
Sending a webmention is just a simple
POST request to the webmention endpoint with two URLs: the URL of your post (source) and the URL of the post you're replying to (target).
curl -si https://webmention.io/aaronpk/webmention \ -d source=https://aaronpk.com/reply.html \ -d target=https://aaronparecki.com/2018/06/30/11/your-first-webmention
The only significant part of the response is the HTTP response code. Any
2xx response code is considered a success. You'll most often receive either a
202 which indicates that the webmention processing is happening asynchronously, or if the receiver processes webmentions synchronously and everything worked, you'll get a
In practice, you'll probably use a library for discovering the endpoint and sending the webmention, so here are a few pointers to start you out in a variety of languages.
Hopefully this guide was helpful to get you going in the right direction!
When you want to put your automatic webmention sending implementation to the test, try sending webmentions to all of the links on the test suite, webmention.rocks!
If you have any questions or run into any issues, feel free to ping me or anyone else in the IndieWeb chat!