Aaron Parecki

I realized I have been avoiding leaving tips in Foursquare because I didn't have a good way to post them from my website. So today I added the ability to syndicate my posts as Foursquare tips.

I have three different ways my posts can end up as tips on Foursquare:

• I can reply to one of my checkins
• I can reply to a Foursquare venue URL directly
• I can send the text of a checkin itself as a tip

That last one is interesting, because it doesn't quite map to the normal way tips end up on Foursquare. But I realized that sometimes I write text in my checkins that is actually more like a tip anyway, so I wanted to give myself that option.

I'm probably most likely to use the first option of replying to one of my own checkin posts to leave a tip. Since my checkins appear in my reader, I can actually post directly inline from the reader to go back and leave tips.

I also made the decision to not try to syndicate these automatically, instead I have to manually click the syndication option before my site will attempt to syndicate the post.

Here's what a checkin post on my website looks like:

And here's what that tip looks like syndicated to Foursquare:

Overall this was pretty straightforward to do. The hardest part was dealing with finding the Foursquare venue URL from my own internal storage of my posts for the three different options.

The Foursquare API for this is reasonably well documented (Add a tip to a venue), but there were a few gotchas. It's not immediately obvious from that page that you need to include the access token as a post body parameter, and you also need to include an API version string. These are documented elsewhere in their docs, so it didn't take too long to find, but if you just start out reading the specific API method you want to use, it's not mentioned there at all.

There is also a parameter in the API docs called "url" which is described as "A URL related to this tip" with an example value of "http://blog.zagat.com/taco-survey-results-are-here". At first I was excited that I may be able to link my Foursquare tips back to my website. But every time I tried including a URL in the request, I got back a generic error "bad tips/add request". Leaving out that parameter made the API call work. Oh well.

So hopefully this will encourage me to leave more tips on Foursquare! I find the tips very helpful when I travel, so I'm happy to contribute back to this data set!

At IndieWebCamp Amsterdam I finally implemented unlisted posts for my website! This has been on my list for a while, and is a prerequisite for being able to do fancier things like posting full photo albums to my site, so I decided to take the plunge and finally get this done.

There has been some brainstorming about this for other Micropub clients and servers already, which was a very helpful start.

I made a couple decisions about how this should work on my site that I am not necessarily completely set on. I wanted to document this before I forget and look back at my code months later and think "why did I do it this way?"

On my website, an "unlisted" post:

• Does not show up on my home page, or any "channel" page like /notes or /articles
• Does not show up on date or month permalinks
• If the post is tagged, it does not show up on the tag pages
• The post will still show up in searches
• The post will still send webmentions for any links in the post
• The post will not be delivered to my ActivityPub followers at all
• The post still uses my same URL scheme, which uses sequential numbers after the date, so these unlisted posts are still enumerable

That last point is very intentional. I don't want to accidentally start treating unlisted posts as a form of access control. I don't want to pretend they are not public. If I had used some sort of longer non-enumerable string in the URL, then it has the risk of looking like a secret URL, and I may be tempted to treat it as a form of private post. Instead, I want to implement private and protected posts as truly protected by requiring the viewer to log in to view them.

I'm on the fence about whether unlisted posts should show up in search. My reasoning is that the act of typing in a search term means you're looking for a particular thing, which is very different from browsing my home page or even tag pages. On the other hand, YouTube implements unlisted posts by making them truly not discoverable unless you know the URL (hidden from search, lists, recommendations, etc). I think I'll leave it like this and see if I'm comfortable after using it for a while.

Implementing this on my site wasn't very hard, because I had quite a bit of ground work laid already. The interesting thing was the cascade effect it had once I wanted to go and actually post an unlisted post. I decided to add a dropdown to Quill, the app I use to post most of my content on my site. However, since many other people use this app too, I didn't want to just add this dropdown if other websites won't recognize the field. So I had to add a mechanism to Quill to ask your website whether it supports unlisted posts.

We have some precedent in Micropub with this kind of thing around querying the Micropub endpoint for which kinds of posts it supports. Quill uses this to disable some of the posting interfaces for servers that don't support those kinds of posts. Micro.blog added this earlier this year so that if you're using Quill with Micro.blog, you will only see the posting interfaces that Micro.blog can handle.

This is the new dropdown in Quill:

If your Micropub endpoint returns "visibility" in the config query, then Quill will show this new dropdown in the note interface!

Special thanks to @sebsel for talking through some of this brainstorming with me during IndieWebCamp!

The new draft spec at OAuth.xyz outlines a potential way to completely re-think OAuth from the ground up.

The XYZ spec, also nicknamed Transactional Authorization, effectively consolidates all of the different variations of OAuth that have evolved over the last decade, and plugs up a few security holes as well.

OAuth 2.0 has served us well over the last 10+ years, and has even been extensible enough to evolve to meet the new requirements of things like mobile apps and apps running on devices that don't have a browser or keyboard. It's hard to imagine that OAuth was originally created before the iPhone existed! That said, this extensibility has sometimes come at the cost of additional complexity, or at the cost of security. Many of the original assumptions that were made while designing OAuth 2.0 no longer apply, and the world of online security is very different today than it was 10 years ago.

The XYZ spec takes everything the industry has learned from OAuth 2.0 and OpenID Connect over the years and consolidates them into a uniform pattern, reducing reliance on less-secure communications channels, and building in flexibility in a consistent way.

If you're interested in the details of XYZ, the website oauth.xyz does a great job of laying out all the use cases with examples.

This post is not an overview of what XYZ is. Instead, what I'd like to talk about today is a suggestion for extending XYZ even further to encompass some additional use cases that have emerged, specifically about adding the user's identity information to the core protocol.

Identity has always been part of OAuth

OAuth was never intended to communicate information about the user who logged in to the app. This is a common point of confusion, usually wrapped up in the "authorization vs authentication" discussion. OAuth was originally created to give third party apps access to a user's account to read or write data at the API.

Naturally, many apps that want access to a user's resources also want to know who that user is. What ended up happening is most OAuth APIs added an endpoint that returns profile information about the user when requested with the access token. You can see this in common APIs like GitHub, Instagram, and Twitter. These were all developed independently, so they all work slightly differently and return data in slightly different formats.

GitHub

https://api.github.com/user
Authorization: Bearer XXXX

{
  "login": "aaronpk",
  "id": 1234,
  "node_id": "MDQ6VXNlcjE=",
  "avatar_url": "https://avatars3.githubusercontent.com/u/113001?s=460&v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/aaronpk",
  ...
}

Instagram

https://api.instagram.com/v1/users/self/?access_token=XXXX

{
  "data": {
    "id": "1574083",
    "username": "aaronpk",
    "full_name": "Aaron Parecki",
    "profile_picture": "https://scontent-dfw5-2.cdninstagram.com/vp/a37666da4d6b670a849e7b77235e4d56/5DCD3710/t51.2885-19/s320x320/52837124_579812379154942_8897415153306304512_n.jpg?_nc_ht=scontent-dfw5-2.cdninstagram.com",
    "bio": "OAuth hacker",
    "website": "https://aaronparecki.com",
    "is_business": false,
    "counts": {
      "media": 1572,
      "follows": 285,
      "followed_by": 505
    }
  }
}

Twitter

https://api.twitter.com/1.1/account/verify_credentials.json

[
  {
    "id": 14447132,
    "id_str": "14447132",
    "name": "Aaron Parecki",
    "screen_name": "aaronpk",
    "location": "30,000 feet",
    ...
  }
]

OpenID Connect

OpenID Connect is built on top of OAuth 2.0 and provides a mechanism for applications to get the identity of the user that signs in. It does this in two ways.

The idea of returning profile information given an access token was standardized in OpenID Connect in the "UserInfo Endpoint". When a provider supports this endpoint, a client can send an access token and get back a representation of the authenticated end user.

OpenID Connect UserInfo Endpoint

https://authorization-server.com/userinfo
Authorization: Bearer XXXXX

{
  "sub": "https://aaronparecki.com/",
  "name": "Aaron Parecki",
  "email": "aaron@parecki.com",
  "preferred_username": "aaronpk",
  ...
}

This ends up looking very similar to the provider-specific endpoints that we looked at above.

OpenID Connect ID Tokens

The other way OpenID Connect deals with identity is by defining a new type of token, an "ID token". The ID token is defined as a JSON Web Token (JWT) that encodes the user's profile information. This ID token is then returned in response to an OpenID Connect request, either returned in the redirect URL using the implicit flow, or returned alongside the access token using the authorization code flow.

In the case of the implicit flow, the application that receives the ID token needs to validate the JWT signature and all the JWT claims, because otherwise anyone could drop an ID token into an application to be signed in. This is a standard part of the plain OpenID Connect workflow.

Applications that also need an access token in addition to the ID token should be using the authorization code flow to obtain the access token, otherwise they are at risk of the access token leaking.

So if an application is getting both an access token and ID token, the simplest way is to get both via the authorization code flow, so that both tokens end up being returned in the response to exchanging the authorization code at the token endpoint. This ends up looking like the below.

POST /token
Content-type: application/x-www-form-urlencoded

grant_type=authorization_code
&code=XXXXXXXXX
&redirect_uri=https://example-app.com/redirect
&client_id={CLIENT_ID}
&client_secret={CLIENT_SECRET}

{
  "access_token": "SlBV32hkMG",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "2xMBxBtEp3",
  "id_token": "eyJhbGciOiJSUzI1..."
}

The interesting part about this is that in this case, the ID token validation step can be skipped completely, since validating it would provide no additional benefit. The application already knows it's talking to the right server, it knows the response is coming from that server, and the connection can't be tampered with since it will be made over HTTPS. This means the application can just extract the data it needs from the ID token and use it directly.

The problem with this approach is that this is essentially providing the application with a point-in-time snapshot of the user's profile information. If their profile information changes, there is no way for the application to get that update unless it either fetches the information from the userinfo endpoint or if the user signs in again. While this may only be a minor annoyance for things like the user's name, it is potentially dangerous for other profile information such as what groups a user belongs to, or other things the application may be relying on to modify its behavior based on the logged-in user.

It also means that the entire profile information for the user is pushed to the application whether it needs it or not. And again, while this may not be a big deal if the only profile information sent is the user's name and email address, it can be potentially a lot of data such as groups, roles, or other provider-specific information that the authorization server may have about the user.

Identity in XYZ

While we're re-thinking OAuth from the ground up, we should also re-think how identity is handled within the system, while drawing from the experience of widely deployed systems. Rather than returning an ID token with potentially stale or large amounts of user information in the authorization response, let's return only the minimum necessary for the application to function, and let it request additional information that it needs when it needs it.

Transaction Request (Authorization Code Exchange)

In XYZ, the application takes the interact handle it got from the redirect and makes a POST request back to the authorization server to get an access token. (Described in Transaction Request.)

POST /authorize
Content-type: application/json

{
  "handle": "80UPWY2NM33OMUKMKSKU",
  "interact_handle": "CuD2MrpSXVKvvI6dN2awtNLx-HhZy46hJFDBicG4KoZaCmBofvqPxtm7CDMTsUFuvcmLwi_zUN70cCvalI6ENw"
}

We'll extend the response as described next.

Transaction Response (Access Token and User Identity)

In addition to the access token being returned, the user identifier can also be returned in the same response.

The transaction response can include just the unique user identifier along with the access token.

{
  "access_token": {
    "value": "UM1P9PMHKUR64TB8N6BW7OZB8CDFONP219RP1LT0",
    "type": "bearer"
  },
  "user": {
    "id": "5035678642"
  }
}

This property is analogous to the "sub" property in an ID token. This value MUST uniquely identify this user within the system, and MUST be consistent when the same user logs in to the same application. The value MAY vary for the same user when logging in to different applications in order to preserve the user's privacy, preventing different applications from cross-correlating users between apps. (This is based on the deployed behavior of several APIs such as Facebook's "app-scoped IDs" and Apple's app-specific user ID and proxy email address.)

If the application needs to know profile information about the user, it can get that from the userinfo endpoint using the access token it just obtained.

We can go one step further and also return the userinfo endpoint at the same time as returning the user's unique identifier, avoiding the need for the app developer to hard-code this value in the app or find it in a separate discovery spec.

{
  "access_token": {
    "value": "UM1P9PMHKUR64TB8N6BW7OZB8CDFONP219RP1LT0",
    "type": "bearer"
  },
  "user": {
    "id": "5035678642",
    "userinfo": "https://authorization-server.com/user/5035678642"
  }
}

This provides the additional advantage of the opportunity for the authorization server to use a unique URL per user if it wants (although that is by no means a requirement, since the access token will need to be included in the request too).

If the application then wants to discover profile information about the user, it can make a request to the userinfo endpoint with the access token:

GET /user/5035678642
Authorization: Bearer UM1P9PMHKUR64TB8N6BW7OZB8CDFONP219RP1LT0

{
  "id": "5035678642",
  "name": "Aaron Parecki",
  "nickname": "aaronpk",
  "email": "aaron@parecki.com",
  "photo": "https://aaronparecki.com/images/profile.jpg",
  "zoneinfo": "America/Los_Angeles"
}

This ensures that the application knows that it can get the latest version of the user's profile information whenever it needs to by continuing to make requests to the userinfo endpoint, and prevents the temptation to treat the initial snapshot of the profile information as authoritative.

If we don't address this need in XYZ, we'll risk ending up in the same situation we're in with OAuth 2.0 where providers end up implementing their own snowflake APIs to let apps find the user's profile information anyway.

tl;dr This is a good move for users in the iOS ecosystem, and is primarily designed as an alternative for apps that currently use "Sign in with [Facebook/Twitter/Google]" to avoid leaking sensitive user info.

Yes, Apple is entering the OAuth ecosystem as a new identity provider. Turns out every iOS user already has an Apple account, so why not enable users to sign in with an account they already have?

Most of the time the way apps use OAuth providers is just to identify users. This is designed to be an alternative to using Facebook/Twitter/Google for that purpose.

This is distinctly different from the case where an app wants you to sign in with your Google account so that it can manage your calendar. Or sign in with Snapchat to apply a filter to your profile picture.

Those use cases are more along the lines of what OAuth was originally intended for: letting apps access your account without giving them your password.

Over the years, apps started to use OAuth to identify users because it's a quick way to find out and verify someone's Twitter/Facebook/etc account without having them type it in. This turned out to be bad for users' privacy:

Once an app knows your Twitter username or your email address, they can sell it to advertisers, or track your activity across other apps. Apple's approach provides a unique scrambled email address to the app, preventing this.

Now you may have heard people concerned by this clause from the new App Store Review Guidelines:

Sign In with Apple [...] will be required as an option for users in apps that support third-party sign-in when it is commercially available later this year.

Sign In with Apple is a good thing for users! This means apps will no longer be able to force you to log in with your Facebook account to use them.

This does not mean that Apple is requiring every app to use Sign in with Apple. This does not mean that apps that want to manage your Google Calendar will have to also add Sign in with Apple.

Yes, this is a little additional work for app developers to support another OAuth provider, but is really not that different from supporting both Twitter and Facebook, or Snapchat and Instagram.

At the end of the day, the benefit of signing in to apps is to be able to save stuff to your account so you can restore it later, and to get email notifications.

"Sign In with Apple" provides apps with both those features without revealing any more information about you than necessary.

So yes, Sign In with Apple is a good thing for user privacy, and will be a better user experience overall.

Is Apple using their position as gatekeepers of the App Store to force adoption of "Sign In with Apple"?

Yes.

Is this a bad thing?

No.

Does this affect you if you don't use an iOS device?

No.

Does this benefit people who have an iOS device?

Yes.

Will we see other OAuth providers follow suit and start randomizing email addresses and user IDs returned to apps? I hope so!

Ironically, Facebook first started doing this a few years ago when they launched app-scoped user IDs.

Anyway, if you're curious about what this will look like, I wrote a sample app that uses Sign In with Apple so you can see how it works.

What the Heck is Sign In with Apple?

Sign In with Apple is based on OAuth 2.0 and OpenID Connect, and provides a privacy-friendly way for users to sign in to websites and apps

developer.okta.com

So you just published your conference talk videos on YouTube, and you want to add captions. YouTube will use speech to text and try to add their own captions for you, but it's pretty hit or miss, especially for technical talks where there's a lot of proper nouns or jargon in the talk. The good news is there's a much better solution, and one that provides benefits to your in-person audience as well!

You can hire a captioning service to transcribe the speakers' presentations live during the event, providing a display of the captions on a separate display at the conference as well. 

There's a real person at the other end of the screen, listening to everything the speakers are saying and typing really quickly. (There's a special keyboard and years of training that go into this, but there are plenty of resources online if you're interested in learning!)

I've worked with a few people who do captioning for these events, and they're always fantastic. If you run a conference, check out StenoKnight and White Coat Captioning and hire them for your next event! They can either send someone out in person to be at the event and do the captioning right there, or they can connect via Skype and do it all remotely.

Typically they'll provide you with a web page that you can pull up on a large monitor at the venue to display the captions in real time for the in-person audience. This is of course great for people who are deaf or hard of hearing, but is also really helpful for audience members who are not native English speakers, since often times the presenters will talk too quickly to be understood.

At the end of the conference, ask the captioner to send you the plain text files of everything they typed. This will just be a regular text file, no fancy formatting. Here's where the magic happens.

Adding Captions to YouTube Videos

YouTube has a special feature which will match up typed text with the speech it recognizes in videos, doing all the work of syncing your captions with the video timings automatically. If you have a clear audio recording and accurate text, it does a surprisingly good job. Here are the detailed steps to take your plain text file and turn it into captions on YouTube.

First, launch YouTube Studio.

Click on "Videos" to show the list of all the videos uploaded.

Choose the video you want to add captions to.

Click the sidebar option "Transcriptions".

Then click "Add Language". We don’t want to use the autogenerated captions at all, since the live captionist does a way better job.

That adds a new row to this table. Click "Add" under "Subtitles".

A new window opens. Choose the "transcribe and auto-sync" option. That will let us paste in the transcribed text.

Copy the captions of the presenter into the box. Make sure the text you paste from starts and ends with the words the video starts with, or YouTube gets confused trying to line things up.

Once you do that, click "Set Timings".

This part takes a few minutes, so go make a coffee or tea, since you’ll need it for the next step. You can refresh this page to check if it’s done. It will look like this while YouTube is busy with it.

Finally when YouTube has finished thinking, it will appear as a draft you can edit.

Now YouTube has done its magic, and matched up the typed text with the spoken words! It usually does a pretty good job of it. Click on the draft and you can see what it's done.

You could probably publish this at this point, but I like to do a manual review of everything to make sure it looks good. You can play the video to review the captions and timings (check out the keyboard shortcuts which can really help speed up this step).

Keyboard shortcuts:

• SHIFT+SPACE - start/stop
• SHIFT+LEFT / SHIFT+RIGHT - skip forward or backwards by a second

While reviewing, I’m mainly looking for the following:

Did YouTube leave any dangling words that could otherwise fit into the previous caption?

For example, this would look better if the word "fit," was in the previous caption frame instead of starting a new caption frame with the end of that phrase.

In that case, just move the word to the other caption frame.

Are there any obvious typos on technical terms or proper nouns? 

The live captionists do a pretty good job, but occasionally some typos slip through.

If the presenter has any long gaps in between sentences, sometimes YouTube gets confused about the timing.

You can find some of these spots by visually looking at the waveform compared to the length of text in the caption. (In this particular example this happens to be accurate but it usually looks similar to this when it’s wrong.)

Once you’re happy with the transcript, click "Save Changes".

Now you need to delete the auto transcript so that only the good transcript is left.

You first have to click "Unpublish".

Then you can click "Delete Draft".

And confirm.

Now there is only one set of captions, the good one! You are finished, congrats!

Now when people watch the video on YouTube and enable captions, they'll be seeing what the live captionist typed during the event! 

I've been helping Donut.js record their meetup talks for about 3 years now. It's been a fun way to experiment with various video recording setups, and the talks are always great so it's nice to have those recorded.

I've always used a smaller kit compared to the rig I bring to large multi-day conferences. But lately I've had to hand off some camera gear to their volunteers since I've been out of town on work trips for most of the meetups this year.

I've been looking for smaller and smaller gear to make this easy to transport, cut down on setup time, but also so that it can be operated by someone without a lot of experience with the gear. I've tried a few iterations of kits for them, but so far the most reliable way to record the meetup is using a camcorder, a separate device to capture the presenter's slides, and a separate audio recorder, then sync up everything in post.

In an ideal world, I'd be able to hand them a small box and a video camera, they could plug everything in, and it would record a single stream mixing in the presenter's slides, the video camera, and the audio. Here's a little diagram showing what I'd like in an ideal world.

The video camera and presenter's laptop connect to the "magic box" via HDMI. We use a PA in the venue, which has an XLR output, and that needs to connect to the magic box to get audio into the mix. Lastly, I want the presenter's laptop HDMI to pass through to an output so that we can feed the venue's projector from the box. The box should be able to record the video output to an SD card or hard drive. I don't need it to be able to livestream, but bonus points if it can.

Part of the goal is also to cut down on post-production time. Right now I have to sync the audio and video (automatically) in Final Cut or Premiere, then line up the recording of the slides manually. It ultimately isn't that bad, but does mean the whole process takes around an hour for the three talks. Ideally I could record a mix of the video and slides already combined into a single video so that the only work is trimming the start and end, and adding the title slides. Here's a snapshot of the kind of layout I'd like to make.

This means scaling the slides and scaling and cropping the video. I would settle for a side-by-side layout like the below, where both are scaled but not cropped.

My last requirement, and one that rules out a few otherwise great devices, is that I need the device to be simple enough to operate to explain in a single page of instructions. I need it to be plug the HDMI and audio inputs in, then turn on the device, and hit record. Once I've set it up once, it can't require any configuration on site.

So far, I haven't been able to find a device that can do all of these things at the same time. Here's a list of all my requirements:

• Two video inputs (camera and presenter's laptop)
• Presenter's laptop needs a scaler
• Pass-through from the presenter's laptop input to an output to feed the projector
• Audio input via XLR
• Can do side-by-side video from two inputs, or at least picture-in-picture
• Can record locally to an SD card or hard drive
• Simple enough to operate by someone not trained on the device with just a one-sheet of instructions

Here are a few setups that I've tried or investigated.

Epiphan Webcaster X2

This device is so close to being perfect. It has only one HDMI input, but you can also plug in a USB webcam as a second camera. While that's obviously not going to be as good quality as a real camera, I would consider it good enough for this use case. The Webcaster X2 is the device that made the screenshot above with the text "My Great Presentation", so you can see that it's able to scale both the HDMI input as well as the USB webcam. Here's where it fails:

• It can't record locally at all, even though it has an SD card slot. It can only stream, and I'm not willing to risk the recording based on the venue's internet connection.
• It doesn't output audio over the HDMI out, so you can't even connect an external recorder to it.
• Audio input is limited to either the HDMI input or the USB webcam, no option to feed a separate audio source in. I would need to use an HDMI audio injector with it to accomplish the diagram above.

I'm also not a huge fan of the fact that it's actually an Android device, but it is pretty well done anyway, and mostly you can ignore that fact while using it.

Total cost: $300 for the Webcaster, but this doesn't really apply because it can't record locally at all so it's not really an option.

DIY Box

At a recent conference I recorded, I hacked together a DIY version of the box using a few components.

• Atomos Ninja Blade HDMI recorder
• Decimator MD-HX Scaler
• Decimator DMON-QUAD multiviewer
• Decimator MD-LX HDMI/SDI Cross-Converter

The inputs are connected with short HDMI extenders to expose them to the outside of the box.

This makes setup super easy, since you just plug in the three HDMI connectors and you're good to go. In this conference we were using a lav mic that fed into the camera, so the audio was coming in via that HDMI.

The scaler handles taking whatever resolution peoples' laptops throw at it and convert it to 1080p, plus outputs that back out for the projector. The multiviewer then takes the scaled computer output and the HDMI camera and creates an image with two smaller windows of each video. It's also able to select which one to use audio from. The output of the multiviewer goes into the Atomos recorder to record the final output.

This worked well, but is kind of a clunky solution, plus wouldn't work for Donut.js where the audio needs to get fed in separately from an XLR cable. That'd require a few more pieces of hardware such as an HDMI audio injector or such.

Total cost: $300 scaler, $300 multiviewer, $100 cross-converter, $500 HDMI recorder: $1200 plus some cables and maybe also an audio injector.

Roland VR-1HD

I haven't actually used this device yet, but it comes very very close to being a perfect solution based on all the videos and reviews I've seen.

It has three HDMI inputs, and one pass-through port which is perfect for feeding the projector.

It even has an XLR input on the side which we can use to input the feed from the PA.

But here's what it's missing:

• It can do picture-in-picture, but one source will be full screen and the second will be small. This means the camera would always overlap the slides rather than be side by side.
• It has no built-in recording capability, so it would require connecting an external HDMI recorder to capture the program out, or connect it to a computer and use that to record.

Switching between picture-in-picture and one of the HDMI sources can be done with the physical buttons on top, and would be easy enough to instruct people how to do.

I suppose I could live with picture-in-picture instead of side by side, but I would feel better about that if it also had built-in recording to an SD card.

By the time you add an external recorder, you're spending $1500 on the VR-1HD and $500 on the recorder, for a total of $2000. 

Tricaster Mini

The Tricaster is definitely an all-in-one solution, but I'm ruling it out immediately since it requires quite a lot of configuration to get running and isn't something I would consider handing off to a volunteer. It's also quite expensive at a baseline price of $6000.

Blackmagic ATEM Television Studio

The Blackmagic ATEM is my goto for larger events, and I do really like it. However, it's still a bit too complicated to hand off to someone to use. It doesn't have built-in recording, so you have to pair it with an external recorder. It also doesn't have a built-in scaler so you need that for the slides too. 

It also can only do picture-in-picture, not side by side video. In order to do that you need to step up to the much larger and more expensive devices.

I haven't actually put together a complete parts list for what it would cost for this option because I don't think it's viable at all. The ATEM TVS is $1000, the recorder is $600, and the scaler is $300, so the base cost before all the other accessories you'd need is $1900.

Computer with OBS

I'm including this option in the list just so people don't tell me I forgot it. It turns out this isn't actually a very good solution, because it won't be an all-in-one box, and also is kind of complicated to operate, requiring a monitor and keyboard and mouse. 

Trying to do this on a laptop isn't really feasible since it requires two HDMI capture cards plus a USB audio interface. I wouldn't trust Windows to do this since it's very easy for a Windows computer to accidentally start running auto-updates at inopportune times. Running Linux is an option, but would then likely require more explanation to people using it. 

I would want to be able to configure the computer to launch OBS on boot and restore a saved configuration, so that there is no fiddling with software to get it running.

Overall I feel like there are too many moving parts and different ways this can fail, and also would require a lot of plugging wires in so the setup time would actually be pretty long.

SlingStudio

I've actually used the SlingStudio at Donut.js quite a bit myself, and it is again almost perfect. 

• Good: It can record to an SD card as well as stream live. 
• Okay: It has only one HDMI input, but the cameras connect wirelessly from a transmitter. 
• Okay: It has an 1/8" audio input, so you need something to convert the XLR input to 1/8", which is annoying but acceptable.
• Bad: Like most products, it can do picture-in-picture but not side-by-side video.
• Bad: The iPad controller is decent, but is not quite simple enough to hand off to someone else without a bit of training. Mainly the awkwardness comes from toggling into picture-in-picture mode.

The total cost of this setup is:

• $1000 for the SlingStudio
• $300 for an HDMI transmitter
• ~$100 for something to convert the XLR to 1/8" audio
• $300 for an iPad to control it
• Total: ~$1700 

Epiphan Pearl Mini

The Epiphan Pearl Mini sure looks like a fantastic device. I haven't tried it out myself, but I've looked at a bunch of reviews of it. It actually seems like it's the only thing that actually ticks all of the feature boxes.

• It can do side-by-side layouts from two inputs
• It has an XLR input for audio
• It can record to an SD card
• It can be operated from the touch screen, where you tap on the layout you want to switch to
• The HDMI output can be configured to pass through one of the inputs, so that I could feed the projector
• You can even add images and text graphics would would be perfect for producing the final layout in the device

The only thing I am not clear on is what happens when you first boot it up. I am hoping that it would restore the last used configuration and be ready to go immediately.

Really the only downside to this is the cost. It's a $3500 device, which is good for what it can do, but also still quite a lot of money. After this much research though, I'm coming around to the idea that maybe it's worth it.

So I think out of all of these options, the best is the Pearl Mini ($3500), and second best is the Roland VR-1HD with external recorder ($2000).

I'd be curious to hear your thoughts on this! Did I forget about any options? Is there a new device that's come out that I don't know about yet?

Write a blog post response or ping me on Twitter to get in touch!

If I ever do find the perfect solution, I will be sure to post a review video on my YouTube channel!

My previous avatar was almost 3 years old, and I was getting tired of it. I decided to replace my avatar on my website for my IndieWebCamp Austin hack day project. But if you know me, you know I can't do anything the easy way. For my hack day project I made the avatar on each post in my website change depending on the emoji I use in the post!

I've had a page on my website for a while now that shows all the emoji I use in my posts. It's sorted by most frequently used emoji.

If you click on an emoji, you can see all the posts that include that emoji, kind of like a tag. This has been pretty fun, but I wanted to take it one step further.

Before I left for Austin, I did a little photo shoot at home. I took pictures of myself making a bunch of faces, and even with a few different props like a mug of beer, eating a pizza, holding an umbrella, etc. I added all these pictures to my admin interface and mapped them to the emoji I frequently use.

(There's a bunch more, but I'll leave that to you to find them!) And now, whenever a post includes an emoji, it will swap the photo that's used as the avatar! Here are a few highlights.

I'm looking forward to being surprised when I forget this exists and randomly use an emoji in a post and see a different photo!

If you'd like to see the video of my demo of this feature, and the rest of the demos at IndieWebCamp, you can watch them here!

I had a great time at IndieWebCamp Austin this weekend! Thanks to everyone for joining us this weekend, and special thanks to Manton for organizing!

It's already been a year since IndieAuth was published as a W3C Note! A lot has happened in that time! There's been several new plugins and services launch support for IndieAuth, and it's even made appearances at several events around the world!

• Micro.blog added native support for IndieAuth, so your hosted micro.blog account is now also an IndieAuth provider
• Dobrado launched native support for IndieAuth
• The IndieAuth Plugin for Drupal launched in the beginning of the year, and has had several releases since then
• The IndieAuth Plugin for Grav adds support to your Grav site by delegating to indieauth.com
• The IndieAuth Plugin for WordPress had a major rewrite and supports IndieAuth natively now

I presented IndieAuth at the W3C Workshop on Strong Authentication & Identity in December, and even published a video of the talk afterwards!

At API Days Global, oauth.io presented a session including IndieAuth.

Josh Hawxwell gave a talk at NottsJS called Indie What? where he covered several IndieWeb building blocks including IndieAuth.

In July, I wrote a blog post called OAuth for the Open Web, where I detailed the technical solutions IndieAuth provides on top of OAuth to enable it to work in a more open and less corporate environment.

In October, I published Dweb: Identity for the Decentralized Web with IndieAuth on the Mozilla Hacks Blog.

So here's to a productive year for IndieAuth in 2018! Looking forward to seeing what new developments come up in 2019!

Inspired by cleverdevil's monthly summary pages, I started working on my own version of them on my website!

My month permalinks have shown a calendar grid of all the posts I made during that month for a while, but it's not particularly easy to skim that and actually understand anything from it.

I had recently added a summary of my modes of transport for the month, which is a fun way to see how much I bike in a month compared to how much I'm in airplanes.

Today I added a few new sections below the calendar.

First is a list of all the photos I've posted that month. They are displayed in the same style as my photo albums, which are full-width justified, uniform height. It's a neat trick I learned from Flickr, and provides a nice looking photo grid of uniform height without cropping any images.

The next section shows a map view of all my checkins for that month. I didn't want to show a pin for every checkin, since that would be too many pins on the map clustered too close together. So instead I started by grouping by city name. That works pretty well when I travel to a few different cities in the month. 

The problem with that approach is if I don't leave Portland for a whole month (it rarely happens these days but still), then the map looks like I didn't even leave a single spot. So instead, I actually group by latitude and longitude rounded to the nearest 0.1 decimal. 

That way when I don't leave Portland, at least it shows a few map pins around the city. 

I should probably switch this to use an actual map pin clustering algorithm, but this was easy and is good enough.

Below that is a section that shows a list of my "popular posts". I decided to rank my posts based on the number of interactions they've received. Replies are weighted the highest, followed by reposts then likes. So if something gets 10 replies, it will beat out a post with only 1 repost. I chose these weightings pretty arbitrarily, spot-checking a few months of data at a time. We'll see how I feel about it after some time.

So I'm pretty excited to launch this! Thankfully I already had enough things indexed in my database that it didn't require a lot of backend changes, it was mostly just UI and design work. We'll see how this feels for a few months and maybe there will be more things I want to add to it later, but this feels like a good start!

Thanks to cleverdevil for the inspiration!

For the past week or so, I've been getting a series of Pingbacks from a spam blog that reposts a blog post a couple times a day as a new post each time. It's up to about 220 copies of the post, each one having sent me a Pingback, and each one showing up in my reader as a notification, which also causes it to be sent to my phone.

Since I use webmention.io to handle my incoming Webmentions (and Pingbacks), this would be the best place to block the site, rather than filtering it out in my reader or my website. 

Webmention.io previously had no way to actually completely block a domain. As Webmentions have started growing in popularity, it's become obvious that we need more tools to combat spam and abuse. While this site was actually sending me Pingbacks, the same applies to Webmentions.

Today I added a new feature to webmention.io to allow people to entirely block a domain, and delete any webmentions received from that domain. 

From the dashboard, you can click the "X" on any recent webmention, or you can paste a URL from one you've received in the past. You'll be taken to this screen where you can either delete just the one webmention, or entirely block the domain.

Once you've blocked the domain, it will show up in your blocklists page!

I hope this helps others keep out spam as well! I'm sure looking forward to never seeing that notification on my phone again!

Today I launched some updates to OwnYourSwarm, the service that sends your Swarm checkins to your own website. It does this by watching your Swarm account and sending checkins to your site via Micropub. 

Private Posts

I made two changes to how OwnYourSwarm can handle private posts. Private posts are currently an experimental feature in Micropub, accomplished by adding a new property to posts called "visibility". The WordPress Micropub plugin already has support for this property, so if you use WordPress you can start using this feature today!

If you mark a checkin in Swarm as "off-the-grid", now OwnYourSwarm will include visibility=private in the Micropub request. Your website can recognize this property and handle setting the post as private.

There is an additional checkbox to have OwnYourSwarm always mark the post as private. You can use this to import all your checkins to your site as private posts in case you don't want all your checkins public on your website.

Automatically Add Tags to Posts

Like OwnYourGram, there is now a way to have OwnYourSwarm always include a list of tags in your checkins. You can use this to automatically set your checkins into a specific page on your website by adding the tag "checkin" for example.

Webmention Settings for Comments

By default, OwnYourSwarm will always try to send Webmentions for any responses to your checkins, such as when someone likes or comments on your checkin, as well as when Swarm itself shows the little tidbits like "11 weeks in a row at coffee shops" awarding coins.

There are now two additional settings you can use to decide whether you want to receive Webmentions for either type of responses. If you'd prefer to not get the Swarm coins, you can disable that. If you'd prefer not to have anybody else's comments appear on your checkins, you can disable that as well.

I also added a user agent string when OwnYourSwarm sends webmentions so that you'll be able to identify the HTTP requests in your server logs.

Hope you enjoy continuing to own your checkins!

I've been looking for ways to slim down the amount of equipment I need to bring to record conferences talks, both to make it easier to travel to other cities, as well as to speed up the setup time during an event.

This post outlines my current favorite set of hardware for recording and livestreaming conference talks and meetups. I'm able to fit all of this into a backpack and carry it on my bicycle to local events, or take it on planes to events in another country.

Video Switcher

The SlingStudio system has been a total game changer, packing an unbelievable amount of power into a tiny box.

At the core of the system is the SlingStudio Hub. This is the brains of the operation: a video switcher, encoder, and recorder. This device broadcasts its own wifi hotspot, which you can then connect your cameras and controllers to. You can use any HDMI camera as a video source by using a CameraLink to wirelessly connect the HDMI device, or you can use any iOS or Android device as a camera as well. This provides a great opportunity to have a super compact rig, since iPhone cameras are actually pretty good now.

For small productions, I will bring one iPhone camera, and one camcorder connected via HDMI. The iPhone provides a good enough picture for a wide shot of the room, and the camcorder provides good optical zoom and low light capability for a close-up of the presenter.

The rear of the device provides a few ports, most importantly an HDMI input and audio input, which means you have a built-in way to grab the slides from the presenter as well as a good audio feed.

I usually plug in an HDMI 1x2 splitter into the input, so that I can send the presenter's computer to the house projector as well as this device. This makes the computer show up as a camera angle in the switcher. The hub can accept a huge variety of HDMI resolutions as the input and it handles scaling itself. I haven't yet found a computer that this device couldn't handle.

Switcher Controller

The Hub is controlled via an iPad or Mac app. I usually use an iPad since it's nice having a dedicated device with a touch screen for this, plus it's easier to walk around with the iPad.

You start by connecting the iPad to the Hub's wifi hotspot that it broadcasts, then when you launch the SlingStudio Console app, it will connect to the Hub and provide you with a controller interface to see all the camera angles and switch between them.

Cameras

Since my goal is to have this pack up entirely in a backpack, I wanted to find the smallest options for cameras, even if it comes somewhat at the expense of quality. I typically use one or two iPhone SEs ($100 used), and a Canon Vixia HF R500 ($200). (The R500s are discontinued, replaced by the Canon Vixia HF R800, which is only a minor upgrade).

Close-Up

I use the Canon for a close-up shot of the presenter, since it has optical zoom and is pretty good at low light. The camera itself doesn't weigh much, so it can fit on a small tripod. I use a short tripod with a monopod extension, which has the benefit of having a pretty small footprint. This wouldn't hold up a DSLR when fully extended, but handles the Canon just fine.

The Canon camera also has an audio input, so I can connect a wireless microphone receiver to this such as the super compact Sony ECMAW4 Bluetooth microphone.

The Canon provides a mini HDMI output, which connects to the micro HDMI input on the SlingStudio CameraLink transmitter. The CameraLink connects wirelessly to the Hub, so I can place the camera wherever. The battery in the CameraLink lasts a couple hours, long enough that I don't need to worry about it for a short talk, but if I'm going to be filming for a whole evening I'll make sure to connect it to micro USB power or at least an external battery pack.

Wide/Rear View

With one good close-up view of the presenter, I just need a wide view as a secondary or fallback camera angle. I usually place this camera in the back of the room so that the audience as well as the projector are visible. Since I have both a close-up of the presenter as well as their slides brought in directly, I can get away with using an iPhone as this camera angle despite its slightly reduced quality.

The iPhone SE has a pretty decent camera. It's the same cameras as in the iPhone 6s line, but you can get a used iPhone SE for about $100 now, making this the cheapest way to get another camera angle into the mix.

To use the iPhone as a camera, you first connect the iPhone to the Hub's wifi hotspot that it broadcasts. Then you launch the SlingStudio Capture app and it will instantly show up as a camera angle in the Hub. You can long-press on the iPhone screen to lock the focus and exposure as well, which is useful when your presenters have slides that switch between white and black backgrounds, which would otherwise confuse the auto-focus and auto-exposure that the phone does.

Running the camera and wifi on the iPhones the whole event would drain the battery in about a half hour, so I always make sure to plug in the phone during the event. The stock charger cable is usually too short to do anything with, so I bring a 10' lightning cable which gives me enough length to run it to a power outlet somewhere nearby.

Audio

Audio is of course a huge part of getting a good quality recording of a presentation. The camera mics built in to camcorders or the iPhone will not get good results at all, since they aren't that good to begin with plus the devices will be typically 10+ feet from the presenter. Instead, you need a microphone super close to the presenter, like a lav mic or handheld mic.

Depending on the situation, I have a few different setups I use for capturing audio. If the venue is providing amplification for the presenter, then I first try to find a way to tap into the house audio. The Hub has an 1/8" input, so I just need to make sure to place the Hub close enough to the house mixing board to run an audio cable to it.

If I need to bring my own microphones and audio gear, or if I'm recording a discussion around a table, then I'll bring either a wireless mic or wired stage or boundary microphones.

Wireless Microphones

The smallest wireless mic I've found is the Sony ECMAW4 Bluetooth microphone. The transmitter and receiver are both the same shape, both only slightly larger than the AAA battery that powers them. It's relatively inexpensive too, at $150. There is a microphone built in, but it also has an 1/8" jack to connect a lav mic. You definitely get better audio using a lav mic, so if you can manage asking the presenters to wear that I highly recommend it.

You can connect the receiver to either the SlingStudio Hub or the Canon camera using a 1/8" cable.

Tabletop Microphone

If you need to capture audio around a whole table, or if you can't get your presenters to wear a microphone, then the best option is to place a boundary microphone on the podium.

Audio Technica makes a fantastic wireless mic system that runs on 2.4ghz rather than a dedicated wireless mic frequency. There are some new FCC regulations coming that will re-allocate the frequencies that many wireless mics use, so it will no longer be legal to operate many of those. This system uses 2.4ghz, the same frequencies that Bluetooth and Wi-Fi use, so will always be safe to use.

Pair the ATW-T1006 Boundary Mic with the ATW-R1700 Receiver and connect the receiver to the Hub. The benefit of using a wireless mic for this, of course, is you don't need to worry about placing the Hub nearby the stage or podium. This also cuts down on the amount of wires you need to bring, which can save a lot of packing space.

Wired Microphones

For one reason or another, you may find it better to use a wired microphone. A wired microphone will usually provide better audio quality and be more stable than wireless mics, though it does come at the cost of more wires to carry and more setup time to connect them.

The Saramonic SmartRig+ 2-Channel Mixer is a small mixer that provides phantom power so you can use nice microphones with it. It plugs into the 1/8" jack on the SlingStudio hub. It does run on a 9V battery with no external power option, although I haven't been able to drain the battery during a normal one-day event yet.

I have two microphones I usually use depending on the situation.

• Shure Boundary Condenser Microphone - This microphone will pick up everyone sitting around a table. You can also set it on a podium to not worry about your presenters fiddling with microphones. If you have a really large room, connect two of these to the mixer.
• Shure SM58 - The Shure SM58 is a long-time standard in stage microphones. It works best when the speaker is a few inches away from the mic, and it does an amazing job of isolating sounds to avoid background noise. You'll need to either ask your presenters to hold the microphone, or give them a stand that places the mic close to their mouth.

Livestreaming

Once all the audio and video sources are connected to the hub, you can press record and everything gets recorded as both the mixed track as well as individually. If you make a mistake while live cutting between camera angles, you can always recover by grabbing the original footage from the camera angle you need.

The other amazing thing is the SlingStudio Hub also has built-in streaming capabilities. It can connect to the venue's wifi, and then connect to Facebook or YouTube to broadcast a livestream. This makes it super easy to both stream an event live, while also recording the raw video for later editing.

There's not much more to say other than that, pretty much you just connect to a wifi hotspot and press stream. I've had good luck even streaming from an iPhone's wifi hotspot. You can choose the bitrate to stream at, anything from 2-5 mbps will give you a good result.

I'm super impressed that this device lets me pack so much into a single package instead of using separate devices for each. This all fits into a backpack, along with my computer and other electronics I normally bring.

Published Videos

Here are some videos I've produced with this rig so you can see the final results!

• Donut.js April 2018
• IndieWeb Summit 2018

Ever since Donut.js has had live captions for their talks every month, I've been curious about how people can learn to type that fast. That led me down quite the rabbit hole of internet research, where I stumbled into the open source stenography community, largely pioneered by Mirabai Knight, who does the captions for Donut.js! (small world!)

I thought it would be fun to try to learn some basic stenography, so I set out to find a keyboard for this. There are several existing QWERTY keyboards you can use, (the keyboard needs to be able to support pressing several keys at the same time, which most keyboards, especially laptop keyboards, do not), and there are a few custom keyboards you can build as well.

I first learned about the Gherkin keyboard from the Plover Blog. It's a tiny keyboard with 3 rows of 10 keys, where the keys are arranged in a grid rather than staggered like typical keyboards. It's commonly referred to as a "30% keyboard" in the mechanical keyboard community. This key arrangement lends itself well to the steno key layout, which looks like this.

The steno keyboard requires 24 or 26 keys (depending on whether you count the S and * as one or two keys), so this 30-key keyboard has more than enough to handle it.

Being brand new to stenography, as well as the mechanical keyboard community, it was a real trick to learn all the new terminology needed to piece this together. I haven't actually bought a keyboard in a long time, much less assembled one from scratch.

The Gherkin kit is really well done, but that's still way more soldering than I am comfortable doing myself. Thankfully, Paul on the Plover chat was offering to do all the soldering if anyone would just ship all the parts to his address. I gladly took him up on that offer, and quickly placed an order from three different online stores for all the pieces.

Electronics Part List

• Gherkin Kit from Spacecat.design (just the PCBs)
• Gateron Switches (30 Clear) from Flashquark (this is the actual key mechanism)
• Arduino Pro Micro
• 1N4148 Diodes (50)

Eventually these all arrived at Paul's house, and he sent some in-progress pictures while he was soldering.

In order to get the whole thing to be thinner, he even filed down the solder joints on the Arduino that's on the bottom, to save a few millimeters.

Final Assembly

The soldered board showed up in my mailbox, which was pretty magical. (Thanks again Paul!) 

I then needed to attach the two plates and add the keycaps.

Mechanical Part List

• Blue and Grey G20 Keycaps from pimpmykeyboard.com (30 Blue BCT, 10 Grey GDE)
• M2 x 4mm Screws (28)
• M2 x 5mm Standoffs (14)
• Clear rubber feet

The keycaps were the hardest part to figure out, since there are so many options. I needed ones compatible with the switches I got, which are the "Cherry MX" style. I settled on the "G20" style, which are slimmer than normal keyboards, and perfectly square and flat. I bought two different colors, using the blue ones for the keys that are actually on the steno keyboard, and grey for the unused keys.

It took me a few tries to get screws and standoffs of the right size, since I wanted this to be as thin as possible. I ended up with 5mm standoffs, and 4mm screws. Adding the thickness of the PCB, the screws end up almost meeting in the middle of the standoffs, and it leaves just barely enough room for the Arduino sandwiched between the two boards.

Lastly, I wanted to find a nice case for this. Turns out there is a case for a portable hard drive that fits this surprisingly well, once you cut out the little middle flap.

So that's been an exciting adventure so far! Now I just have to get through the somewhat daunting number of steno lessons and practice practice practice!

OAuth has become the de facto standard for authorization and authentication on the web. Nearly every company with an API used by third party developers has implemented OAuth to enable people to build apps on top of it.

While OAuth is a great framework for this, the way it has ended up being used is much more centralized and closed than prior efforts like OpenID 1. Every service that spins up an OAuth-enabled API ends up being its own isolated system. For example, if I want to build an app that can read someone's step count from FitBit, I have to first go register as a developer on FitBit's website in order to get API keys to use with their OAuth API. 

This works okay for major services like Google, Twitter, Facebook, and even FitBit, but breaks down when you start to consider use cases like having someone's personal WordPress blog be its own OAuth server. If I want to build an app that lets you upload photos to your WordPress site, I'm obviously not going to be able to register for API keys on everyone's own WordPress installations. Enabling third party clients to be built against systems like WordPress or Mastodon opens up a huge possibility for some really interesting things. The trick is always how do these apps authenticate the user or obtain a token they can use to access those APIs.

This post details a few specific challenges with OAuth preventing it from being used by independent websites, as well as the solutions to each.

Client Registration

The first major hurdle to overcome is the need for the developer to register to get API keys for the service. In a world where everyone's own website is its own OAuth server, it's obviously not practical to have an app developer register API keys at each.

In OAuth, client registration gives us a few specific things:

• Provides a unique ID that is used to identify the app throughout the OAuth process, called the client ID
  
• Provides a place to enter the name and icon for the app which is displayed during login
• Registers one or more redirect URLs for security
• For "confidential clients" (web server apps), registration also provides the client with a client secret

Note that in traditional OAuth, client secrets are not used by mobile apps or JavaScript apps, and OAuth servers will often not even issue secrets to those types of apps. Since we're trying to avoid registration entirely, we can also just avoid using client secrets at all, and leverage the same protections OAuth already has in place for clients that can't use a secret.

In order to avoid registration, we need a solution for the first three bullet points above.

Client ID: Every application needs a unique identifier. If we're talking about turning every website into an OAuth provider, we need a way to have globally unique identifiers for every OAuth app. It turns out we already have a mechanism for this: URLs! In this Open Web version of OAuth, client IDs can be the application's URL. For web-based apps, this is straightforward, as it's simply the website the app is running on. For native apps, this can be the application's "about" page.

Application name and icon: Since the application's client ID is a URL, we can assume every application has a web page that talks about it, and treat that web page as the place the client defines its own metadata like name and icon. A simple way to accomplish this is with Microformats, so that the application's web page just needs to add a couple classes around the name and icon of the app. This is currently documented and implemented as the h-app microformat.

Redirect URL registration: This one is a bit more subtle. The purpose of redirect URL registration is to prevent an attacker from tricking an authorization server into sending authorization codes to the attacker. This becomes especially important when we aren't using client secrets. The trick is that since client IDs are already URLs, we can shortcut the normal registration process by declaring a rule that redirect URLs have to be on the same domain as the client ID. This way, we can avoid a situation where an application claiming to be good.example.com sets a redirect URL to attacker.example.org and steals the authorization code. The only way to get the authorization code to attacker.example.org would be to set the client ID to that domain as well, which a user would hopefully notice.

User Accounts

There are two different situations to consider with regards to user accounts: authentication and authorization. Authentication is the process of proving the identity of the person signing in. Authorization is how an application obtains permission to do something to someone's account.

When we talk about authentication, we are talking about wanting to allow an unknown user to identify themselves to the site they're logging in to. Common examples of this are using your email address as your identity to sign in to a website. You bring an existing identity (your email address) and then authenticate (usually by clicking a link that was sent to your email). The original version of OpenID was created to solve this problem on the web. People identified themselves with a URL, which they were able to prove they controlled using OpenID. This allows a new user to log in to a site without needing a prior relationship with the site.

When we talk about authorization, the situation is subtly different. In this case, we're talking about a user of a website wanting to give permission to a third-party app to access some part of their account. We're very used to this pattern now, which is the typical OAuth use case of granting an application the ability to access your Google Calendar, or logging in to a third party Twitter app.

Authorization: There isn't really a challenge unique OAuth on the Open Web with regards to authorization. Once the client registration problem is solved, everything else falls into place nicely. It is assumed that users are authorizing an application to access an account they already have, so the application will just end up with an access token that works with their existing account.

Authentication: Where we need to define some new behavior is talking about authentication. In this case, we want users to be able to bring an existing identity and use it to log in to other places. This means we need a way to uniquely identify users across the entire web. We can again use URLs as the solution! Every user is identified by a URL. This can be a short URL like someone's domain name, e.g. https://aaronparecki.com/, or for a site with multiple users, can be a URL that contains a path specifying a particular user on the site, e.g. https://github.com/aaronpk. 

Discovery

With traditional OAuth services, discovery is not needed since the application author knows which OAuth server they're talking to before they start building the app. There is typically a "Sign in with ____" button in the application that begins the authorization process. In the case of using OAuth for authentication, the common pattern is to include buttons for several common "social login" providers such as Facebook, Google, Twitter and LinkedIn. Before the "social login" space essentially consolidated to these four, there were sometimes a dozen of these buttons on an application's login page, which eventually became known as the "NASCAR problem".

In a world where every WordPress or Gitlab site is its own OAuth provider, there obviously can't be a button for each on a login screen. Instead, we need to find out from the user which server to use to authenticate them. 

Since we previously stated that every user identifier is a URL, we can ask the user to enter their URL in the sign-in screen, and then fetch that URL and discover their authorization server from there.

Once we've found the user's authorization endpoint, we can start a normal OAuth request and send them to their server to authenticate. When the server redirects back to the application, it will go and verify the authorization code with their authorization endpoint just like normally happens with OAuth.

Knowing Who Logged In

While knowing any user identity information is technically not part of OAuth, we do need the server to return a user identifier when using OAuth for authentication. In practice, most applications also want at least a unique user identifier in the authorization case as well.

We've previously said that user identifiers are URLs, which solves the global user identity problem, and gives us a mechanism to discover the user's OAuth server. So all we need is a way to return this information to the application after the user has authenticated. 

OAuth gives us an easy opportunity to return this to the application: in the access token response when the application sends the authorization code to obtain an access token. The server can at that point return the full user identifier of the user that logged in. As long as the domain name matches the domain that the user entered at the start, the application can consider it successful. This also gives the authorization server the opportunity to canonicalize the user identifier, correcting "http" to "https", or adding a path component to the user's profile URL.

Let's do this!

By now, hopefully you're thinking "this sounds great, Aaron, someone should write this up as a OAuth extension!" I'm glad you asked!

Earlier this year, I wrote this all up as an extension to OAuth 2.0, called IndieAuth. IndieAuth encapsulates these small additions needed for OAuth 2.0 to work in the Open Web.

Despite this spec being published in January, it has actually been implemented for several years before that. There are many implementations of this extension on the server side, everything from standalone authorization server projects, to a WordPress plugin, and it's even implemented by a commercial service, Micro.blog. As far as consuming apps, nearly every Micropub app has implemented this for logging users in.

For further details on implementing this extension, there are several guides available depending on whether you're writing a client, a server, or just part of a server.

• Authenticating users with IndieAuth
• Obtaining an access token with IndieAuth
• Creating an Authorization Endpoint
• Creating a Token Endpoint

There are a few existing open source projects you can use to get started if you don't want to write your own!

• selfauth - a standalone authorization server using a simple password login
• IndieAuth for WordPress - a plugin that turns your WordPress install into an OAuth 2.0 server
• IndieAuth for Drupal - a Drupal plugin that provides a built-in OAuth 2.0 server
• Acquiescence - an authorization endpoint written in Ruby that authenticates users via GitHub

For further reading, check out the IndieAuth spec. Feel free to drop in to the IndieWeb chat if you'd like to talk about this, or you can reach me on Twitter or from my website.

Webmention is one of the fundamental indieweb building blocks. It enables rich interactions between websites, like posting a comment or favorite on one site from another site. This post will walk you through the simplest way to get started sending webmentions to other sites so that you can use your own site to join the conversations happening on the Indie Web.

So what do you need to walk through this tutorial? We'll use static files and simple command line tools so that you can easily adapt this to any environment or programming language later.

Get started

First, we'll create a new HTML file that we'll use to contain the comment to post. At the very minimum, that file will need to contain a link to the post we're replying to.

<!doctype html>
<meta charset="utf-8">
<title>Hello World</title>
<body>
  <p>in reply to: <a href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p>
  <p>Trying out this guide to sending webmentions</p>
</body>

Go ahead and copy that HTML and save it into a new file on your web server, for example: https://aaronpk.com/reply.html. Take your new post's URL and paste it into the webmention form at the bottom of this post. After a few seconds, reload this page and you should see your post show up under "Other Mentions"!

Making it look better

That's a great start! But you might be wondering where your comment text is. To make your comment show up better on other peoples' websites, you'll need to add a little bit of HTML markup to tell the site where your comment text is and to add your name and photo.

Let's take the HTML from before and add a couple pieces.

<!doctype html>
<meta charset="utf-8">
<title>Hello World</title>
<body>
  <div class="h-entry">
    <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p>
    <p class="e-content">Trying out this guide to sending webmentions</p>
  </div>
</body>

Note the parts added in green. These are Microformats! This tells the site that's receiving your webmention where to find specific parts of your post. We first wrap the whole post in a <div class="h-entry"> to indicate that this is a post. Then we add a class to the <a> tag of the post we're replying to, as well as a class to the element that contains our reply text.

Now, take your URL and paste it into the webmention form below again. After a few seconds, reload the page and your reply should look more complete here!

Now we see the text of the reply, and also notice that it moved out of the "Other Mentions" section and shows up along with the rest of the replies!

Of course this web page still looks pretty plain on your own website, but that's up to you to make it look however you like for visitors visiting your website! As long as you leave the h-entry and other Microformats in your post, you can add additional markup and style the page however you like!

Adding your name and photo

Let's make the comment appear with your name and photo now! To do this, you'll need to add a little section to your web page that indicates who wrote the post.

In Microformats, the author of a post is represented as an h-cards. An h-card is another type of object like h-entry, but is intended to represent people or places instead of posts. Below is a simple h-card that we'll add to the post.

<div class="h-card">
  <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40">
  <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a>
</div>

When we add this h-card into the post we've written, we need to tell it that this h-card is the author of the post. To do that, add the class u-author before the h-card class like the example below.

<!doctype html>
<meta charset="utf-8">
<title>Hello World</title>
<body>
  <div class="h-entry">
    <div class="u-author h-card">
      <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40">
      <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a>
    </div>
    <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p>
    <p class="e-content">Trying out this guide to sending webmentions</p>
  </div>
</body>

Now when you re-send the webmention, the receiver will find your author name, photo and URL and show it in the comment!

Great job! If you've successfully gotten this far, you're now able to comment on things and even RSVP to events using your own website!

One more detail that you'll want to include on your posts is the date that your post was written. This will ensure the receiving website shows the correct timestamp of your post. If you eventually incorporate this into a static site generator or CMS where you show a list of your replies all on one page, then you'll also want to add a permalink to the individual reply in this post. Typically an easy way to solve both is with the markup below.

<a href="https://aaronpk.com/reply.html" class="u-url">
  <time class="dt-published" datetime="2018-06-30T17:15:00-0700">July 30, 2018</time>
</a>

We can add that to the post below the content.

<!doctype html>
<meta charset="utf-8">
<title>Hello World</title>
<body>
  <div class="h-entry">
    <div class="u-author h-card">
      <img src="https://aaronpk.com/images/aaronpk.jpg" class="u-photo" width="40">
      <a href="https://aaronpk.com/" class="u-url p-name">Aaron Parecki</a>
    </div>
    <p>in reply to: <a class="u-in-reply-to" href="https://aaronparecki.com/2018/06/30/11/your-first-webmention">@aaronpk</a></p>
    <p class="e-content">Trying out this guide to sending webmentions</p>
    <p>
      <a href="https://aaronpk.com/reply.html" class="u-url">
        <time class="dt-published" datetime="2018-06-30T17:15:00-0700">July 30, 2018</time>
      </a>
    </p>
  </div>
</body>

Automatically sending webmentions

The last piece to the puzzle is having your website send webmentions automatically when a new post is created.

This part will require writing some code in your particular language of choice. You'll start by making an HTTP request to get the contents of the page you're replying to, then looking in the response for the webmention endpoint.

We can simulate this on the command line using curl and grep.

curl -si https://aaronparecki.com/2018/06/30/11/your-first-webmention | grep rel=\"webmention\"

The response will include any HTTP Link headers or HTML <link> tags that have a rel value of "webmention".

Link: <https://webmention.io/aaronpk/webmention>; rel="webmention"
<link rel="webmention" href="https://webmention.io/aaronpk/webmention">

If you get more than one, the first one wins. You'll need to extract the URL from the tag and then send the webmention there.

Sending a webmention is just a simple POST request to the webmention endpoint with two URLs: the URL of your post (source) and the URL of the post you're replying to (target).

curl -si https://webmention.io/aaronpk/webmention \
  -d source=https://aaronpk.com/reply.html \
  -d target=https://aaronparecki.com/2018/06/30/11/your-first-webmention

The only significant part of the response is the HTTP response code. Any 2xx response code is considered a success. You'll most often receive either a 202 which indicates that the webmention processing is happening asynchronously, or if the receiver processes webmentions synchronously and everything worked, you'll get a 201 or 200.

In practice, you'll probably use a library for discovering the endpoint and sending the webmention, so here are a few pointers to start you out in a variety of languages.

• Ruby
• PHP
• Node
• Python
• Go
• Elixir
• ...more on indieweb.org

Hopefully this guide was helpful to get you going in the right direction!

If you want to dive into the weeds, check out the Webmention spec as well as more details on reply posts.

When you want to put your automatic webmention sending implementation to the test, try sending webmentions to all of the links on the test suite, webmention.rocks!

If you have any questions or run into any issues, feel free to ping me or anyone else in the IndieWeb chat!

Using the HTML <input type="url"> field is normally a good idea when you're asking the user to enter a URL. It doesn't make a huge difference on desktop browsers, but makes it a lot easier on mobile browsers. On iOS, when you've focused on a URL input field, the keyboard switches to a slightly different layout with different keys optimized for entering URLs.

This is great for things like web sign-in where you're asking the user to enter their domain name to sign in. However, for some reason, browsers have implemented the URL validation a bit too strictly. If you don't enter a URL scheme, you'll get an error like the below if you try to submit the form.

This is pretty irritating because it forces the user to enter the URL scheme http:// or https://, which ironically on the special iOS URL keyboard, requires tapping the "123" button in order to get to the screen to type the ":" character. It would be nice if the URL field accepted plain domain names and defaulted to http://.

I wrote a bit of JavaScript that will prepend http:// to the value of any <input type="url"> form fields on blur.

<script>
  /* add http:// to URL fields on blur or when enter is pressed */
  document.addEventListener('DOMContentLoaded', function() {
    function addDefaultScheme(target) {
      if(target.value.match(/^(?!https?:).+\..+/)) {
        target.value = "http://"+target.value;
      }
    }
    var elements = document.querySelectorAll("input[type=url]");
    Array.prototype.forEach.call(elements, function(el, i){
      el.addEventListener("blur", function(e){
        addDefaultScheme(e.target);
      });
      el.addEventListener("keydown", function(e){
        if(e.keyCode == 13) {
          addDefaultScheme(e.target);
        }
      });
    });
  });

</script>

I wish browsers would implement this themselves, but in the mean time you can use this bit of JS to provide a bit better user experience when asking your users for URLs.

I've made the difficult decision to drop support for Twitter authentication on IndieAuth.com. Some time last week, Twitter rolled out a change to the website which broke how IndieAuth.com verifies that a website and Twitter account belong to the same person.

Since I am already in the process of replacing IndieAuth.com with two new websites (lots of discussion on the wiki), it is not worth the effort to do what it would take to fix this for IndieAuth.com.

What Changed on Twitter.com

In order to verify that you are the person behind the URL you initially type in, IndieAuth.com checks your website to find a link to a Twitter profile, then checks that Twitter profile to see if it links back to your website. If there is a match, then you'll see the green button for Twitter on IndieAuth.com.

Twitter rolled out a change that prevents normal HTTP requests from returning actual HTML on Twitter profiles. I'm assuming this is part of their effort to fight bots, but it's unfortunate this use case got caught up in that mess. If you visit your Twitter profile in a browser and click "view source", you'll see something like this now.

This is a delightful bit of HTML that sets a cookie via Javascript and then reloads the page. Presumably this happens so quickly that normally you won't notice it.

Fetching a profile URL with curl now returns an empty HTTP body.

Even if I go through the hoops to make IndieAuth.com set cookies and refresh the page, there's no guarantee that they won't just change this again next week. I don't like playing these games, so instead I am just shutting off Twitter support in IndieAuth.com.

Replacing IndieAuth.com

The new version that you'll eventually use to sign in to the IndieWeb wiki is called IndieLogin.com. It is currently in beta, and is not available to other developers, but you can try signing in to the test page there right now. This new version gets around this Twitter problem by not even attempting to fetch Twitter profile pages in the first place.

The new login flow works like this:

• You enter your website on IndieLogin.com
  
• IndieLogin.com finds your Twitter profile by checking all rel=me links for one matching twitter.com
• IndieLogin.com shows you a button to authenticate with Twitter immediately (rather than first checking that your Twitter profile links back)
• After you authenticate on Twitter and are redirected back to IndieLogin.com, it fetches your Twitter profile from the Twitter API
• If your Twitter profile as reported by the API includes the initial website you started with, then you're authenticated

This avoids the problem because IndieLogin.com never tries to fetch your Twitter profile HTML. Instead, it uses the API directly. This does mean that you can get into a situation where IndieLogin.com may prompt you with a Twitter button that can fail (if you are logged in to a different Twitter account than the one your website links to). However, it also speeds up the initial login prompt since it doesn't have to go check Twitter before showing you the login button first.

Hopefully I'll be able to launch IndieLogin.com soon so that the lack of Twitter support on IndieAuth.com isn't too annoying. In the mean time, you can authenticate via GitHub or email on IndieAuth.com.

IndieWeb Summit is soon, and is shaping up to be an exciting event! We're hosting IndieWeb Summit the same week as the (final) Open Source Bridge, in case you needed another reason to visit Portland! IndieWeb Summit will be Tuesday-Wednesday June 26-27th, with a pre-party the Monday evening before.

If you're at all interested in taking back ownership of your online data, decentralizing the web, independent blogging, or any aspect of having a website, you should consider joining us for this event!

As gRegorLove said so well:

It’s a really friendly, collaborative group of people and it is always inspiring to see what people are making.

You don’t need to be a programmer! In fact, I would love to see more non-programmers attending. We need writers, graphic artists, designers, UX engineers, and anybody that wants to reclaim some of their online presence with a personal website.

Keynotes

One of the distinguishing features of IndieWeb Summit compared to the IndieWebCamp events we run in many other cities throughout the year is we begin day 1 with a few keynote presentations to help set the stage for the two days. This year we're featuring a few special guests during the keynotes.

Manton Reece will give a talk about how Micro.blog works with open standards to encourage people to own their data while also making a service that is incredibly fun and easy to use.

William Hertling, the author of Kill Process, a book that features the IndieWeb, will talk about his inspiration for writing the book and where he sees the future of the IndieWeb heading.

We've been seeing some exciting progress with IndieWeb readers over the last few months, between my reader Monocle, Eddie's iOS app "Indigenous", and Jonathan and Grant's app "Together". We'll be sharing the latest developments along that front as well!

Related Events

In addition to IndieWeb Summit, the whole week will be a great lineup of events!

• Monday, June 25th 5:30pm - Pre-summit meetup at Pine Street Market
• Tuesday, June 26th 9am-5:30pm - IndieWeb Summit Day 1 - Keynotes and Discussions
• Tuesday evening 6:30pm - Donut.js
• Wednesday, June 27th 9am-5:30pm - IndieWeb Summit Day 2 - Create, Hack, Demos!
• Friday, June 29th - Open Source Bridge unconference and party

I hope to see you there! You can register now at 2018.indieweb.org!