Aaron Parecki

If you know me, you probably know that I log everything I eat and drink and post it to my website. A couple years ago, I wrote a small Pebble app that allowed me to quickly post common food and drink from my watch! Coincidentally around the time Pebble announced that FitBit had acquired their assets, my Pebble stopped working completely. This meant I no longer had a quick way to log food, and have to pull out my phone again to make log entries.

This afternoon, Tantek suggested that I use my Amazon Alexa to post food and drink to my website instead! Of course this will only work when I'm at home, but it turns out that I'm home a lot of the time I'm eating and drinking. I also eat tacos every day, so it'd be great not to have to get out my phone during breakfast.

So today, I launched Alexa integration for Teacup, the app I use to track my food.

This was quite a challenging project given all the moving parts involved. I started by defining the voice interface I wanted to use. 

Interaction Model

Voice interactions for Alexa apps have to follow a pretty strict structure. Alexa doesn't support interpreting fully unstructured text, so app developers have to define patterns that Alexa can match on. Invoking any Alexa app involves first speaking the trigger word, followed by a keyword such as "ask" or "tell" followed by the app name, and then the pattern of text the app wants to match. So for Teacup, this results in speaking sentences such as:

• "Alexa, tell Teacup I drank coffee"
  
• "Alexa, tell Teacup I ate tacos"

This gets turned into what Amazon calls the "Interaction Model", and is a list of "slots" along with corresponding keywords for each slot, as well as writing out some sample sentences.

• LIST_OF_ACTIONS = ate | drank
• LIST_OF_FOOD = Coffee | Cocktail | Beer | Tacos | Mac and Cheese | ...

Sample utterances:

• "I {Action} {Food}"
• "I {Action} a {Food}"

It isn't clear to me whether the list of keywords I provided is the complete set, because while I was testing, it managed to post the word "on" for the food, which is not in my list.

Authentication

The next challenge was linking user accounts between Amazon users and Teacup users. Amazon provides great documentation on this, and thankfully it's all based on OAuth 2.0 rather than having made up some other model themselves. Essentially, the Amazon Alexa app on your phone acts as an OAuth 2.0 client, and you have to build an OAuth 2.0 server into your app that it works against. This is a pretty clever solution actually. Luckily, I'm pretty familiar with OAuth 2.0, so I was able to build this out pretty quickly.

One thing struck me about Amazon's recommendations about building OAuth support in your app. They say that they'll launch your authorization URL from inside the iOS app, which is a known antipattern for apps in general. In Amazon's docs, it says "The user logs in using their normal credentials for your site." This is a really bad idea. You never want to train your users to enter their passwords into random apps. This is the whole reason we have OAuth in the first place!

Rather than asking people to enter passwords in the Alexa app, I opted to solve this a different way while still being compatible with Amazon's service.

When you connect Teacup with the Alexa app, instead of asking for your Teacup password, it asks you to sign in from a real browser and generate a temporary device code.

So you launch a real browser where you can be sure you're logging in to Teacup and not a phishing website, and then visit your settings screen which shows you this.

Clicking the button generates a temporary code.

Then you enter that code in the Alexa app, and it's able to connect the app session to your user account! This way you've never risked typing passwords into the Alexa app. Teacup takes the code entered, verifies that it's active, and then completes the OAuth 2.0 handshake with Amazon.

Now you're logged in to Teacup in the Alexa app! This means the Alexa service now has an access token that it can use to talk to Teacup, which is how the two accounts are associated.

It Works!

Once I got everything hooked up, I had to put it to the test! Of course I can't risk posting fake data, so I had to open a beer and try it out.

Here's the beer it posted to my site!

If you want to see the code involved in making this work, head over to the Teacup GitHub repo. 

In the mean time, I'm looking forward to telling Alexa I ate tacos tomorrow morning!