It’s like #100DaysOfCode but where you only build dumb shit 💁♀️ I won’t post every day, but when I have something dumb and (mostly) finished - I promise to share 💅🏼
Today I made a few changes to XRay to make it easier to deploy in more kinds of environments. I also removed a bunch of CSS/JS dependencies and simplified the UI a bit.
I dropped the CSS framework I was using, and dropped jQuery. All I was using that for was the silly tab interface on the home page, and I figured it wasn't worth all that extra CSS/JS just for that effect.
I rewrote the CSS for the home page inline, to avoid needing to even fetch an external stylesheet.
I then set out to see what it would take to be able to deploy this to shared hosting, especially in a subfolder. I had to do a few things to make that work.
I wanted it to be as easy to install as "download this zip, extract to a folder on your webserver and run." Since I suspect people might not configure their web server to point to the "public" folder as the root, I had to add a new index.php file to the root of the project which just includes the "public/index.php" file where all the magic happens. I also added .htaccess files in all of the other folders to prevent those files from being run by requests. (The web server should never serve files out of the "vendor" or "views" folder directly.)
I published a zip file in the "Releases" section on GitHub which includes all the necessary composer dependencies already bundled in the file. This means you can just extract the zip and run it!
I tested this out by installing it on my Dreamhost account, and it works great!
You can download the latest version of XRay here:
Continuing yesterday's work, today I added support for parsing Twitter URLs to XRay.
There were a couple tricks to make this work. I wanted to make sure that Tweets are always expanded to include the most data possible, and also wanted to avoid needing to make a bunch of HTTP requests. Scraping from the twitter.com website wasn't an option, since some of the data isn't available or would require additional HTTP calls to fetch. (For example I would have to fetch every t.co URL to expand them.) So I set to work using the Twitter API to fetch the tweets.
I didn't want to hit Twitter rate limits by sharing all XRay access from a single account, and I also didn't want to add a database to XRay so that it can continue to be stateless. This meant that the only option was for the XRay client to pass in its own Twitter credentials when fetching twitter.com URLs. This is an acceptable compromise for me, since it keeps XRay simple, and also avoids me needing to officially get a Twitter app approved. If you want to use this feature, you can go to dev.twitter.com and create an app and access tokens for your account right there, which doesn't even involve writing any code. I've updated the XRay readme with further instructions.
Now p3k will include my Twitter credentials when making a request to XRay for a twitter.com URL, and XRay uses my Twitter credentials to fetch the tweet from the API.
So now, whenever I repost something on twitter.com, the contents are expanded and my website shows the full Tweet!
Today I added support for XRay to extract data from Instagram URLs!
This means anything that uses XRay will now return structured data when given an Instagram URL, just like how it parses h-entry and other Microformats.
Unfortunately, Instagram does not provide timezone data for the published date, only a Unix timestamp. So if the photo is tagged at a location, then XRay will look up the appropriate timezone for that location and adjust the timezone of the published date accordingly!
Here's what the parsed JSON looks like for this photo. Note that the timezone is set to East Coast because this photo was taken at MIT.
"name":"Massachusetts Institute of Technology (MIT)",
In addition to my website using this for reposts and comments, when I paste that URL into IRC, Loqi uses XRay to expand it and provide a little text preview.
Quill started out as a sample Micropub client with a lot of useful debugging information. As it's grown, I've been slowly adding more posting interfaces to Quill, and tweaking them to look less like a debugging tool and more like a real app. Instead, micropub.rocks has taken over as a more detailed debugging utility for building a Micropub endpoint, so I can now optimize Quill for being more user-friendly and less developer-focused.
At IndieWebCamp Germany in 2015, I added a rich editor to Quill, inspired by the clean Medium interface. You can even use the Quill rich editor without being logged in! It will save the post to your browser's localstorage as a draft, and let you post it after you log in. I wanted to make an interface that anybody would feel comfortable writing in, and be able to show it to people without them first logging in, as an introduction to what it might look like to have nice writing applications that aren't tied to a specific backend like Medium.
In order to help people get started with Quill, today I wrote some more documentation for it!
When you're logged out and click the "publish" button, it prompts you to log in and links out to the documentation home page.
The documentation page contains a short introduction, targeted at general users of Quill rather than developers, and then links to the specific pages that you'd need to read for developing a Micropub server that can work with Quill to learn about the specific properties it sends.
So far I've written the overview page, and documented the Rich Editor and basic note interface, as well as written about syndication and the new "post status" option.
Happy reading! https://quill.p3k.io/docs
Today I started autolinking the text in comments on my website, and made a couple other minor improvements to how they look.
Previously, links, @-names and hashtags were rendered as plaintext, so comments weren't clickable.
With the new autolinking in place, these now look like the below:
I had to make a backend change to p3k to support autolinking properly. It turns out I hadn't been storing whether the original comment text was HTML or plaintext, and autolinking HTML is very hard I was running into all sorts of edge cases when I was trying to use the autolinker on HTML input. So now I store whether the comment text came in as HTML or plaintext, and only run the autolinker on plaintext comments. This does mean there are some comments where @-mentions or hashtags aren't linked, because the author did not link them up. I've decided it's going to be a separate project to work on autolinking text in HTML.
If comments were written as HTML, I now also display that HTML! Of course the HTML is thoroughly sanitized with HTML Purifier, which is done in XRay. XRay allows a small subset of HTML tags, and removes all attributes except for Microformats 2 values in class attributes. I do display images in comments now which is fun too!
The other improvement I made is if a blog post mentions a post, then I only show the name of the post and don't show any content. This cleaned up the list of mentions a lot, since before I was showing a (poorly) truncated version of the content as well.
Next I'll have to work on autolinking and truncating text in HTML comments, which it turns out is super tricky! I think the correct thing to do is to parse the HTML tree, and run the plaintext autolinker on the text value of each node, except for text inside specific elements such as <a>, <button>, etc.
I've been using Quill to write all these #100DaysOfIndieWeb posts, which is a great way to find the pain points in the interface. After having written 14 articles in the last 14 days, the main thing I want to be able to do is start an article here in the Quill editor, save it as a draft to my website, then open up the raw HTML to make fine-grained edits, and only after I'm done, actually publish it.
I also added tags and slug fields so that I don't have to add those after the fact too!
The "status" property is an experimental extension to Micropub, being documented here: https://indieweb.org/Micropub-extensions#Post_Status I started out this morning by documenting how Wordpress handles post status in their interface. That research has been captured on the wiki.
In order to support this on my website, I first had to recognize the property in the Micropub request to set an internal flag for whether a post is a draft. When a post is a draft, it is not shown in lists to logged-out users, although if you know the permalink, you can see the post when you're logged out.
I also never send Webmentions or publish to the WebSub hub for draft posts. I only send Webmentions after the post is published and publicly viewable.
I'm also interested in incorporating the idea of "visibility" of posts into Micropub and my website. Flickr has a great implementation of this known as the "Guest Pass". I plan to research this more in the future to see if I can incorporate some of that into my website.
If you know me, you probably know that I log everything I eat and drink and post it to my website. A couple years ago, I wrote a small Pebble app that allowed me to quickly post common food and drink from my watch! Coincidentally around the time Pebble announced that FitBit had acquired their assets, my Pebble stopped working completely. This meant I no longer had a quick way to log food, and have to pull out my phone again to make log entries.
This afternoon, Tantek suggested that I use my Amazon Alexa to post food and drink to my website instead! Of course this will only work when I'm at home, but it turns out that I'm home a lot of the time I'm eating and drinking. I also eat tacos every day, so it'd be great not to have to get out my phone during breakfast.
So today, I launched Alexa integration for Teacup, the app I use to track my food.
This was quite a challenging project given all the moving parts involved. I started by defining the voice interface I wanted to use.
Voice interactions for Alexa apps have to follow a pretty strict structure. Alexa doesn't support interpreting fully unstructured text, so app developers have to define patterns that Alexa can match on. Invoking any Alexa app involves first speaking the trigger word, followed by a keyword such as "ask" or "tell" followed by the app name, and then the pattern of text the app wants to match. So for Teacup, this results in speaking sentences such as:
This gets turned into what Amazon calls the "Interaction Model", and is a list of "slots" along with corresponding keywords for each slot, as well as writing out some sample sentences.
It isn't clear to me whether the list of keywords I provided is the complete set, because while I was testing, it managed to post the word "on" for the food, which is not in my list.
The next challenge was linking user accounts between Amazon users and Teacup users. Amazon provides great documentation on this, and thankfully it's all based on OAuth 2.0 rather than having made up some other model themselves. Essentially, the Amazon Alexa app on your phone acts as an OAuth 2.0 client, and you have to build an OAuth 2.0 server into your app that it works against. This is a pretty clever solution actually. Luckily, I'm pretty familiar with OAuth 2.0, so I was able to build this out pretty quickly.
One thing struck me about Amazon's recommendations about building OAuth support in your app. They say that they'll launch your authorization URL from inside the iOS app, which is a known antipattern for apps in general. In Amazon's docs, it says "The user logs in using their normal credentials for your site." This is a really bad idea. You never want to train your users to enter their passwords into random apps. This is the whole reason we have OAuth in the first place!
Rather than asking people to enter passwords in the Alexa app, I opted to solve this a different way while still being compatible with Amazon's service.
When you connect Teacup with the Alexa app, instead of asking for your Teacup password, it asks you to sign in from a real browser and generate a temporary device code.
So you launch a real browser where you can be sure you're logging in to Teacup and not a phishing website, and then visit your settings screen which shows you this.
Clicking the button generates a temporary code.
Then you enter that code in the Alexa app, and it's able to connect the app session to your user account! This way you've never risked typing passwords into the Alexa app. Teacup takes the code entered, verifies that it's active, and then completes the OAuth 2.0 handshake with Amazon.
Now you're logged in to Teacup in the Alexa app! This means the Alexa service now has an access token that it can use to talk to Teacup, which is how the two accounts are associated.
Once I got everything hooked up, I had to put it to the test! Of course I can't risk posting fake data, so I had to open a beer and try it out.
Here's the beer it posted to my site!
If you want to see the code involved in making this work, head over to the Teacup GitHub repo.
In the mean time, I'm looking forward to telling Alexa I ate tacos tomorrow morning!
Yesterday, Amy was asking if there was a library for drawing curved lines between two points on a map. She wants to use this for her travel posts, which say for example "Tokyo to Boston". We noticed that Facebook draws curved lines when they show a planned trip. Martijn was able to figure out how to reproduce this on Facebook consistently. You have to start by creating a "checkin" post, then add a "traveling to" activity to it. Notice that the line from Portland to Seattle is curved rather than straight, although it doesn't strictly follow a Great Circle line either.
I liked this idea, and realized that my own travel posts would benefit from images as well, so I set to work adding this to the library that I use to generate the maps for my posts. Before, my posts just look like this. Wouldn't this look better with a map?
My static maps are generated using Atlas, a service I wrote for handling all the geo stuff on my website. The static map generation is actually a surprisingly simple chunk of code, which assembles all the necessary map tiles and then draws points and lines on top of them using PHP GD and ImageMagick.
The first step was to figure out how to draw curved lines. There was more to it than I originally anticipated. I knew roughly the shape of the curve I wanted, and after researching a bit, determined that a Quadratic Bézier Curve would give me the shape that I want.
A Quadratic Bézier Curve differs from a standard one in that only one control point (P1 in the image above) is used instead of two. Luckily, the PHP ImageMagick library has a function to draw that curve. If I use a control point at the midpoint of the start and end of each leg, I'll get a nice symmetric curve. I can vary the height of the curve by using a control point farther away from the line connecting the two points. Here's approximately what I'm going for, where A and B are the start and end points of the line segment, and P is the control point.
Given A and B, I can find the mid-point M easily. Half the distance is d, so I know one side of the triangle, and I choose the angle alpha. It's been a long time since I've had to use any trig, so my memory of all this is rusty, but I did remember that it should be possible to find point P given one angle and one side of a right triangle. It took a while to dig up all the identities and formulas for this. Ultimately this stackexchange answer explained things well enough for me to reverse engineer a generic formula out of it.
This means I can now vary the angle alpha to adjust how steep to make the curve. This translates into the following ImageMagick PHP code.
We start at point A, then move through point P to land at B. With an angle of 25 degrees, this gives a pretty great looking curve!
Once I started running this code for other trips from various points on the globe, I immediately ran into some interesting issues. It turns out these curves look best when the horizontal curves go upwards and the vertical curves go to the right, and when there are multiple hops, the lines must not overlap. Here are some images from my library with curves I'm relatively happy with.
It took quite a bit of trial and error to determine the rules for making these render properly. The main trick is this: If the line is wider than it is tall, then draw the line from left to right. If the line is taller than it is wide, then draw from top to bottom. ("Draw from left to right" means choosing the leftmost point as point A in the equation above.)
This rule ends up with pretty good results in most cases, as you can see above. However there are still a couple of cases I'm not 100% happy with, although I can't quite come up with a rule to fix them so I'm going to leave them as is for now.
I'm reasonably happy with the results regardless that I'm going to leave it alone for now. If you have any thoughts on fixing it, you're welcome to submit a pull request to Atlas!
If you want to use this yourself, you're welcome to run the Atlas API on your own server, or just copy the code and use it as a library! To generate these curves, pass a parameter bezier=25 into the request when you are generating a line between two points.
Now my itinerary posts look a lot better!