Aaron Parecki

Now that I have pages showing all posts that use a specific emoji such as 👍, I found myself wanting to be able to click the emoji in my posts to see other posts with that emoji. This is the same as how my autolinked hashtags work, where you can click on any hashtag in a post and see other posts with that hashtag.

This was relatively straightforward to accomplish with my new Emoji Detector library, there was only one trick needed. Because some emoji are composed of others (such as 🏳️‍🌈 being composed of 🌈 plus 🏳), the particular byte sequence for 🌈 actually appears twice in a string that has both of those emoji. Once for the standalone rainbow, and another as part of the rainbow flag. This means if we tried to replace all instances of 🌈 with a hyperlink, we'd actually catch the 🌈 byte in 🏳️‍🌈 as well, breaking it into separate emoji!

So to avoid this problem, I used two tricks. 

First, since the Emoji Detector library returns the number of code points in each emoji found, I first sort the list so that the longer emoji strings are first in the list.

$emojis = \Emoji\detect_emoji($text); 
usort($emojis, function($a,$b){ 
  return $a['num_points'] < $b['num_points']; 
});

Second, instead of doing an inline replace of the post text with the hyperlink, I create a mapping of placeholder strings for each emoji found, and replace the emoji with those placeholders instead.

$map = [];
foreach($emojis as $emoji) {
  if($emoji['short_name']) {
    $k = md5(mt_rand().$emoji['emoji']);
    $map[$k] = $emoji['emoji'];
    $text = str_replace($emoji['emoji'], $k, $text);
  }
}

Then, in a separate loop, I loop through the mapping and replace each random string with the emoji including a hyperlink wrapping it.

foreach($map as $k=>$emoji) {
  $text = str_replace($k, '<a href="/emoji/'.$emoji.'" class="emoji">'.$emoji.'</a>', $text);
}

This way, I avoid the "clobbering" issue of double matching the duplicate byte sequences.

Lastly, I noticed that when I use several emoji in a row, they end up appearing squished together like this:

Since I was already wrapping each emoji with a hyperlink, I was then able to style them to add a bit of spacing around them, so now they appear a little nicer:

I wanted to find all emoji in a string, including info about them, for my next #100Days project. However I couldn't find a library that does this. The closes I found was iamcal's Emoji conversion library, which can replace emoji in a string with HTML tags, as well as the EmojiOne library which can replace emoji in a string with shortcodes.

I started down a path of attempting to understand unicode encoding. A very helpful resource is this post from 2003 titled "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)". It's worth a read if you have to deal with user input at all.

If you aren't familiar with the details of Emoji, Unicode and UTF-8 encoding, then what you probably don't realize is that an emoji character such as 👨‍👩‍👦‍👦 is actually composed of seven unicode characters. Each person is a separate character, and they are all connected with the "Zero-Width-Join" (ZWJ) character. This ends up being seven code points in total: 👨 [ZWJ] 👩 [ZWJ] 👦 [ZWJ] 👦. There are also skin tone modifiers which are their own character. So an emoji like 👍🏼 is actually two characters, the 👍 plus the skin-tone-3 modifier.

To further complicate things, I've been talking about unicode code points, but it turns out these code points can be represented in any number of ways in a string depending on the string encoding. Typically we only need to worry about handling UTF-8 encoded strings now, so that's where I started. The UTF-8 encoding of a character like "A" is the same as the ASCII encoding of the character, using only one byte. However a character such as 👍 requires more than one byte to represent. This means actually finding meaningful emoji in a string is not as simple as reading byte by byte, and is not even as simple as reading UTF-8-character by character. 

Thankfully, EmojiOne has done the hard work of finding the Emoji characters in a string. However their library doesn't have a way to return the Emoji found, it can only be used to replace them. I also didn't like the list of short names they use, I prefer the Slack names instead.

What I ended up with was putting together the parsing regex from EmojiOne with the Emoji data from Slack's data set. I turned this into a library that returns the data I want to use. Here's how it works.

Given an input string that may contain emoji characters, this function will find any emoji in the string and return an array with information about each character.

  $input = "Hello 👍🏼 World 👨‍👩‍👦‍👦";
  

  $emoji = Emoji\detect_emoji($input);

• emoji - The emoji sequence found, as the original byte sequence. You can output this to show the original emoji.
• short_name - The short name of the emoji, as defined by Slack's emoji data.
• num_points - The number of unicode code points that this emoji is composed of.
• points_hex - An array of each unicode code point that makes up this emoji. These are returned as hex strings. This will also include "invisible" characters such as the ZWJ character and skin tone modifiers.
• hex_str - A list of all unicode code points in their hex form separated by hyphens. This string is present in the Slack emoji data array.
• skin_tone - If a skin tone modifier was used in the emoji, this field indicates which skin tone, since the short_name will not include the skin tone.

This package is now available on GitHub, and via Composer!

composer require p3k/emoji-detector

🎉👍

When someone types an emoji in Slack, it shows up in the API as a text description surrounded by colons. For example, 😻 is :heart_eyes_cat: when it comes through the API. Previously, the gateway would just send the text through to IRC, which doesn't look great for IRC users.

Today I updated the gateway to translate those emoji text to unicode so that they appear as native emoji in IRC.

It was actually a surprisingly small amount of code overall. Thankfully, Slack publishes the data they use for their emoji names, so I basically added that to the gateway and look up the mapping there. I had to use a separate library to translate the unicode hex values to an actual unicode string.

It even supports the emoji with skin tone variations! 👍🏻👍🏿

For now, I just left the custom emoji unreplaced, since they are images in Slack and there isn't a good way to send image data in IRC. For our web logs, I might in the future replace those with images so that they look right there, but for now just getting the standard emoji replaced is an improvement for our IRC channel as well as the web logs!