The personal blog of AJ Jenkins

Why you should care about Ranked Choice Voting

2020-08-29T16:00:00Z

UPDATE (11/4/20): The ballot initiative for ranked choice voting in Massachusetts has failed. You can read my thoughts on the outcome here.

This November Americans will be voting on who will be our next president for the next four years. If you're like many voters, the choice between Biden and Trump feels like choosing between the lesser of two evils. Many Americans will not be voting for the candidate that they want to be President, but instead voting against the candidate that they don't want to be President. At the same time, many have observed that the country feels more divided than it's ever been since the Civil War. The solution I've heard most often is "a call for unity." But politicians have been calling for unity since our nation's founding, and "calling" for unity doesn't accomplish anything. Unfortunately, our political system doesn't allow for unity. The systems of checks & balances laid out by our founding leaders have proven utterly useless in the face of the eternal war between Democrats and Republicans. Instead of the three branches of government keeping each other in line like you learned in your social studies class, Democrats and Republicans are engaged in a tug-of-war that has infected the entire government and has no winners. Our founding leaders thought they had given us a tool to get us out of any mess: the constitutional amendment. Unfortunately, passing a constitutional amendment requires a two-thirds majority in both houses of Congress as well as ratification by three-fourth of the states. Given the current deadlock between the Democratic and Republican Party at both the federal and state levels, passing a constitutional amendment on anything is a pipe dream.

It sounds hopeless. But it's not. In November voters in Massachusetts will have the chance to vote on something that I genuinely believe can cure America of the illnesses that I described. I'm referring to Choice 2, which, if passed, would adopt Ranked Choice Voting for most state and federal elections in the state. You're probably wondering how changing the way we elect candidates could possibly fix all of those problems. Or you might not even know what Ranked Choice Voting is (according to a recent poll, 32% of MA residents don't understand it). In this essay I'll explain what Ranked Choice Voting is, how it works, and how it could save our political system.

Ranked Choice Voting (RCV) is an alternative method for electing political candidates that replaces plurality voting. If you've never heard of plurality voting, it's the basic voting system that you probably just think of as "voting." First I'll explain how plurality voting works so you can see how RCV is different. In plurality voting voters pick their one, favorite candidate from a list of possible candidates. The winner is whoever receives the most votes (a plurality of the votes), but they don't need a majority (more than half).

RCV works a little bit differently. Instead of only choosing your one, favorite candidate, you rank all (or some) of the candidates in the order of who you most want to win. So you would rank your first-choice candidate first, your second-choice candidate second, etc. and finally your least-desired candidate last.

Once everyone's ranked ballots have been collected, it's time to pick the winner. Unlike plurality voting where you just count up the votes and you're done, to determine the winner in RCV you need to analyze the ballots repeatedly over several rounds. Don't worry, it's easier than it sounds.

Ok, so here are the steps for figuring out the winner:

In the first round you only look at voters' first-choice candidate. If one candidate receives more than half of all of the first-choice votes, that candidate is declared the winner!
However, if there are more than two candidates running, there probably won't be a single candidate with more than half the votes. In this case, there's no clear winner after Round 1 and it's time for Round 2.
Before Round 2 begins, one candidate is eliminated from the running. The candidate who received the fewest votes in Round 1 is eliminated.
Now that there's one fewer candidate, Round 2 can begin. Round 2 works exactly like Round 1 – only the first-choice of each voter is used and if one candidate receives a majority of the votes, they win – but with one exception. This is the cool part of RCV. For the voters who had picked as their first-choice the candidate that was just eliminated, instead of counting their first-choice candidate, you count their second-choice candidate.
So in Round 2 you count up the first-choice votes from everyone whose top-choice is still in the running, and the second-choice vote from everyone whose top-choice was eliminated.
If there's one candidate who now has more than half of the total votes, that candidate is the winner!
If there's still no candidate with a clear majority, you eliminate the candidate with the fewest votes like we did before Round 2. Then you count everyone's votes again, but always using voters' top-candidate who hasn't been eliminated yet as their vote.
Eventually you're guaranteed to have a winner, at least by the time when all but two candidates have been eliminated.

And that's it! Sorry if that wasn't as simple as I promised. But to summarize, in each round you count up everyone's votes using their top-choice that's still in the running. If one candidate has more than half the votes, they win. If not, the candidate with the fewest votes is eliminated and then you repeat the process with the new set of candidates. Tallying the votes by hand is a little tedious, but should be something that anyone with basic math skills could do. But luckily, computers can calculate the winner very quickly so RCV shouldn't cause any delays in the election process.

So, why is Ranked Choice Voting better than plurality voting? Rather than go into a technical deep dive of all the benefits of RCV, in this article I'll just talk about the biggest advantage: it eliminates the "spoiler effect" when similar candidates run against each other. When political pundits talk about a "spoiler candidate", they mean a candidate that runs in an election (usually a third-candidate entering a two-candidate race) and then "spoiling" the election for whichever candidate they're more similar to, thus handing the election to the candidate that neither of them are similar to. For a real-world example of this, in 2000 many people blamed the loss of Al Gore to George W. Bush on Ralph Nader, who ran as a Green Party candidate in the presidential election. Basically, people believe that if Ralph Nader hadn't run, the people who voted for Nader would've instead voted for Gore (since the two ran on relatively similar platforms), thus securing the win for Gore instead of Bush. Of course, it's impossible to say what Nader voters would have done had Nader not been a candidate, but it's certainly plausible that his presence resulted in Bush winning.

Let's imagine how things might've played out if RCV was used in the 2000 election. For simplicity, let's pretend that Bush, Gore, and Nader were the only candidates in the election, even though there were actually other candidates. Instead of voters being forced to choose one candidate, they would instead rank all three candidates in order of preference. Let's assume that 80% of Nader supporters rank the candidates Nader → Gore → Bush, and 20% of his supporters rank them Nader → Bush → Gore. Since Nader only received 2.74% of the popular vote, we can assume that he would have been eliminated from the running before Bush or Gore. Therefore, the votes for Nader would then be distributed to the second-choice candidate of his voters. So 80% of the Nader supporter votes would go to Gore, and 20% would go to Bush. In this scenario, Nader was able to safely run without spoiling the election for either candidate, so maybe Gore would've won the election. Of course, if you're glad that Bush won the election in 2000, maybe you're thinking that it's good that we didn't use RCV. But if a majority of the nation (technically, electors in the Electoral College) would've preferred Gore over Bush as president, don't you think he deserved to win?

I want to make it clear that RCV is not a partisan issue. RCV will benefit conservatives and liberals alike because it will empower people to vote for the candidate that they actually want to win the election, not just the candidate who they think has the best chance of beating the person they don't want to win.

Here's another example from more recent history, that I hope will show how RCV could benefit Republican voters too. This is based on my own speculation, but I don't think it's too farfetched. If you remember the 2016 Republican Primary process, there were initially a sea of candidates and Trump was not considered a serious candidate (more details on the primary). Let's fast-forward to the end of that primary. At the end, four front-runners emerged: Donald Trump, Marco Rubio, John Kasich, and Ted Cruz. Donald Trump ended up getting 44.9% of the popular vote and clinching the nomination. But another way of saying that is 55.1% of Republicans voted for not Trump. Yet since Trump had the most votes (not a majority though), he ended up winning the primary and going on to be President. Now, I don't have any hard evidence supporting this theory, but I believe that many of the Republicans who didn't vote for Trump would've actually preferred any of the candidates over Trump. Unfortunately, voters couldn't agree on which of the samey Republicans to throw their weight behind to beat Trump, so their votes were split across Rubio, Cruz, Kasich and others, thus clearing the way for Trump to take the nomination. I believe that if voters were able to vote in that primary using RCV, many would have ranked Trump last on their ballot (I'm thinking of the "never Trumpers") and perhaps a candidate that was actually preferred by a majority of Republican voters would've been nominated instead.

Another cool thing about RCV is that primaries themselves are actually unnecessary. If you've ever wondered why we have to have primary elections before the main election, the main reason is because if multiple candidates from the same party ran against each other in the main election, the vote would be split across all of the candidates in that party. But imagine if RCV had been used in the 2016 presidential election without primaries. Voters would have had the opportunity to rank Trump, Clinton, Sanders, Rubio, Kasich, and Cruz in a single ballot. I have no idea who would've won that election, but I wouldn't necessarily assume that it would have been Trump or even Clinton. For Americans who felt dissatisfied with their choices in that election, that should be great news.

I promised that I would tell you how Ranked Choice Voting would fix the political problems I described in the first paragraph. To be clear, I'm not saying that adopting RCV in MA will be sufficient to save democracy in America. But I hope that if we adopt it in MA, we can serve as a model for the other states who will follow suit. And as more and more states start using RCV to elect members of Congress, I believe that we'll start electing representatives who actually represent us instead of their party. Anyway, time to deliver on that promise.

Here's how RCV will address the issues I mentioned in the introduction:

Voters are forced to vote for the "lesser of two evils" - Plurality voting does not permit more than two candidates to run in a general election because other candidates could "spoil" the election. Voters must vote tactically in a primary, and vote for the candidate they think is most likely to beat their opponent rather than voting for their true favorite candidate. But with RCV, more than two candidates can run without fear of similar candidates spoiling the election for each other. With RCV, voters can rank the candidates according to their true preferences and feel confident that their vote will automatically go to their preferred candidate who is capable of winning.
The country is more divided than ever - I actually don't believe this is true. I think that most of the country falls somewhere in the middle on most issues, but people are forced to pick a side in the battle between Democrats and Republicans which makes us feel divided. I know many Democrats who can't imagine how someone could have voted for Trump, but I bet that many Trump voters are more moderate than they might expect. I think many people voted for Trump, not because they support his agenda or even like him, but because they just happened to not want Clinton to be president for some reason.
Congress is locked in a stalemate where everyone loses - Our government is currently controlled by a duopoly: the Republican and Democratic parties. I believe that RCV has the potential to disrupt this power dynamic by helping other political parties put representatives in Congress and end the tug-of-war. RCV would make it more likely for third-party candidates to win elections because voters could safely vote for those candidates without feeling like they're throwing away their vote. As an example, many Americans politically identify as libertarian, but neither of the main parties fully supports a libertarian agenda. With RCV, true libertarian candidates would likely be elected to office in some races. If there were true libertarians in Congress, they could vote with Democrats in some votes while voting with Republicans in others. This would make it easier for a majority vote to be achieved (even overcome a filibuster) and would also better reflect the true preferences of the constituents.
The checks & balances described in the Constitution are not working - I think one mistake that our founding leaders made when writing the Constitution is that they assumed that politicians would act honorably and behave like proper gentlemen. Unfortunately, civility has been abandoned over the course of our nation's history as a result of the bitter struggle between the Democrats and Republicans. In 2019 when President Donald Trump was impeached, it was a forgone conclusion that nothing would happen to Trump as long as Republicans held a majority in the Senate. Congress is supposed to act as a check on the president's power, and one of their strongest checks is the power of impeachment. But instead of carrying out an unbiased investigation into the crimes alleged against the president, the impeachment process devolved into another tug-of-war between the Republicans and Democrats. Just like in my last point, I believe that RCV can fix this problem by injecting representatives from third-parties into Congress, and also by ensuring that the representatives who get elected actually reflect the preferences of their constituents.

Thanks for reading, and if you live in MA I hope you vote YES on Choice 2 on November 3! If you'd like to get involved with the RCV campaign in MA, you can learn more here.

Doing It for the Likes

2020-09-27T16:00:00Z

Rewinding to 2007

This is the story of the first and hopefully last time in my life I did something just to get approval from strangers on the internet.

In 2007 I was 15 years old and Halo 3 was my favorite video game. The campaign mode wasn't great (especially compared to Halo 2), but it was the best multiplayer experience I had ever played. I spent almost every afternoon after school playing local multiplayer with my friends, and I spent a bit of time playing online too. I was the worst player among my friends, but I still loved it.

Back in 2007, micro-transactions weren't a thing yet in video games, so the only way to unlock stuff was by playing the game. In Halo 3 when it launched there wasn't much to unlock, but you could unlock a few different armor variants for your Spartan in multiplayer. However, there was one armor variant that you couldn't unlock just by playing the game: the Recon Armor.

Image source

Originally, Recon Armor was only available to employees of Bungie, the developer of the game. So on the rare occasion when you saw someone with Recon Armor in the online multiplayer mode, it was really exciting because you knew you were playing with someone who actually helped make the game. But shortly after it was available to Bungie employees, they decided to start gifting the armor to a handful of members of the community. Getting the armor wasn't a matter of skill or level of commitment – the only way to get the armor was to catch the eye of a Bungie employee and hope that they deemed you worthy of the Recon Armor. I remember reading this blog post from Bungie where they talked about how to get Recon Armor. They made it very clear that you would not get it by nagging them, and they also highlighted two members of the community who they had given it to.

The one I remember is this player, who uploaded a video to YouTube of him shooting an exploding barrel, which caused a nearby traffic cone to fly towards him so fast that it killed him. If not the first documented occurrence, it was definitely the most famous case of someone killing themselves with a traffic cone in the game. Here's the original video:

https://www.youtube.com/watch?v=ym0BtwmCvoc

So, to summarize, I was a teenager and my favorite game was Halo 3. The developers created an extremely exclusive unlock which could not be earned by conventional means. The combined effect of these was that I decided that I had to do whatever it took to get this armor. Getting Recon Armor became my life mission.

Becoming worthy

So, my first step was to brainstorm what I could do that would get the attention of a Bungie dev and be deemed worthy of receiving Recon Armor. It was obvious that making a video of me getting killed by a traffic cone wouldn't work, because that was only funny the first time it happened. Also, the reason that video was funny was because it was (presumably) completely unplanned. So I could just keep playing and hope something crazy like that happened while I was playing, but I wanted to do something I had control over, so that plan was out.

Here's what I settled on. In Halo 3 they had added a new game mode called Forge. It was basically just a custom level editor. The way it worked is you controlled a little flying robot called a Sentinel and you could create and drop any object in the game (structures, vehicles, weapons, etc.). There was also local multiplayer so you could have multiple people all working together on the same creation at the same time. It was kind of like Minecraft, but 4 years earlier and you used objects from Halo instead of blocks.

On the internet I had seen people who had creatively used Forge Mode for drawing things. They would drop weapons around a flat part of the map, then fly up to take a birds-eye-view photo of their drawing. People would draw things like an outline of Master Chief, but most of what I saw wasn't very good. I believed that I could create something much better than what I had seen so far. And of course, my hope was that if my creation was good enough and it got enough traction on the internet, Bungie would see my work and grant me Recon Armor.

Here's what I made. I decided to recreate the Halo 3 logo in Forge using every single weapon in the game. I don't think I was the first person with this concept, but I remember my version being better than anything else I had seen.

It took about two days to make (less than 10 hrs). I used the box that the game came in as a reference and dropped the weapons wherever I thought they would look good. My younger brother was helping me by controlling a second Sentinel so I could see what I was doing from multiple perspectives. The hardest part was I couldn't freeze the weapons in place once I placed them. I had to carefully drop them on top of each other and hope that the weight of each weapon wouldn't disturb the position of the other weapons. The ground also wasn't perfectly flat, so I had to balance things on top of the ridges in the sand.

When I was done, I was very proud of my creation. To date, I think it's the most artistic thing I've ever done. My friends were all impressed when I showed it to them.

Getting Recon

I thought that if a Bungie employee saw the screenshot of what I had built in Forge, I would get the Recon Armor for sure. So all I had to do was get their attention.

Bungie had a forum where players could talk about the Halo games. I decided that my best bet was to upload my screenshot there and catch the eye of one of the community moderators. Which I did, but not in the way I had hoped, unfortunately.

The way forums work is the most recently updated post is displayed at the top of the board. I created a new post and my creation was at the top of the board for all of the world to see. Pretty quickly, I got some positive responses from other community members, but no word on Recon Armor. As the days passed, people stopped replying to my post so my post started falling down in the board, until it was no longer on the first page. And once your post isn't on the first page, it's very unlikely that people will see your post and reply to it, thus beginning the downward spiral into obscurity.

Now what? I wasn't ready to give up yet. I was still convinced that my art piece was worthy of Recon Armor. It was just a matter of getting the right person to see it. But what should I do? I couldn't just upload my picture again and hope for a better outcome. Also, I didn't want to lose the goodwill I had already earned on my first post.

So I asked my younger brother what I should do. He had a simple suggestion: "bump" it. I asked what "bumping" meant. He said that all I needed to do to get back on the first page is reply to my own thread (even just with the single word "bump") so my post would again be the first and most recently updated thread on the board. I'm not sure how my 12 year old brother knew about this trick, but it seemed like a great idea.

So, every day after school I would look at where my post was positioned in the board, and if I wasn't satisfied I would write a reply to my post. Sometimes I would try to write something that looked meaningful, like tagging someone who had previously commented and ask them some question. But I also remember sometimes just writing the word "bump" if I couldn't think of anything.

This went on for about a week until I received an email from a forum moderator saying that I had been banned from the forum (I can't remember if it was temporary or permanent). I was shocked. Not only did I not receive Recon Armor (which I felt I deserved), but they were actually banning me from the forum just for trying to highlight my accomplishment. Apparently, "bumping" was against the rules of the forum, which I guess I neglected to read. I asked my brother if he knew "bumping" was against the rules. He said something like "Yeah, of course it's against the rules. I thought that was obvious." 😖

Looking back

Throughout my life, I've thought about this experience a lot. Any time I'm thinking about doing something that will require me to get approval or validation from strangers on the internet, I remember my Halo 3 logo. What I learned is that if you're creating or doing something just because you want to impress other people, you're doing it for the wrong reason and you'll probably end up disappointed. This is one of the main reasons I don't use social media. It often feels like it's less about sharing text and media, but getting as many Likes as possible on what you post. I've had friends demand that I Like their post and I reply "Why do I have to Like your post? Isn't enough that I like it?"

The Halo 3 experience is also one of the reasons I don't have analytics on my blog. By not knowing how many people are reading my blog posts, it lets me continue writing because I enjoy it instead of writing clickbait and getting that dopamine hit when I see the page view count go up on Google Analytics.

Anyway, thanks for reading.

Bonus: Actually getting Recon

Eventually I did get the Recon Armor in Halo 3. Everyone did. In 2008 Bungie released an update which included the Vidmaster Challenges, which upon completion would unlock the Recon Armor. Getting the armor was still difficult, but now there was a clear path anyone could follow to get the armor. This made the armor much less exclusive, so I stopped caring about getting it. I'm not even sure if I tried to complete the challenges. But in 2012 another update for the game was released which gave the Recon Armor to all players. But since my obsession with the armor stemmed only from its exclusivity and not the actual aesthetics, I don't think I even used the Recon Armor at that point. I had moved on to caring about other things.

Why I don't have analytics on my website

2020-10-04T16:00:00Z

UPDATE (5/30/22): Since writing this post, I've reversed my decision and added analytics to my website. You can read why here.

Originally I was planning on writing a blog post about how I chose "X" service for analytics on my website. I was going to go over the pros and cons of each service I considered and then dramatically tell you the one, best web analytics service.

But after a lot of thought, I made the unconventional decision to not use any analytics service at all for my website. I haven't heard of anyone else who's done this, so I decided to write why I decided to do this.

Wait, what do you mean "analytics"?

If the title of my post is self-explanatory to you, feel free to skip this section.

So, here's the deal: on basically every single website you visit there's code embedded on the website used to track how you use the website. This code can track how many users view each page, what other website they were referred from, and what they click on in each page.

By far, the most popular user analytics tool is Google Analytics. Not only is Google Analytics one of the most advanced user analytics tools available, it's also very easy to set up and completely free (at least the basic version). Google makes this free because when a website uses Google Analytics, they agree to share all of their usage data and traffic patterns with Google, and the more Google knows about how people use the internet, the better they can serve targeted ads. One cool/creepy feature of Google Analytics is it gives you very detailed information about the demographics of your users. It will show you the age, gender, and location of everyone who visits your website. It can even show you what types of things they're interested in, like sports, travel, food & drinks, etc. If you're a marketer, this data is invaluable because you can ensure that your content and marketing is optimized to appeal to the demographics of your users. It's also very useful for showing targeted ads on your site if you use ads to generate revenue. And of course Google offers a companion service, Google AdSense, which is used for serving ads from Google and integrates with Google Analytics.

One tool that Google uses to learn all of this detailed information about you is the cookie. Cookies have been a feature of web browsers since the 90s, and allow websites to store information specific to you in your browser that can persist even after you leave the website or close your browser. Cookies can be used for lots of different purposes, such as keeping you signed into a website so you don't need to reenter your username and password every time you visit the site.

Typically, the "benign" cookies (i.e., not a privacy concern) are ones that are implemented by the website you're using, like the aforementioned cookie used for keeping you signed in. These are "first-party" cookies since they come from the website itself. But there are also "third-party" cookies, which are set and used by companies external to the website you're on. One of the most common third-party cookies on the web is the Google Analytics user token, which is automatically set by Google Analytics as soon as you visit the website and is used to track your entire history of actions across all your sessions on that site. Third-party cookies are "bad", because when you use a website your data is not only shared with the website you're on, but also with third-parties that you may not want your data shared with, like Google. Websites used to do this all the time without telling you, but now with regulations like GDPR, websites are required to inform you about how they share your data with third-parties and give you the option to opt-out.

The passage of GDPR is the reason that all websites you visit now have that annoying banner at the bottom with some fine print about sharing data with third-parties and a giant button that usually says ACCEPT ALL COOKIES, which most people click because it's the easiest way to make it go away.

Ok, that's pretty much everything you need to know about web analytics. Now I'll explain why I decided not to add analytics to my website. Before that though, I'll briefly go over why I would even want analytics on my personal blog to begin with.

Why I considered adding analytics to my blog

It's just how you do things

As I mentioned above, pretty much all websites have some kind of user tracking built into them. Adding analytics to your website is definitely the default decision. In fact, in every website I've made previously, adding Google Analytics was one of the first things I did. It's also incredibly easy to do – adding analytics (at least, Google Analytics) takes only 5 minutes and is simple enough that anyone can do it.

So not only is omitting analytics an unusual decision in general, but it's even an unusual decision for me.

Tailoring my content based on what's popular

This was my original reason for wanting to use analytics on my website, and probably the main reason why most blogs use analytics. It's not uncommon for a single blog post to "go viral" and be the most-read post on the site by a significant margin. And if one of your posts goes viral, you probably want to make your future blog posts more similar to that post. Or maybe you'll write a follow-up post to try to entice the people who read your first post back onto your blog.

A less extreme possibility is that certain types of posts attract more readers, so it would make sense to mostly write those kinds of posts going forward. For example, if my posts about software engineering are getting more views than my posts about video games, maybe I'll want to focus my blog on software engineering instead of just writing about whatever I feel like.

Knowing if I'm famous yet

Another thing I worried about was how will I know if I'm "famous" if I don't know people are reading my blog posts? A million people could be reading this right now, but I'd have no idea.

Why I decided against adding analytics to my blog

I hate cookie banners with a passion

Just to make sure we're on the same page, I'm talking about those banners at the bottom of basically all websites that mentions something about sharing data with third-parties and has a big button with the text ACCEPT ALL COOKIES.

Yeah, I hate those things. Websites added them in response to the passage of GDPR, which was a law designed to protect the privacy of EU citizens while browsing the web. But as far as I'm concerned, all it did was add another annoying thing to websites that you have to click on before you can use it. I read that some updated regulations will eventually make these banners useful and more than just a nuisance, but it doesn't seem like that's happened yet.

As someone who hates these banners, I decided that there was no way I would put one of these things onto my website. Theoretically, I bet I could get away with using third-party tracking cookies without displaying the legally required banner, because I doubt EU regulators would bother coming after me. But I'd prefer to do things the right way.

So, if I did want analytics on my website, I would need to find a privacy-focused analytics service that would be GDPR compliant and exempt me from needing to display the dreaded cookie banner.

Note: Although I'm a big proponent of online privacy, I have to be honest – the main reason I was looking at privacy-focused analytics solutions was so I wouldn't have to put that stupid banner on my website. Protecting your privacy as a visitor to this website was just an added bonus. Sorry 😞

Privacy-focused analytics services that I considered

To some people, this will be the most interesting part of the blog post. To others, it will be useless and boring. Feel free to skip this.

Here's the best list of privacy-focused analytics services that I found. There aren't actually that many services like this, because most analytics services actually differentiate themselves from Google Analytics by giving you even more detailed information about the users on your website. Of the privacy-focused solutions, many of them seemed very "enterprisey" (like Matomo, Plausible and Fathom), which I ruled out because they didn't seem like a good fit for a personal blog.

Here's the list of the finalists I considered using:

Simple Analytics - Doesn't use cookies, so it's automatically compliant with GDPR and doesn't require a cookie banner. From the screenshots and marketing materials, it looks like they have great UI/UX. It also sounded like a really novel approach to analytics, which appealed to me. The downside is that the starting price is $19/month.
GoatCounter - Open-source, bare-bones analytics tool. GDPR compliant, so no cookie banner required. Made and run by one person, who seems like a nice guy. Free, with the option to pay if you want to. Created exactly for the use case of a personal website. The downside is that the UI/UX looks pretty bad (sorry, Martin).
Netlify Analytics - This is only an option if you're hosting your site on Netlify, but I am. Great UI/UX. Supported by a relatively successful company, so it probably won't disappear any time soon. All analytics are server-side, which means no cookies or adding any tracking code to your website. Getting started is as simple as flipping a switch from the admin dashboard. The downside is it costs $9/month.

If I had to pick one of these, I would've gone with Netlify Analytics. The fact that it's completely server-side appeals to me, because that's how I think analytics should work. Although GoatCounter is free, the simplistic UI bothers me. And although Simple Analytics seems cool, I can't justify spending $19/month on a website that doesn't make any money. But I could almost justify spending $9/month on a website that doesn't make any money, which is why Netlify Analytics is the winner for me. I probably would have used Netlify Analytics, despite its cost, if it weren't for the other reason I decided against using analytics on my blog...

I'm better off not knowing how many page views I'm getting

Going back to my earlier discussion about why I might want analytics on my site, the three reasons I gave were making money, tailoring my content effectively, and knowing if I had become famous. But other than boosting my ego, the only value of knowing if I had become famous would be profiting off of my blog somehow, which goes back to the money thing.

So really, having analytics could help me in three ways:

Making me money
Influencing what I write about
Boosting my ego

I'll go over the first two points in detail, but regarding the ego thing, I'll just say that having my ego boosted would probably do me more harm than good 🙂

Making money

First of all, if you didn't know, making money from a blog is hard. If real news sites can't figure out how to make their businesses profitable online, what chance do I have? Sure, there are some notable authors who have built highly profitable blogs, like Ben Thompson (Stratechery). But I think they're the exception. Also, the people making money from a newsletter are typically writing about topics that will help their readers make money directly or indirectly, which justifies the price for the newsletter.

But I don't want to be locked into writing about "business" things. Also, even if I could turn my blog into my full-time job, I don't think I would enjoy that. Blogging is fun as a hobby, but I bet it would be a lot less fun (and more stressful) if it was my primary source of income.

Alternatively, I could just slap some ads on my blog for some passive income, but with the traffic I assume I'm getting, I wouldn't make much money that way. I'd also have to give up on protecting my reader's privacy so I could have targeted ads. It just doesn't seem worth it for a few buck (or pennies) a month.

Writing things that people want to read

To some extent I want to write things that people will want to read. When I'm thinking of what to write about, I definitely consider what topics people will find interesting. My strategy is I try to write articles (and headlines) that sound like something I would read if I saw it while browsing Hacker News.

But I'm only guessing. With analytics, I could cut out the guesswork and know exactly which articles and topics attract the most views.

My fear is that, with that level of insight, it would unconsciously affect my writing. Over time, I would lose sight of my original reason for creating this blog (because I enjoy writing) and begin writing just to maximize my page views, even if I'm not making money from the additional page views.

For more on this, check out my post on "Doing It for the Likes".

Final thoughts

Although I've personally decided not to use analytics on my website, don't take this as a rallying-cry against analytics. Analytics are great, and are vital for most websites. Analytics enable companies and creators to make data-driven decisions on the best way to improve their website, product, or service, which is how it should always be done.

But for my own blog, my goal is to write for my own enjoyment. And that's something that no analytics tool can help me with.

Closing-closing thought

Actually, this post is a rallying-cry against something. Cookie banners. Cookie banners suck 😅

Achieving a perfect PageSpeed score with Eleventy

2020-10-11T19:34:00Z

Update: I decided to rewrite this post as a guide instead of a story. If you want the full story of how I improved the performance of my website, keep reading. But if you just want the quick tips on how to make your Eleventy website faster, read this post instead.

Intro

As a web developer, I was feeling a little guilty about the performance of my blog. Just so we're on the same page, when I'm talking about "performance" in this post, I'm referring to how quickly the page loads.

My site was already pretty fast to begin with because of the tools I used to build my website, like a static-site generator (Eleventy), and a CDN for caching static assets (Netlify). I also don't use any client-side Javascript on my website or large CSS libraries like Bootstrap. I use Javascript, but only for building my site, and for styling I just use a single 100-line CSS file I wrote myself. Downloading large JS and CSS files (and blocking scripts) are some of the main sources of modern website slowness, which is why I chose to avoid them from the start.

But I knew there was still an opportunity to make my site faster. The one thing I felt bad about was not resizing my images to match the size I was displaying them at. Although I never display images wider than 640px on my site, I was loading images that were much bigger than that. This is wasteful because your browser downloads the full-resolution image, but then downsamples it before rendering it.

Of course, I could always manually shrink my images, but that's a pain to do for all of the images on your site, and plus, computers are smart and should be able to do that automatically. Although I've never set it up myself, I knew this was a pretty standard feature in modern tools for making websites.

Although I knew about one bottleneck making my website slow (unoptimized images), I knew there could be others. Rather than just make random optimizations to my site, I did the right thing and used a tool for analyzing the performance of my website.

Using PageSpeed Insights

From the get-go, I knew that I wanted to use a Google tool for evaluating my site's performance. That's because I wanted to test my site using the Core Web Vitals metrics that Google recently started advocating for. They clearly put a lot of thought into the selection of these metrics, and they align with my subjective experience of what makes some websites great to use and some websites terrible. My favorite metric they included is Cumulative Layout Shift, which measures how much the page "jumps" when viewed on mobile. There's nothing worse than reading an article on mobile, only for the page to scroll to a new position for no apparent reason, forcing you to scroll back to where you were.

Google actually has a bunch of different tools for measuring your site performance. You can see an overview of them here. I chose to use PageSpeed Insights because it measures all of the Core Web Vitals and is very easy to use – all you need is a URL.

For testing, I decided to use this post I wrote about Halo 3 because it has large images and an embedded YouTube video. My other posts are mostly text, so I assumed that page would have the most room for improvement.

Here are my results for that page before I made any optimizations (all results are from the Mobile test):

View PDF of the full report

Not bad! As I expected, just by using a static site with only a few images and no CSS or JS, my site is automatically pretty fast. For comparison, when I ran this test on the YouTube homepage, it got a score of 41. And that's a website that Google made, and they made the test!

After giving myself a pat on the back, I dug into the specific suggestions they gave. Here's what they suggested:

Remove unused JavaScript*
Properly size images
Serve images in next-gen formats
Remove unused CSS*
Eliminate render-blocking resources
Defer offscreen images
Ensure text remains visible during webfont load*
Reduce the impact of third-party code*
Does not use passive listeners to improve scrolling performance*

But here's the kicker: five of those suggestions are solely the result of embedding that YouTube video (the issues caused by YouTube are indicated with an asterisk*). Come on, Google, that's not fair. You can't yell at me for how slow my website is when the only reason it's slow is because I'm embedding a video from the site that you made!

The other problems are legit though and fully within my control. The rest of this post shows how I addressed each of these suggestions (including the ones related to the YouTube video). Most of my performance improvements came from adding stuff to my Eleventy config, so if you just want to see the final result, you can view my Eleventy config file here.

Inlining my CSS (Eliminate render-blocking resources)

I was surprised to see my CSS listed as a culprit for making my website slow. I was pretty proud that I managed to fit all of the CSS for my entire website into a single, 100 line file. But because that CSS file gets loaded in the page head, it blocks the rest of the page from rendering until the file finishes downloading. And apparently, waiting for my CSS file to download was adding 180 ms to the overall page load time. Not much, but these things do add up. It's not that big of a deal though, because browsers cache CSS files anyway, but that only helps if the user has already visited your site before.

Anyway, even though it wasn't causing significant performance issues, it's a very easy thing to fix so I decided to just do it.

The solution Google suggests is inlining "critical" JS and CSS. When it comes to CSS, "critical" means that the CSS is necessary for styling the content that's "above the fold." And "above the fold" means the content that you see on a website right when the page loads, before you scroll down (you can read a deeper dive on critical CSS here). Based on that definition, all of my CSS is critical because most of my styles are for things at the top of the page, like the nav and page header. But that's ok, because the rule with inlining CSS is that you can only inline a small amount of CSS (whatever that means), and I figured my 100-line was probably small enough to qualify.

Luckily for me, I happened to remember that the official Eleventy docs have a quick tip on how to inline your CSS. When I first read that tip, I remember thinking that it sounded like overkill. But after seeing it come up in my PageSpeeds report, I changed my tune.

I followed the docs to the letter, so I'm not going to rehash the steps here. The only thing that tripped me up was that I had to move my CSS file. Originally I kept my CSS file in _pages/static/, but I had to move it to the _includes/ directory with my template files. I don't understand why, but it is implied by the docs so I was able to figure it out without too much trouble. You can see exactly how I implemented this change by viewing this commit on my GitHub repo.

Resizing and compressing images (Properly size images)

This was the most difficult change to implement, by far. I didn't feel like this was something I could skip though, because this was the original performance optimization I had planned to do. Also, the actual code change I made for this was pretty simple, but it just took a long time to figure out the exact incantation I needed to use to get the plugin to work properly.

I'm getting ahead of myself though. The first way I tried to do this was through the Netlify build settings. First I tried installing a community-made plugin that promised it would do everything I needed, but when I tried running a build with it, it didn't seem to do anything. Then I found a setting in Netlify (Site settings → Build & deploy → Post processing → Asset optimization) that promised to compress my images. I tried that, but that didn't do anything either. In retrospect I should've realized that my images were already compressed, but needed to be resized, which is technically different.

Next I did some DuckDuckGoing and found that Eleventy has a first-party plugin for resizing and compressing images. Perfect! It only took me a few minutes to copy the example code from the README into my project and then... nothing. No images displaying on the page and no magically resized image files showing up in my output folder 😞

The first error I got was AssertionError: tag myImage not found. Searching for that error online wasn't helpful, unfortunately. But then I realized that I was trying to use a Nunjucks shortcode inside of a Markdown file (that's what I write my blog posts in). So I tried pasting the same code into a Nunjucks file that I had. That fixed that error, but then I got a new error: Input file is missing. This seems to be a pretty common error if you look at the GitHub issues for the plugin. Basically Eleventy is telling you that it can't find the image you're trying to use.

Unfortunately, it doesn't give you any more information than that. Although I knew what the problem was, finding the correct combination of values for the image source and the plugin config based on the directory structure defined by my custom directory structure was no easy task. After an hour or more of trying every permutation of file paths I could think of and one git reset --hard after I introduced an infinite loop into the build pipeline, I finally figured it out.

To try and help anyone else struggling with this plugin, I'll explain what I think you're supposed to do:

Set urlPath to be the directory where your source images are, relative to your input directory that you configured in the return object in the Eleventy config file (the default input directory is .).
Set outputDir to where you want the converted images to be put, relative to the root of your project (probably where your package.json and .gitignore files are). It should go into your output directory with your other compiled files (the default output directory is _site).
When you're using your newly created shortcode, the path to the file you should use is the full path from your project root to wherever the source image is (including its file extension). So it will look something like ./{inputDir} /{pathToImages}/ {ImageWithExtension}.

After I got that working, I decided to go back and see if I could get the plugin to work with a Markdown file. If I couldn't, I would either need to find a totally new solution, or commit to writing all of my future blog posts using Nunjucks, which seemed terrible. Thankfully, I figured out how to make the plugin work with Markdown files. This wasn't in the documentation for the plugin, but I found out you can configure Eleventy to compile your Markdown files using Nunjucks, which enables the use of Nunjucks shortcodes within Markdown files. I go into the process in more detail here in the project's GitHub Issues.

You can see exactly what I did to implement image resizing here.

Using WebP for images (Serve images in next-gen formats)

I'm including this in the same section because the eleventy-img plugin can also convert your images into these "next-gen formats" like WebP. If you've never heard of WebP, it's a relatively new compression algorithm for images, comparable to JPEG or PNG, invented by Google for use on the web. Although it feels a little sneaky that Google uses its performance analyzer tool to push its own image format onto you, WebP is actually pretty cool. Basically, without any noticeable loss in image quality, it gives you at least as good compression as JPEG while supporting image transparency like PNG. It's the best of all possible worlds when it comes to image formats.

It's even supported on all modern browsers!... except Mac OS Safari 😩 This was actually really confusing to me because when you search for "Mac OS Safari WebP" one of the first results is an article from MacRumors stating that Apple would be adding support for WebP images to Safari in Safari 14. So then I checked what version of Safari I'm running right now: Version 14.0. Yay!

But when I tried switching my images over to WebP, they didn't load in Safari. Weird. Maybe I'm doing something wrong. So then I tried my site in Chrome, and the images load fine. I even tried loading a random WebP image from the web in Safari, but that didn't work either.

Now I'm thinking that MacRumors must've had it wrong about when Apple was adding WebP support. So I go to the official release notes for Safari, only to discover that, yes, support for WebP images was added in Safari 14. But I'm using Safari 14 and it's not working!

After even more DuckDuckGoing, the mystery is finally revealed. I go to the ever-useful Can I Use site's page for WebP. There it is, in Note 3: "Partial support in Safari refers to being limited to macOS 11 Big Sur and later." Come on Apple, you're better than this. Don't have different features in Safari 14 depending on which OS you're running. That's not how versions are supposed to work. Just give the versions different names, like 14.0-catalina and 14.0-bigsur or something 😞

Anyway, after all of that, I decided not to use WebP for any of my images until Mac OS Big Sur has been out for a while and most people have upgraded to it.

UPDATE (11/21/21): I'm now using WebP (and SVG) for all of the images on my website.

Defer offscreen images

This sounded hard, so I decided not to do it. Next!

Fixing all of the YouTube-induced performance issues

Using one-weird trick (i.e., Eleventy plugin) I was able to fix all of the YouTube issues at once.

Looking at the suggestions PageSpeed was making for the YouTube stuff, I realized that all of the complaints boiled down to the YouTube stuff being loaded first before the rest of the page loaded. Because I'm an Experienced Frontend Engineer 😉, I knew that the solution would be to load the YouTube video into the page asynchronously. Which is just a fancy way of saying: load the other stuff on the page first, then pop that bad-boy in there whenever it's ready.

After another quick search, I found that YouTube actually does support asynchronous loading of an embedded video. Here are the official docs on how to do that. Great, except this looks like a lot of code that I don't particularly want to write.

Thankfully, I didn't have to. On the Eleventy docs I discovered that someone had already made a plugin for embedding YouTube videos (Thanks, Graham!). It even has a convenient "lite" mode, which automatically does all that yucky async loading stuff I didn't want to figure out.

The best part of all was that after following the directions in the README, everything worked on my first try! Even "lite" mode worked without a fuss. The only weird thing I noticed was that in Safari I needed to click on the video twice to play, but my research suggested that this was an issue with Safari, not YouTube or the plugin I was using.

You can see the commit where I implemented this change here.

Results

View PDF of the full report

Booyah! Perfect score! Improvements in all metrics across the board, except for Cumulative Layout Shift but that's only slightly worse and it's still in the green.

Now the only suggestion I get is to use those fancy "next-gen" image formats. Which I'll do at some point, but not until Safari adds support for WebP as I discussed above.

Bonus: addPassthroughCopy

This doesn't have anything to do with performance, but it's a really useful Eleventy feature that I discovered while configuring the YouTube embed stuff, and I wish I had known about it earlier.

Here's what happened. When I was configuring the "lite" mode for YouTube embeds, I decided I wanted to be "cool" and use my own self-hosted copies of the needed JS and CSS files rather than depending on an externally controlled CDN. Hosting your own static assets (especially JS) is a security best-practice because if that external CDN ever got hacked, the hackers could replace the file you're loading with a malicious script and potentially do bad things to the users of your website. It's unlikely, but it's such an easy thing to mitigate and the exact steps were in the README, so I thought I might as well do it the safe way.

I downloaded the two files I needed, dropped them both in my trusty static/ directory and configured the plugin to look for them there instead of the external CDN. But it didn't work. When I checked the Network tab in the Developer Console, I noticed that the CSS file was loading fine, but the JS file wasn't found. Which seemed weird, because I put them both in the same folder. Why would Eleventy find one file, but not the other?

This took me a while to figure out, but I eventually realized that I didn't actually understand how Eleventy works. For the most part, Eleventy seems to just take all of the files I feed it in the input directory, process them, and then spit them back out in the output directory. But it only does this for files that match the template formats you've specified in your Eleventy config. And .js was not one of the file formats I had listed in the eleventyConfig.setTemplateFormats() array. This also explained why a lot of other static files I had weren't getting copied over, like my robots.txt file.

But thankfully, Eleventy provides a built-in solution to this problem. While you could just add every single file extension to the list of template formats, that's not the best way to do it. What you're supposed to do, is for any folder where you just want Eleventy to blindly copy over all of the contained files regardless of file type, use the addPassthroughCopy function.

You can see how I use it in my .eleventy.js file here.

Ta da! Adding that one line fixed my YouTube problem and resolved other issues too, like some of my favicon files not being copied over correctly 🎉

Three simple tricks for a lightning-fast Eleventy website

2020-10-12T13:44:00Z

Originally I wrote this guide as a story, but I thought some people would prefer a condensed version that just teaches you the three tricks I used for making my website faster. If you'd rather read the original version of this post, you can read it here.

Using the three tricks I'll show you in this guide, I was able to improve my PageSpeed Insights score for this blog from 86 to 100 (on mobile), which is a perfect score.

Each of these tricks is actually just an Eleventy plugin, so I can't take much credit for these tricks. But did run into some difficulties while setting them up, so hopefully my step-by-step guide will help anyone else trying to use these plugins.

I'll review the three plugins I used in order of how much they improved my site's performance, from most to least.

Async loading of YouTube videos (eleventy-plugin-youtube-embed)

In my PageSpeed Insights report, 5 out of the 9 suggestions were a direct byproduct of embedding a YouTube video on my page. I was using the standard embed code that YouTube generates when you click the Embed button from a YouTube video, which loads the video player in an iframe. I was using the privacy-improved version which doesn't use cookies, but I don't think that has any impact on performance.

If you look at the specific suggestions made in the report, you'll see that I just needed to defer the loading of all of the YouTube assets, so they don't block the rest of the page from loading. Luckily, there's a community-built plugin that can do this automatically.

That plugin is called eleventy-plugin-youtube-embed and is included in the official list of Eleventy plugins. The default functionality simply allows you to put a link to a YouTube video in a Markdown file, and the plugin will automatically replace the URL with the embed code for the video. This on its own is pretty cool, but won't improve the performance of your site, because it's still using the same embed code as before.

But this plugin also supports embedding a "lite" version of the YouTube player, which is just a static image that looks like a YouTube player, and only loads the real player when the image is clicked on. Enabling lite-mode fully resolved all of the performance issues I was seeing in my PageSpeed Insights report.

The documentation for the plugin is really good, and I didn't have any issues getting it to work, but if you really want to see the code I used, here it is (full code on Github):

const embedYouTube = require('eleventy-plugin-youtube-embed');

module.exports = function (eleventyConfig) {

    // Lots of other configuration stuff here...

    eleventyConfig.addPlugin(embedYouTube, {
        lite: true
    });

    return {
        // No need to make any changes here
    };
};

(I actually decided to host the lite JS and CSS files myself, so this isn't exactly what my config looks like. For more info, see my Bonus Trick at the end of this post.)

Resizing images (@11ty/eleventy-img)

Resizing (i.e., shrinking) images can greatly improve the performance of your website. It's common for people to upload high resolution images to their website, but then use CSS or the image width setting to display the image at a much lower resolution. This is wasteful because your browser still downloads the image at full resolution, but then downsamples the image before you ever see it. You can also reduce the file size of your images by using better compression formats, like JPEG or WebP. If you haven't heard of WebP, it's a next-gen image format designed by Google for displaying images on the web. Without any loss in perceptible image quality, it offers at least as good compression as JPEG while still supporting transparency like PNG.

You can set up your Eleventy site to automatically resize your images as well as convert them into an image format like WebP using the official plugin for images, @11ty/eleventy-img. Even though it's an official plugin, I had the most trouble getting this one set up. Hopefully my learnings will save you some time if you set this up in your own project. I'll also show you how you can use this plugin for images in your Markdown files too.

By the way, you can use this plugin to generate simple <img> tags, or you can generate the more advanced <picture> tags which support fallback or responsive images that display the correct image for your browser and viewport. For my purposes though, I only needed to use <img> tags, so I don't cover the use of <picture> tags here.

Generating <img> tags in Nunjucks templates

There's an example of how to do this in the README for the plugin, so I won't rehash that here. However, I did have a lot of trouble getting this to work, so I'll share some advice that may help you if it's not working for you. If you're getting an error in the build console Input file is missing, this might help.

Set urlPath to be the directory where your source images are, relative to your input directory that you configured in the return object in the Eleventy config file (the default input directory is .).
Set outputDir to where you want the converted images to be put, relative to the root of your project (probably where your package.json and .gitignore files are). It should go into your output directory with your other compiled files (the default output directory is _site).
When you're using your newly created shortcode, the path to the file you should use is the full path from your project root to wherever the source image is (including its file extension). So it will look something like ./{inputDir} /{pathToImages}/ {ImageWithExtension}.

The reason I got tripped up is because the plugin doesn't actually convert your images until you've correctly configured everything in both your .eleventy.js file and your Nunjucks template where you're using the image. That's because it doesn't convert all of the images in the directory you give it. It only converts the images that are specifically referenced from one of your Nunjucks templates using the Nunjucks shortcode, myImage (or whatever you call it).

You can see all of the changes I made to get this to work in this commit.

Using the plugin in a Markdown file

This is not currently covered in the README, so I had to figure this out on my own. Luckily, it's pretty easy to do. All you need to do is update your Eleventy config to use Nunjucks for parsing your Markdown templates. Here's the official doc on how to configure that. If it's not clear from the docs, you need to add a new property to the return object in your Eleventy config function. Like this:

module.exports = function (eleventyConfig) {

    // Lots of other configuration stuff here...

    return {
        // Enables the use of Nunjucks shortcodes within Markdown
        markdownTemplateEngine: 'njk',
        dir: {
            // You don't need to change this part
        }
    };
};

Then in your Markdown files you can use the myImage shortcode, exactly the same way it's used in the examples in the plugin README. Like so:

This will be interpreted as normal Markdown.

<!-- Ignore the backslashes -->
\{\% myImage "./src/images/cat.jpg", "photo of my cat" \%\}

This will _also_ be interpreted as normal Markdown.

Converting your images to WebP

I actually decided against this because Mac OS Safari doesn't support WebP images yet (they're adding support in Mac OS Big Sur which should be released soon). If you want to be fancy, you can use a <picture> tag to load either a WebP or other image format, depending on what the browser supports. I decided to keep things simple though and just use JPEG images for most things, and use PNGs for images that needed transparency.

But if you'd rather use WebP images, here's how you set that in the config:

eleventyConfig.addNunjucksAsyncShortcode('image', async function (
    src,
    alt,
    outputFormat = 'webp' // This is the only line you have to change
) {
    if (alt === undefined) {
        throw new Error(`Missing \`alt\` on image from: ${src}`);
    }

    let stats = await Image(src, {
        widths: [640],
        formats: [outputFormat],
        urlPath: '/static/',
        outputDir: './_output/static/'
    });

    let props = stats[outputFormat].pop();

    return `<img src="${props.url}" width="${props.width}" alt="${alt}">`;
});

Making your images scale proportionally on mobile

Using the <picture> tag, you can actually load a lower resolution image on mobile devices to save even more bandwidth. I decided not to bother with that though, and just use a normal <img> tag and scale the image based on the width of the viewport.

The first time I tried this using the example code from the README, my images looked stretched when viewed on a mobile device. To fix it, all I had to do was remove the height property from the <img> markup I was returning from the plugin function.

// Change this...
return `<img src="${props.url}" width="${props.width}" alt="${alt}">`;

// To this.
return `<img src="${props.url}" width="${props.width}" height="${props.height}" alt="${alt}">`;

Inlining CSS (clean-css)

My CSS file was already pretty small to begin with (1 kb), so originally I wasn't worried about it. But according to my PageSpeed report, the extra network request required to download my CSS file was adding about 180 ms to the overall page load time. Which isn't much on its own, but these things do add up, so I decided to look into it.

The solution the report suggests is inlining "critical" JS and CSS. When it comes to CSS, "critical" means that the CSS is necessary for styling the content that's "above the fold." And "above the fold" means the content that you see on a website right when the page loads, before you scroll down (you can read a deeper dive on critical CSS here). Based on that definition, all of my CSS is critical because most of my styles are for things at the top of the page, like the nav and page header. But that's ok, because the rule with inlining CSS is that you can only inline a small amount of CSS (whatever that means), and I figured my 100-line was probably small enough to qualify.

Luckily for me, I remembered that the official Eleventy docs have a quick tip on how to inline your CSS.

Summary

After making these three changes, I was able to boost my PageSpeed Insights score from 86 to 100 and shave 0.9 sec off my page load time, which is a 50% increase in speed!

Here's a PDF of my results after making these changes.

After Apple releases Mac OS Big Sur, I'll update my website to use WebP images which should make things even faster.

Bonus trick: addPassthroughCopy

This doesn't have anything to do with performance, but it's a really useful Eleventy feature that I discovered while configuring the YouTube embed stuff, and I wish I had known about it earlier.

You can see how I use it in my .eleventy.js file here.

Ta da! Adding that one line fixed my YouTube problem and resolved other issues too, like some of my favicon files not being copied over correctly 🎉

Ranked Choice Voting: Why we lost in 2020

2020-11-04T20:00:00Z

If you haven't heard, the 2020 ballot initiative to use ranked choice voting (RCV) in Massachusetts has failed. The count isn't finalized yet, but roughly 45% voted for the measure, while 55% voted against it. There was a similar ballot initiative in Alaska for ranked choice voting that also failed, but I know much less about it.

I've spent the last few months volunteering to get ranked choice voting passed in MA, so this is obviously very disappointing for me. While I've known for a while that this was a possible outcome, I genuinely believed that victory was possible. And I think I was right – it was still pretty close.

While I'm disappointed by the result, I feel ok. I feel like I personally did everything I could to help get it passed, and I don't think anything else I could have done would have made a meaningful difference. Given how strongly I believe in ranked choice voting, I know that had I not volunteered and donated towards the initiative, I'd be kicking myself now wondering if I could have made a difference. At least now I don't have to wonder and I have no regrets.

When I realized that the initiative had failed, my first question was "How did this fail?" Massachusetts is a very progressive state and I thought that if RCV could pass anywhere, it would be here. Plus, a similar measure passed in Maine in 2016, so I knew a victory was possible. Early voter polls also showed a nearly 10-point lead of Yes-voters over No-voters. What went wrong?

Regarding the early poll results, I don't actually think they were wrong. They were just misleading. The two most recent polls showed 45% and 48% of voters saying they would vote Yes, which is approximately how many people did vote Yes. What seems to have happened is that nearly 100% of people who said they were undecided in the poll translated into No voters in the actual election.

My sense is that many people went into this election mostly to vote on the presidential election and didn't put any thought into the ballot questions until they were reading their ballot on election day. This was my fear from the beginning, because I fully admit that ranked choice voting sounds like a strange idea when you first hear about it and the benefits are not immediately obvious. The ballots also get printed a long time before the election and don't show who has endorsed the measure. Ranked Choice Voting was endorsed by an amazing list of groups and people: Elizabeth Warren, Ed Markey, Joe Kennedy, Former Republican governor of MA Bill Weld, The Boston Globe – even Jennifer Lawrence officially endorsed it. But none of that was shown on the ballot, unfortunately.

The only prominent politician to oppose it was current MA Governor Charlie Baker. His criticism boiled down to: RCV would make elections more complicated and our current election system is fine the way it is.

So although this loss could have been predicted by the results of early polls, it still doesn't explain why more voters weren't in favor of it. After all, a similar measure passed in Maine in 2016 with 52% support.

This thinking led me to reframe my initial question of "How did RCV fail in MA (and AK)?" to "How did RCV pass in Maine? What makes Maine special?" I'm only speculating, but I think I have a pretty good guess. The reason RCV was popular in Maine was because their 2016 ballot question came on the heels of two frustrating gubernatorial races that resulted in an unpopular governor. In 2010, Paul LePage was elected governor with only 38.1% of the vote due to a split election where he ran against four other candidates. And in 2014, he won re-election with only 48.2% of the vote. While in office his approval rating was only 41%, which was one of the lowest of any governor in the country (Wikipedia).

Having such an unpopular governor win two consecutive elections without majority support provided a simple narrative for ranked choice voting: if Maine had RCV, then they wouldn't have Paul LePage. So it's likely that RCV passing in Maine was mostly a referendum on Paul LePage himself, rather than a reflection of genuine support for RCV. I imagine the campaign messaging for RCV in Maine was basically "Do you hate Paul LePage? Do you wish literally anyone else was our governor? Then vote Yes on Question 5!" Maybe it's not a coincidence that LePage's disapproval rating was 53% and RCV passed with 52% of the vote.

Maine had a straightforward story for why they needed ranked choice voting. But what was the story in Massachusetts?

I think this is the real reason why RCV didn't pass in Massachusetts – the argument for RCV was intellectual rather than a direct response to real experiences. One counterargument I heard from several voters regarding RCV is that most elections only have two candidates anyway, so RCV wouldn't actually be any different than plurality voting. My response to that was that the reason there are usually only two candidates in elections is because we use plurality voting. A third candidate running would only spoil the election, therefore they usually don't run. The problem with this line of reasoning is that it requires you to imagine all of the great candidates who aren't running but we could be electing and is not grounded in people's real experiences. I still think I'm right on this, but counting on people's ability to imagine better candidates is risky.

I've also thought a lot about Governor Baker's assertion that we don't need RCV in MA. A week ago, I scoffed at that statement (of course, things could be better), but I now realize that he's kind of right. If you're happy with the politicians that are currently being elected, then there's no incentive to change the way we elect them. If you can't imagine anyone you'd rather see in power, then changing the voting system could only make things worse. And boy are people happy with the politicians in MA: Charlie Baker's approval rating sits at 80% and he consistently had the highest approval rating of any governor in the US for nearly three years! (Wikipedia)

When you compare the favorability of Paul LePage in Maine with Charlie Baker, it makes total sense why more voters weren't itching for RCV in Massachusetts. Why adopt a new voting system when our current system is already giving us wonderful politicians like Baker?

I'm still very much in favor of ranked choice voting, but I'm skeptical that it can pass in a state unless there's deep dissatisfaction with their current politicians like there was in Maine. Perhaps we'll just have to wait for voters to realize the flaws with plurality voting before we can get traction for ranked choice voting. I just hope we can afford to wait.

I'm making a pandemic dating app, Part 1

2020-11-08T21:00:00Z

Well, technically I haven't started yet. But I plan to start building it soon.

Why I'm making a dating app now

I've dreamed of making a dating app for the past ten years, but haven't actually made one yet. My phone is filled with notes on different ideas for dating apps, but for one reason or another I've never actually followed through with any of them.

In the past I've hesitated for two reasons: I'm afraid of starting things and dating apps depend on network-effects to be successful. Because of my personality I'm reluctant to start anything. I'm definitely a "Let's just stay home" kind of person, so it's hard for me to take initiative and start something on my own, especially something as daunting as starting a business.

Creating a dating app is an especially scary prospect for me because it depends on network-effects to be successful. What this means is that its value to users hinges on how many users are actively using it. If you're the only person on a dating app, it's useless. The same is true for social media services, which is why you don't see many serious challenges to Facebook and Twitter's dominance. In terms of dating apps, the best app is arguably Tinder. Not because it has the best features, but just because it has the most users.

So to make a "good" dating app, it needs to be great and offer something new, otherwise it will never be able to grow its user base to a critical mass and actually become useful.

So why am I serious about making a dating app now? First, this pandemic has created a unique situation that I believe calls for a new type of dating app from what currently exists, and I think I can make that app. And second, I've got nothing better to do right now. There's a pandemic, I'm single, and I'm not interested in baking bread 😁

Do we really need another dating app?

I think there are two ways to think about that question. On the one hand, with all of the problems that exist in the world, is another dating app really what we need right now? No, of course not.

Although dating apps aren't the most important thing in the world, I still think they have the potential to be valuable. As Anne Hathaway awkwardly put it in Interstellar, "Love is the one thing we're capable of perceiving that transcends time and space." Putting aside cringe-worthy movie lines, I do believe that love is real and I would feel proud if I could make something that helped some people find it.

I'm not naïve though – I know that many people use dating apps for hook-ups. But I don't have anything against people looking for a hook-up. As long as it's consensual and people are honest about what they're looking for, I'm all for it.

Anyway, there's another way of looking at the question. Is there room for a new dating app to offer something that the existing apps don't already offer? In my opinion, the answer is an absolute yes.

I've downloaded and tried lots of different dating apps (Tinder, Bumble, OkCupid, CMB, Hinge, Feeld, Pure, and more). I've also talked to anyone willing to talk to me about their own experiences with dating apps.

From my own experiences and what I've heard from others, I've learned one thing: everyone hates dating apps. You might be wondering how dating apps could be simultaneously extremely popular and extremely disliked. I think what's happening is people download them because they feel like they're the only way to meet people anymore, but they leave much to be desired. I mean, if you're not in college, what are you supposed to do? Go to a bar and hope for the best? If this was ever an effective strategy, it's certainly not possible or advised now in the time of COVID. So people use dating apps because they don't have any other options.

I'll get to what people don't like about the current dating apps soon. But the important thing is that the current apps are bad. And I think this is a missed opportunity. After overcoming its initial stigma, online dating should have been the panacea for all single people in the world. With so many options for online dating, how could they all be so bad?

Why the existing dating apps are bad

Full disclosure: I've never tried Match.com or any other paid dating app. I've also never tried exclusive dating apps like The League. Maybe they're great. Maybe the problems below don't apply to them. But even if these apps are better than the free and open options, I don't care. I'm not interested in any dating app that filters its potential user base by charging a subscription or excluding people they deem unworthy.

The number one complaint I've heard about dating apps is that they waste your time. This isn't how it's usually phrased to me, but that's what it boils down to. This problem is also not limited to one gender – it's a problem for everyone.

Not trying to stereotype, but from what I've heard many men complain that they don't get enough matches on dating apps. They spend hours swiping through profiles, yet never manage to land a date with someone.

The women I've talked to have the opposite problem. They get flooded with matches. But the bulk of those matches are "junk". Most conversations fizzle out and never lead to a date, or end when they're asked for the umpteenth time to "send nudes."

And if you do manage to actually make it through the swiping-and-messaging tar pits and go on a date, there's a high likelihood that the the person will turn out to not be as great as you had hoped. And now you've wasted one of your precious free nights on that person.

It's exhausting.

The second problem I've heard about dating apps, and which I've personally experienced, is that they lower your self-esteem. People often talk about dating apps being a "numbers game," meaning that you have to work your way through a lot of people to find a good one. But it's hard not to feel bad about yourself when you're confronted with so much failure. You'll try to tell yourself not to take it personally, and there's nothing wrong with you and that this is how dating apps feel for everyone. But everyone has that one friend who found a serious relationship after downloading an app and casually using it for a week. Why do dating apps work for some people but not for me? What's wrong with me?

On the other hand, some people actually use dating apps for a self-esteem boost. If you're someone who gets a lot of matches, it's nice to see all of the people who liked your profile. Maybe this is actually effective for some people, but my understanding is that this is not a good way to develop lasting self-esteem.

Also, maybe it's just me, but I feel like dating apps don't actually care whether or not I find a date. It seems like the actual goal of dating apps is to make me so frustrated that I cave and buy more Super Likes or their premium subscription.

If you've used dating apps before, I hope that these problems align with your own experience. But they may not. Most of what I know comes from anecdotes shared by my friends, who are mostly cis-gendered, White and Asian, men and women. None of my friends are non-binary, and I would imagine that the dating experience of a trans woman of would be very different from my own. I'm planning to add comments to my blog soon, and then you'll be able to tell me what your experience with dating apps has been.

Why my dating app will be better

Sorry, but I think this post has gone on long enough. In Part 2, I'll go into detail about how my app will be different and how it will address some of these problems. I'll link to it here once I've written it.

New features: Comments and RSS feed

2020-11-11T21:00:00Z

New features on the blog

Comments

You can now add comments to any blog post I've written, old or new. Just scroll down to the bottom of the page and put your name and comment. Feel free to make up a name or leave it blank.

I'm not doing any moderation of the comments because I'm hoping it won't be necessary, so don't make me regret it 😅

I'm using Staticman for the comments. This lets me have comments without adding a database to my site. Setting it up wasn't as straightforward as I expected, so maybe I'll write a post about it later.

RSS feed

You can now subscribe to get every update to my blog by adding https://ajenkins.me/feed.xml to your RSS reader of choice. If you don't know what an RSS feed or reader is, it's a way to get updates from different websites aggregated in a single place. It's like subscribing to a website's newsletter, except the updates get pushed to a special app instead of your email inbox.

If you're looking for a recommendation for an RSS reader app, I use Feedly which has been working pretty well for me.

More features coming soon

Email updates

I wouldn't call it a newsletter, but for people who don't want to use an RSS reader, I'd like to offer the option to get an email every time I publish a new post.

Rather than use some fancy service like MailChimp, my plan is to just put a form on my home page where you can give me your email address, which would then be added to a list of email subscribers. And whenever I publish a new post, I would just use my personal email to inform everyone on the list.

I wanted to do it this way because MailChimp and its competitors brag about all of the insights you get into your newsletter's performance, but it just sounds like an invasion of privacy to me. See my post on why I don't have analytics on my site.

By the way, if this sounds like a terrible idea to you, please let me know... in the comments! 😉

This might be a bit farther out because I don't think it's as important. But it would be nice on some of my longer posts to be able to jump to a particular heading.

...And more

I have a few other ideas, but that covers the major ones. Thanks for reading!

I'm making a pandemic dating app, Part 2

2020-11-27T20:15:00Z

Sorry for the long wait since Part 1. In Part 2, I'll talk about the main features of the app and how it's different from existing dating apps (and hopefully better). This is the second and final part of this post, but I am planning on writing more posts about this project as I work on it.

I have a name in mind for what I want to call the app, but I'm going to keep it a secret for now. So instead of putting the name in this post, I'll refer to it as XXXXXXX. Your computer isn't broken, it's supposed to look like a black rectangle.

Elevator Pitch

XXXXXXX is a video-centric dating app that respects your time. XXXXXXX was designed from the ground up around the question "What is the optimal dating experience when you can't safely meet other people in-person?" The "goal" of most dating apps is to schedule an in-person meeting with someone, so what does a dating app look like when that's no longer possible? And although the concept for this app was born out of the circumstances of the current pandemic, I believe that XXXXXXX will offer a superior user experience over existing dating apps even when things go back to normal.

How it works

After you create your account, you're brought to a list of all users who match your preferences and are currently online. Rather than swiping through profiles one at a time, you'll scroll through a list of users.

When a user's profile catches your eye, just tap the 📞 button to start a video call with them 😱

I know that calling a complete stranger might seem a little scary to some of you. But the good news is your other option is you can always just wait for someone to call you instead 😌

If someone calls you, you'll see the incoming call pop up on your screen and you can decide whether or not to accept it. Before you accept, you can view the person's profile. If you don't accept the call within 30 seconds, it will automatically be declined.

Once a call is accepted, you'll immediately begin a 3 minute video call where you can get to know each other. After the call ends, you'll be able to message each other through the app or set up another video call.

If you don't want to hear from that user anymore, you can block or report them.

To make the initial video call a little less intimidating, everyone will have the option to write their own conversation starters which are displayed in their profile. For example, you could put "What's the last thing you watched on Netflix?"

Why I think it's better than existing apps

As far as I know, there's no app that works quite like this. It does share many similarities with other dating apps though. I think of it as a hybrid between "traditional" swipe-based dating apps like Tinder and virtual speed-dating apps. I'll explain why I think XXXXXXX is better than either of those types of apps.

Tinder-like apps

Most popular dating apps have adopted the Tinder-style swiping system. The hallmark of these apps is that you see profiles one at a time, and you are forced to make a decision about the person you're looking at (swipe left or swipe right) before you're presented with the next user-profile. Some examples of this type of app are Tinder, Bumble, Hinge, and OkCupid. Here are the ways that XXXXXXX is better than those apps.

Doesn't waste your time

This is the big one. In Part 1 of this post, I said that this is the main shortcoming of current dating apps. With most dating apps, you don't actually find out if you have chemistry with a person until your first date. But getting on that first date takes a lot of work: you need to write a compelling profile and find flattering photos of yourself, then you need to spend hours swiping through profiles hoping you get matched, next you need to do your best to have an interesting text-based asynchronous conversation with your matches (which can drag on for more than a week), and finally you need to schedule a date and commit an entire evening to getting to know that one person. And if the date doesn't go well, you're back to square one and need to repeat the entire process. It's exhausting.

XXXXXXX, on the other hand, bypasses all of those steps. In my opinion, you can learn a lot more about someone from a 3 minute video call than you would learn from a week of text messaging. Although you can set up a video call with someone you've matched with in a traditional dating app (and some people do), my instinct is that asking someone to do a video call right off-the-bat would be a turn-off for some people. In dating apps there's an unspoken etiquette that you need to message someone for a certain amount of time before you can propose moving onto the next step, be it a video call or an in-person meeting. Hinge actually added a feature that lets you privately indicate when you're ready for one of those things, and the app notifies you when you've both stated your readiness for taking the next step. This takes the guesswork out, and is a great improvement, but it still likely requires you to spend some time messaging the other person before they feel ready to move on to a video call.

In my opinion, needing to send any text messages is a waste of time. It's unlikely that you'll learn anything of value from those messages, and you'll only know if you like the person after a video call or meeting. So why not just skip straight to that? XXXXXXX solves this problem by forcing everyone to have the video call first, and then making text messaging an optional second step. There's a stigma with asking for a video call or in-person meeting in your first message in a traditional dating app, but this stigma is avoided by forcing everyone to interact in a more efficient way.

No fake profiles

There are two types of "fake" profiles you'll encounter in dating apps. The first type, the real fake profiles, are profiles for people who don't exist. A well-known form of this is catfishing. There are multiple reasons why someone would make a fake profile, but in my experience they're mostly scammers who try to get you to send them money either by offering some service or trying to blackmail you.

The second type of "fake" profile are real people who are intentionally dishonest or misleading in their profile. Everyone does this to some extent (choosing the most flattering photos to use in their profile), so my point is not to judge people who do this. But not knowing exactly what you're getting into until you meet the person on the first date is a problem that many dating app users face.

XXXXXXX solves both of these problems by making the first interaction you have with a user a video call. Without wearing an elaborate disguise, it's very hard to fool someone about your identity or appearance over a real-time video call.

As an added bonus, I believe that prioritizing video-based interaction would reduce the pressure to spend a lot of time hunting through your Facebook or Instagram profile for that perfect dating app profile picture. My hope is that people would worry less about their photos if a video call that let's you see what the person actually looks like is just a click away.

Speed-dating apps

This is a relatively new category of dating app, and so far none have managed to become very popular. These apps seek to replicate the format of in-person speed dating events, but online. After creating a quick profile (age, gender, name, picture) and setting some basic preferences (gender, age range, max distance), you get randomly paired with people who match your criteria and engage in a brief video call with them. If you both indicate that you liked the other person, you get connected so you can continue communicating or schedule a meeting. Also, to ensure that there are as many users as possible for you to meet with, these apps typically schedule specific times when you can use them, like Thursdays and Saturdays at 7pm.

XXXXXXX is very similar to these types of apps. The main difference is that it doesn't randomly pair you with users but instead lets you choose who you talk to. This is important because it gives users a greater sense of agency in the dating process, and also ensures that you're not wasting your time talking to people you have no interest in. Liking specific users (i.e., swiping-right) is also one of the few fun parts of dating apps, so XXXXXXX preserves that element of the experience.

If these apps are the digital equivalent of speed-dating, XXXXXXX is the digital equivalent of meeting people at a bar. Like in a bar, you choose who you approach and try to strike up a conversation with and immediately begin with a face-to-face conversation. XXXXXXX is better than a bar though because you know that everyone you see in your list is someone looking for someone like you, conversation starters are built in (no need for corny pick-up lines), and you don't have to worry about someone physically harming you.

Risks

There are some risks with XXXXXXX over other dating apps. I've done my best to think through all of the things that could go wrong with an app like this or ways it could be abused, but I'm sure there are others I haven't thought of yet. Below are what I view as the two biggest risks to XXXXXXX.

Flashers

Sooner or later, a user will expose themselves to another user during a video call. Unfortunately, if you create a service that lets people video-chat with strangers on the internet, it's inevitable that some of them will use it as a tool for flashing people. I would obviously forbid this kind of behavior in XXXXXXX, but that won't stop most people who want to expose themselves to strangers. I need to do everything I can to ensure that flashing is rare on XXXXXXX, both for ethical reasons and to ensure that XXXXXXX is an app that people want to use.

There are two ways to limit flashing: preventing people from getting flashed in the first place, and making sure that people who are caught flashing are prevented from doing it again.

There's no bulletproof way to prevent people from flashing other users, but I've seen some technical solutions to the problem. One measure some apps use is they require your face to be in frame at all times, otherwise it blurs your video. This would definitely make it harder to flash someone, but I'm sure any system like that could be outsmarted.

For that reason, I'll also need to figure out what to do when someone does flash another user. My plan is to report flasher, and then after the claim has been verified, I would permanently ban the flasher from the app. But how do you stop that person from just making a new account? I have a few different ideas on how to stop banned users from creating new accounts, so maybe I'll talk about them in a future post.

Not enough users online

In the "How it works" section I mentioned that you would only be able to see the profiles for users who are currently online. This is different from apps like Tinder where you can view the profiles for all users in your area at any time, even if they're not currently using the app. So, if there aren't enough people using XXXXXXX when you're using it, it will feel very lonely. If you open the app and only see one person in your list, and they either don't answer your call or it's not a good call, now what? If this happens repeatedly, you would most likely delete the app.

If XXXXXXX has as many users as Tinder, this wouldn't happen very often. No matter when you're using the app, there will always be people online. But of course, XXXXXXX won't have many users as Tinder in the beginning, so how do you ensure it doesn't feel like you're the only person using the app?

One solution is to restrict when people can use the app to specific times. This is what speed-dating apps do, and may be what I do for XXXXXXX. Another option is to send a push notification whenever a certain number of people are using the app. That way you don't need to randomly check the app all the time to see if people are using it. But the downside is it requires an initial group of people to start using the app to get the ball rolling. My least favorite solution is to make XXXXXXX asynchronous like most dating apps and show you offline users too. But what do you do with an offline user? Leave a message? Swipe right and hope they swipe-right you back? My fear is that if I go down this road, I'll sacrifice all of the key benefits of XXXXXXX, like not wasting your time.

Next steps

I have a pretty good idea of what the initial version of this app will look like and what features I'll include, so the next step is to start building it. First I'm going to create a minimal backend API for the app to use (the "brain" of the app), then I'll start working on the code for the app itself. The first feature I'll implement is the video-calling, because the success of the app hinges on the video-calling working flawlessly. Once I've got the minimum functionality working, I'll release an alpha version of the app to my friends and family for early testing. Getting to that point is at least a few months away, so I'll provide another update once I've hit that milestone with further next steps. While I'm working on the app, I plan on publishing updates on this website. I have an RSS feed you can subscribe to if you want to be notified when there's a new post.

Thanks for reading, and let me know what you think about XXXXXXX in the comments!

Review of AWS AppSync and Amplify in 2019

2020-12-02T02:00:00Z

UPDATE: I'm using Amplify just for authentication in my new app. You can find my updated opinion of Amplify at the end of this post.

In 2019 I started a personal project using AWS AppSync, AWS Amplify, and React Native. AppSync and Amplify may have gotten better since I tried using them, but at that time they were so painful to use that I abandoned the project. I thought about migrating to a different backend framework, but that would've sucked, and it just didn't seem worth the effort.

Although some things were great about AppSync/Amplify, my overall experience was so negative that I would strongly urge anyone considering it for a personal project to reconsider.

AppSync vs. Amplify

AppSync and Amplify are closely related, but are actually distinct Amazon products. AppSync is a tool for building a GraphQL API, while Amplify is a command-line tool and frontend framework for building apps that talk to AppSync. AppSync creates the /graphql endpoint and the GraphQL resolvers. Amplify has a command-line tool you use to generate your Amplify app (supports mobile or web) and automates a lot of the stuff you would need to do with AppSync, like writing resolvers. When I used Amplify, you were required to use DynamoDB, which is Amazon's performance-optimized NoSQL managed database solution. Amplify also automatically generates your GraphQL queries and mutations for your client-side code based on what it thinks you'll need given your GraphQL schema.

Initial impression

My initial experience with Amplify was awesome. After running just a few commands from the command-line, I had a React Native app running on my iPhone (using Expo) that was talking to a real GraphQL backend running in Amazon's cloud. With a little more work, I had a fully functional Sign In/Sign Out/Forgot Password screen in my app backed by AWS Cognito. I was also able to add row-level permissions to my GraphQL API that restricted users access to their own resources just by adding an @auth directive to my GraphQL schema. The early promise of Amplify was that I could make any backend changes I needed just by updating my GraphQL schema, and then run the Amplify regenerate command which would magically update all of my client-side code and push out updates to AppSync and DynamoDB.

This all sounded great in theory, but in the end Amplify wasn't able to live up to that promise.

Many-to-many relationships

Setting up a many-to-many relationship is very easy to do in SQL databases. For some reason, many-to-many relationships are a little tricky to represent in a GraphQL schema, so each GraphQL framework has settled on their own way to handle them.

At the time when I was using Amplify, the did not natively support many-to-many relationships. Instead, they offered a hack you could use where you simulate the many-to-many relationship using two many-to-one relationships. Unfortunately, when you used this hack, the client-side code that Amplify generated for you was not what you actually needed, so to get it to work fully I had to do a lot of extra legwork.

For some reason, Amplify had no problem with create-mutations for many-to-many relationships, but was completely unable to handle deletes. For my project, I needed a many-to-many relationship between Movies and Lists of Movies (MovieList). Implementing the ability to delete a Movie from a MovieList required two weeks of frustrating work. I had to write my own resolver using an Apache VTL template (seriously, what is this?) and fight with DynamoDB. When AppSync works it's great, but debugging it when it's not is a nightmare.

If you want to go deep on this, take a look at the Github issue for adding many-to-many relationships to Amplify. The issue was open for more than a year and is filled with comments from developers trying to get Amplify to do simple things and who were just as frustrated as I was.

Final thoughts

I've been told by more experienced engineers that when it comes to frameworks, "magic" is bad. In this context, "magic" refers to a framework's ability to automate away writing boilerplate code which is supposed to save you time. The problem with "magic" frameworks is you're screwed if that framework doesn't have a "spell" (i.e., documented solution) for that one special thing you need to do. The magic of the framework becomes your enemy, and you need to fight against the framework or write complex low-level code to solve your problem.

This is exactly what happened to me with Amplify. Everything was a breeze until I needed to delete a Movie from a MovieList. By the time I realized the difficulty of this task, I was already too deep with Amplify to consider abandoning it. So I had to waste weeks solving a problem that I wouldn't have had in the first place if I was using a more basic, less magical framework.

Also, don't use DynamoDB unless you're really, really sure it's right for your use-case. I assumed that a NoSQL database would be simpler than a SQL database (No rules! 🥳), but it's actually the opposite. Database migrations are not possible in DynamoDB tables without manually iterating through every row and updating it, so you better be sure you know what your schema is going to be when you start. As far as I can tell, the only reason you would use DynamoDB over a SQL database is if you need a Google-scale database solution with massive horizontal scaling and extremely low latency responses. For 99% of use cases, a simple SQL database running on RDS is easier to work with and will perform just fine.

If you're curious what I would use instead of Amplify if I were to start that project today, you can read my post on that here.

Building a modern API in 2020

2020-12-03T03:00:00Z

Intro

In 2020, you have more options than ever for building a backend API for your app or website. Choice is good and the newest generation of backend frameworks are amazing, but having so many choices to consider can be overwhelming.

Before starting work on my dating app, I decided to research these next-gen tools for building a backend to make sure I'm using the right tool for the job. If you're planning on creating an app that needs a backend, you'll likely be considering many of the same options I was considering. Although I don't have hands on experience with most of these tools, I did learn a lot about them by carefully reading their docs to understand what they do and don't do. I hope that by sharing what I've learned with you, I can clear some things up that confused me and save you some time.

I'll also tell you which services I'm planning to use for my app. But keep in mind that I may change my mind once I get started, or I may regret some of these decisions. In a year I'll give an update and let you know how these tools worked out for me 🙂

Components of backend stack

In my mind, these are the main components you need when building a backend API:

Hosting (e.g., Netlify, AWS EC2, serverless options)
Database (e.g., MongoDB, PostgreSQL)
(Optional) ORM (e.g., SQLAlchemy, Prisma Client)
API (REST or GraphQL, lots of choices)
Auth service (e.g., Auth0, AWS Cognito)

A complete backend stack needs most if not all of these things, but some frameworks cover multiple items in this list. Like Amplify + AppSync, which takes care of everything in this list.

I'll walk you through my own decision process for what I would use for each of these components in the sections below. Before that though, I need to talk about AWS Amplify. In theory, it sounds like a great tool because it handles all of the above components for you. But I used Amplify for a project I worked on last year, and my experience with it was so negative that I swore to never use it again. You can read my full review of Amplify/AppSync here.

Hosting

Cloud providers

Technically, one could decide to host their API on-premises, but I don't think that's a serious consideration for any new project being built in 2020. The question in 2020 isn't whether or not you're going to build in the cloud, but which cloud you're going to use.

The big players in the cloud space are Amazon (AWS), Microsoft (Azure), and Google (GCP). Heroku is still a good option, but can get a little pricey. There are also lots of special purpose hosting options that might make more sense for your use case. For example, the backend needs for this blog are very simple, which is why I'm hosting it with Netlify.

I don't have a strong opinion on Azure, AWS, or GCP, but I have the most experience with AWS so that's my starting point when I'm deciding what to use. Those three cloud providers are similar enough in terms of features and pricing that if you have experience with one of them, that's the one you should use.

Self-hosted vs. Managed

If a tool is self-hosted, that means that you're responsible for supplying your own hardware (typically an EC2 instance or Heroku server) to run and serve up the tool. A managed service (often branded as the Cloud version), is run on some company's servers, and you pay them a subscription to get access to the tool through them.

Some tools only offer a self-hosted option, some only offer a managed option, and some offer both. Like Jira, which has two versions you can buy: Jira Server (self-hosted) and Jira Cloud (managed).

Typically, self-hosted tools are open-source and completely free to use forever. All you need to pay for is the server that's running it. However, there's also the cost of being completely responsible for that piece of infrastructure. You need to make sure that the server(s) you're using is fast enough to deliver acceptable performance and figure out how to scale it if it's not. If there's data involved, you're also on the hook for keeping that data secure – if you get hacked that's on you. You also have a lot more control over how the tool works though. Often if you don't like how it works, you can always just change the code that's running and make it work the way you want it to.

With managed solutions, you pay a company to worry about all of that stuff for you. They usually provide you with a nice UI for doing admin tasks, and handle things like making sure the service is performant, secure, and keeping the software up to date.

Like all things in software engineering, which of these options is better for you will depend on your use-case. I think personal preferences also plays a big role in how people make this decision. I know some developers who are great at DevOps and always prefer to self-host everything.

But for me and for this project, I decided early on that I would go down the managed route for everything. A classic piece of advice is to maintain control over the parts of your app that are unique to the problem you're solving, but outsource everything else. For example, my user auth needs are very standard, so I immediately disregarded any auth provider that didn't offer a fully managed solution. That narrowed my options down to basically Auth0 or AWS Cognito.

Serverless?

I don't know what the word is for the opposite of serverless (serverful?), but a lot of the next-gen tools tout their ability to run on serverless infrastructure. The advantage of a serverless backend over a traditional server is that you only pay for what you use. When you use a traditional cloud server, like an EC2 instance from Amazon, you pay a fixed rate as long as it's up and running, even if it's sitting idle and not doing anything. With a serverless backend, you only pay for when it gets used, and you pay per action performed. This can make serverless infrastructure much cheaper to run than a traditional server, since you only pay for what you use. Serverless is also much better at dealing with spiky traffic because serverless functions are extremely parallelizable and can run as many instances at once as needed to get through the influx of requests.

You don't need to commit wholesale to making your entire backend serverless or not serverless though. Hybrid solutions are possible where your main server does most of the work, but ancillary tasks are handled by short-lived serverless functions.

The main downside of serverless is it may require you to adapt the way your code works to function in a serverless environment. In the past, APIs were always built with the assumption that they would be running on a persistent server, so you usually need to use a special-purpose API framework optimized for serverless deployments.

For my purposes, although I could save money by making my backend serverless, as a team-of-one I decided I'm better off sticking with a traditional "serverful" architecture. That being said, since I'm mostly using managed services for my backend, making things serverless honestly wasn't something I really had to consider. When you pay for a managed service, you don't need to worry about whether or not the company is using serverless. All that matters is that the service works when you need it to. I have no idea if Auth0's underlying architecture is serverless or not, but as long as they give me a JWT when I ask for it, I'm happy.

What I'm using

I plan on hosting my database on AWS RDS, and my API using Hasura's managed cloud offering (spoilers, I'm using Hasura). For auth, I'm using a managed solution like I mentioned above.

Databases

Let's get this out of the way: I know very little about databases. A month ago, I wouldn't have been able to tell you the difference between an INNER JOIN and OUTER JOIN if my life depended on it. So, you should take my opinions on this topic with a grain of salt. The only databases I've used are PostgreSQL, MongoDB, and DynamoDB.

I have the most experience with Postgres and it seems to be the gold-standard for SQL databases and the most common choice for a database. So going into this I was leaning towards Postgres, but I don't particularly enjoy writing SQL so I was on the lookout for options that would solve my data persistence problem without needing to write a line of SQL.

Rather than lecture you on the pros and cons of SQL vs. NoSQL, I'm just going to skip to the choices I actually considered using. Regarding SQL vs. NoSQL, all I know is that I much prefer using PostgreSQL over MongoDB or DynamoDB (especially DynamoDB, see my review of Amplify).

The way I chose what database to use was I was committed to using whatever database was most compatible with the API framework I wanted to use. Most of the API frameworks I was considering offered the best compatibility with Postgres, so that seemed like the obvious choice.

Near the end of my search though, I actually did consider using another database: Dgraph. Dgraph is a relatively new company which offers several related products, one of which is a database. They call their database Graph DB, and it's a distributed graph database optimized for building a GraphQL API.

This was extremely enticing to me, and I was actually very close to using Dgraph to build my backend.

But after a sober assessment of Dgraph, I decided against using it. My fear with Dgraph is that I would spend a lot of time building my backend with it, but then eventually find something that Dgraph can't do or doesn't do well. If I wanted to switch to a tried-and-true database like Postgres, migrating would be extremely painful. DuckDuckGoing "migrate from dgraph to postgres" didn't return any results, so I assume I would need to crawl the GraphQL API to get all my data out, dump it to a file, then load it into Postgres. And that's assuming I can massage the data in Dgraph to conform to a relational database model, which isn't a guarantee because Dgraph is a graph database and not relational.

With an API framework like Hasura, if I decide to use something else, at least I'll have my Postgres database that I can reuse with a new framework.

What I'm using

Plot twist! I've decided to use AWS Aurora as my database. Supposedly it's a drop-in replacement for Postgres, so I don't think it's a stretch to say I'm using a Postgres database. The reason I want to use Aurora instead of true Postgres is because it works exactly like Postgres but is cheaper and faster. That's what Amazon says anyway, so I'll take their word for it.

Prisma

When I started researching backend options, I was really hoping to learn that Prisma would be the best choice for my app. I've been following Prisma's development for a while, and I've been itching to finally use it for a project. In 2019 when I was screaming at Amplify for ruining my life, I would often look longingly at the Prisma docs and fantasize about how much easier things would have been for me if I had used Prisma instead. So Prisma was definitely my number one choice going into this.

Unfortunately, after doing a lot of research on Prisma, I realized that it's not the magic bullet I thought it was. It took me a long time to come to grips with what Prisma even is now, so I'll explain what I thought Prisma was, why I thought that, and what it actually is.

The History of Prisma

The original incarnation of Prisma was actually called Graphcool. Graphcool was already considered a legacy product by the Prisma team when I first encountered it, so I don't know a lot about it.

But it seems like it was a full-stack, managed service for building a GraphQL API, similar to what DGraph offers now. It had everything you needed for building a GraphQL backend, including user auth. But for some reason they decided it was bad, and pivoted to developing Prisma.

Now known as Prisma 1, Prisma had a narrower scope than Graphcool. Prisma 1 consisted of three main parts: the Prisma data model schema, the Prisma server, and the Prisma client. First you would define your data model using their schema file, and then you could have it run migrations in your database to create the needed tables. You were responsible for providing your own hosted database, but it was compatible with most SQL databases. The schema file also generated the Prisma client code you used for querying and mutating the data, and the Prisma server was just a thing that facilitated communication between the Prisma client code and the database.

When they abandoned Graphcool in favor of Prisma, I was confused but impressed. Graphcool did more things than Prisma does, so wouldn't that make it better. But I knew that they must have put a lot of thought into the decision, and surely it was the right move. I inferred that if they were discontinuing Graphcool in favor of a product that did less, well then Graphcool must have done too much. Sometimes a framework that does too much than a framework that does too little (I'd rather build an app with Express than Ruby on Rails), so that must have been the case here. It's better to do the important things really well, than to try to do everything.

But as you probably guessed from the name Prisma 1, now there's a Prisma 2! Based on the name, I assumed that Prisma 2 would just be a better version of Prisma 1. But in fact, Prisma 2 is actually another pivot almost as significant as the pivot away from Graphcool. For that reason, they're actually moving away from the Prisma 2 moniker now in favor of calling it Prisma Client. And like the last pivot, Prisma 2 does less than Prisma 1. Prisma 2 (Prisma Client) is just an ORM. Graphcool was a fully managed GraphQL API – Prisma 2 is an ORM you use to build a GraphQL API.

It took me a really long time to understand that though, because I was thinking of Prisma Client as an alternative to AppSync, Hasura, and Dgraph. But Prisma Client is solving a different problem, and occupies a much thinner slice of the stack than those tools.

Like their first pivot, I assumed that this pivot was motivated by an epiphany about backend developments, and was just a level of genius that I couldn't comprehend. I'm not being sarcastic by the way, this is literally what I thought. My obsession with finding silver bullets in software engineering makes me eternally optimistic that someone will finally crack the hard problems and find a revolutionary way to do something better.

Another misconception I had about Prisma was I thought that Prisma Client was something you used in your client-side code. I was thinking of it as a super-powered version of Apollo Client, except instead of talking to a GraphQL API it magically talks directly to the database. But even though the Prisma Client code is in Javascript, it's not intended to run on the client-side. Prisma Client is used within a NodeJS server, like Express.

I also wasted a lot of time trying to figure out what Prisma 2's auth story is. Hasura has a page in their docs about auth and so does Amplify, Dgraph, and even Prisma 1. But the only reference to auth I could find in the Prisma 2 docs was an example repo that implemented user auth. I looked at that code but I was confused because the auth code seemed completely disconnected from the Prisma code. In the end, I finally realized that the reason they didn't have any docs about auth is because Prisma 2 just doesn't have an auth story like the frameworks I was comparing it to. This makes sense when you realize that Prisma 2 is... Just. An. ORM.

Why I'm not using Prisma

Although in my head I still picture the Prisma devs as enlightened, 10x, levitating engineers who have discovered the ultimate truth for building backends, I've decided that Prisma 2 is not the right tool for my project

If the backend API was the interesting part of my project, then I would consider using Prisma. But it's not, so I want a tool that will give me a fully functional API from the least amount of effort. Prisma Client is just too low level of a tool for me – if it's just a sub-tool for building out a GraphQL API, then I'd rather use one tool that can give me a complete GraphQL API.

Maybe next time, Prisma 😢

API frameworks

Here are the API frameworks I considered using for this project, in no particular order: PostgREST, Supabase, Hasura, GraphQL Nexus, FastAPI, Flask, Express, PostGraphile, and Dgraph.

These tools occupy a wide spectrum of API solutions. Here are a few ways to categorize them:

GraphQL or REST

GraphQL

GraphQL Nexus
PostGraphile
Dgraph
Hasura

REST

PostgREST
Supabase
FastAPI
Flask
Express

Self-hosted or managed

Self-hosted

PostgREST
GraphQL Nexus
FastAPI
Hasura Core
Flask
Express
PostGraphile

Managed

Supabase
Hasura Cloud
Dgraph

Postgres or a different database

All of the options I considered either require Postgres (you can spot them based on their name) or are compatible with Postgres, with the exception of Dgraph which uses its own database system.

GraphQL vs. REST

REST is still the gold-standard for API design and is how most APIs built in the last 10 years have been modeled. GraphQL is a relative newcomer originally developed by Facebook but that has been embraced by the open-source community. After my negative experience with Amplify and AppSync, I came into this project assuming I would build a REST API. But then I read a Medium post about GraphQL and walked away convinced I was better off using GraphQL. The advantage he described that was convincing to me was that with a GraphQL API you don't need to waste time designing your endpoints in accordance with REST best-practices. Once you've got a spec-compliant GraphQL endpoint, you're done – it will do everything you need it to do. The main downside of GraphQL is that it can be hard to migrate from REST to GraphQL, but that's not a concern with a brand new project. So for me, there's not much of a downside to using GraphQL.

Self-hosted vs. Managed

In the Hosting section I go into more detail about the differences between them. I also said that I prefer managed solutions whenever possible. That being said, in this case I was very open to self-hosting the API. As you can see in the above lists, most of the frameworks I considered were self-hosted and self-hosting my API seems very natural to me, unlike self-hosting an auth service.

Although I was open to self-hosting my API, the managed solutions were ultimately more appealing to me. Not because I didn't want to host the API, but because the managed options also promised the ability to manage the API and database through an admin UI rather than needing to write code. Even though I know how to write code, I still try to avoid writing code unless I need to. Because while writing code is a lot of fun, maintaining code is not. So the less code there is to maintain, the better.

Postgres vs. Something else

Based on the above two considerations, that only left Hasura and Dgraph as potential options. In the end I decided I was better off with Hasura, even if Dgraph would have offered a better developer experience. Knowing that the entire backend was underpinned by a Postgres database was just more important. See the Databases section above for more on this.

What I'm using

Hasura Cloud. Read the previous few sections for why.

User Auth

Like I mentioned in the Hosting section, user auth is one thing that I really want a managed solution for. Handling user data scares me, and I don't want to have to be responsible for keeping sensitive data like that secure.

Of the managed offerings, the two main options I found were Auth0 and AWS Cognito. Here are the pros and cons of each of them:

Auth0

Pros

Free for up to 7000 users
Best-in-class developer experience
Great documentation
Very popular - easy to find help online
SDKs in lots of languages for generating log-in flows

Cons

Expensive past 7000 users

AWS Cognito

Pros

Made by Amazon, so you know it's probably decent and won't close up shop
Very affordable pay-per-user pricing model, no big jumps in price

Cons

AWS docs are usually bad
No drop-in code for login-flow unless you're using Amplify

What I'm using

AWS Cognito. Even though Auth0 would have been way easier to get started with, their pricing model is a no-go for me. I'm ambitious and hope to have many more than 7000 users (it's a free consumer app), and I don't want to have to give that much money to Auth0. I considered starting with Auth0 and then moving to Cognito down the road once I got more than 7000 users. But after reading Auth0's docs on how you migrate from one auth backend to another, the process sounded too hard and fraught with peril. I wouldn't trust myself to do that without screwing up and deleting or exposing everyone's passwords.

So I decided I'm better off biting the bullet and spending some extra effort in the beginning to get Cognito set up.

Summary

Putting it all together, here's what my backend stack will look like:

Database - AWS Aurora (Postgres flavor)
API - Hasura Cloud
Auth service - AWS Cognito

On the frontend, I'm going to use Apollo Client and React Native.

That's it! Thanks for reading this very long post. This post was good motivation for me to add a table of contents plugin to my blog, so maybe I'll do that soon.

What backend tools are you excited about? Do you think I missed the mark on something? Will I rue the day I decided to build my backend with Hasura? Let me know in the comments!

Dating App: Progress Update 1

2020-12-27T14:00:00Z

In my last post I talked about some of the technologies that I was planning on using for building my dating app. Since then, I've completed the minimum functionality for the app and wanted to share what I've done so far.

On the backend

Skip to the next section if you just want to see some screenshots.

Behind the scenes, I've set up an Aurora (Postgres) database running on Amazon RDS. On top of that, I'm using Hasura Cloud to serve a GraphQL API. So far, I'm loving Hasura. Everything just works, I haven't run into any roadblocks yet, the documentation is great, and I've been able to fully set up my API and manage my database using their UI. The only hiccup I've run in to with Hasura was in the initial set up. They have a guide specifically on how to set up Hasura with Aurora, but the guide doesn't mention that it doesn't work with the serverless option and you need to set up your database as provisioned. I had to contact support, but they figured it out pretty quickly and I just had to create a new database.

For user authentication I'm using AWS Cognito and Amplify. This might be surprising to you if you read my review of Amplify, but it seems like it's evolved a lot since I last used it. Using Amplify also seemed like the easiest way to integrate Cognito into a React Native app. The even have specific instructions on how to use Amplfiy with an app managed with Expo, which I was using in the beginning until I had to eject (more on ejecting later). Also, you can now use Amplify just for one function, like auth, and you don't need to use Amplify for your API if you don't want to. I go into more detail about my experience using Amplify at the end of this post.

The frontend

Once I got the backend working, the next step was to build the mobile app itself which would talk to the backend. I decided to use React Native to build my app because I have a lot of experience with React and I didn't want to have to write my app twice, once in Swift for iOS and once in Java/Kotlin for Android.

I also wanted to use Expo, because I had a great experience using it with my last React Native project. It's pretty magical to have hot-reloading on an app running on my actual iPhone and not needing to use XCode or the iOS Simulator ✨

I recently needed to eject my Expo app from the managed workflow to the bare workflow, which was an educational experience. You can read about my experience ejecting at the end of this post.

The app

The first screen you'll see when you open the app.

You sign in using a phone number and password. No email required.

On this tab, you can see your profile information. There's no way to edit it yet.

Here you can see all of the currently online users (including yourself). This is real and dynamically updates.

I have the minimum functionality working now. You'll notice that the app looks... rough. That's intentional. My strategy when working on a software engineering project is to do the hardest thing first. In this case, the part of the app I was most worried about was the live video-calling. The reason you should always do the hardest part first is because there's a chance that you might not be able to even to build it at all. Live video-calling is the central feature of my app, and if I can't figure out how to make that work and make it work well, then the whole app is dead on arrival. Imagine if I had spent months implementing every other feature of the app (photos, messaging, etc.), only to discover that I can't get the video part working and I would need to give up or make a significant pivot. All of that time and effort would've been wasted.

Once I have all of the core functionality working, I'll go back and style everything and make it look like a real app.

What you see in the screenshots above are the features that I felt were absolutely necessary to get working before tackling video. To reach that conclusion, I first imagined what the absolute minimal video-call experience could be.

The minimum video chat app I could imagine was an app that just displays a list of the currently online users and lets you call them. So, that means I would first need a way to display the list of online users and even before that I would need to set up user accounts and login. The reason you need user accounts and login is because the app has to somehow uniquely identify the person currently using the app. At one point I toyed with just asking users to provide a username every time they open the app, and then they would just "become" that user. Meaning that if you wanted to impersonate another user, you would just type in their public name when you open the app. This is obviously absurd from a security perspective, but could be acceptable for a prototype. I decided against it though because I was worried I would need to write extra code to accommodate this bizarre workflow that I'd end up throwing away later. So I decided to bite the bullet and set up the entire user authentication flow as the first step of building the app.

Setting up the authentication flow was pretty challenging because I had to set up all the inputs, transitions, and error-handling myself. Amplify actually offers a pre-built component that fully handles all of this, but I couldn't use it because it only supports email-based authentication, but I decided I wanted to use phone numbers for auth instead. Therefore I had to use the Amplify JS APIs instead.

Once I got the auth flow working, creating the two tabs with the online users and profile data was relatively easy. I'm using Apollo Client to interact with the Hasura GraphQL I set up at the beginning, so getting and showing data is straightforward.

To show the list of online users, I'm not using WebSockets or anything fancy like that. In my users table in my database I have a column called last_seen which is a timestamp. When you're signed in, the app automatically sends a POST request to the GraphQL API that updates that time with the current time. I made a custom hook to do handle this. To use the hook, I just call useKeepAlive() with no arguments inside of <ApolloProvider>.

Here's the code for my custom hook:

import { useEffect } from 'react';
import { useMutation, gql } from '@apollo/client';

const UPDATE_LAST_SEEN = gql`
  mutation UpdateBasicProfile($userId: uuid!, $now: timestamptz) {
    update_users(where: { id: { _eq: $userId } }, _set: { last_seen: $now }) {
      returning {
        last_seen
      }
    }
  }
`;

export default function useKeepAlive(user) {
  const now = new Date();
  const userId = user.username;
  const [keepAlive, { error, data }] = useMutation(UPDATE_LAST_SEEN, {
    variables: { userId, now }
  });

  useEffect(() => {
    const timer = setInterval(() => keepAlive(), 15000);
    return () => clearInterval(timer);
  });

  if (data) {
    console.log("I'm still alive");
  }

  if (error) {
    console.warn('Error updating last_seen:', error.message);
  }
}

Then in the Online Users tab, I query the list of all users filtered down to people who were last seen in the last 30 seconds. Originally I was going to set up a subscription to the users object for automatically refreshing the user list, but Apollo recommends avoiding subscriptions unless you really need the data to be extremely fresh, like if you're building an instant messaging app. For my purposes, I actually don't need the data re-fetched that often. I define a user as being online if they were online within the past 30 seconds, so I really don't need to re-check that information any more than every 15 seconds. Apollo conveniently exposes a parameter pollInterval that tells it to re-fetch the data according to whatever interval you specify.

I tested this feature by opening the app on my phone and on my computer using the iOS simulator. On my phone, I signed in using my actual phone number. On my computer, I signed in using a phone number generated by Google Voice. I had to use real phone numbers because the login flow texts you a 6-digit code to confirm you actually have access to that phone number every time you sign in.

Next up

I've now finished everything I need to support video calls. The way it will work is when you tap on one of the online users in the list, it will place a call to that user. That user will then see an incoming call within the app and they can either accept or decline the call. If they accept the call, it will open a full-screen video chat with the other user. I only have one phone to test with, so hopefully I'll be able to place a call from my phone to the iOS simulator or vice versa.

By the way, I plan on using Agora for the video calls. I evaluated several video-call vendors, and Agora was the only one that had affordable pricing, good React Native support, and good developer docs. If I'm not happy with Agora, my backup plan is to setup a WebRTC server with Amazon Kinesis Video Streams and do the WebRTC stuff by hand using react-native-webrtc.

Hopefully I'll have video calls up and running, and maybe more, when I publish my next update.

Bonus 1: Mini-review of Amplify in 2020

My updated perspective on Amplify is that it's gotten much better since 2019, but still feels like a beta product. I got stuck almost immediately because when you do the initial setup of your local environment you need to authenticate your app using a browser. I use Safari for all of my personal work, but apparently the authentication only works with Chrome and Firefox. If this was communicated in an error message, it wouldn't have been a problem but I had to dig through the Github issues to figure out why the authentication wasn't working. They also misspelled "communicate" with two "n"s in their unhelpful error message (see Github issue).

I also ran into an issue using Cognito that was significantly more challenging to resolve because I was using Amplify. When you set up auth through Amplify, it automatically generates two app clients, one with a secret key and one without. I didn't originally understand the difference between these two app clients, so I mistakenly deleted one of them. Cognito warns you that this action is irreversible, but I thought I knew what I was doing (I didn't). Once I realized my mistake, I decided that the simplest way to fix it would just be to start over. So I tried to delete the Cognito stuff through the Amplify web console, but the delete failed without any explanation. Eventually I realized the delete was failing because I first had to manually delete the domain I created in Cognito. I then manually deleted the CloudFormation stack created by Amplify, thinking I was making Amplify's job easier (I wasn't). If you're using Amplify don't try to manually delete something yourself. Always let Amplify delete its own resources. If you don't, you'll get stuck in a Catch-22 situation where it will become impossible to use your Amplify app.

After I deleted the CloudFormation stack, I noticed that Amplify was still showing that I had auth setup. Amplify must have its own state about your setup, so I figured that when I hit Delete it would just notice that the CloudFormation stack had already been deleted and it would just update its own state to reflect that auth had been removed.

Instead, when I tried to remove auth through Amplify, it complained that it couldn't find the stack to delete, so the operation failed. And if you try to add auth to the app it complains that you already have auth, so that fails too. And when I tried just deleting the whole Amplify app, that failed too because it tries to delete all of the sub-components, like the auth stuff, before deleting the top-level app. Basically, it can't delete the auth component because the auth component has already been deleted and it can't delete the app because it can't delete the auth component 🤦‍♂️

Eventually, I figured out that I could trick Amplify by creating a new CloudFormation stack with the exact name that it was looking for when it tried to delete the auth stack. I just made a minimal stack that created a random S3 bucket, but because the name matched the name it was expecting to exist, it happily carried out the delete operation and let me fully delete my Amplify app. I could've just abandoned my broken Amplify app and created a new one, but then I would've needed to create a new app with a new name and I know that I would constantly mix up the two apps every time I tried to use Amplify.

In conclusion, Amplify is still bad, but if you want to use Cognito or AppSync with a web or native app, you're still better off using Amplify than trying to implement everything yourself from scratch.

Also, if you're using Amplify Auth with React or React Native, I highly recommend reading this blog post by Sophia Brandt. She wrote a really good React Hook for updating your app state based on whether or not you're logged in. This is definitely something you wouldn't want to implement yourself, and should really be included in the core Amplify React library.

Bonus 2: Ejecting Expo

Expo offers two workflows: managed and bare. With the managed workflow, you can develop your app entirely with Javascript/Typescript and can mostly ignore platform-specific concerns with iOS and Android. This is the recommended workflow for most use-cases, but the drawback is that some React Native libraries either don't support it or don't document how to use it with Expo. The other limitation is that certain native APIs, like Bluetooth and WebRTC, are not supported in the managed workflow. If you start off using the managed workflow but realize later that you need to use the bare workflow (like me), then you'll need to "eject" your app to convert your app over to the bare workflow.

If you're starting a new project with Expo, I highly recommend reviewing the limitations of the managed workflow and deciding upfront which workflow you want to use, rather than ejecting later. Unfortunately, the ejection process isn't as smooth as I'd hoped and I ended up needing to spend a few days of debugging to get the app up and running again. I assume things would've worked much better if I had started off with the bare workflow rather than ejecting to it later.

That being said, most apps should be fine using the managed workflow. And if you can use the managed workflow, I highly recommend it over the bare workflow. The only reason I had to eject for this app is because I need live video calling, which means that I need to use WebRTC which is not supported in the managed workflow.

If you're wondering about the bugs I encountered when I ejected, I eventually determined that the source of all of my bugs was something called Flipper, which is a plugin for debugging mobile apps. I think I was having the same problem as this person, but they closed their own issue because it magically disappeared on its own. Unfortunately, my issue did not resolve itself, and none of the solutions I found online fixed it. Eventually, I was able to fix the problem by just deleting all of the Flipper-related code from AppDelegate.m. I guess that means I won't be able to use Flipper in my app, but at least my app builds again.

I also had to install a few extra things to get Amplify Auth working again. That was pretty easy though because it was obvious what I needed to do from the Amplify docs. If you compare the instructions for Expo and React Native, you'll notice that the React Native instructions includes an additional dependency to install, amazon-cognito-identity-js, and an additional step of running npx pod install afterwards. After I did that, everything in my app started working again.

However, I do still have one outstanding bug I haven't been able to fix, but I'll probably just live with it. When I try to run my app directly on my iPhone using the Expo server, the login step fails with the error getRandomBase.default is not a function. I figured out that this is coming from the react-native-get-random-values package, which is a dependency of aws-amplify. The only solutions I could find online to this problem are uninstalling and reinstalling aws-amplify or react-native-get-random-values or deleting my node_modules/ directory and re-running npm install. None of that fixed the problem, of course. This also happens to be one of those fun bugs that disappears when you run the app in debug-mode 🥲

The silver-lining is that this issue only manifests when I run the app using Expo, not when I run it with the iOS simulator, which implies to me that it's not a "real" bug and would never be encountered by a real user. There's also a pretty simple workaround: run the app in debug mode, log in, then disable debug mode. Once you're logged in, you stay logged in, so it's not like I need to do this every time I run the app. That being said, I don't think I would've hit this issue if I had started off using the bare Expo workflow, so this is yet another reason to avoid ejecting if you can.

Dating App: Alpha Preview

2021-01-26T02:00:00Z

Sorry for the extended hiatus. I haven't had time to write much on here recently because I've been spending all of my free time working on my dating app. I also took a break to take a crash course in how to get better at using dating apps, but that's a story for another day 😉

Since my last update almost a month ago, I've actually made a lot of progress. As of this past weekend, my app is officially up on TestFlight and is being tested on real iOS devices by my friends and family!

Here's a preview of all of the features I've implemented so far. There's not much yet, but the standout feature is the video calling which is real (I'm testing it with two iPhones).

https://youtu.be/Rfri4qkrrqY

Obviously there's still a long way to go before it's ready for real use. The main things it's missing right now are:

A fresh coat of paint. It looks pretty rough right now. Profile photos would be nice too.
Preferences and filtering by matching users. Right now you see all online users.
Messaging. You'll need some way to follow up with people to build the connection.
Time restrictions. Since the first step is calling other users (who need to be online), I'll need to force everyone to use the app at the same time.

But before I do any of that, I decided that I'm going to go back and write automated tests for all of the current functionality. This is to ensure I don't break existing functionality when I'm adding new features or refactoring the code. It's a pretty time consuming endeavor though – I spent about 8 hours yesterday writing tests just for signup/login! But in the long run having the tests will actually save me time because I'll be able to make changes with confidence and without needing to manually test that I didn't break anything.

By the way, I'm using Jest and React Native Testing Library for my tests. It worked great for testing the login code, but I ran into issues when I tried to integrate React Navigation into my tests. If you have any tips, let me know in the comments!

Also, if you'd like to be an alpha tester for my app, let me know! The easiest way to let me know is by giving me your email address using the form below. I promise I won't use it for anything not related to the dating app.

Fixing npm install on an M1 Mac

2021-11-20T13:20:00Z

I just bought a new Macbook with an M1 chip to work on my dating app. I got the basic Macbook Air but with 16 GB of RAM if you're wondering. I heard that there were some issues with software compatibility with software built for Intel-based Macs, but I figured all of that would have been smoothed out by now given that these Macs have been out for a full year. However, when I re-cloned the repo for this blog and ran npm install I kept getting errors. I'll tell you how I fixed it so you know what to do if this happens to you.

Before you get started, I recommend you install three things on your new Mac that should make things easier:

The Xcode developer tools. Run xcode-select --install.
Homebrew for installing command line utilities on Mac.
nvm for installing node, npm, and switching node versions.

Once you have that installed you should use nvm to install some version of node. I installed the latest version (17) which ended up working for me, but you may need a different version for your project. Now, cd into your project folder and run npm install. If it works, great! You can stop reading now 😄

Ok, so npm install didn't work. Now what? First I'll tell you the last thing I did, because I suspect that if I had just done that I could have skipped all the other steps.

When I was trying to debug this I was confused because when I would Google the errors I was seeing I would find it mentioned in a closed Github Issue that said the error was fixed in a new version of the package. In retrospect it seems obvious, but I was able to make all of my npm install errors go away by upgrading all of my packages to their latest version.

Here's an example of an error I was getting that went away after updating my packages:

npm ERR! code 1
npm ERR! path /Users/aj/Development/blog/node_modules/esbuild
npm ERR! command failed
npm ERR! command sh -c node install.js
npm ERR! Unsupported platform: darwin arm64 LE

Rather than figure out the latest version for each package and upgrading them one-by-one I wanted to upgrade everything to their latest version all at once.

My understanding is that this is what the npm update command does, so that might work for you, but I used a third party package called npm-check-updates instead. You can install it with npm install -g npm-check-updates (luckily npm install worked for this package 😅) and run it by just running ncu in the folder where your package.json file is or ncu -u to actually modify your package.json file.

Once my package.json file was updated with the latest version for each package I ran npm install again and everything installed without errors. In my case, I didn't even need to change any of my code – everything just worked.

So hopefully that's all you need to do, but I actually did do one other thing before I updated the packages, so I'll mention it in case you run into the same error I did even after updating your packages.

Here's the error I was getting:

npm ERR! code 1
npm ERR! path /Users/aj/Development/blog/node_modules/sharp
npm ERR! command failed
npm ERR! command sh -c (node install/libvips && node install/dll-copy && prebuild-install --runtime=napi) || (node-gyp rebuild && node install/dll-copy)
npm ERR! CC(target) Release/obj.target/nothing/../node-addon-api/nothing.o
npm ERR!   LIBTOOL-STATIC Release/nothing.a
npm ERR!   TOUCH Release/obj.target/libvips-cpp.stamp
npm ERR!   CXX(target) Release/obj.target/sharp/src/common.o
npm ERR! info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.9.1/libvips-8.9.1-darwin-arm64v8.tar.gz
npm ERR! ERR! sharp Prebuilt libvips 8.9.1 binaries are not yet available for darwin-arm64v8
npm ERR! info sharp Attempting to build from source via node-gyp but this may fail due to the above error
npm ERR! info sharp Please see https://sharp.pixelplumbing.com/install for required dependencies
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@8.2.0
npm ERR! gyp info using node@16.13.0 | darwin | arm64
npm ERR! gyp info find Python using Python version 3.8.9 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
npm ERR! gyp info spawn /Library/Developer/CommandLineTools/usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/Users/aj/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/aj/Development/blog/node_modules/sharp/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/aj/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/Users/aj/Library/Caches/node-gyp/16.13.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/Users/aj/Library/Caches/node-gyp/16.13.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/Users/aj/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/Users/aj/Library/Caches/node-gyp/16.13.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/Users/aj/Development/blog/node_modules/sharp',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! warning: /Library/Developer/CommandLineTools/usr/bin/libtool: archive library: Release/nothing.a the table of contents is empty (no object file members in the library define global symbols)
npm ERR! ../src/common.cc:23:10: fatal error: 'vips/vips8' file not found
npm ERR! #include <vips/vips8>
npm ERR!          ^~~~~~~~~~~~
npm ERR! 1 error generated.
npm ERR! make: *** [Release/obj.target/sharp/src/common.o] Error 1
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/Users/aj/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
npm ERR! gyp ERR! System Darwin 21.1.0
npm ERR! gyp ERR! command "/Users/aj/.nvm/versions/node/v16.13.0/bin/node" "/Users/aj/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /Users/aj/Development/blog/node_modules/sharp
npm ERR! gyp ERR! node -v v16.13.0
npm ERR! gyp ERR! node-gyp -v v8.2.0
npm ERR! gyp ERR! not ok

It's a little hard to parse this error message, but basically npm can't install a package called sharp which I believe is a low-level utility package. And the reason the install is failing is because "Prebuilt libvips 8.9.1 binaries are not yet available for darwin-arm64v8" and "fatal error: 'vips/vips8' file not found". When I Googled this error I found a lot of different solutions, but the one that worked for me was running brew install vips, then re-running npm install. Once I did that this error went away, but then I got the esbuild error I first mentioned. I suspect that if I had updated my packages from the get-go I wouldn't have needed to install vips at all.

If you're trying to run npm install on an M1 Mac and updating the packages doesn't fix it or you're getting different errors, I probably won't be of much help. I recommend Googling the errors you're getting to see if anyone else ran into that same error and was able to find a solution.

Hope this helped and good luck!

[1]:

How to fall asleep

2021-11-25T13:40:00Z

NOTE: I'm using Amazon Affiliate links for some of the product recommendations in this post, which means that if you buy something using that link I might get money. But everything I recommend are actual products that I own and use and no one paid me or gave me anything to recommend these specific products.

The title of this post isn't a clever metaphor – this is just my advice on how to fall asleep. I'm not a doctor or an expert on sleep. I'm just a person who's struggled to fall asleep for most of my life and only figured out how to reliably fall asleep at a reasonable hour relatively recently (in the last few years). For most of my life I would lie awake for hours in bed unable to fall asleep. But now that I've discovered this system, I consistently fall asleep within 15 minutes of when my head hits the pillow. And if I go to bed when I'm supposed to I often wake up naturally before my alarm goes off. I've always thought of myself as a night-person, but I've trained myself to become a morning person.

Also, if your issue is staying asleep, this post isn't going to help you. I've never had a problem staying asleep, so you'll need to look elsewhere for advice on that.

Rules for sleeping

Later on I'll explain the actual process I follow to fall asleep, but first I want to cover some general rules I follow for good sleep hygiene.

Only go to bed when you're tired

This is a tip I got from reading other lists online on tricks for sleeping. The rationale is that you want to classically condition your brain to associate your bed with sleeping. Lying in bed not sleeping messes with that association, so you should always wait until you're actually tired before getting in bed.

This advice sounds obvious to me when I say it, but it's actually not what I did for the majority of my life. When I was a kid I went to bed when my parents told me it was bed time. I carried this idea of "going to bed at bed time" into adulthood but it just doesn't work. I used to think that if I just went to bed at a reasonable hour I would get enough sleep, but you'll get more hours of sleep if you wait until you get tired first then go to bed.

Don't bring your phone to bed

This is another common tip you'll see online. People say that the light from your phone's screen will confuse your brain into think it's daytime or something. They usually blame the blue light from your screen, which has led to phones adding modes like Night Shift that filter out blue light and give your display a yellowish hue. I can't say for sure whether or not those night modes make a difference, but I know that I have trouble falling asleep if I'm on my phone in bed even with Night Shift on.

I used to bring my phone to bed with me and play on my phone (read news articles, etc.) until I got tired and fell asleep. But this usually took 2+ hours, which meant significantly fewer hours of sleep each night.

I know a lot of people use their phone as an alarm, so keeping your phone out of the bedroom may not be an option. As a compromise, at least make sure your phone is out of reach from your bed. I actually use my Apple Watch as my alarm, which I'll discuss later.

At the end of the day though (pun?), what really matters is just that you're not using your phone in bed or just before bed. I personally don't have the discipline to not use my phone if it's near me, but maybe that's not a problem for you.

Stick to a consistent sleep schedule

If you're reading this article, you've probably heard of circadian rhythms. I'm not going to offer a technical definition of what that means, but the basic idea is that your body likes to follow natural cycles for when to do things like sleeping.

The important thing is just that you try to go to bed at the same time every night and wake up at the same time every morning.

(Optional) Avoid caffeine

I'm very sensitive to caffeine and if I have anything with caffeine in it after 10am, there's a good chance I'll have a hard time sleeping that night. This is abnormal though and most people I know drink coffee every day (even late in the day) and have no problem falling asleep. But if you're having trouble falling asleep, it might be because of caffeine or having caffeine too late in the day.

Personally I don't have any caffeine at all unless I'm really, really tired or hungover on a work day. If you are tired during the day, I've found that a 20 minute nap has the same effect as a cup of coffee without interfering with my sleep that night (assuming taking a nap is an option for you).

How I fall asleep

Step 0: Creating a sleep schedule and setting up my alarm

I work Monday-Friday from 9-5, but I work out before work three days a week. So on days when I work out I need to wake up at 6:30am. However, I sleep better when I stick to a consistent sleep schedule so I try to wake up at 6:30 even on weekdays when I'm not working out. I've found that I need 8 hours of sleep to feel well rested the next day, which means that I need to be asleep (asleep, not just in bed) by 10:30pm. It takes me about 15 minutes to fall asleep so my goal is to be in bed by 10pm (but I don't go to bed until I'm tired, see above). On weekends I'm usually going out and staying up much later (I go to bed between 12-3am) so I let myself sleep in on Saturday and Sunday.

Based on all this, the sleep schedule I've come up with is: On M-F start preparing to go to sleep at 9pm, go to bed at 10pm, alarm goes off at 6:30am, snooze 0-2 times, wake up and get out of bed. On weekends I go to bed whenever I want and don't set an alarm.

You might think there wouldn't be much to say about setting an alarm, but I actually have a pretty complicated alarm system configured on my iPhone. Since iOS 14 iPhones have had a feature called Sleep that includes an alarm for waking you up, a reminder when it's time to go to sleep, and sleep tracking through your Apple Watch.

You can read how to set it up here.

This is the sleep schedule I have configured through the Health app:

My Sleep Goal is 8 hrs and I have Wind Down set to 1 hr, which will remind me to go to sleep at 9pm every week night. I also have it set up so all notifications on my phone are silenced while my phone is in Sleep mode, except for Slack notifications which are important for my job.

Another feature I love in Sleep is the ability to skip my alarm for one day. In the old days if I wanted to sleep in the next day I would turn off my alarm altogether, but then inevitably I'd forget to turn it back on later and would be late for work. With Sleep you can either skip your alarm for the next day, or set it to wake you up at a different time the next day only. It will automatically go back to its original time the following day.

Step 1: Take a melatonin one hour before I want to go to sleep

My goal is to go to sleep at 10pm, so I take a melatonin pill every weeknight at 9pm. It takes about an hour to kick in, so a little earlier than 9pm is fine for me, but if I don't take it until 9:30 I probably won't be sleepy until 10:30. I use the Wind Down notification on my iPhone as my reminder that it's time to take a melatonin pill.

I'm not a doctor, so I can't tell you what dose you should take, but I think it's common sense that you should take the lowest dose that's effective for you. For me, that's 3 mg but you may be fine with a lower dose or need a higher dose. I recommend starting with a low dose and only upping the dose if necessary. I've found that with higher doses I'll have weird dreams, which is apparently a side-effect of melatonin for some people. I don't have any weird dreams when I take 3 mg though.

I don't know if there's any difference between different brands of melatonin, but this is what I buy.

Step 2: Watch anime until I get sleepy

Sometimes (read: often), I'm already watching anime before 9pm, but regardless of what I'm doing I always start/continue watching anime at 9pm. Starting at 9pm I'll binge watch whatever anime TV show I'm watching at the moment until I start feeling sleepy. Each episode is ~20 minutes, so it usually works out to 2-4 episodes. When I feel my eyelids getting heavy, I notice I'm drifting in and out of sleep, or I'm pausing the episode to figure out how much time is left in the episode, it's time to go to bed. If I go to bed after I've reached this state I can reliably fall asleep within 15 minutes.

Ok, so you're probably wondering why anime. Or if there's something special about anime that induces sleep. No, I don't think there's anything special about anime. The goal of this step is to relax and stop thinking. If I was good at meditation that would probably work even better. But I'm not, so I use anime to get me into a pseudo-meditative state.

If meditation works for you, great, do that. But if you're not good at meditation and don't want to learn, I think there are other ways to relax and stop thinking somewhat. Other types of TV shows may work better for you, so just choose whatever you think will work best for you.

The reason why anime works better for me than other TV genres is because (don't hurt me 😳) it's repetitive, predictable, and has enough padding that you're not missing much if you close your eyes for a few minutes. Don't get me wrong, I love anime, but even the most action-packed animes usually have a ton of slow parts. I imagine binge watching reality-TV would work well too, but I'm not enough of a masochist to binge watch reality-TV. Cool, I think I've offended everyone now 😌

Step 3: Using a white noise generator

This step is the most optional, but I find it helpful. Over the summer I use a desk fan pointed at my bed to stay cool (see next step), and when it got colder and I stopped using the fan I found it was harder for me to fall asleep. At first I assumed it was a temperature thing, but then I thought maybe the constant noise of the fan was helping me fall asleep.

I bought a white noise machine on Amazon and thought I would try it to see if it made a difference. This is the one I bought. That's what The Wirecutter was recommending when I bought it, but now they recommend a newer model that's a bit cheaper, so maybe get that one instead.

At first using it felt kind of silly, and I found the noise a little annoying to be honest. But after using it for a week I got used to it. Now I use it every night it's hard for me to sleep without it. I'm not sure why background noise makes it easier to sleep, but it seems to work. Also, I think now I'm classically conditioned to associate the sound with sleeping, so it now acts a cue for my brain telling me to sleep.

One critique you may hear about white noise machines is that although they may help you fall asleep, the constant noise throughout the night actually lowers the overall quality of your sleep. I don't know if this is legit or not, but the white noise machine I bought has a setting that makes the noise automatically fade out after one hour. If you get the same one I bought, you just press the top of the leftmost button to turn it on, then press the bottom of that same button to set it to fade out and turn off after one hour.

Step 4: Getting comfortable

If you're not comfortable, it's going to be harder to fall asleep and stay asleep. I have some more tips on this in the next section, but I think the most important and hardest thing is getting your body to the right temperature. If any part of your body is too hot or too cold, sleeping can be a challenge.

Everyone's needs in this regard is unique so I'll just tell you what works for me, because it's kind of weird. For me to be comfortable I need to have a blanket or sheet covering my torso. But if I just put a full blanket over my whole body I get too hot and I can't sleep. So I'll either fold up the bottom corner of the blanket so my legs are exposed or rotate the whole blanket by 30° so my torso is covered but my legs are exposed. This seems like as good as time as any to mention that I currently don't share my bed with anyone, which, if you do, can definitely complicate some of these things 😅

I also only sleep in boxers (unless it's really cold), and I hate when the sheets are tucked under the mattress. I'm a back sleeper, so the tucked in sheets put too much pressure on my feet.

Anyway, that's a lot of random information that's highly specific to me and probably not what you should be doing. The point I'm trying to get across is that everyone's needs are different and may be very specific. Just figure out what's comfortable for you, don't be afraid to experiment, and don't worry if what's most comfortable for you sounds weird to other people.

Other stuff I find helpful for sleeping

A good mattress

I have the Saatva Classic. I'm happy with it, but I don't have strong opinions on mattresses. You can see what The Wirecutter recommends here.

A good pillow

I went through a phase where I tried a bunch of different pillows. Someday maybe I'll write a blog post on all the pillows I tried. Until then, I'll just say that I'm a back-sleeper and I'm really like the Ecosa Pillow. More recommendations from The Wirecutter here.

Earplugs

I live in a city and it can get pretty loud in the morning with trucks and construction. I don't sleep with earplugs in, but I do keep a pair of earplugs on my nightstand that I can pop-in on particularly noisy mornings.

A water bottle next to my bed

I have trouble breathing through my nose, so I breathe through my mouth when I sleep. Because my mouth is open most of the night it dries out and I get thirsty. Obviously any water bottle will do, but I prefer a bottle that can hold a lot of water so it last through the night and one with a sport top so I don't have to fight with a cap when I'm half asleep. Recently I splurged on a 32 oz Hydro Flask with the Straw Lid and a matching Bottle Boot which protects my nightstand if I put my bottle down too hard. $50 is a lot to spend on a water bottle, but I use it a lot so it's worth it for me.

Final Thoughts

Sleep is really important. Something I picked up on recently is that on days when I feel depressed, I'm usually just tired. Since I started sleeping 8 hours a day I've had a lot more energy and have been a lot more positive and upbeat. I hope this guide was helpful. Good luck!

How to actually build the metaverse

2021-11-28T05:15:00Z

What is the metaverse and why are we talking about it?

If you follow technology news you may have been hearing the word "metaverse" a lot recently. It feels like every day I see a headline on VentureBeat along the lines of "Company X raises $Y million to create the metaverse" (Example 1, Example 2). Or maybe you heard that Facebook recently renamed their company to Meta to better align with their vision of creating the metaverse.

But what actually is the metaverse? First I'll give my definition which is a bit different from what you're likely to read on a lot of sites that are pushing the metaverse as the next big thing.

The metaverse is:

A 3D virtual world (like you'd find in most video games)
All "spaces" in the world are connected (CokeLand, PepsiLand, StarWarsLand, StarTrekLand all exist in a single, connected world)
Anyone can contribute new stuff to this shared world. They can make anything they can imagine, as long as what they make doesn't break any laws.
There's a single, persistent "reality" for the whole metaverse. The metaverse is the same for everyone who interacts with it (like you'd find in an MMORPG like World of Warcraft).

My definition is based on the description of the metaverse in the sci-fi novels Snow Crash (1992) and Ready Player One (2011). I don't think Snow Crash invented the idea of the metaverse, but these books definitely did a lot to popularize the idea.

Something I didn't include in my definition is anything about Virtual Reality (VR) or Augmented Reality (AR). Most visions of the metaverse assume that people would interact with it using a VR or AR headset (both books I mentioned use highly immersive VR headsets). But I don't think VR is necessary for the metaverse or the most interesting thing about it. Don't get me wrong – realistic, immersive VR would be awesome – but I view building better VR as a separate technology challenge from building the metaverse. Better VR technology would definitely make the metaverse more fun, but I think you could still build the metaverse without it. Also, even if the primary way people interact with the metaverse is through VR goggles, I'd expect you to still be able to experience that same world in 2D on a computer monitor and navigate it using a standard mouse and keyboard.

Does the metaverse already exist?

Based on my definition of the metaverse, you might be able think of some things that already exist that seem like they fit that definition. However, I would argue that no one has built the metaverse yet (but I don't think it's far off).

I'll go through some candidates that either come close to being the metaverse or they claim they're building/built the metaverse.

Second Life

Second Life is an online video game that came out in 2003. Of all the candidates I'll talk about, I think Second Life is the closest actually being the metaverse and mostly meets all of the criteria I gave in my definition. But I think they don't qualify based on a technicality. Throughout this post I've been using the phrase "the metaverse", which implies that there's only one metaverse. But I don't think Second Life is nearly popular enough for people to think of it as the metaverse. The record for most players logged in at one time is 88,200 (source). Second Life would need to have hundreds of millions of users for me to think of it as the metaverse.

The Internet

I've never thought of the internet as being the same as the metaverse, but some people think that it meets the definition of metaverse. Like this blog post titled "The Metaverse already exists, it's just currently called the Internet".

But the internet isn't a 3D virtual world. It does meet the other parts of my definition though.

Fortnite

Tim Sweeney, the CEO of Epic Games, likes to say that Epic Games is building the metaverse and the most visible example of this aspiration is their hit game Fortnite. Fortnite is an actual video game (unlike Second Life) but it has become more of a social hub and they've hosted virtual events inside the world like live concerts with Marshmello and Ariana Grande. They also do lots of collaborations with lots of brands and has featured characters like Thanos from Marvel's Avengers, Batman from DC Comics, characters from Star Wars, and characters from other video games like Master Chief from Halo (source).

I don't know if Tim Sweeney has actually ever claimed that Fortnite is the metaverse, but it's definitely metaversey in some ways. It fails my definition though because it doesn't have user generated, there's not a single world that all players occupy together, and it's a bit too niche and specialized right now as a video game targeted at kids.

Meta (Facebook)

Mark Zuckerberg has said that his company Meta (formerly known as Facebook) is building the metaverse. I'm not a big fan of Mark Zuckerberg, so I haven't looked into the details of their plan. They definitely could build the metaverse, but I hope their plan is to contribute to its creation and not try to own the metaverse. A single entity owning the metaverse would not be ideal, but it's definitely a possibility and is what happened in Ready Player One. I guess we'll see.

What's so great about the metaverse anyway?

As I allude to in my definition of the metaverse, a lot of things that already exist that fulfill a lot of the good things about the metaverse. We already have virtual worlds – tons of them actually. Download just about any video game from Steam and boom, virtual world right at your fingertips. A lot of games even let you explore these worlds in VR if you have a VR headset. You can even create your own experiences in some games and share them with other players, like Dreams for PS4. So really, the only advantage of the metaverse is that there's a single, shared metaverse and all of these experiences are connected.

I'll give an example to illustrate the difference. Imagine you're playing GTA V and you want to switch to Fortnite. Right now the way you'd do that is you'd quit out of GTA V then go to your Steam library, find Fortnite and click the start button and load into that. But if GTA V and Fortnite were just different "spaces" in the metaverse, you could go directly from GTA V-land to Fortnite-land by walking, flying, teleporting, etc. without ever leaving the metaverse. Also, your avatar and control settings wouldn't change when you moved from one space to the other (maybe your clothes or art style would change).

That would be pretty cool, right? But is switching games the way we do now that big of a deal? I think it is. I'll draw a comparison to how software worked before and after the internet. The internet is older than me, but I can imagine skeptical people making similar critiques of the internet being overrated. Before the internet, to switch software you would need to eject a disk and put in a new disk to use other software. Or at least switch to a new window for the new app on your computer. Changing programs probably didn't seem like a big problem for people at the time, but could you imagine if every time you wanted to switch websites you had to quit out of a program and switch to a new program? Like you're watching videos on YouTube but want to read something on Wikipedia so you need to close the YouTube app and switch to the Wikipedia app. As soon as I wrote this I realized that this is exactly how mobile apps work, and it's not that big of a deal, haha. I think the bigger thing you'd miss without the internet is the lack of links. Imagine if Google didn't directly link you to websites but instead just gave you the address of the website you want to go to, which you would copy to your clipboard and paste into a new app. That would be really annoying.

I think the metaverse is similar in this regard. It may not sound that special, but once you got used to it, going back to the old way of navigating across virtual spaces would feel tedious.

What's Web3 and what does it have to do with the metaverse?

As I said in the subtitle, I think Web3 is BS so I'm probably not the best person to explain what it is. Most definitions of Web3 sound more like religious prophecies than actual explanations, so my definition might sound kind of boring in comparison. If you would rather read about Web3 from people who are actually excited about it, check out this online compendium of articles on Web3 and related concepts.

Before we get to Web3, let's talk about Web 1.0 and Web 2.0 first. Obviously no one called it Web 1.0 at the time, but you can think of Web 1.0 as static websites that don't offer any means of accepting user input (besides scrolling). If you ignore the comments section, this blog is a perfect example of a Web 1.0 website – I write stuff and you read it. Web 2.0 websites allow the user to contribute things back to the website. Think leaving a comment on a site, editing a Wikipedia article, tweeting on Twitter, uploading a photo to Instagram, etc. So in Web 1.0 information and content only flowed in one direction (website → user), but in Web 2.0 information and content can flow in both directions (website ↔ user).

Web3 (they don't call it Web 3.0 because that doesn't sound cool) is supposedly the next evolution of the internet from Web 2.0. There are two innovations promised by Web3: decentralized ownership of websites and content and the making payments a first-class feature of the web. For example, YouTube (i.e., Alphabet) controls all of the videos on YouTube, even the videos that you uploaded. If they decide that they don't like your video, they can remove it from the website and the only way to get it back is by appealing to them. With a Web3 version of YouTube, you alone own and control the content you create and there's no central authority like YouTube that can take down your video. Regarding the payments thing, right now if you want to buy something online you need to use PayPal or a credit card processing system. But with Web3 payments is a built-in feature in your browser and you can pay for stuff directly without relying on outside tools. You're paying with cryptocurrencies like Bitcoin though, not US dollars. There's a lot of overlap between people excited about Web3 and people excited about cryptocurrencies because Web3 is built on top of the same blockchain tech used for cryptocurrencies.

Would these new features make the web significantly better? I don't think so, but I guess I could see it being helpful for certain niche types of websites. I feel like the web is already decentralized enough as it is and won't benefit from greater decentralization. But maybe in 10 years naysayers like me will sound like fools, just like the naysayers of the internet and computers before me. I guess we'll see.

So... what does any of this have to do with the metaverse?

Right, sorry, got a little off-track there. If you ask me, Web3 is all hype and has absolutely nothing to do with the metaverse. I think the metaverse is a genuinely exciting concept, but I really don't see how Web3 makes it better. But in a lot of corners of the internet (read through that compendium I linked to earlier for examples) Web3 is essential for creating the metaverse and any metaverse that isn't built using Web3 technologies is no metaverse at all.

I strongly disagree with this viewpoint, and my annoyance with this viewpoint is what inspired me to write this blog post. My opinion is that we could pretty much build the metaverse right now without needing any new ideas like Web3, blockchain, etc. There are a few innovations we'd need to make, but it's basically just improvements on well established technologies.

How to actually build the metaverse

There's more than one way you could build the metaverse, so this is actually how I think we should build the metaverse. If someone gave me a billion dollars to go build the metaverse, this is how I would do it.

Basically, I would leverage as much of the current internet infrastructure as possible. So the internet is not the metaverse, nor would the metaverse replace the internet – the metaverse should be built on top of the internet.

Virtual spaces are just websites

Here's a simplified version of what happens when you go to a website on your computer: you type the URL into your browser, hit enter, your browser uses a thing called DNS to map that URL to an IP address of another computer (server) somewhere in the world, your browser says "Give me the website!", the server says "You got it!", then sends you the HTML file for that website which your browser renders on your screen for you.

Now here's the metaverse version of that story: everything is exactly the same except for when the server sends you an HTML file. Instead of sending you an HTML file, which is a mostly human-readable text format, the server would send you a METABLOB file (not an actual thing, I just made that up) which would be used by your browser to render a virtual world instead of a 2D webpage. Your browser would work exactly the same way it does now. The only new thing it needs to do is learn how to render a METABLOB file on your computer so you can actually explore and interact with the virtual world that it encodes.

The other part that might be a little different is if you're using a VR headset to experience the metaverse, you wouldn't want to take the headset off to type a URL into your browser's address bar. Instead you would type it in on a virtual keyboard displayed inside the virtual world, type it in on a virtual keyboard located on your wrist, or maybe say the name of the website you want to go to in your microphone. Alternatively, if the different spaces in the metaverse exist on a cartesian plane, maybe you could enter in the XY coordinates of where you want to go or select the website from a 2D map.

Those are all just UX options though – at the end of the day, the website or address is still going to be converted into an IP address of a another computer on the internet (server) that will send a file back to your browser/VR device.

To make this idea more vivid I'll talk through a specific example. Right now if you go to disney.com your browser will bring you to the Disney website. But in the metaverse, you would go to disney.meta in your "browser" and instead of getting the Disney website when you go there you would get whatever virtual world Disney decides to build. Let's call their slice of the metaverse DisneyLand. DisneyLand would probably be a lot like the actual Disney Land and Disney World – you'd be able to walk around a virtual park, interact with different characters, go on rides, play games, buy physical (and virtual) stuff with real money. The difference between DisneyLand and Disney Land is that until VR tech gets way better, this experience will not be as immersive as actually going to Disney Land in real life. Also, since it's basically a video game, the Disney Imagineers really are only limited by their imagination in terms of what they can build and it's a lot cheaper to build new stuff in the virtual world so things would probably change more often. Also, since it's a virtual world, if they want to expand the "park" they only need to buy more processing power not more land.

And if you get bored of DisneyLand, without leaving the metaverse, you can head directly to UniversalLand by typing/speaking "universal.meta" in your browser/VR device.

But remembering the address of every website in the metaverse would be a pain. You should also be able to walk from website to website on your own two virtual feet.

Virtual spaces are connected with portals, which are links

So walking around within a single virtual space (website) is no problem because the METABLOB file you downloaded contains a whole virtual world (like a video game) that you can walk around in. But that METABLOB file only contains the code for that one world. All of these virtual spaces are supposed to be connected into a single cohesive metaverse.

I can think of two ways this could work. The first, which would be way cooler, is if all of these mini virtual worlds gave the illusion of existing in a single shared space. For example, if you climb to the top of the virtual Cinderella Castle in DisneyLand you could see the T-Rex from Jurassic Park off in the distance at UniversalLand. And in the other direction you could see Taylor Swift performing over in SwiftLand (taylorswift.meta). This would be awesome, but would require you to download (stream actually) the entire metaverse, or intelligently download whatever part of the metaverse you're looking at or may look at soon.

My other idea for connecting these virtual spaces (websites) builds on top of another familiar web technology: links. Links are how we currently connect websites on the web, so all we need to do is translate the idea of a link into an equivalent that works for the virtual world. My proposal is to use portals. These portals would appear as 2D "paintings", that either hover in space or are mounted on a virtual wall and you could jump/walk through them to navigate to a new virtual space (website). Yes, I stole this idea from Super Mario 64. The painting would be a still image that gives you a sneak peek of what to expect on the other side and would also have a placard containing the full URL of where the portal/painting will take you. If you want to see a live version of what's on the other side of the painting before committing to walking through it, you can knock on the painting (or tap it) which would transform the painting into an actual window into that other world with parallax effects and everything (like an actual portal). Under the hood, that knock/tap would download the part of that virtual space that would be visible from the other side of the portal. This interaction is analogous to a long-press on a link in iOS Safari where it pops open a preview of the site.

I'll give a couple of concrete examples of how this would work in practice. Imagine you're in the market for a new blender. Being the responsible shopper that you are, you go directly to thewirecutter.meta for reviews and recommendations on the best blender. Once inside you walk over to their blender showroom where they have the best blenders on display. Inside the showroom is an AI avatar that tells you some canned info about the blenders (written by The Wirecutter's review team) and maybe you can even test out the virtual blender with some virtual carrots or something. You like what you see and hear, so you're ready to buy. But The Wirecutter is just a review site – they don't actually sell stuff. You've got to go to Amazon to actually buy the blender. On the wall behind the blender is a painting that will take you to amazon.meta to buy the blender. You walk through the painting and it takes you directly to where that blender is located inside Amazon's gigantic virtual warehouse containing every product they sell. Picture the inside of a Walmart, but it's the size of the sun. Now that you're in AmazonLand you're looking at the blender on the shelf, and it's next to some other blenders (More Products Like This, Sponsored Products, Other Customers Also Viewed, etc.). There's also a book hanging under it that you can flip through with user reviews. Once you're ready to buy it, you hit the big Buy button on the shelf. If you're already signed in, you just confirm the shipping address and then your blender will be mailed to your house in 2 days.

I also wanted to explain how Google would work in the metaverse, but the first example was pretty long. But basically Google would look like an art gallery. Instead of typing your search into a search box you would say something into a microphone. Then all of the paintings on the wall would update to be the top 25 most relevant websites for what you searched for. If you want to see the page 2 results, you can walk over to the next room which has 25 more paintings in it. When a painting catches your eye (seems like the website you want to go to), you just step through it.

Interacting with other people in the metaverse

In most depictions of the metaverse it's like a big video game where everyone has an avatar and you can interact with other people's avatars. Which sounds cool, but I'm not sure if it's something you would always want. If you've been to Disney World or Disney Land, haven't you thought that it would be more fun if there weren't quite as many people or you never needed to wait in line? Do you really want to share disney.meta with all of the other people currently visiting disney.meta? Because if there really is one shared reality, that means that there's only one Space Mountain and you actually would need to wait in line to ride it.

This is sort of a cop-out answer though, because my original definition did say that a consistent, shared reality was a requirement for a true metaverse. It definitely should be an option, but it would be nice to be able to turn this off and experience the virtual world by yourself or just with your friends.

Building this functionality is admittedly a gap in my explanation of how to build the metaverse, but it still could be done using current web technologies. Just not very elegantly. The way I would build this functionality is by putting the responsibility for rendering other people's avatars on the individual website. The website knows who's currently using the site, so they could just use websockets to stream the location and actions of other people/avatars/players and the shared state of the world to you in real time. But every website would need to figure out how to implement this, which might lead to the functionality not being supported on all sites or varying degrees of quality in the implementation.

Your browser creates your avatar

In the beginning of this post I mentioned that when you travel across different virtual spaces in the metaverse your avatar and how you control your avatar should stay the same.

This would be possible because your personal browser is responsible for the creation of your avatar, rendering your avatar inside of the metaverse, and allowing you to move and control your avatar.

If you're interacting with the metaverse through VR, you probably aren't going to be looking at your avatar much anyway (besides your hands). But if you walk past a virtual mirror, you would expect to see your custom avatar staring back at you. To accomplish this, the METABLOB file returned from the website to your browser would just have code for a mirror object and your browser would handle rendering your avatar correctly.

Your browser would also send information about your avatar as part of any request to the server it makes. Your browser already does this with the User-Agent request header, which includes information about your browser. Same idea, but just with a much longer string that fully describes what your avatar looks like. This is important if the website is going to display your avatar to other users. Your browser may know what you look like, but it doesn't know what other users visiting the site look like unless that information has already been shared with the server.

User-generated content

There are two ways you could generate your own content in the metaverse: by creating content inside of a single virtual space using tools provided by that website, or by making your own totally custom virtual space.

The first option would be the more user-friendly of the two and would probably be much more common. For example, in DisneyLand (disney.meta) they could have a workshop where you build your own virtual lightsaber. The tools you use to create your lightsaber would be very simple and would require no coding knowledge. You would either use a virtual menu to select your hilt and kyber crystal or would walk around a virtual workshop and pick up a virtual kyber crystal and slot it into the hilt of your choice. The details aren't important – the point is that the experience of creating this piece of custom content would be a completely custom thing built by Disney. If you want to carry your lightsaber around with you throughout the metaverse, not just in DisneyLand, you could "install" the lightsaber into your browser's custom avatar. This is analogous to installing a third-party Chrome extension. You would just tell disney.meta what browser you're using (or they would detect it), and they would have a special version of the lightsaber that's compatible with your browser so it can be added to your avatar.

Another type of custom content you could build in this way is your own personal space inside the confines of a specific virtual space. For example, you could go to animalcrossing.meta and build your own home or island inside of that world. But you would only be able to use the tools and items provided to you by Animal Crossing and if your creation violates their policies, they could block your content or ban you altogether. The experience of building a custom home/island could be extremely similar to the current experience of playing Animal Crossing.

But if you want complete freedom to build anything you can imagine and you want to be fully in control of your creations, you need to build your own website. If you build your own website, you're responsible for getting your own hosting and writing the code for your virtual world. But because you're writing your own METABLOB file that specifies everything about your virtual world, you can really build anything, assuming it's not illegal (doesn't infringe on other people's intellectual property, etc.).

By the way, user-generated content is usually the part of the metaverse that leads to the claim that Web3 is necessary for building the metaverse. But this proposal doesn't rely on any Web3 technology and is possible with the current web standards that exist today. I can't imagine wanting more flexibility or "ownership" than what I describe here, so I don't see what Web3 would really be adding.

What's stopping us from building the metaverse right now

Even though everything I described is using standard web technologies that have existed for a while, there are a few reasons why this vision isn't feasible (yet).

No common language for building virtual worlds

In my explanation I referenced a file type called METABLOB, which I made up. For this idea to work, there needs to be a single (or very small number) of file formats for describing these virtual worlds. Modern browsers only know how to interpret three languages: HTML, CSS, and JavaScript. You can't use Python code for your website because browsers don't know how to read and use Python. So everyone is going to need to agree on what language to use for creating virtual worlds. Or maybe one company like Meta will create a language along with a metaverse browser that can read it. Then that language will become the de facto standard and everyone else will just adopt it. I think something like this is how Javascript became the default and only programming language for the web.

Faster download speeds

A typical webpage includes about 5 MB of data your browser needs to download. A typical VR video game is probably between 10-100 GB. If your browser needs to essentially download an entire VR game every time you go to a new website, download speeds are going to need to get way faster or we'll need a breakthrough in compression algorithms. Otherwise you won't be walking through portals to go to other websites, but very long hallways.

Summary

That's it! Hopefully I made the idea of the metaverse more concrete for you. I think we'll eventually be able to build the metaverse, and I'd expect it to be achievable within 10 years. Unlike a lot of other hot new things in technology (like self-driving cars) the metaverse doesn't require any breakthroughs in AI or anything like that. Getting faster download speeds is tricky, but gigabit internet already exists, it's just not common and most websites can't offer download speeds like that right now. More immersive VR would require technological breakthroughs, but I view that as a separate problem from building the metaverse

If you have questions or you think I missed an important aspect of how the metaverse should work, leave a comment below.

How to stay motivated when working on a big project

2021-12-18T20:50:00Z

Last winter I started working on a dating app and continued working on it through the spring. But after getting my Covid vaccine and when things started opening back up during the summer, I took a break and stopped working on my app. A few weeks ago I picked it back up, but it was really hard to get back into it.

I've been working on this project for almost a year now and I've picked up some tricks on how to stay motivated on a long-term project and to see it through to completion. I guess I haven't actually finished yet, but I feel confident that I will. Anyway, here are some tricks that I use to stay motivated and to stick with a big project.

General tips

Make a commitment that you're going to finish your project, no matter what.
Tell as many people about it as you can so you feel guilty if they ask "how's that thing going?" and you haven't made any progress since the last time they asked about it.
Stay caffeinated.
Make a playlist of songs that motivate you.
Play your music through (good) speakers, not headphones, if you don't have roommates. You want to keep working as long as possible, and headphones get uncomfortable if you wear them for too long.
Make a to-do list. I use Trello. Check things off as you go.
Focus on one thing at a time. Just focus on that one thing and don't worry about anything else.
Work like a sculptor. Get the general shape of the whole thing done before you start refining the details.
If working on something is sapping your motivation (like writing tests), stop. Work on something else, even if it's less important.
Work on the most fun stuff first. If you look at something on your to-do list and it makes you groan or gives you a sense of dread, skip it. Another day you'll feel up to it, but today is not that day.
The #1 goal is to keep working at all costs. As long as you're constantly making progress, that's all that matters. Everything needs to get done eventually, but the order you do things in matters less than you think it does.
Items on your to-do list must be as small as possible. Don't write full user stories. Each item on my to-do list can be completed in about 30 minutes. This will ensure that you always feel like you're making progress. I think of each item on my to-do list as one git commit.
If starting/picking back up your project seems daunting, work on a "gateway" project for a bit to get yourself (back) in the right mindset. For me, working on my dating app seemed scary, so I wrote a few blog posts instead. Didn't gain me any progress on my dating app, but got me used to sitting at my computer outside of working and tapping on my keyboard.

Software Engineering-specific tips

Don't worry about writing perfect code. Just get something working ASAP. Make the code nice later.
Don't refactor code prematurely, even if you'll need to do it eventually. Wait until it gets annoying.
Don't worry about handling edge cases. Make to-do list items for those as you think of them.
Make the UI pretty (i.e., styling, CSS) last. You can spend an eternity improving the look and feel of your app, so don't do anything more than the bare minimum until the functionality is 100% complete.

Why I changed my mind about Ranked Choice Voting

2022-04-11T15:45:00Z

In my very first post on this blog I endorsed Ranked Choice Voting. At the time, I was campaigning for Question 2 in Massachusetts to replace our current voting system (plurality voting) with Ranked Choice Voting (RCV). That initiative ultimately failed though. You can read my theory on why it failed here.

I still wish RCV had passed in MA, I still think RCV is way better than our current voting system (plurality voting), and I still stand by most of what I wrote in my original blog post.

However, going forward, I will no longer be pushing for adoption of RCV. I'll be supporting approval voting instead. First I'll explain what I think is wrong with RCV, then explain why approval voting is better.

What's wrong with RCV

There are lots of resources online that compare the mathematical pros and cons of different voting systems (here's a good starting point). However, my main problem with RCV doesn't have anything to do with math, or whether it picks the best candidate.

My problem with RCV is that I think it's too confusing for most Americans, and people wouldn't trust/respect the outcome of an RCV election.

This wasn't something I used to worry about. But that changed in the aftermath of the 2020 presidential election. I'm specifically referring to the "Stop the Steal" movement, and the January 6, 2021 attack on the US Capitol.

Here's my summary of what happened: Joe Biden won the presidential election, Donald Trump couldn't accept that he lost (or chose to pretend that he didn't lose), Trump said that the election was rigged and he should have actually won the election, other people believed Trump/couldn't accept that he lost either, some people tried to overturn the election either through "legal" means (suing, recounting ballots) or force (storming the Capitol).

The statistic that blows my mind is that 30% of Americans think that the 2020 election was rigged and Joe Biden won the election illegitimately (article with lots of polls that back this up).

I'm not sure how much support you need to stage a successful coup, but I'd guess it's around 30%, so I'm very worried about this and the potential for something like this to happen again.

Now, all of what I just said happened with plurality voting. That's the voting system we've been using since the Constitution was ratified. Imagine if the 2020 presidential election had been using RCV instead. Of course, I don't think using RCV would have made the outcome of the election illegitimate. But I do think that even more people would have felt that the election was illegitimate if we had used RCV.

That's the gist of what I think is wrong with RCV. Even though it's a pretty good voting system (miles better than plurality voting), the way the winner is determined is more complicated than plurality voting so I think it has a "trustworthiness" problem.

So even if RCV would be better than what we have now, I fear RCV could just be one more tool that conservatives could use to question the outcome of an election and create doubt in people's mind that the election was fair and legitimate.

It's sad that we now have to factor in whether this would be likely to fuel conspiracy theories, but I think it's unavoidable and it's dangerous to ignore conspiracy theorists.

I'd rather keep using a broken voting system than something that would probably just get overruled (either "legally" or by force) by people who don't like the outcome of the election.

Why Approval Voting is better

First I need to explain what approval voting is. Here's how it works: you know how "regular" voting (plurality voting) works, right? Ok, now imagine that, except you can vote for more than one person. Then the person with the most votes wins. That's it.

Literally the only difference between approval voting and plurality voting is that you can vote for more than one candidate. Same ballots, same voting machines (just need to be updated to not reject ballots with more than one choice filled in), same ballot counting system.

You really can't beat it in terms of simplicity. I'd say it's equally simple to plurality voting. I think the only reason plurality voting seems a bit simpler is because that's what we're used to. If you think about it, limiting people to only voting for one candidate feels like an almost arbitrary restriction.

And what's most amazing about approval voting is how much you get from this one little tweak. In fact, all of the benefits I talked about with RCV apply to approval voting too. Reducing strategic voting, allowing third-party candidates to run without spoiling the election, electing candidates who better represent all of the voters, etc. Approval voting is actually even better than RCV at achieving these things.

Here's a chart from equal.vote that compares the "goodness" of different voting systems:

If you're curious about the math behind the chart, just follow that link. But the point of this post is not to compare the mathematical pros and cons of RCV and approval voting. If you want that though, you can read what the RCV people say about approval voting here, and you can read what the approval voting people say about RCV here.

But, again, the math has nothing to do with why I switched my support from RCV to approval voting. The real reason I think approval voting is better than RCV is because it's simpler and stands a better shot of not having its results overturned because it "doesn't feel right."

Of course, approval voting is not completely immune to conspiratorial attacks. If I was an unscrupulous politician that lost an election that used approval voting, I would assert that approval voting is anti-democratic and demand that all ballots with more than one candidate selected are invalid (i.e., probably most of them). That would of course throw the whole election into limbo (assuming enough people sided with me and were equally delusional or immoral), and in the chaos I'd try to seize power.

I know that sounds really bad, and makes our current voting system sound kind of good in comparison. But I think the only way we get out of this mess (toxic factionalism and extreme polarization) is by fixing our voting system. I really believe that the root cause of polarization in US politics is the plurality voting system (see my original post for why). We need to fix our voting system, and I believe the safest (least likely to facilitate a coup) alternative is approval voting.

Summary

RCV is a great voting system (much better than what we have now), but the way the winner is determined is honestly a little confusing and we don't need that right now. Approval voting is also a great voting system, and the way the winner is determined couldn't be simpler, which makes it very attractive in a world where conspiracy theories drive politics.

Extra: Okay, but how would RCV actually lead to a coup?

For brevity, I didn't dig into how using RCV would lead to a coup. If you're pro-RCV, you might think the idea that RCV could be the cause of a coup a bit fantastical. And it kind of is. I was making an unrealistic assumption that RCV would be used to decide the outcome of a presidential election. But if you're familiar with the current spread of RCV, you know that that's nowhere close to happening. Well, Maine does use it to choose their presidential electors, but that's just one state. Even if the RCV movement is successful even beyond the imagination of its leaders, we're still at least 15-20 years away from the day when all or most Americans are using RCV to vote for president.

The strategy of the RCV movement is to get people using RCV at the state and local level. Then once lots of people are using it in lots of places, it will seem less weird and it will be easier to get even more people to use it in other places. Until eventually everyone uses RCV for all elections.

The reason why my critique of RCV is probably unfair is because I'm imagining us using RCV in a presidential election right now. Of course people wouldn't understand or trust RCV now. But maybe in the 15-20 years it takes to build up to the point where most people are actually voting for the president using RCV, it won't seem strange and opaque to people.

Maybe. But on the other hand, even in 20 years I still think there will be politicians and people who will use any tool available to them to cast doubt on an election that they don't like the outcome of. And I think even in this imagined world where RCV is well-known and normal, it would still be an easy target for people who want to paint an election as illegitimate.

Never mind, I added analytics to my blog

2022-05-30T12:30:00Z

A while back I posted about why I don't have page view tracking on my blog.

Well, I decided to add analytics to my blog and figured I should explain why.

I thought of two situations where I should probably care about how many people are reading my posts:

I recently wrote a post about approval voting. I'm volunteering in an organization that's pushing for the use of approval voting for elections in Massachusetts. They liked the post and asked me how many people have read it. I obviously don't know, because I don't have analytics on my blog.
I post infrequent updates about the dating app that I'm building. So far, I've mostly been sharing them because I thought people might find the updates interesting or helpful, but at some point I'll probably start using my blog as a platform for promoting my dating app. I'd like to know whether or not blogging is an effective marketing tool for my app and if it's worth my time.

The common thread between these two example is that I'm starting to use my blog to try to motivate people to do something (support approval voting or download my dating app). With analytics, I'll be able to determine how successfully I'm accomplishing those goals with my posts. If my posts are getting a lot of engagement, that will motivate me to post more frequently.

Important note: I'm still strongly opposed to cookie banners so I will not be adding tracking cookies or any other client-side tracking to my website. Instead, I will be using Netlify's server-side analytics solution, which currently costs $9/month.

The personal blog of AJ Jenkins

Why you should care about Ranked Choice Voting

Doing It for the Likes

Rewinding to 2007

Becoming worthy

Getting Recon

Looking back

Bonus: Actually getting Recon

Why I don't have analytics on my website

Wait, what do you mean "analytics"?

Why I considered adding analytics to my blog

It's just how you do things

Tailoring my content based on what's popular

Knowing if I'm famous yet

Why I decided against adding analytics to my blog

I hate cookie banners with a passion

Privacy-focused analytics services that I considered

I'm better off not knowing how many page views I'm getting

Making money

Writing things that people want to read

Final thoughts

Closing-closing thought

Achieving a perfect PageSpeed score with Eleventy

Intro

Using PageSpeed Insights

Inlining my CSS (Eliminate render-blocking resources)

Resizing and compressing images (Properly size images)

Using WebP for images (Serve images in next-gen formats)

Defer offscreen images

Fixing all of the YouTube-induced performance issues

Results

Bonus: addPassthroughCopy

Three simple tricks for a lightning-fast Eleventy website

Async loading of YouTube videos (eleventy-plugin-youtube-embed)

Resizing images (@11ty/eleventy-img)

Generating <img> tags in Nunjucks templates

Using the plugin in a Markdown file

Converting your images to WebP

Making your images scale proportionally on mobile

Inlining CSS (clean-css)

Summary

Bonus trick: addPassthroughCopy

Ranked Choice Voting: Why we lost in 2020

I'm making a pandemic dating app, Part 1

Why I'm making a dating app now

Do we really need another dating app?

Why the existing dating apps are bad

Why my dating app will be better

New features: Comments and RSS feed

New features on the blog

Comments

RSS feed

More features coming soon

Email updates

Table of Contents

...And more

I'm making a pandemic dating app, Part 2

Elevator Pitch

How it works

Why I think it's better than existing apps

Tinder-like apps

Doesn't waste your time

No fake profiles

Speed-dating apps

Risks

Flashers

Not enough users online

Next steps

Review of AWS AppSync and Amplify in 2019

AppSync vs. Amplify

Initial impression

Many-to-many relationships

Final thoughts

Building a modern API in 2020

Intro

Components of backend stack

Hosting

Cloud providers

Self-hosted vs. Managed

Serverless?