Designing a website to not have 404s

Written by Lilou Artz on June 11, 2024.

#sql, #seo, #ux

When I started working on Pillser, I knew I am not going to get everything right the first time.

This is particularly true about the information architecture of the website.

The idea behind Pillser is simple: associate research with supplements. However, as there aren't many anologous websites, I have to come up with how to present this information. As such, I wanted to reduce lock-in to URL architecture as much as possible. One of the ways I've done it is by using fuzzy matching to redirect users to the right place when they make a typo, a product is renamed, or the logic used to generate the URL changes. (The latter has happened enough times already that I am glad that I've implemented this feature!)

Here is what it looks like from the user's perspective:

/supplements/spore-probiotic-6066 This is the correct URL for the supplement.
/supplements/spore-probiotic This URL is missing the ID. However, the website will redirect the user to the correct page.
/supplements/youtheory-spore-probiotic This URL includes the brand name. However, since disregarding the brand name "spore probiotic" is unique enough to identify the supplement, the website will redirect the user to the correct page.

At the moment, I've applied this logic only to the supplement pages, but I am planning to extend it to the rest of the website.

In practice, the implementation is so simple that I am surprised that more websites do not implement it. It is basically a fallback mechanism that queries the database to find the closest match to the URL.

SELECT
  id,
  slug
FROM supplement
ORDER BY similarity(slug, ${slug}) DESC
LIMIT 1

The similarity function is a PostgreSQL pg_tgrm extension that calculates the similarity between two strings. The closer the value is to 1, the more similar the strings are. Since my goal is to find the closest match, I order the results by the similarity in descending order and pick the first one.

In the backend, I log whenever such a redirect happens. This way, I can manually override the redirect logic if I discover that the chosen supplement is not the correct one or not the most relevant substitute.

The primary goal of this is to provide a better user experience. I am unsure what are the implications for SEO, but I am hoping that it will be positive. I will keep you updated on this.