Process Clients FAQ Team Blog Imprint
Updated on in #devops, #jamstack — 10 Minute Read

URL Normalization with Netlify Redirects

Static site generators are not a new idea, but recently they started to gain more traction. Modern implementations (e.g., Jekyll) are a giant leap forward for web design. They offer many advantages over traditional CMS systems, but I want to emphasize the dead-simple deployment process. Web designers and developers love this because they can focus on what really matters: Their client’s project. The need for DevOps knowledge or even dedicated people is mostly gone. Just copy the generated static files to a web server, and you are done. Also, they are starting to displace traditional content management systems through the integration of user interfaces, allowing laymen to create and edit pages in a WYSIWYG fashion.

This is great, but in certain situations, you may need fine-grained control over your web server. In these cases, the “copy to a web server and forget approach” is not sufficient. One of these cases is if you want to enforce properly normalized URLs for your website. To achieve this, you need to configure redirects on your web server. Static site generators can’t help you with that, but platforms like the amazing Netlify - one of the leading companies in the JAMstack community - make this straightforward.

Canonical

The canonical tag is common knowledge nowadays. For instance, this page has a canonical tag like this:

1
<link rel="canonical" href="/blog/url-normalization-with-netlify-redirects/"/>

The purpose of this is to tell search engines where the page lives. This is necessary because, by default, any given page is accessible through several different URLs: You can navigate to this page via /blog/url-normalization-with-netlify-redirects/ and /blog/url-normalization-with-netlify-redirects/index.html.

This is no concern as far as SEO goes if your website has proper canonical tags. Still, it can be a concern in other areas if people can access the same page using different URLs. Just to name an example, it could confuse your user tracking system.

Personally, I recommend URLs with trailing slash. It doesn’t matter which format you choose, though, as long as you are consistent about it. Just pick one and make sure that your canonical tags match your choice.

Basic Normalization

The best way to enforce your preferred URL style is permanent HTTP 301 redirects on the server-side. Needless to say, static site generators can’t help you in this regard. You could set up your generator to spit out a config file for your web server, but this can be difficult because web server configuration is not famous for being easy to learn. Luckily you don’t have to deal with that because Netlify has already solved the problem for you.

The easiest way to set this up on Netlify is a _redirects file. This is just a text file that looks like this:

1
/blog/url-normalization-with-netlify-redirects/index.html /blog/url-normalization-with-netlify-redirects/ 301!

This instructs Netlify to permanently redirect /blog/url-normalization-with-netlify-redirects/index.html to /blog/url-normalization-with-netlify-redirects/ (with trailing slash). This makes sense if you consider that the page’s canonical tag also points to the latter. Finally, note the exclamation mark. This is required to tell Netlify to perform the redirect even if a file/directory matching the source URL exists, which is, of course, the case here.

Generic Normalization

Manually writing redirect rules like this for all of your pages is very error-prone, so I looked around for a more more generic solution. Fortunately, Netlify’s _redirects feature is expressive enough to make this more elegant. The following config is limited to paths that are five levels deep, but this is probably good enough for most websites:

1
2
3
4
5
6
7
8
# Remove direct references to index.html.
# Concrete Example: /foo/bar/index.html /foo/bar/ 301!
/index.html / 301!
/:a/index.html /:a/ 301!
/:a/:b/index.html /:a/:b/ 301!
/:a/:b/:c/index.html /:a/:b/:c/ 301!
/:a/:b/:c/:d/index.html /:a/:b/:c/:d/ 301!
/:a/:b/:c/:d/:e/index.html /:a/:b/:c/:d/:e/ 301!

After looking through Netlify’s excellent documentation, I found that we can go even further because they also support splats:

1
2
3
4
# Remove direct references to index.html.
# Concrete Example: /foo/bar/index.html /foo/bar/ 301!
/index.html / 301!
/*/index.html /:splat/ 301!

The second config is basically identical to the first one, but it’s no longer limited to five levels! It’s 100% generic.

Testing

During experimentation with _redirects, I needed to make sure that my changes don’t break anything that already worked before. So, I wrote a few quick tests in a shell script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/env sh

base="https://bugfactory.io"

function expect_success {
  local path="$1"
  echo -n "Expecting GET '${path}' to succeed ... "

  headers=$(http --headers GET "${base}/${path}")
  if echo $headers | head -n 1 | grep --quiet 200; then
    echo "SUCCESS"
  else
    echo "FAILURE"
  fi
}

function expect_redirect {
  local path="$1"
  local dest="$2"
  echo -n "Expecting GET '${path}' to redirect to '${dest}' ... "

  headers=$(http --headers GET "${base}/${path}")
  if echo "$headers" | head -n 1 | grep --quiet 301 && \
     echo "$headers" | grep --quiet "^Location: ${dest////\\/}\\s*\$"; then
    echo "SUCCESS"
  else
    echo "FAILURE"
    echo "$headers"
  fi
}

expect_success  ""
expect_success  "/"
expect_redirect "/index.html"                                 "/"
expect_redirect "/blog"                                       "/blog/"
expect_success  "/blog/"
expect_redirect "/blog/index.html"                            "/blog/"
expect_redirect "/blog/telling-docker-who-you-are"            "/blog/telling-docker-who-you-are/"
expect_success  "/blog/telling-docker-who-you-are/"
expect_redirect "/blog/telling-docker-who-you-are/index.html" "/blog/telling-docker-who-you-are/"

Just for the sake of completeness, here is the script’s output:

1
2
3
4
5
6
7
8
9
Expecting GET '' to succeed ... SUCCESS
Expecting GET '/' to succeed ... SUCCESS
Expecting GET '/index.html' to redirect to '/' ... SUCCESS
Expecting GET '/blog' to redirect to '/blog/' ... SUCCESS
Expecting GET '/blog/' to succeed ... SUCCESS
Expecting GET '/blog/index.html' to redirect to '/blog/' ... SUCCESS
Expecting GET '/blog/telling-docker-who-you-are' to redirect to '/blog/telling-docker-who-you-are/' ... SUCCESS
Expecting GET '/blog/telling-docker-who-you-are/' to succeed ... SUCCESS
Expecting GET '/blog/telling-docker-who-you-are/index.html' to redirect to '/blog/telling-docker-who-you-are/' ... SUCCESS

Feel free to copy this script to test your own redirect setup. Just replace the URLs, and you are good to go to write your redirect rules with confidence.

Problem solved. If you are not yet using Netlify, you should give them a try because they have a ton of useful features like this.


Notes:
Software: HTTPie 1.0.3

Feedback or Questions?

Let's Talk on Twitter