Deploying a Hugo website to AWS in 6 steps (CDN+HTTPS)

Hugo meets AWS
Hugo meets AWS

Using Amazon's AWS S3, Cloudfront CDN and HTTPS

  • Published: August 12, 2018

Overview

Step by step guide to deploy your Hugo website to Amazon Web Services using:

  • a private S3 bucket1
  • Cloudfront CDN
  • SSL certificates (URLs with HTTPS)
  • all requests redirects to the non-WWW version https://example.com
  • handling pretty URLs (no .html in URLs)
  • Error pages.

In other words, after following the guide we will have:

  • URLs will “start” with https://, e.g.: https://example.com/about
  • accessing a URL with WWW will redirect us to its non-WWW version, e.g.: https://www.example.com -> https://example.com
  • accessing a URL with HTTP will redirect to its HTTPS version, e.g.: http://example.com -> https://example.com
  • URLs can only be accessed through Cloudfront’s CDN, there is no direct access to the S3 bucket objects. For example, for the /sitemap.xml object:
  • Accessing a non existing page, will load Hugo’s custom error /404.html with 404 HTTP status.

1. Set up hosting

To host the website we set up a private S3 bucket and then configure Cloudfront to redirect requests to that bucket.

1.1 non-WWW bucket

One bucket for the naked domain called: example.com. This bucket will hold our static website files.

  1. Go to https://console.aws.amazon.com/s3/home
  2. Press Create bucket button 2.1 Enter bucket name: example.com 2.2. Press Next, Next and Create bucket.
    • Now your bucket list should contain the example.com bucket
  3. Click example.com bucket and select the Properties tab.
  4. Select Static website hosting
  5. Select Use this bucket to host a website 5.1 In Index document enter: index.html
    • There is no need to set up an error document as this will be handled by Cloudfront.

1.2 WWW bucket

We create a new bucket just to redirect WWW to non-WWW requests.

  1. Go again to https://console.aws.amazon.com/s3/home
  2. Press Create bucket button 2.1 Enter bucket name: www.example.com 2.2. Press Next, Next and Create bucket.
  3. Click www.example.com bucket and select the Properties tab.
  4. Select Static website hosting 4.1 Select Redirect requests 4.2 In Target bucket or domain enter our previously created bucket: example.com 4.3 Set Protocol as: https

2. Set up CDN

2.1 CDN for example.com

Using Cloudfront we use their CDN, enforce only access the bucket throught Cloudfront and HTTPS usage, and handle pretty URLs in combination with a Lambda Edge function.

  1. Go to https://console.aws.amazon.com/cloudfront/
  2. Click Create distribution
  3. In the Delivery method for your content page, select the Web/Get Started button
  4. Enter ONLY the following data:
    • origin: Click inside the text input and a list of your buckets will appear, select example.com.s3.amazonaws.com.
    • Restrict Bucket Access: Select Yes; more options will appear
      • Origin Access Identity: Select or create an Access Identity
      • Grant Read Permissions on Bucket: Select Yes, Update Bucket Policy so Amazon automatically handle your S3 bucket permissions
    • In Default Cache Behavior Settings section:
      • *Viewer Protocol Policy: Select Redirect HTTP to HTTPS
      • Compress objects automatically: Select Yes to Compress Content when possible
    • In Distribution Settings section:
      • Alternate Domain Names (CNAMEs): enter example.com.
    • SSL Certificate:
      • Select Custom SSL Certificate (example.com)
      • And press the Request or Import a Certificate with ACM button.
        • You will be redirected to AWS Certificate Manager to create a new certificate, in this page add the two domain names:
          • example.com
          • www.example.com
          • Then click Next and validate your certificate.
      • After you have the certificate, go back to Cloudfront settings page and select your newly created certificate from the list at Distribution Settings/SSL Certificate/Custom SSL Certificate.
  5. Click Create distribution at the bottom of the page
To require that users always access your Amazon S3 content using CloudFront URLs, you assign a special CloudFront user - an origin access identity - to your origin.
Cloudfront documentation

2.2 CDN for www.example.com

Create another distribution for the www.example.com:

  1. Go to https://console.aws.amazon.com/cloudfront/
  2. Click Create distribution
  3. In the Delivery method for your content page, select the Web/Get Started button
  4. Enter ONLY the following data:
    • origin: Click inside the text input and a list of your buckets will appear, select www.example.com.s3.amazonaws.com.
    • In Default Cache Behavior Settings section:
      • *Viewer Protocol Policy: Select Redirect HTTP to HTTPS
    • In Distribution Settings section:
      • Alternate Domain Names (CNAMEs): enter www.example.com.
    • SSL Certificate:
      • Select Custom SSL Certificate (example.com)
      • Select your previously created certificate from the list at the Custom SSL Certificate input box.
  5. Click Create distribution at the bottom of the page

3. Set up DNS

  1. Go to your hosted zones in Route53 console https://console.aws.amazon.com/route53/home?#hosted-zones:
  2. Click Create Hosted Zone button and enter example.com.
  3. Click Create Record Set button.
  4. Leave Name empty so we are setting up example.com..
  5. Select Type: A - IPv4 address
  6. Select Alias: Yes 6.1. In Alias target, select the example.com Cloudfront distribution.
    Be sure to select the example.com “Cloudfront distribution” and not the “S3 endpoint”.
  7. Click Save record set

Now we create another one for the www version:

  1. Click Create Hosted Zone button and enter example.com.
  2. Click Create Record Set button.
  3. Enter Name: www
  4. Select Type: A - IPv4 address
  5. Select Alias: Yes 5.1. In Alias target, select the www.example.com Cloudfront distribution.
  6. Click Save record set

4. Handle Pretty URLs

Hugo by default generates web pages like <content-title>/index.html at its /public directory.

The default root object feature for CloudFront supports only the root of the origin that your distribution points to. CloudFront does not return default root objects in subdirectories. For more information, see Specifying a Default Root Object (Web Distributions Only).

To assign a default root object for your CloudFront distribution, be sure to upload the object to the origin that your distribution points to.

As we have our S3 bucket private, accessing a webpage like example.com/hello/ won’t request our example.com/hello/index.html. To handle this we use a Lambda@Edge function.

Lambda@Edge lets you run Lambda functions to customize content that CloudFront delivers, executing the functions in AWS locations closer to the viewer. The functions run in response to CloudFront events, without provisioning or managing servers.

This function does (source):

  • URI paths that end in .../index.html are redirected to .../ with an HTTP status code 301 Moved Permanently. (This is the same as an “external” redirect by a webserver).

  • URI paths that do not have an extension and do not end with a / are redirected to the same path with an appended / with an HTTP status code 301 Moved Permanently. (This is an “external” redirect)

4.1 Lambda@Edge Function Installation

We use the function standard-redirects-for-cloudfront, to install it via the Serverless Application Repository:

  1. Go to AWS Serverless Application Repository
  2. Press the Deploy button to use the application standard-redirects-for-cloudfront.
  3. It opens a description of the app, hit Deploy again to finish deploying it.
  4. After it has been created, locate the button View CloudFormation stack or go directly to the Cloudformation Console
  5. In the Resources tab, locate the AWS::IAM::Role and open the Physical ID, it will open up the IAM console
  6. Edit the trust relationship to allow CloudFront to execute this function as a Lambda@Edge function., set the policy to:
   {
	 "Version": "2012-10-17",
	 "Statement": [
	   {
		 "Effect": "Allow",
		 "Principal": {
		   "Service": [
			 "lambda.amazonaws.com",
			 "edgelambda.amazonaws.com"
		   ]
		 },
		 "Action": "sts:AssumeRole"
	   }
	 ]
   }
  1. Go back to the Cloudformation’s Stack Detail page and in the Output tab, copy the Value, this is the ARN (Amazon Resource Name) for the Lambda function that we will use in Cloudfront.
  2. Go back to the CloudFront console, select the example.com distribution
  3. Edit the redirect Behavior
  4. In Lambda Function Association select Event Type/Origin Request and enter the Lambda function ARN from the previous step.
  5. Wait for the CloudFront distribution to deploy.

5. Error page

If we try to access a URL that doesn’t exist on our S3 bucket, like https://example.com/not-existing-page we will get a 403 Forbidden error code because Cloudfront tries to access a object2 that doesn’t exists.

Setting up the error page on S3 wouldn’t have any effect because this is an error that should be handled by Cloudfront.

To do this, we configure CloudFront to respond to requests using Hugo’s custom error page located at /layouts/404.html, when your origin returns an HTTP 403 permission denied.

  1. Go to Cloudfront console: https://console.aws.amazon.com/cloudfront
  2. Select your example.com distribution
  3. Choose the Error Pages tab.
  4. Press Create Custom Error Response button.
    • In HTTP Error Code, select: 403: Forbidden
    • Customize Error Response: Yes
    • Response Page Path: /404.html
    • HTTP Response Codeo: 404: Not Found

6. Deploy command

Finally, after everything is set up, we use the AWS Command Line Interface to copy files to the S3 bucket, specifically, the sync high-level S3 command.

aws s3 sync public/ s3://example.com/ --size-only --delete
aws sync <LocalPath> <S3Uri>:
Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination. Only creates folders in the destination if they contain one or more files.

6.1 (Optional) Makefile

A great build automation tool for deploys is GNU make.

A simple Makefile containing all recipes, that provides these commands:

  • build-production: build the Hugo website using the environment variable HUGO_ENV set to production so you can develop your website choosing which parts of the code to avoid being used in development (like Google Analytics or displaying ads).
  • deploy:
    1. builds the website,
    2. copy new files to the S3 bucket
    3. remove files from bucket that are not present in the newly generated site
    4. uploaded and removed files are refreshed in CDN
    5. notifies Google and Bing that a new sitemap of your site is available
  • aws-cloudfront-invalidate-all: refresh CDN contents

In /Makefile:

SHELL := /bin/bash
AWS := aws
HUGO := hugo
PUBLIC_FOLDER := public/
S3_BUCKET = s3://example.com/
CLOUDFRONT_ID := ABCDE12345678
DOMAIN = example.com
SITEMAP_URL = https://example.com/sitemap.xml

DEPLOY_LOG := deploy.log

.ONESHELL:

build-production:
	HUGO_ENV=production $(HUGO)

deploy: build-production
	echo "Copying files to server..."
	$(AWS) s3 sync $(PUBLIC_FOLDER) $(S3_BUCKET) --size-only --delete | tee -a $(DEPLOY_LOG)
	# filter files to invalidate cdn
	grep "upload\|delete" $(DEPLOY_LOG) | sed -e "s|.*upload.*to $(S3_BUCKET)|/|" | sed -e "s|.*delete: $(S3_BUCKET)|/|" | sed -e 's/index.html//' | sed -e 's/\(.*\).html/\1/' | tr '\n' ' ' | xargs aws cloudfront create-invalidation --distribution-id $(CLOUDFRONT_ID) --paths
	curl --silent "http://www.google.com/ping?sitemap=$(SITEMAP_URL)"
	curl --silent "http://www.bing.com/webmaster/ping.aspx?siteMap=$(SITEMAP_URL)"

aws-cloudfront-invalidate-all:
	$(AWS) cloudfront create-invalidation --distribution-id $(CLOUDFRONT_ID) --paths "/*"

Now you can run each recipe with make: make build-production, make deploy or make aws-cloudfront-invalidate-all.

In particular, we are going to run make deploy each time we update our website locally.

Example usage


$ make deploy
HUGO_ENV=production hugo

                   | EN
+------------------+-----+
  Pages            | 290
  Paginator pages  |   0
  Non-page files   |   6
  Static files     |  45
  Processed images |   0
  Aliases          | 100
  Sitemaps         |   1
  Cleaned          |   0

Total in 2733 ms
Running ["ImageCheck", "LinkCheck", "ScriptCheck", "HtmlCheck"] on ["public"] on *.html... 


Ran on 266 files!


HTML-Proofer finished successfully.
echo "Copying files to server..."
aws s3 sync public/ s3://example.com/ --size-only --delete | tee -a deploy.log
grep "upload\|delete" deploy.log | sed -e "s|.*upload.*to s3://example.com/|/|" | sed -e "s|.*delete: s3://example.com/|/
|" | sed -e 's/index.html//' | sed -e 's/\(.*\).html/\1/' | tr '\n' ' ' | xargs aws cloudfront create-invalidation --distribution-id ABCDE12345678 --paths
curl --silent "http://www.google.com/ping?sitemap=https://example.com/sitemap.xml"
curl --silent "http://www.bing.com/webmaster/ping.aspx?siteMap=https://example.com/sitemap.xml"
Copying files to server...
upload: public/example-page/index.html to s3://example.com/example-page/index.html
{
    "Location": "https://cloudfront.amazonaws.com/2017-03-25/distribution/XXXXXXXXXXX/invalidation/I1XO2I27BX1UAY",
    "Invalidation": {
        "Id": "XXXXXXXXXXX",
        "Status": "InProgress",
        "CreateTime": "2018-08-13T04:31:52.659Z",
        "InvalidationBatch": {
            "Paths": {
                "Quantity": 1,
                "Items": [
                    "/example-page/"
                ]
            },
            "CallerReference": "cli-1234-56789"
        }
    }
}
<html><meta http-equiv="content-type" content="text/html; charset=UTF-8">
<head><title>Google Webmaster Tools
-
Sitemap Notification Received</title>
<meta name="robots" content="noindex, noodp">
<script src="https://ssl.google-analytics.com/urchin.js" type="text/javascript">
    </script>
<script type="text/javascript">
      _uacct="UA-12345-1";
      _utcp="/webmasters/";
      _uanchor=1;
      urchinTracker();
    </script></head>
<body><h2>Sitemap Notification Received</h2>
<br>
Your Sitemap has been successfully added to our list of Sitemaps to crawl. If this is the first time you are notifying Google about this Sitemap, please add it via  <a href="http://www.google.com/webmasters/tools/">http://www.google.com/webmasters/tools/</a>  so you can track its status. Please note that we do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about
when or if they will appear.</body></html><html><body>Gracias por enviar tu Sitemap.  Únete a las <a href="/webmaster">Herramientas de administrador web de Bing</a> para ver el estado de tus Sitemaps e informes sobre tu progreso en Bing.</body></html>

A Makefile with a more complete list of recipes is available at: https://github.com/marcanuy/hugo-deploy-aws-makefile

Considerations

Why a private S3 bucket?

When serving a website from S3 bucket, each request of an object has a cost. If someone wants to harm your business, it could start an attack downloading a lot of files from different servers and you will be billed for that.

When serving them from Cloudfront, costs are much more cheaper and fast as the distributed nature of a Content Delivery Network.

References


  1. A bucket is a container for objects stored in Amazon S3. Every object is contained in a bucket. For example, if the object named photos/puppy.jpg is stored in the johnsmith bucket, then it is addressable using the URL http://johnsmith.s3.amazonaws.com/photos/puppy.jpg
    [return]
  2. Objects are the fundamental entities stored in Amazon S3. Objects consist of object data and metadata.
    [return]
Uruguay
Marcelo Canina
I'm Marcelo Canina, a developer from Uruguay. I build websites and web-based applications from the ground up and share what I learn here.
comments powered by Disqus

Guide to deploy your Hugo website to AWS using a private S3 bucket, Cloudfront CDN, SSL certificates, handling pretty URLs and error pages.

Except as otherwise noted, the content of this page is licensed under CC BY-NC-ND 4.0 ·