Deploying a Hugo website to AWS in 6 steps (CDN+HTTPS)

Overview

Step by step guide to deploy your Hugo website to Amazon Web Services using:

• a private S3 bucket¹
• Cloudfront CDN
• SSL certificates (URLs with HTTPS)
• all requests redirects to the non-WWW version https://example.com
• handling pretty URLs (no .html in URLs)
• Error pages.

In other words, after following the guide we will have:

• URLs will “start” with https://, e.g.: https://example.com/about
• accessing a URL with WWW will redirect us to its non-WWW version, e.g.: https://www.example.com -> https://example.com
• accessing a URL with HTTP will redirect to its HTTPS version, e.g.: http://example.com -> https://example.com
• URLs can only be accessed through Cloudfront’s CDN, there is no direct access to the S3 bucket objects. For example, for the /sitemap.xml object:
  • https://s3.amazonaws.com/example.com/sitemap.xml -> Code: AccessDenied
  • https://example.com/sitemap.xml
• Accessing a non existing page, will load Hugo’s custom error /404.html with 404 HTTP status.

1. Set up hosting

To host the website we set up a private S3 bucket and then configure Cloudfront to redirect requests to that bucket.

1.1 non-WWW bucket

One bucket for the naked domain called: example.com. This bucket will hold our static website files.

1. Go to https://console.aws.amazon.com/s3/home
2. Press Create bucket button
   1. Enter bucket name: example.com
   2. Press Next, Next and Create bucket.
      • Now your bucket list should contain the example.com bucket
3. Click example.com bucket and select the Properties tab.

2. Set up CDN

2.1 CDN for example.com

Using Cloudfront we use their CDN, enforce only access the bucket throught Cloudfront and HTTPS usage, and handle pretty URLs in combination with a Lambda Edge function.

1. Go to https://console.aws.amazon.com/cloudfront/

2. Click Create distribution

3. In the Delivery method for your content page, select the Web/Get Started button

4. Enter ONLY the following data:

   • origin: Click inside the text input and a list of your buckets will appear, select example.com.s3.amazonaws.com.
   • Restrict Bucket Access: Select Yes; more options will appear
     • Origin Access Identity: Select or create an Access Identity
     • Grant Read Permissions on Bucket: Select Yes, Update Bucket Policy so Amazon automatically handle your S3 bucket permissions
   • In Default Cache Behavior Settings section:
     • *Viewer Protocol Policy: Select Redirect HTTP to HTTPS
     • Compress objects automatically: Select Yes to Compress Content when possible
   • In Distribution Settings section:
     • Alternate Domain Names (CNAMEs): enter example.com.
   • In Default Root Object: index.html.
   • SSL Certificate:
     • Select Custom SSL Certificate (example.com)
       • And press the Request or Import a Certificate with ACM button.
         • You will be redirected to AWS Certificate Manager to create a new certificate, in this page add the two domain names:
           • example.com
           • www.example.com
           • Then click Next and validate your certificate.
     • After you have the certificate, go back to Cloudfront settings page and select your newly created certificate from the list at Distribution Settings/SSL Certificate/Custom SSL Certificate.

5. Click Create distribution at the bottom of the page

6. Make CloudFront able to access to your S3 bucket by going to the Origins and Origin Group tab, edit the existing origin, in Grant Read Permissions on Bucket select Yes, Update Bucket Policy, and save changes, it will automatically generate the following policy:

   {
      "Version":"2012-10-17",
      "Id":"PolicyForCloudFrontPrivateContent",
      "Statement":[
        {
     	 "Sid":" Grant a CloudFront Origin Identity access to support private content",
     	 "Effect":"Allow",
     	 "Principal":{"CanonicalUser":"CloudFront Origin Identity Canonical User ID"},
     	 "Action":"s3:GetObject",
     	 "Resource":"arn:aws:s3:::example.com/*"
        }
      ]
   }
   

Now when you visit the S3 Bucket Policy tab, it will be already generated with the values for your bucket.

This is a very important step or you will get a 404 error page when trying to access your site because CloudFront won’t be able to read your files, read more about it at Granting Permission to an Amazon CloudFront Origin Identity

To require that users always access your Amazon S3 content using CloudFront URLs, you assign a special CloudFront user - an origin access identity - to your origin.

2.2 CDN for www.example.com

Create another distribution for the www.example.com:

1. Go to https://console.aws.amazon.com/cloudfront/
2. Click Create distribution
3. In the Delivery method for your content page, select the Web/Get Started button
4. Enter ONLY the following data:
   • origin: Click inside the text input and a list of your buckets will appear, select www.example.com.s3.amazonaws.com.
   • In Default Cache Behavior Settings section:
     • *Viewer Protocol Policy: Select Redirect HTTP to HTTPS
   • In Distribution Settings section:
     • Alternate Domain Names (CNAMEs): enter www.example.com.
   • SSL Certificate:
     • Select Custom SSL Certificate (example.com)
       • Select your previously created certificate from the list at the Custom SSL Certificate input box.
5. Click Create distribution at the bottom of the page

3. Set up DNS

1. Go to your hosted zones in Route53 console https://console.aws.amazon.com/route53/home?#hosted-zones:

2. Click Create Hosted Zone button and enter example.com, point your domain to Route 53 DNS servers showed at Record Set NS type list.

3. Click Create Record Set button.

4. Leave Name empty so we are setting up example.com..

5. Select Type: A - IPv4 address

6. Select Alias: Yes

   1. In Alias target, select the example.com Cloudfront distribution.
      Be sure to select the example.com "Cloudfront distribution" and not the "S3 endpoint".

7. Click Save record set

8. Update your DNS nameservers to point to the new name servers.

Now we create another one for the www version:

1. Click Create Hosted Zone button and enter example.com.
2. Click Create Record Set button.
3. Enter Name: www
4. Select Type: A - IPv4 address
5. Select Alias: Yes
   1. In Alias target, select the www.example.com Cloudfront distribution.
6. Click Save record set

4. Handle Pretty URLs

Hugo by default generates web pages like <content-title>/index.html at its /public directory.

The default root object feature for CloudFront supports only the root of the origin that your distribution points to. CloudFront does not return default root objects in subdirectories. For more information, see Specifying a Default Root Object (Web Distributions Only).

To assign a default root object for your CloudFront distribution, be sure to upload the object to the origin that your distribution points to.

As we have our S3 bucket private, accessing a webpage like example.com/hello/ won’t request our example.com/hello/index.html. To handle this we use a Lambda@Edge function.

Lambda@Edge lets you run Lambda functions to customize content that CloudFront delivers, executing the functions in AWS locations closer to the viewer. The functions run in response to CloudFront events, without provisioning or managing servers.

This function does (source):

• URI paths that end in .../index.html are redirected to .../ with an HTTP status code 301 Moved Permanently. (This is the same as an “external” redirect by a webserver).

• URI paths that do not have an extension and do not end with a / are redirected to the same path with an appended / with an HTTP status code 301 Moved Permanently. (This is an “external” redirect)

4.1 Lambda@Edge Function Installation

We use the function standard-redirects-for-cloudfront, to install it via the Serverless Application Repository:

1. Go to AWS Serverless Application Repository

2. Press the Deploy button to use the application standard-redirects-for-cloudfront.

3. It opens a description of the app, hit Deploy again to finish deploying it.

4. After it has been created, locate the button View CloudFormation stack or go directly to the Cloudformation Console

5. In the Resources tab, locate the AWS::IAM::Role and open the Physical ID, it will open up the IAM console

6. Go to Trust Relationship tab and choose Edit the trust relationship to allow CloudFront to execute this function as a Lambda@Edge function., set the policy to:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
    	 "Effect": "Allow",
    	 "Principal": {
    	   "Service": [
    		 "lambda.amazonaws.com",
    		 "edgelambda.amazonaws.com"
    	   ]
    	 },
    	 "Action": "sts:AssumeRole"
       }
     ]
   }
   

7. Go back to the Cloudformation’s Stack Detail page and in the Output tab, locate the key StandardRedirectsForCloudFrontVersionOutput and note down its Value (it will look something like: arn:aws:lambda:us-east-1:XXXXXXXXXXX:function:aws-serverless-repository-StandardRedirectsForClou-XXXXXXXXXXXX:2 ). We will use it in the next steps as this is the ARN (Amazon Resource Name) for the Lambda function that we will use in Cloudfront.

8. Go back to the CloudFront console, select the example.com distribution

9. Go to the Behaviour tab and edit the default Behavior.

10. Now we use the Lambda function, in Lambda Function Association select Event Type/Origin Request and enter the Lambda function’s StandardRedirectsForCloudFrontVersionOutput ARN value from the previous step.

11. Wait for the CloudFront distribution to deploy.

5. Error page

If we try to access a URL that doesn’t exist on our S3 bucket, like https://example.com/not-existing-page we will get a 403 Forbidden error code because Cloudfront tries to access a object² that doesn’t exists, so to properly handle this, we should return a 404 error response to the request.

Setting up the error page on S3 wouldn’t have any effect because this is an error that should be handled by Cloudfront.

To do this, we configure CloudFront to respond to requests using Hugo’s custom error page located at /layouts/404.html, when your origin returns an HTTP 403 permission denied.

1. Go to Cloudfront console: https://console.aws.amazon.com/cloudfront
2. Select your example.com distribution
3. Choose the Error Pages tab.
4. Press Create Custom Error Response button.

• In HTTP Error Code, select: 403: Forbidden
• Customize Error Response: Yes
• Response Page Path: /404.html
• HTTP Response Codeo: 404: Not Found

6. Deploy command

Finally, after everything is set up, we use the AWS Command Line Interface to copy files to the S3 bucket, specifically, the sync high-level S3 command.

aws s3 sync public/ s3://example.com/ --size-only --delete

Command aws sync <LocalPath><S3Uri> explanation:

Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination. Only creates folders in the destination if they contain one or more files.

6.1 (Optional) Makefile

A great build automation tool for deploys is GNU make.

A simple Makefile containing all recipes, that provides these commands:

• build-production: build the Hugo website using the environment variable HUGO_ENV set to production so you can develop your website choosing which parts of the code to avoid being used in development (like Google Analytics or displaying ads).
• deploy:
  1. builds the website,
  2. copy new files to the S3 bucket
  3. remove files from bucket that are not present in the newly generated site
  4. uploaded and removed files are refreshed in CDN
  5. notifies Google and Bing that a new sitemap of your site is available
• aws-cloudfront-invalidate-all: refresh CDN contents

In /Makefile:

SHELL := /bin/bash
AWS := aws
HUGO := hugo
PUBLIC_FOLDER := public/
S3_BUCKET = s3://example.com/
CLOUDFRONT_ID := ABCDE12345678
DOMAIN = example.com
SITEMAP_URL = https://example.com/sitemap.xml

DEPLOY_LOG := deploy.log

.ONESHELL:

build-production:
	HUGO_ENV=production $(HUGO)

deploy: build-production
	echo "Copying files to server..."
	$(AWS) s3 sync $(PUBLIC_FOLDER) $(S3_BUCKET) --size-only --delete | tee -a $(DEPLOY_LOG)
	# filter files to invalidate cdn
	grep "upload\|delete" $(DEPLOY_LOG) | sed -e "s|.*upload.*to $(S3_BUCKET)|/|" | sed -e "s|.*delete: $(S3_BUCKET)|/|" | sed -e 's/index.html//' | sed -e 's/\(.*\).html/\1/' | tr '\n' ' ' | xargs aws cloudfront create-invalidation --distribution-id $(CLOUDFRONT_ID) --paths
	curl --silent "http://www.google.com/ping?sitemap=$(SITEMAP_URL)"
	curl --silent "http://www.bing.com/webmaster/ping.aspx?siteMap=$(SITEMAP_URL)"

aws-cloudfront-invalidate-all:
	$(AWS) cloudfront create-invalidation --distribution-id $(CLOUDFRONT_ID) --paths "/*"

Now you can run each recipe with make: make build-production, make deploy or make aws-cloudfront-invalidate-all.

In particular, we are going to run make deploy each time we update our website locally.

Example usage

$ make deploy
HUGO_ENV=production hugo

                   | EN
+------------------+-----+
  Pages            | 290
  Paginator pages  |   0
  Non-page files   |   6
  Static files     |  45
  Processed images |   0
  Aliases          | 100
  Sitemaps         |   1
  Cleaned          |   0

Total in 2733 ms
Running ["ImageCheck", "LinkCheck", "ScriptCheck", "HtmlCheck"] on ["public"] on *.html... 


Ran on 266 files!


HTML-Proofer finished successfully.
echo "Copying files to server..."
aws s3 sync public/ s3://example.com/ --size-only --delete | tee -a deploy.log
grep "upload\|delete" deploy.log | sed -e "s|.*upload.*to s3://example.com/|/|" | sed -e "s|.*delete: s3://example.com/|/
|" | sed -e 's/index.html//' | sed -e 's/\(.*\).html/\1/' | tr '\n' ' ' | xargs aws cloudfront create-invalidation --distribution-id ABCDE12345678 --paths
curl --silent "http://www.google.com/ping?sitemap=https://example.com/sitemap.xml"
curl --silent "http://www.bing.com/webmaster/ping.aspx?siteMap=https://example.com/sitemap.xml"
Copying files to server...
upload: public/example-page/index.html to s3://example.com/example-page/index.html
{
    "Location": "https://cloudfront.amazonaws.com/2017-03-25/distribution/XXXXXXXXXXX/invalidation/I1XO2I27BX1UAY",
    "Invalidation": {
        "Id": "XXXXXXXXXXX",
        "Status": "InProgress",
        "CreateTime": "2018-08-13T04:31:52.659Z",
        "InvalidationBatch": {
            "Paths": {
                "Quantity": 1,
                "Items": [
                    "/example-page/"
                ]
            },
            "CallerReference": "cli-1234-56789"
        }
    }
}
<html><meta http-equiv="content-type" content="text/html; charset=UTF-8">
<head><title>Google Webmaster Tools
-
Sitemap Notification Received</title>
<meta name="robots" content="noindex, noodp">
<script src="https://ssl.google-analytics.com/urchin.js" type="text/javascript">
    </script>
<script type="text/javascript">
      _uacct="UA-12345-1";
      _utcp="/webmasters/";
      _uanchor=1;
      urchinTracker();
    </script></head>
<body><h2>Sitemap Notification Received</h2>
<br>
Your Sitemap has been successfully added to our list of Sitemaps to crawl. If this is the first time you are notifying Google about this Sitemap, please add it via  <a href="http://www.google.com/webmasters/tools/">http://www.google.com/webmasters/tools/</a>  so you can track its status. Please note that we do not add all submitted URLs to our index, and we cannot make any predictions or guarantees about
when or if they will appear.</body></html><html><body>Gracias por enviar tu Sitemap.  Únete a las <a href="/webmaster">Herramientas de administrador web de Bing</a> para ver el estado de tus Sitemaps e informes sobre tu progreso en Bing.</body></html>

Testing

Testing the new setup:

The main URL: https://example.com.

<pre class="shell">
<samp>
<span class="shell-prompt">$</span> <kbd>curl -I https://example.com</kbd>
HTTP/2 200 
etag: "XXXXXXXXX"
server: AmazonS3
x-cache: Miss from cloudfront
via: 1.1 XXXXXX.cloudfront.net (CloudFront)
(...)
</samp>
</pre>

Redirecting from http to https:

<pre class="shell">
<samp>
<span class="shell-prompt">$</span> <kbd>curl -I http://example.com</kbd>
HTTP/1.1 301 Moved Permanently
Server: CloudFront
Content-Type: text/html
Location: https://example.com/
X-Cache: Redirect from cloudfront
Via: 1.1 XXXXXXXXX.cloudfront.net (CloudFront)
(...)
</samp>
</pre>

Considerations

Why a private S3 bucket?

When serving a website from S3 bucket, each request of an object has a cost. If someone wants to harm your business, it could start an attack downloading a lot of files from different servers and you will be billed for that.

When serving them from Cloudfront, costs are much more cheaper and fast as the distributed nature of a Content Delivery Network.

References

• https://docs.aws.amazon.com/AmazonS3/
• https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html
• https://aws.amazon.com/premiumsupport/knowledge-center/secure-s3-resources/
• How do you set a default root object for subdirectories for a statically hosted website on Cloudfront?
• Why does AWS recommend against public S3 buckets?
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cnames-and-https-procedures.html
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DownloadDistS3AndCustomOrigins.html#adding-cloudfront-to-s3
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html#private-content-creating-oai

Changelog

• 2019-08-11
  • Fixed access from Cloudfront only (set default root object index.html).
  • Removed Static Website Hosting in S3 specification.