How to Tell if a Webpage Can Also Be Delivered Gzipped - Command Line With Curl

Overview

First of all, why would you want to have Gzipped content? Let’s have a look at the HTTP protocol to understand how this works and then the command to test it is working.

Understanding gzip compression is also an important factor when analyzing web-pages with speed measurement tools like Google’s PageSpeed Insights, as it has a special Enable Compression rule to detect that compressible resources were served with compression, and get rid of the following message:

Enable compression

Compressing resources with gzip or deflate can reduce the number of bytes sent over the network.

Theory

The HTTP protocol has the capability to transfer compressed content, this improves transfer speed and bandwidth utilization.

Negotiation in HTTP goes like this:

1. First the web client, makes a request to the server notifying which compression schemes it supports. This is done by including a list of tokens in the HTTP request. Compression in HTTP can be done in two ways:

   • Transfer-Encoding header field:
     • Used at lower level.
     • Field: TE.
   • Content-Encoding header:
     • Used at higher level and the most common one.
     • Field: Accept-Encoding. Example: Accept-Encoding: gzip, deflate

2. If the server supports any of the requested methods, it delivers the compressed data using Content-Encoding or Transfer-Encoding field in the HTTP response with the used schemes. - Example: Content-Encoding: gzip

Browsers that do not support compliant compression method will download data without compression.

The most common compression schemes include gzip, Deflate and Brotli.

Commands

The command line utility we use to test for gzip compression is curl, a flexible tool to transfer data from a server.

To just get the header info, we use the -I parameter. (equivalent to --head)

$ curl -I http://example.com

Which gives us:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
Date: Sat, 18 Aug 2018 18:04:04 GMT
Etag: "1541025663"
Expires: Sat, 25 Aug 2018 18:04:04 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (mic/9B22)
X-Cache: HIT
Content-Length: 1270

Note the above Content-Length of 1270.

So now we test the web page was served using compression, including the Content-Encoding header in the request with the -H parameter:

$ curl -I http://example.com -H "Accept-Encoding: gzip,deflate"

And expect to have Content-Encoding in the response:

HTTP/1.1 200 OK
Content-Encoding: gzip
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html
Date: Sat, 18 Aug 2018 18:04:32 GMT
Etag: "1541025663+gzip"
Expires: Sat, 25 Aug 2018 18:04:32 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (mic/9A89)
X-Cache: HIT
Content-Length: 606

Content-Length is 606, which gives us a content that is 48% the size of the uncompressed page size.

To just get the web page download size of the compressed and uncompressed content:

$ curl http://example.com/ --silent -H "Accept-Encoding: gzip,deflate" --write-out "%{size_download}\n" --output /dev/null
606
$ curl http://example.com/ --silent --write-out "%{size_download}\n" --output /dev/null
1270

Also if your web-pages are effectively using gzip compression, in PageSpeed Insights you will get the following message: Enable compression: You have compression enabled.

References

• https://en.wikipedia.org/wiki/HTTP_compression
• https://developers.google.com/speed/docs/insights/EnableCompression
• https://tools.ietf.org/html/rfc2616
• How can I tell if my server is serving GZipped content?