Creating SSL/TLS Certificates for Docker and Docker Swarm

Reading time ~17 minutes

Docker1 has one of the most gentle learning slopes of a new technology to enter the mainstream in a long time. A developer can get up and running in a very short amount of time2 and begin realizing value almost immediately with Docker, but the hard part comes when trying to secure the new technology for use in a production like environment. Production has a much higher standard when it comes to availability, security, and repeatability. This can lead to problems as the differences between the development and production environment are both:

  • Fairly complex to replicate in a secure way: It is not feasible to pass around the private key for your production certificate in the name of development environment automation. On the other hand, to generate a full CA and all of the certificates and keys required can be daunting.
  • Functionally quite a large delta: The difference between an insecure, non-tls environment and an SSL one can be significant. For instance, at this exact exact point in time it appears that Ansible does not yet support a TLS enabled Docker host3.

In order to attack this problem, we should attempt to replicate the prod environment when it is feasible and especially if it is easy and cheap. To this end, let’s create the full certificate chain needed to run a secure Docker Swarm4 cluster. I think you will find that it is both easy and cheap :)

The script

This is a bash script I used5 that will output everything within the directory it is run. It accomplishes the following things:

  • Creates a Certificate Authority
  • Creates a cert/key for Docker Swarm (supporting both client and server auth)
  • Generates 3 certificates for the individual Docker hosts with SAN IPs

It is required to set a config as we need to add a SAN IP Address entry to the certificate and CSR. This is required because without it, Swarm will spit out the following error:

ERRO[0282] Get https://10.100.199.201:2376/v1.15/info: x509: cannot validate certificate for 10.100.199.201 because it doesn't contain any IP SANs

Disclaimer: I am no SSL wizard and so some of the settings in the openssl.cnf may be insecure, not needed, or even both. In addition, you can see that this is totally insecure as

  • the password is in the script
  • the passwords are removed from the keys
  • many other reasons

Please don’t use these exact script or the generated certs for production use!

gen_ssl.sh

#!/bin/bash

export OPENSSL_CONF=openssl.cnf


echo 'Creating CA (ca-key.pem, ca.pem)'
echo 01 > ca.srl
openssl genrsa -des3 -passout pass:password -out ca-key.pem 2048
openssl req -new -passin pass:password \
        -subj '/CN=Non-Prod Test CA/C=US' \
        -x509 -days 365 -key ca-key.pem -out ca.pem


echo 'Creating Swarm certificates (swarm-key.pem, swarm-cert.pem)'
openssl genrsa -des3 -passout pass:password -out swarm-key.pem 2048
openssl req -passin pass:password -subj '/CN=dockerswarm01' -new -key swarm-key.pem -out swarm-client.csr
echo 'extendedKeyUsage = clientAuth,serverAuth' > extfile.cnf
openssl x509 -passin pass:password -req -days 365 -in swarm-client.csr -CA ca.pem -CAkey ca-key.pem -out swarm-cert.pem -extfile extfile.cnf
openssl rsa -passin pass:password -in swarm-key.pem -out swarm-key.pem

# Set the default keys to be Swarm
cp -rp swarm-key.pem key.pem
cp -rp swarm-cert.pem cert.pem

echo 'Creating host certificates (dockerhost01-3-key.pem, dockerhost01-3-cert.pem)'
openssl genrsa -passout pass:password -des3 -out dockerhost01-key.pem 2048
openssl req -passin pass:password -subj '/CN=dockerhost01' -new -key dockerhost01-key.pem -out dockerhost01.csr
openssl x509 -passin pass:password -req -days 365 -in dockerhost01.csr -CA ca.pem -CAkey ca-key.pem -out dockerhost01-cert.pem -extfile openssl.cnf
openssl rsa -passin pass:password -in dockerhost01-key.pem -out dockerhost01-key.pem

openssl genrsa -passout pass:password -des3 -out dockerhost02-key.pem 2048
openssl req -passin pass:password -subj '/CN=dockerhost02' -new -key dockerhost02-key.pem -out dockerhost02.csr
openssl x509 -passin pass:password -req -days 365 -in dockerhost02.csr -CA ca.pem -CAkey ca-key.pem -out dockerhost02-cert.pem -extfile openssl.cnf
openssl rsa -passin pass:password -in dockerhost02-key.pem -out dockerhost02-key.pem

openssl genrsa -passout pass:password -des3 -out dockerhost03-key.pem 2048
openssl req -passin pass:password -subj '/CN=dockerhost03' -new -key dockerhost03-key.pem -out dockerhost03.csr
openssl x509 -passin pass:password -req -days 365 -in dockerhost03.csr -CA ca.pem -CAkey ca-key.pem -out dockerhost03-cert.pem -extfile openssl.cnf
openssl rsa -passin pass:password -in dockerhost03-key.pem -out dockerhost03-key.pem

# We don't need the CSRs once the cert has been generated
rm -f *.csr

openssl.cnf

#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

# This definition stops the following lines choking if HOME isn't
# defined.
HOME			= .
RANDFILE		= $ENV::HOME/.rnd
oid_section		= new_oids
extensions		= v3_req

[ new_oids ]
tsa_policy1 = 1.2.3.4.1
tsa_policy2 = 1.2.3.4.5.6
tsa_policy3 = 1.2.3.4.5.7

####################################################################
[ ca ]
default_ca	= CA_default		# The default ca section

####################################################################
[ CA_default ]
dir		= ./tls		# Where everything is kept
certs		= $dir/certs		# Where the issued certs are kept
crl_dir		= $dir/crl		# Where the issued crl are kept
database	= $dir/index.txt	# database index file.
new_certs_dir	= $dir/newcerts		# default place for new certs.
certificate	= $dir/cacert.pem 	# The CA certificate
serial		= $dir/serial 		# The current serial number
crlnumber	= $dir/crlnumber
crl		= $dir/crl.pem 		# The current CRL
private_key	= $dir/private/cakey.pem# The private key
RANDFILE	= $dir/private/.rand	# private random number file
x509_extensions	= usr_cert		# The extentions to add to the cert
name_opt 	= ca_default		# Subject Name options
cert_opt 	= ca_default		# Certificate field options
default_days	= 365			# how long to certify for
default_crl_days= 30			# how long before next CRL
default_md	= default		# use public key default MD
preserve	= no			# keep passed DN ordering
policy		= policy_match

[ policy_match ]
countryName		= match
stateOrProvinceName	= match
organizationName	= match
organizationalUnitName	= optional
commonName		= supplied
emailAddress		= optional

[ policy_anything ]
countryName		= optional
stateOrProvinceName	= optional
localityName		= optional
organizationName	= optional
organizationalUnitName	= optional
commonName		= supplied
emailAddress		= optional

####################################################################
[ req ]
default_bits		= 1024
default_keyfile 	= privkey.pem
distinguished_name	= req_distinguished_name
attributes		= req_attributes
x509_extensions	= v3_ca	# The extentions to add to the self signed cert
string_mask = utf8only
req_extensions = v3_req # The extensions to add to a certificate request

[ req_distinguished_name ]
countryName			= Country Name (2 letter code)
countryName_default		= AU
countryName_min			= 2
countryName_max			= 2
stateOrProvinceName		= State or Province Name (full name)
stateOrProvinceName_default	= Some-State
localityName			= Locality Name (eg, city)
0.organizationName		= Organization Name (eg, company)
0.organizationName_default	= Internet Widgits Pty Ltd
organizationalUnitName		= Organizational Unit Name (eg, section)
commonName			= Common Name (e.g. server FQDN or YOUR name)
commonName_max			= 64
emailAddress			= Email Address
emailAddress_max		= 64

[ req_attributes ]
challengePassword		= A challenge password
challengePassword_min		= 4
challengePassword_max		= 20
unstructuredName		= An optional company name

[ usr_cert ]
basicConstraints=CA:FALSE
nsComment			= "OpenSSL Generated Certificate"
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ v3_ca ]
subjectAltName = @alt_names
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
basicConstraints = CA:true

[ crl_ext ]
authorityKeyIdentifier=keyid:always

[ alt_names ]
# The IPs of the Docker and Swarm hosts
IP.1 = 10.100.199.200
IP.2 = 10.100.199.201
IP.3 = 10.100.199.202
IP.4 = 10.100.199.203

Installation

Once you have generated the TLS keys and certificates they must be installed on the target machine. I prefer to just copy the certificate and the key files into /etc/pki/tls/certs/ and /etc/pki/tls/private/ respectively. Once they are installed, you can then fire up your Docker and Swarm daemons like so:

Docker

/usr/bin/docker -d \
  --tlsverify \
  --tlscacert=/etc/pki/tls/certs/ca.pem \
  --tlscert=/etc/pki/tls/certs/dockerhost01-cert.pem \
  --tlskey=/etc/pki/tls/private/dockerhost01-key.pem \
  -H tcp://0.0.0.0:2376

Swarm

/usr/local/bin/swarm manage \
  --tlsverify \
  --tlscacert=/etc/pki/tls/certs/ca.pem \
  --tlscert=/etc/pki/tls/certs/swarm-cert.pem \
  --tlskey=/etc/pki/tls/private/swarm-key.pem  \
  --discovery file:///etc/swarm_config \
  -H tcp://0.0.0.0:2376

Using the TLS enabled Docker daemon

Now in order to use the Docker daemon, you will have to present a client cert that was generated from the same CA as the certificate Docker/Swarm is using. We have generated one here and more can be made if needed. Set the following environment variables in order to tell the Docker client what to use for the TLS config:

export DOCKER_HOST=tcp://dockerswarm01:2376
export DOCKER_CERT_PATH="`pwd`"
export DOCKER_TLS_VERIFY=1

This will now enable the Docker client to communicate ‘securely’ with Docker Swarm and Docker Swarm to communicate securely with the Docker nodes behind it.

  1. https://www.docker.com ↩

  2. http://goo.gl/QlZ5qv ↩

  3. https://github.com/ansible/ansible/issues/10032 ↩

  4. https://github.com/docker/swarm ↩

  5. https://github.com/technolo-g/docker-swarm-demo/blob/master/bin/gen_ssl.sh ↩

Read More

Update from Matt

Matt has been working on big art recently, including Double Diamond and Moonrock Mountain. They are both large-scale sculptures that incorporate everything he has learned throughout his career. Continue reading

Containerizing local commands

Published on August 28, 2018

Making HTTP GET and POST Requests to Docker Hub in a Jenkinsfile

Published on August 12, 2017