Deploy and configure a local Docker caching proxy

Recently I was looking into caching for Docker layers downloading for the Fabric8 development environment, to allow me to trash the vms where my Docker daemon was running and still avoiding me to re-download basic images each single time I recreate my vm.

As usual, I tried to hit Google first, and was pointed to these couple of pages:

http://unpoucode.blogspot.it/2014/11/use-proxy-for-speeding-up-docker-images.html

and

https://blog.docker.com/2015/10/registry-proxy-cache-docker-open-source/

Since I use quite often Jerome Petazzoni’s approach for a transparent Squid + iptables to cache and sniff simple http traffic (usually http invocation from java programs), I’ve found that the first solution based made sense, so I have tried that first.

It turned out that the second link was what I was looking for; but I have still spent some good learning hours with the no longer working suggestions from the first one, learning the hard way that Squid doesn’t play that nice with Amazon’s Cloudfront CDN, used by Docker Hub. But I have to admit that it’s been fun.
Now I know how to forward calls to Squid to hit an intermediate interceptors that mangles with query params, headers and everything else.
I couldn’t find a working combination for Cloudfront but I am now probably able to reproduce the infamous Cats Proxy Prank. =)

Anyhow, as I was saying, what I was really looking for is that second link that shows you how to setup an intermediate Docker proxy, that your Docker daemon will try to hit, before defaulting to the usual Docker Hub public servers.

Almost everything that I needed was in that page, but I have found the information to be a little more cryptic that needed.

The main reason for that is because that example assumes I need security (TLS), which is not really my case since the proxy is completely local.

Additionally, it shows how to configure you Docker Registry using YAML configuration. Again, not really complex, but indeed more than needed.

Yes, because what you really need to bring up the simplest local (not secured) Docker proxy is this oneliner:

docker run -p 5000:5000 -d --restart=always --name registry   \
  -e REGISTRY_PROXY_REMOTEURL=http://registry-1.docker.io \
  registry:2

The interesting part here is that registry image, supports a smart alternative way to forward configuration to it, that saves you from passing it a YAML confguration file.

The idea, described here, is that if you follow a naming convention for the environment variables, that reflects the hierarchy of the YAML tree, you can turn something like:

...
proxy:
  remoteurl: http://registry-1.docker.io
...

That you should write in a .yaml file and pass to the process in this way:

docker run -d -p 5000:5000 --restart=always --name registry \
  -v `pwd`/config.yml:/etc/docker/registry/config.yml \
  registry:2

Into the much more conventient -e REGISTRY_PROXY_REMOTEURL=http://registry-1.docker.io runtime environment variable!

Let’s improve the example a little, so that we also pass our Docker proxy a non-volatile storage location for the cached layers, so that we are not losing them between invocations:

docker run -p 5000:5000 -d --restart=always --name registry   \
  -e REGISTRY_PROXY_REMOTEURL=http://registry-1.docker.io \
  -v /opt/shared/docker_registry_cache:/var/lib/registry \
  registry:2

Now we have everything we needed to save a good share of bandwidth, each time we need to get some Docker image that had already passed through our local proxy.

The only remaining bit is to tell our Docker daemon to be aware of the new proxy:

# update your docker daemon config, according to your distro
# content of my `/etc/sysconfig/docker` in Fedora 23
OPTIONS=" --registry-mirror=http://localhost:5000"

Reload (or restart) your Docker daemon and you are done! Just be aware that if you restart the daemon you might need also to re-start the Registry container, if you ware working on a single node.

An interesting discovery, it’s been learning that Docker daemon doesn’t break if it cannot find the specified registry-mirror. So you can add the configuration and forget about it, knowing that your interaction with Docker Hub will just benefit of possible hits your caching proxy, assuming it’s running.

You can see it working with the following tests:

docker logs -f registry

will log all the outgoing download requests, and once the set of requrests that compose a single pull image operation will be completed, you will also be able to check that the image is now completely served by your proxy with this invocation:

curl http://localhost:5000/v2/_catalog
# sample output
{"repositories":["drifting/true"]}

The article would be finished, but since I feel bad to show how to disable security on the internet, here’s also a very short and fully working and tested example of how to implement the same with TLS enabled:

# generate a self signed certificate; accept default for every value a part from Common Name where you have to put your box hostname
mkdir -p certs && openssl req  -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key  -x509 -days 365 -out certs/domain.crt

# copy to the locally trusted ones, steps for Fedora/Centos/RHEL
sudo cp certs/domain.crt /etc/pki/ca-trust/source/anchors/

# load the newly added certificate
sudo update-ca-trust enable

# run the registy using those keys that you have generated, mounting the files inside the container
docker run -p 5000:5000 --restart=always --name registry \
  -v `pwd`/certs:/certs \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
  registry:2

# now you just need to remember that you are working in https, so you need to use that protocol in your docker daemon configuration, instead of plain http; also use that when you interact with the API in curl