create a slim and lean docker image for your go/golang application

all well and dandy - we have our app, it is running inside a docker image - yet there is still something we could do better.

hit 'docker images' and take a look at the output

REPOSITORY   TAG        IMAGE ID       CREATED         VIRTUAL SIZE
deligo       latest     11e4c299c1e0   39 minutes ago  755.8 MB

anything bothering you? this 755MB for the image kindda bothered ME.
so much space for an executable, which is less then 1% of this.
lets see how we can improve this

of course, this problem is not something new and many have looked and found solution that satisfies their case. in most cases the solution was using an image of an extremely slim linux distribution - alpine linux

it comes with only about 5 megabytes and through its packaging system - apk - allows you to only install packages you need

now, in my original Dockerfile i was actually building the executable inside the container, but we don't really need this right now, do we? in many cases it would probably make sense - thus the image will be able to grab latest code and build and run the latest executable, but on the other hand you may not wish to have your source code out, even in a secure environment like a docker image.

so, we will just ADD our executable to docker and it will run it away.
here is one of the benefits of compiled go executables - the practically have no dependencies (alright, this is only in my silly case, but i think this is also pointed out by golang dev community as well).

so lets do this.

if we sticked with building our app inside the image we'd have put in the Dockerfile something like

RUN apk add --update go && \
    rm /var/cache/apk/*

but we won't, so we won't: we're are only adding our executable with an ADD command

ADD deligo /go/bin/

and setting the entrypoint

ENTRYPOINT /go/bin/deligo

build and run away!
by the way - here are two useful switches to docker build cmd:

--no-cache=true
--rm

no-cache is especially useful when your executable has changed. i had some problems until realized that docker uses a cache for each step when building the image - which is good, but this cache doesn't seem too smart if it cannot realize a file has changed. so now i use this cmd arg for every build
rm of course removes not needed working data created during the build, and i believe essentially not needed

so, we built the image, we start it with docker run (don't forget to supply -p 8080:8088 or whatever, if your app has IO over ports), and then...
well in my case it was something like

deligo not found

but how it may not be found when i see it in the folder on the image and i can even sh to the image and run the file and..not found!? oh

so while we copied our executable to the image, there still seem to be some dependencies to syslibs which our slim alpine linux doesn't include. we can probably research which and add them, but this will inevitably make our image bigger. may be lets just sigh heavily and do it - include the whole go build chain etc - but turns out this might not even work, because at least for me alpine's

apk --update add go

only added go v1.4, even though i see go v 1.5.1 at their packages site. i found some workarounds for this, but none worked for me.
yet i found an excellent tip on codeship.com 's site, on how to build our go app with all libs statically linked into the executable:

CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o deligo .

further in the article there were some discussions on SSL which i didn't bother to read, instead hurrying to build my app with the tip above and rebuild the docker image, and start it and...yes, it runs!

actually, no - it doesn't :( when my app tries to read/write through the ports, some mysterious message was dumped:

x509: failed to load system roots and no roots provided

what?

nicely, the first search result was this - https://github.com/docker/docker/issues/3825, and mentioning SSL, i remembered the text on codeship's site and so quickly switched to it, to find the final solution: you need to provide certificates to your app, in order to satisfy go’s x509 library

ADD ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

phew
(by the way - is it safe giving out certificates like that, any ideas?)

with this done - the rest is safe sails, and favorable winds...

yes, btw - our docker image got from 750MB to only 15 (15!!!!!) MB!!!
not too bad, eh?

docker image, ports and remote connections

soooo, very quickly i stumbled on a problem with my app i was deploying with docker.
the app opens a port and docker allows to redirect ports on the image to different ports on the host, so i was running

docker run -t -i -p 8090:8080 deligo

but i couldn't connect to 8090.
or rather netcat/nc was showing it as busy, but it wouldn't open in a browser.
so what was happening?

luckily the very first search result on duckduckgo provided the insight (stackoverflow.com):

Docker container published ports not accessible

i quickly checked and indeed in my go app i had started my http listener for 127.0.0.1 which as it turns out does not allow remote connections!
quickly changed to 0.0.0.0 and the whole chain works fine now, yeey!
 

create docker image for your go application

just wanted to give a try to docker so i gave a try J

somehow i thought that docker is a really magical tool and you just point it to your executable and it gathers all dependencies and pops the image wrapped in a pink pandelka.
well - not so J

but it really is not too difficult to make a docker image - there are just a few not too difficult steps.

my first impulse of course was to snap an ubuntu image and set up everything there manually:

docker run ubuntu:14.04 -t -i /bin/bash

and then install go, setup paths blah blah

luckily i noticed somewhere that there is a handy golang docker image already so i grabbed it and tenderly started it

docker run -t -i golang  /bin/bash

not many thing around, but enough to get us started

now, you can of course can still setup the golang image to your needs manually, but it is so much clever to have docker do this automatically for you!
for this purpose we will use a Dockerfile and build our image from it.
this will actually enable us to start as many instances of our app simultaneously in dedicated docker images with a wave of the hand.

so what do we need to put in this file?
first we need to tell docker which base image we will use; in our case this is the golang image:

FROM golang

i will deploy the app i think i mentioned in another article, which is hosted at https://github.com/esdee-git/deligo.
but for this example i won't grab it from github, but from my local drive; and i want docker to copy everything from the current folder to a specified location:

ADD . /go/src/deligo

the golang image hierarchy root is at /go, so this is where i place my stuff in my image too

now, i recently was bothered a bit for mainenance, so i started using glide (https://github.com/Masterminds/glide/) which seems a nice way for dependencies management, until golang team comes up with a native solution - if ever.
so we need to grab and install glide, and from then on we will build throuh this tool

RUN go get github.com/Masterminds/glide
RUN go build github.com/Masterminds/glide

actually this of course is not needed, if we want to only export the executable 'deligo' - for which the ADD command would've been enough, but since this is an exercise in docker lets have as many even redundant steps as bearable.

if we want to do (or rather RUN) something in a different folder we can get away with something like

RUN cd src/deligo && [some command]

but the nicer way to do it is with

WORKDIR src/deligo

which of course changes the current folder temporarily

next  we'll use glide to check what is in its glide.yaml file and download and setup dependent libraries:

RUN GO15VENDOREXPERIMENT=1 glide install
RUN GO15VENDOREXPERIMENT=1 go build

i didn't bother to setup the proper enviroment variable for this test, so i'm adding it to the command line; glide needs this so go tools are aware that we're using the new (for go 1.5) vendor approach to dependencies

finally we must tell docker what to start when the image is loaded.
remember that the value of docker is its container-ness (aahem). i.e. it is meant to run a service in a container.
of course you can run as many services in a container as you like, but really for decoupling one container=one service is probably preferable

so...

ENTRYPOINT ["./deligo"]

if you want to check how your image was built, you can comment the above and instead

ENTRYPOINT /bin/bash

which will allow you to look around and test the new environment as much as you like, but this will require to rebuild the image in order to enable the true entry point after testing

the better way is to override the entry point when starting the docker image:

docker run -t -i --entrypoint /bin/bash deligo

so after you're done exploring justrun

docker run -t -i deligo

 which will start the image with the assigned entry point

ah, lets not forget the build step prior to running, which is...ta-daaa!:

docker build --rm -t deligo .

"--rm" will auto-remove some temporary containers build during construction of our image

so this is all about this short intro to docker.
as always i'm writing this to document my own efforts, so i don't have to go searching too far for these steps :)

anyway, hope it be helpful for somebody else too - next stop: Heroku!

chaooo