create a slim and lean docker image for your go/golang application

all well and dandy - we have our app, it is running inside a docker image - yet there is still something we could do better.

hit 'docker images' and take a look at the output

REPOSITORY   TAG        IMAGE ID       CREATED         VIRTUAL SIZE
deligo       latest     11e4c299c1e0   39 minutes ago  755.8 MB

anything bothering you? this 755MB for the image kindda bothered ME.
so much space for an executable, which is less then 1% of this.
lets see how we can improve this

of course, this problem is not something new and many have looked and found solution that satisfies their case. in most cases the solution was using an image of an extremely slim linux distribution - alpine linux

it comes with only about 5 megabytes and through its packaging system - apk - allows you to only install packages you need

now, in my original Dockerfile i was actually building the executable inside the container, but we don't really need this right now, do we? in many cases it would probably make sense - thus the image will be able to grab latest code and build and run the latest executable, but on the other hand you may not wish to have your source code out, even in a secure environment like a docker image.

so, we will just ADD our executable to docker and it will run it away.
here is one of the benefits of compiled go executables - the practically have no dependencies (alright, this is only in my silly case, but i think this is also pointed out by golang dev community as well).

so lets do this.

if we sticked with building our app inside the image we'd have put in the Dockerfile something like

RUN apk add --update go && \
    rm /var/cache/apk/*

but we won't, so we won't: we're are only adding our executable with an ADD command

ADD deligo /go/bin/

and setting the entrypoint

ENTRYPOINT /go/bin/deligo

build and run away!
by the way - here are two useful switches to docker build cmd:

--no-cache=true
--rm

no-cache is especially useful when your executable has changed. i had some problems until realized that docker uses a cache for each step when building the image - which is good, but this cache doesn't seem too smart if it cannot realize a file has changed. so now i use this cmd arg for every build
rm of course removes not needed working data created during the build, and i believe essentially not needed

so, we built the image, we start it with docker run (don't forget to supply -p 8080:8088 or whatever, if your app has IO over ports), and then...
well in my case it was something like

deligo not found

but how it may not be found when i see it in the folder on the image and i can even sh to the image and run the file and..not found!? oh

so while we copied our executable to the image, there still seem to be some dependencies to syslibs which our slim alpine linux doesn't include. we can probably research which and add them, but this will inevitably make our image bigger. may be lets just sigh heavily and do it - include the whole go build chain etc - but turns out this might not even work, because at least for me alpine's

apk --update add go

only added go v1.4, even though i see go v 1.5.1 at their packages site. i found some workarounds for this, but none worked for me.
yet i found an excellent tip on codeship.com 's site, on how to build our go app with all libs statically linked into the executable:

CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o deligo .

further in the article there were some discussions on SSL which i didn't bother to read, instead hurrying to build my app with the tip above and rebuild the docker image, and start it and...yes, it runs!

actually, no - it doesn't :( when my app tries to read/write through the ports, some mysterious message was dumped:

x509: failed to load system roots and no roots provided

what?

nicely, the first search result was this - https://github.com/docker/docker/issues/3825, and mentioning SSL, i remembered the text on codeship's site and so quickly switched to it, to find the final solution: you need to provide certificates to your app, in order to satisfy go’s x509 library

ADD ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

phew
(by the way - is it safe giving out certificates like that, any ideas?)

with this done - the rest is safe sails, and favorable winds...

yes, btw - our docker image got from 750MB to only 15 (15!!!!!) MB!!!
not too bad, eh?

docker image, ports and remote connections

soooo, very quickly i stumbled on a problem with my app i was deploying with docker.
the app opens a port and docker allows to redirect ports on the image to different ports on the host, so i was running

docker run -t -i -p 8090:8080 deligo

but i couldn't connect to 8090.
or rather netcat/nc was showing it as busy, but it wouldn't open in a browser.
so what was happening?

luckily the very first search result on duckduckgo provided the insight (stackoverflow.com):

Docker container published ports not accessible

i quickly checked and indeed in my go app i had started my http listener for 127.0.0.1 which as it turns out does not allow remote connections!
quickly changed to 0.0.0.0 and the whole chain works fine now, yeey!
 

create docker image for your go application

just wanted to give a try to docker so i gave a try J

somehow i thought that docker is a really magical tool and you just point it to your executable and it gathers all dependencies and pops the image wrapped in a pink pandelka.
well - not so J

but it really is not too difficult to make a docker image - there are just a few not too difficult steps.

my first impulse of course was to snap an ubuntu image and set up everything there manually:

docker run ubuntu:14.04 -t -i /bin/bash

and then install go, setup paths blah blah

luckily i noticed somewhere that there is a handy golang docker image already so i grabbed it and tenderly started it

docker run -t -i golang  /bin/bash

not many thing around, but enough to get us started

now, you can of course can still setup the golang image to your needs manually, but it is so much clever to have docker do this automatically for you!
for this purpose we will use a Dockerfile and build our image from it.
this will actually enable us to start as many instances of our app simultaneously in dedicated docker images with a wave of the hand.

so what do we need to put in this file?
first we need to tell docker which base image we will use; in our case this is the golang image:

FROM golang

i will deploy the app i think i mentioned in another article, which is hosted at https://github.com/esdee-git/deligo.
but for this example i won't grab it from github, but from my local drive; and i want docker to copy everything from the current folder to a specified location:

ADD . /go/src/deligo

the golang image hierarchy root is at /go, so this is where i place my stuff in my image too

now, i recently was bothered a bit for mainenance, so i started using glide (https://github.com/Masterminds/glide/) which seems a nice way for dependencies management, until golang team comes up with a native solution - if ever.
so we need to grab and install glide, and from then on we will build throuh this tool

RUN go get github.com/Masterminds/glide
RUN go build github.com/Masterminds/glide

actually this of course is not needed, if we want to only export the executable 'deligo' - for which the ADD command would've been enough, but since this is an exercise in docker lets have as many even redundant steps as bearable.

if we want to do (or rather RUN) something in a different folder we can get away with something like

RUN cd src/deligo && [some command]

but the nicer way to do it is with

WORKDIR src/deligo

which of course changes the current folder temporarily

next  we'll use glide to check what is in its glide.yaml file and download and setup dependent libraries:

RUN GO15VENDOREXPERIMENT=1 glide install
RUN GO15VENDOREXPERIMENT=1 go build

i didn't bother to setup the proper enviroment variable for this test, so i'm adding it to the command line; glide needs this so go tools are aware that we're using the new (for go 1.5) vendor approach to dependencies

finally we must tell docker what to start when the image is loaded.
remember that the value of docker is its container-ness (aahem). i.e. it is meant to run a service in a container.
of course you can run as many services in a container as you like, but really for decoupling one container=one service is probably preferable

so...

ENTRYPOINT ["./deligo"]

if you want to check how your image was built, you can comment the above and instead

ENTRYPOINT /bin/bash

which will allow you to look around and test the new environment as much as you like, but this will require to rebuild the image in order to enable the true entry point after testing

the better way is to override the entry point when starting the docker image:

docker run -t -i --entrypoint /bin/bash deligo

so after you're done exploring justrun

docker run -t -i deligo

 which will start the image with the assigned entry point

ah, lets not forget the build step prior to running, which is...ta-daaa!:

docker build --rm -t deligo .

"--rm" will auto-remove some temporary containers build during construction of our image

so this is all about this short intro to docker.
as always i'm writing this to document my own efforts, so i don't have to go searching too far for these steps :)

anyway, hope it be helpful for somebody else too - next stop: Heroku!

chaooo

simple BroadcastReceiver for Android missed calls app + tips for remote/wireless debuging on the phone

i wanted to play a bit with Android programming (again), so first i decided to try the latest AndroidStudio. I was originally Android-ing on Eclipse and gave only a quick test to the then-new AndroidStudio (neg, mostly because of Gradle probs, i think).
Now I see much has changed, AndroidStudio is very stable and doesn't get in the way - at all - even helping quite a bit.
The best improvement in the Android development environment, for me - and i don't know if it is nothing new - is the ability to run/test and debug (!) remotely - in my case wirelessly - on an actual physical device.

so no slooow Android emulator, no genymotion (which is actually good) - just write your code, immediately deploy/run/test/debug on a real device - so good!
basically what you have to do is plug the phone via usb and type

./adb tcpip 5555

then ask nicely adb to connect to your device (you must know the IP by which the device is connected to your same network, for example 222.778.1.21, but more likely something like 192.168.0.301):

./adb connect 222.778.1.21:5555

then unplug the phone, check if adb sees it via:

./adb devices

and off you go - start run/debug in AndroidStudio and select your phone/tablet/watch/car whatever from the list

this article helped a lot to clear this out - www.jessechen.net/blog/debugging-your-android-app-wirelessly-on-an-android-smartphone/

note that i don't think you actually need the apps that run your phone mentioned in this article, since it seems they only serve to show the IP of your device, which can be retrieved in other ways
 

coding the app itself was easy, except for a bug i made in the end, which caused some &^#$%&$^%$:
i open a cursor into the handy CallLog.Calls, but immediately did moveToNext(), which was jumping to the wrong record....blast.

i didn't bother for UI for the app, since everything is done behind the scenes with BroadcastReceiver and AlarmManager.
Programming was fun, there is so much info on the net that for simple things everyone should be able to write himself nowadays, programming is really quickly turning into a household skill so to say :) oh, what the future will bring with the internet of things!


just to get a glimpse of the code and approach:

we set a BroadcastReceiver for android.intent.action.PHONE_STATE and when notification arrives, we plug a listener into the call state and in its onCallStateChanged try to determine if the call was answered or not.
Then we set an alarm with AlarmManager and then we define a new receiver for the alarm event.
Now this is where we turn "professional" since the user might have reviewed the missed calls before the alarm rang, so we have to check this first, before ringing any bells for missed calls that the user is already aware of.

How? By opening the log and checking the status of missed calls - there is a flag "NEW" which is 1, if the user had not acted on the system missed calls notification, so we check if any of the call records in the log have this set, if yes - we play our own notification and set a new one, which will follow the same path of logic.
There is also another interesting flag/field there: IS_READ, but it seems it is set to 1 only when user acted on the call specifically, like calling back, from what i could gather. In any case flag new proved to be the reliable one.

i actually pushed the new product to github at https://github.com/esdee-git/Micalls, and might even try to push it to Google Play store, just to get a feel of the experience