GitLab Runners Using Containers
A local runner in a terminal on my local development machine triggered by a commit or merge request is better than running test manually from the same terminal in my IDE. But it not a very big step toward the automation required to support continuous integration. It doesn’t scale and it relies on my development box being turned on and online.
We need to consider other options for a practical executor for our runner. Our criteria are that it must scale horizontally (as opposed to queuing, which will stack up commits and merges) and it needs to be hosted where it is always available, such as on the cloud. In my case it makes sense to use AWS to host a docker image that can serve as a container for my runner.
Getting the infrastructure set up to make all of the parts work together requires many steps involving Amazon AWS service configuration, Writing a Gitlab gitlab-ci.yml script and a docker file. I did not find an end-to-end guide covering every step required to make it happen. The best guides I found still required additional tweaks and changes due to the fluid nature of cloud services. I made a few mistakes too. Hopefully someone reading this has some depth in at least one of the areas: I did not. I just did my best to remain patient, humble and determined, and I reminded myself that others had already done this many times.
The best guide I found to set things up is pretty good. If you have a free GitLab account and a free tier AWS account you can follow Valentin Despa’s guide like I did. There were only a few things that I had to adjust or change along the way.
As he points out right at the beginning, this is not a production-worthy setup. It’s more of an exploratory foray to travel down the entire path. From there you can build up to an enterprise-level setup with the appropriate infrastructure. In my case I find it useful to demonstrate the capability in action to stoke the required resources to have others come in and expand something I start.
Once I had everything set up on my AWS I ran into issues getting the same integration to work on the internally hosted Gitlab setup at my company. Specifically, the step where you register the gitlab-runner that is hosted on the Docker image to the internally hosted GitLab instance. The runner can’t find Gitlab via DNS to post the key, and I can’t seem to find it’s IP address to possibly bypass DNS. Stay tuned…