Setting up Gitlab to AutoDeploy a website

Posted by Michael Osburn on Wed 08 February 2017

Recently I have been looking more at GitLab and it's build in CICD pipelines and decided to see what it takes to push out a pelican based website though this process. After following several documents I found the process requires a few things to be in place first.

  • GitLab runners need to be configured to listen to the server for a project
  • a .gittlab-ci.yml file needs to be created to do the build and deployment
  • Setup a ssh deployment key to the restricted user on the shared server
  • Finally, I had to set some project variables in place to handle the ssh key pairs and host ssh fingerprint

As I already maintain a gitlab server with multiple builders, the first part was already done for me but in the event that it is not already completed for your installation, simply spin up a new host and run though this ansible playbook https://github.com/haroldb/ansible-gitlab-runner and get your gitlab_runner_registration_token from the page at https://your.gitlab.server/admin/runners.

Since I am generating this project with Pelican, I chose the easy method of using the built in .gitlab-ci.yml template from the Set up CI link on your main project page. Note: this link only works if you do not have an existing .gitlab-ci.yml file in the root of your git repo.

Once you are in the editor for .gitlab-ci.yml select the template for pelican and commit this file. After a few minutes, this will produce a broken build as the default template does not actually produce a working build, but it does allow you to validate that a commit to master will trigger a build to the runner.

This is the full template that is included with gitlab, however alpine is missing quite a few utilities and dependencies for a modern version of pelican.

# This file is a template, and might need editing before it works on your project.
# Full project: https://gitlab.com/pages/pelican
image: python:2.7-alpine

pages:
  script:
  - pip install -r requirements.txt
  - pelican -s publishconf.py
  artifacts:
    paths:
    - public/

As it turns out, you need to add a lot more to this file in order to create a full working build. After way to many iterations to get it all working, although this needs a major rewrite using docker best practices, here is the result of my working .gitlab-ci.yaml file:

image: python:2.7-alpine

pages:
  script:
  - apk add --update alpine-sdk readline-dev bash ncurses ncurses-dev
  - pip install -r requirements.txt
  - pelican-themes -l
  - pelican -s publishconf.py
  artifacts:
    paths:
    - public/

deploy:
  stage: deploy
  script:
    - apk add --update rsync openssh
    - apk add --update alpine-sdk readline-dev bash ncurses ncurses-dev
    - pip install -r requirements.txt
    - pelican -s publishconf.py
    - echo "$deploy_key" > deploy_key
    - chmod 0600 deploy_key
    - mkdir ~/.ssh
    - echo "$host_ssh_key" > ~/.ssh/known_hosts
    - rsync -avrte 'ssh -i deploy_key' output/* mosburn@mosburn.com:~/michaelosburn.com/
    - rm -f deploy_key

Creating the ssh-keypairs was a bone stock ssh-keygen and copying id_rsa.pub over to ~/.ssh/authorized_keys on the remote server. Of note in this, I have two variable setup in the gitlab server to provide the contents of $deploy_key and $host_ssh_key until I get this rewritten using my vault setup.

If I was to do this project again I would make a few changes to the infrastructure in general. For one instead of hosting the website on a server, I would push the deployment to a S3 bucket and use CloudFront to host the actual website in a serverless fashion. I have yet to get around to migrate this website over to AWS simply because I do not have the traffic to warrant a monthly charge instead of paying my host every two years a flat rate. Additionally, as my day job is DevOps, having one part of my infrastructure someplace that I do not have to get up in the middle of the night to fix and have some one else do that for me means that I can potentially get more sleep and will not have my website down until some one points out to me that it is broken.