CI/CD with Gitlab

What is CI/CD?

CI/CD stands for continuous integration and continuous delivery (or deployment). These practices essentially automate the integration of code from different developers and the deployment of the new changes.

CI/CD reduces the time from when a feature is completed on a developer's machine, to when the feature is deployed and reaches the customer, all whilst maintaining code quality. By streamlining the integration and deployment process, a business can react to changing needs, new features, or fix bugs very quickly and have the final product deployed to customers as soon as the code has been finalised.

Continuous integration can help ensure code quality. Thorough tests can be written beforehand and can be set to run every time a new change is pushed to the repository. These tests can ensure functionality or enforce code style (and thus improve maintainability) with tools like Pylint for Python.

Continuous delivery may involve writing scripts that are executed after the testing phase passes. The aim is to deploy the new changes to production quickly, and this step is often triggered manually after a human review.

Continuous deployment takes this one step further and automates the deployment process in a sustainable way. This may involve creating a thorough testing and reviewing process, while also having a quick way to revert changes in case a bug slips through.

GitLab's CI/CD Pipeline

.gitlab-ci.yml

GitLab's CI/CD Pipelines are configured by adding a YAML file called .gitlab-ci.yml to the project's root. This file determines

• What to execute using the GitLab Runner
• What decisions to make when specific conditions are encountered

Daruweb's .gitlab-ci.yml will be provided in the next section as an example.

Pipelines

A pipeline consists of

• Jobs, which define what to do. For example, jobs that compile or test the code
• Stages, which define when to run the jobs. For example, the test stage should run after the build stage.

Usually, a pipeline only proceeds to the next stage once all jobs in the current stage succeed. Otherwise, the pipeline terminates early. For example, if any job in the build stage fails, the test stage will not run.

A typical pipeline might consist of four stages, executed in this order:

1. A build stage, with a job that compiles the code
2. A test stage, with jobs that test the code
3. A staging stage, with a job that deploys to staging
4. A production stage, with a job that deploys to production

Further Reading

The information in the secion above is based on the CI/CD section of GitLab's Docs.

Documentation about the .gitlab-ci.yml file can be found here.

Further documentation about GitLab Pipelines can be found here.

GitLab Runners run the jobs in the pipeline. Further documentation can be found here.

Daruweb

This website (Daruweb) uses GitLab's CI/CD pipeline to ensure changes can be pushed to production quickly and easily. Daruweb is containerised in a Docker container. To deploy a change without CI/CD, one would have to manually

1. Build the container
2. Push the container to Docker Hub
3. SSH into the staging VM
4. Stop the existing container
5. Remove the existing container
6. Pull the new image from Docker Hub
7. Start a new container with the updated image
8. Repeat steps 3 to 7 with the production VM once the changes are tested and verified

With some simple scripting and the use of GitLab's Pipelines, this entire process can be done in one click straight from GitLab's web GUI.

Daruweb utilises a build and deploy stage with an additional before stage that sets up the runner's environment. Steps 1 and 2 are done in the build stage and are automatically executed whenever a change is pushed to GitLab.

Once the build succeeds, the developer can choose to deploy to staging (or UAT in this case) or production via a single click on GitLab's web interface. Once the deploy stage is triggered, the runner copies and executes a script from the repository that performs steps 3 and onwards onto the respective VM. One advantage of this last step is that the script can be modified without needing to SSH into the VM.

Daruweb's .gitlab-ci.yml

The following example is the actual .gitlab-ci.yml file used by Daruweb as of September 2020. Comments have been added to better explain the contents of the file. Sensitive data like passwords and keys can be configured as environmental variables from within GitLab. One such example is the key stored in $daruweb.

# Use the official Docker image
image: docker:latest

services:
  - docker:dind

# before_script runs before the other jobs and is used to set up the runner's environment
before_script:
  # Log in to Docker
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY


  # Install ssh-agent if not already installed, it is required by Docker
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'

  # Run ssh-agent (inside the build environment)
  - eval $(ssh-agent -s)

  # Add the SSH key stored in the daruweb variable to the agent store
  # tr is used to fix line endings for ed25519 keys
  - echo "$daruweb" | tr -d '\r' | ssh-add -

  # Create the SSH directory and give it the right permissions
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh

  # ssh-keyscan scans the keys of the UAT and production servers
  - ssh-keyscan uat.darumatic.com >> ~/.ssh/known_hosts
  - ssh-keyscan darumatic.com >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts

# This job builds the Docker image and pushes it up to Docker Hub
build:
  stage: build

  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

# This job deploys to the UAT server
deploy-uat:
  stage: deploy

  script:
    # Copy the update_container script up to UAT
    - scp update_container.sh [email protected]:~

    # SSH into UAT, login to Docker and run the update_container script
    - ssh [email protected] "docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY && ~/update_container.sh $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

# This manual job deploys to the production server
deploy-prod:
  stage: deploy

  script:
    # Copy the update_container script up to prod
    - scp update_container.sh [email protected]:~

    # SSH into prod, login to Docker and run the update_container script
    - ssh [email protected] "docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY && ~/update_container.sh $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

  # Set deploy-prod to be manual so it can be triggered on GitLab's web GUI
  when: manual

Conclusion

GitLab's Pipelines allows CI/CD principles to be set up and applied to any project. Daruweb uses this feature to deploy new changes to production quickly and effortlessly.