8. Github Actions CI-CD Introduction

Github secrets and variables

They get combined into the .env file made using workflows and can be accessed from the server.

Secrets

You can't read these after setting or updating them and they can't be used in commands (they get censored)
You can set these at https://github.com/fiitka/SmartRent/settings/secrets/actions

COMPANY_DJANGO_SECRET_KEY
COMPANY_MYSQL_PASSWORD
COMPANY_MYSQL_ROOT_PASSWORD
PRESENTATION_DJANGO_SECRET_KEY
PRESENTATION_MYSQL_PASSWORD
PRESENTATION_MYSQL_ROOT_PASSWORD
WORKFLOW_PROD_SERVER_CERT
WORKFLOW_PROD_SERVER_KEY
WORKFLOW_PROD_SSH_PRIVATE

Variables

You can read these at https://github.com/fiitka/SmartRent/settings/secrets/actions

COMPANY_MYSQL_DATABASE
COMPANY_MYSQL_USER
NGINX_ALLOW_EXTERNAL_IP_RANGE
NGINX_ALLOW_SERVER_DOMAIN_COMPANY - first domain will be prepended with dev branch name except production, other space-delimited values will be ignored
NGINX_ALLOW_SERVER_DOMAIN_PRESENTATION - first domain will be prepended with dev branch name except production, other space-delimited values will be ignored
PRESENTATION_MYSQL_DATABASE
PRESENTATION_MYSQL_USER
WORKFLOW_PROD_SERVER_IP
WORKFLOW_PROD_SSH_PORT

First time setup on a new server

Make sure to update the Github secrets on site and workflow .yml file based on .env.example in root of the repo

Make sure to create SFTP and SSH credentials on the production server

Install Docker engine and Docker Compose
Create the github-actions user. You can specify the home directory while creating it:

sudo useradd -m -d /home/github-actions -s /bin/bash github-actions

Set a password for the user:

sudo passwd github-actions

Grant the user access to Docker. This involves creating the docker group, making sure Docker uses the correct group and adding the user to it:

sudo groupadd docker
sudo usermod -aG docker github-actions

Set up SFTP and SSH with a keypair-only access. Edit sudo vi /etc/ssh/sshd_config and add:

Match User github-actions
    ChrootDirectory /home/github-actions
    X11Forwarding no
    AllowTcpForwarding no
    PasswordAuthentication no
    PubkeyAuthentication yes

Ensure the home directory has the correct permissions:

sudo chown github-actions:github-actions /home/github-actions
sudo chmod 755 /home/github-actions

Create a new keypair for the github-actions user, and allow access with it:

sudo mkdir -p /home/github-actions/.ssh
sudo ssh-keygen -t rsa -b 4096 -C "github-actions@launchrentals" -f /home/github-actions/.ssh/id_rsa -N ""
sudo chown github-actions:github-actions /home/github-actions/.ssh/id_rsa
sudo chmod 600 /home/github-actions/.ssh/id_rsa
sudo chmod 644 /home/github-actions/.ssh/id_rsa.pub
sudo cp /home/github-actions/.ssh/id_rsa.pub /home/github-actions/.ssh/authorized_keys
sudo chmod 644 /home/github-actions/.ssh/authorized_keys

Restart Docker to apply new docker group and test docker with hello-world + cleanup:

sudo service docker restart # or sudo reboot
sudo su - github-actions
docker run hello-world
docker rm -f $(docker ps -a -q)
docker rmi $(docker images -q)

After these steps, restart the SSH service:

sudo service sshd restart

Make sure your server has netstat:

sudo apt install net-tools

Setup gateway nginx reverse proxy docker deployment manually, which will distribute traffic to other deployments based on branch name: see gateway folder

Currently there is no workflow to set this up as its not deemed necessary, only to update it using update.sh

exit # from the ssh to your machine
# Make sure you're in the gateway folder
cd gateway
# .env will be made automatically during first github workflow run (see main.yml ctrl+f _gateway)
# cp .env.example .env
# Set values common for all deployments
# - all domain names will be prepended with "$BRANCH_NAME-" except for "develop" which will be mapped directly on the domains
# - only provide a single domain name for each relevant environment variable
# vi .env
cp -r ../../nginx_ssl gateway
tar -czf gateway.tgz gateway
sftp tpprod
    put gateway.tgz
    exit
ssh tpprod
    tar -xzf gateway.tgz
    mv gateway _gateway
    cd _gateway
    # Make sure you have no existing deployments on host ports 80 and 443
    sh update.sh

Disable MotD for clean ssh logs and exit out of the server

touch ~/.hushlogin
exit

Add the SSH private key as a secret in the Github repo settings (these are not in the .env.example file as they are workflow-specific and pointless on local machines):

Name: WORKFLOW_PROD_SSH_PRIVATE_KEY (in secrets)
- Value: The contents of /home/github-actions/.ssh/id_rsa
Name: WORKFLOW_PROD_SERVER_IP (in variables)
- Value: The public IP address of the production server
Name: WORKFLOW_PROD_SSH_PORT (in variables)
- Value: The port you configured in the sshd_config file

Ideally also update certificate secrets for HTTPS

Name: WORKFLOW_PROD_SERVER_CERT
- Value: Copy the contents of .cert file generated by certbot certonly
Name: WORKFLOW_PROD_SERVER_KEY
- Value: Copy the contents of .key file generated by certbot certonly

Add npm if running on PowerPC to pre-install express in a workflow step from outside Docker (for now)

!! Make sure the versions of node and npm listed below match all frontend versions specified in their package.json files !!

# Make available for github actions user
sudo su ga
cd ~
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
nvm install 20.18.0
nvm use 20.18.0
npm install -g npm@10.8.2
npm -v
node -v
exit

Add root privileges for cleanup operations (due permission issues from to Docker volume bind running as root)

visudo
github-actions ALL=(ALL:ALL) NOPASSWD: ALL

Github Actions main.yml File Sample


# Branches where the workflow will run
on:
  push:
    branches:
      - "*"
      - "!wiki"
      - "!docs"

jobs:
  deploy_job:
    runs-on: ubuntu-latest
    name: build-deploy
    concurrency:
      group: one-workflow-at-a-time 
      cancel-in-progress: false
    steps:
      - name: Checkout the repository
        uses: actions/checkout@v2
        
      - name: Setup Node version
        uses: actions/setup-node@v4
        with:
          node-version: ...

      - name: Load and adjust environment variables from Github secrets
        run: |
          set -x

      - name: Copy SSL certificates from Github secrets
        run: |
          set -x

      - name: Setup target SSH credentials
        run: |
          set -x

      - name: PowerPC Target only - Install and build Nuxt applications (prebuilding, Nuxt dependencies not supported) using x64
        run: |
          cd presentation_frontend
          npm install --no-package-lock
          npm run build-ci-prod
          ...

      - name: Save all relevant files into .tgz and purge irelevant
        run: |
          set -x
      
      - name: Cleanup and copy .tgz to server over SFTP
        run: |
          sftp production <<EOF
          EOF

      - name: Purge old and untar new Docker deployment on the server over SSH
        run: |
          ssh -tt production <<EOF
          EOF

      - name: PowerPC Target only - Preinstall Express on PPC in frontends for Docker containers to work
        run: |
          ssh -tt production <<EOF
          EOF

      - name: Build and start Docker deployment on the server over SSH
        run: |
          ssh -tt production <<EOF
          EOF

      - name: Here are your domains!
        run: |
          echo ...