How I got here...

Part of the whole effort to mess around with Docker was to get a better understanding of what people do for continuous integration and continuous deployment - CICD. There are no shortage of services online that do this for you - Github, Azure Devops, Bitbucket.. are all names that are familiar in terms of being able to do CICD.

I wanted to wire it up on my own, and least understand what the moving parts are.

Self-hosting a git repository with Gitea

I long ago moved my mess of source code from a folder in my Nextcloud sync'd folders to a local install of Gitea. It's basically "good enough" - I wish I could push a new repository to it and it just create the repository for me, instead of it having to exist first... but that's about the only complaint I have. For all I know, this is an option that I can tweak or a plugin I can add that would fix the complaint.

I was also aware of git post-commit hooks. Thinking this was the direction I needed to go, I started writing some bash script. The objective was simple: When I pushed a commit to the blog's repo, it would generate the site using the hugo Docker container, then put the data where it needed to be.

This worked, but I learned quickly this isn't how it's done in general: I needed the ability to modify folders on the host OS, I needed to manage some SSH keys, and I was routinely fighting permission issues - Docker would create files under it's own user, which my user account didn't have rights to. These were all fixable issues, but it was evident that this isn't how it's done.

Drone

A bit of learning later, I find myself trying to wire up Drone. The idea of Drone is simple: Add a webhook from your repository that points to Drone, and Drone will light up some workers to perform whatever actions you've asked for. These actions are run in a Docker container, and the output from those actions tell you things are healthy.

It took a little bit of figuring - and after abandoning reading someone's blog (which was either old, or just outright bullshit), the doc's saved the day.

For my own reference later in life, the relevant and important parts from my docker-compose.yml file resembles something like this:

 1version: "2"
 2
 3services:
 4  server:
 5    image: gitea/gitea:latest
 6    container_name: gitea
 7    environment:
 8      ...
 9    restart: always
10    networks:
11      - traefik
12    ...
13
14  drone-server:
15    image: drone/drone
16    container_name: drone-server
17    ports:
18      - 8085:80
19      - 9000
20    volumes:
21      - drone:/var/lib/drone
22    restart: always
23    depends_on:
24      - server
25    environment:
26      - DRONE_OPEN=true
27      - DRONE_GITEA=true
28      - DRONE_NETWORK=traefik
29      - DRONE_DEBUG=true
30      - DRONE_ADMIN=timatlee
31      - DRONE_GIT_ALWAYS_AUTH=true
32      - DRONE_USER_CREATE=username:timatlee,admin:true
33      - DRONE_DATABASE_DRIVER=mysql
34      - DRONE_DATABASE_DATASOURCE=...
35      - DRONE_GITEA_SERVER=...
36      - DRONE_GITEA_CLIENT_SECRET=...
37      - DRONE_GITEA_CLIENT_ID=...
38      - DRONE_RPC_SECRET=...
39      - DRONE_SERVER_HOST=...
40      - DRONE_SERVER_PROTO=http
41      - DRONE_TLS_AUTOCERT=false
42      - DRONE_AGENTS_ENABLED=true
43    networks:
44      - traefik
45
46
47  drone-agent:
48    image: drone/drone-runner-docker
49    container_name: drone-agent
50    command: agent
51    restart: always
52    depends_on:
53      - drone-server
54    volumes:
55      - /var/run/docker.sock:/var/run/docker.sock
56      - drone-agent:/data
57    environment:
58      - DRONE_RPC_HOST=...
59      - DRONE_RPC_PROTO=http
60      - DRONE_RPC_SECRET=...
61      - DRONE_RUNNER_CAPACITY=1
62      - DRONE_RUNNER_NETWORKS=traefik
63    networks:
64      - traefik
65
66volumes:
67  drone: {}
68  drone-agent: {}
69
70networks:
71  traefik:
72    external: true

• DRONE_DATABASE_DATASOURCE was defined from https://docs.drone.io/server/reference/drone-database-datasource/.
• DRONE_GITEA_SERVER refers to my Gitea installation.
• DRONE_GITEA_CLIENT_SECRET refers to the OAUTH2 secret that was created when setting up Drone in Gitea.
• DRONE_GITEA_CLIENT_ID is the OAUTH Client ID was that was generated at the same time the OATUH was created.
• DRONE_RPC_SECRET is the secret key that's used to communicate with the Drone agents.
• DRONE_SERVER_HOST is the server name that Drone runs at. This wound up needing to be a FQDN, and didn't like running as a folder under an existing domain.

Traefik is handling the HTTP/HTTPS rewriting, so I didn't think it was necessary to have Drone handle it's own certificate.

Within the Agent configuration, similar properties are seen - RPC server and RPC_HOST, which refers to the Drone server host.

The big takeaway here was that Drone, like a lot of these projects, like having their own subdomain.

Drone Containers

Somewhere buried in the documentation, the Drone docs tell you that it will use the containers you specify in the configuration, but then replace the entrypoint with the commands that are specified. In my case, I am using a Hugo container whose entrypoint is Hugo. I would normally just run it from the commandline:

docker run --rm --volume="/tmp/website:/src" klakegg/hugo:latest-ext

This would generate the output to public/ in the /src/ folder.

It took me a bit to figure out that Drone makes use of the container you specify, but overrides the entrypoint (to? a shell?) and adds the repository to the container. Any command in a step is run within the defined container. I am able to run hugo check to make sure the environment is healthy (is it not in a container?) before building.

Assuming that if a command generates a non-zero return means that the workflow errors out... this is kinda neat.

Drone.yml

Figuring out the Drone file was a bit of an adventure too. I'm using Hugo to build this site, so I wanted to make use of the existing Hugo Docker image - it's frequently updated, has a lot of use, and I know it works.

Back when I was learning Hugo, I opted to use a theme called Clarity, which is what you see as if this post. I added it to my site using the documentation - which has you add the theme as a submodule. Cool, easy enough - except the default behaviour on git checkout does not recursively check out submodules.

One of Drone's first actions is to check out the source code, and since it also doesn't check out submodules, the theme layer wasn't getting generated.

So, early in the Drone configuration, I add the step:

1steps:
2- name: submodules
3  image: alpine/git
4  commands:
5  - git submodule update --init

which gets the theme submodule for me, and away it goes!

Rsync to a destination

Drone has an rsync plugin that will copy the content from within the running container to a target server and path. This thing is really just a Docker container running rsync, so it should be reasonably straightforward, right? Surprisingly it was - The only gotcha here that I encountered was that I was trying to reuse an existing public/private key and I failed to add the public key to the list of authorized keys. An easy fix, at least.

In the end, my Drone configuration (.drone.yml) reads as follows:

 1---
 2kind: pipeline
 3type: docker
 4name: default
 5
 6workspace:
 7  path: /src
 8
 9steps:
10- name: submodules
11  image: alpine/git
12  commands:
13  - git submodule update --init
14
15- name: versioncheck
16  image: klakegg/hugo:latest-ext
17  commands:
18  - hugo version
19
20- name: basic checks
21  image: klakegg/hugo:latest-ext
22  commands:
23  - hugo check
24
25- name: build-development
26  image: klakegg/hugo:latest-ext
27  commands:
28  - hugo -D -E -F
29  when:
30    branch: master
31
32- name: rsync-development
33  image: drillster/drone-rsync
34  environment:
35    RSYNC_KEY:
36      from_secret: drone_ssh_key
37    RSYNC_USER: username
38  settings:
39    hosts:
40    - home.timatlee.com
41    source: ./public/*
42    target: /var/www/html/
43    recursive: true
44    delete: true
45  when:
46    branch: master
47
48- name: build-publish
49  image: klakegg/hugo:latest-ext
50  commands:
51  - hugo
52  when:
53    branch: publish

The key, drone_ssh_key, is defined in Drone, and the documentation here sufficient to get you going.

I don't have the "publish" branch actions wired up yet, but that's kinda next on the list - at least until something else more interesting comes along. The nice thing about this approach, versus the script I started with, is that it's remarkably easy to change the criteria of actions. If I don't ever both with the "publish" branch, I can just change the config easy enough.

So that was fun.

Yep.

I think next steps are to figure out what the correct publishing workflow is. Is working on the master branch and making "final" changes on another branch the correct way? Or is publishing from master correct?

Or maybe this whole thing should make up it's own container - publish it to a local container repository, then let Watchtower pull in a new copy of the container when it detects a new one. Right now, the content is just being copied to a folder on the host machine...