Utilizing CI in QGIS plugin development
Testing is essential part of writing good piece of software. Writing unit tests and integration test are good way to test the application easily. It is recommended to run those tests before every commit during the development. However in practice it might be forgotten and eventually one might notice that some of the tests have been broken at “some point”.
This is where Continuous integration (CI) comes to the aid! CI aims to make developing easier and avoiding conflicts by integrating regularly and often, even after every commit. This involves compiling and building application as well as quality assurance by running tests.
There are a lot of CI tools out there. Especially Open Source projects can use many tools for free! One of the newest tools is Github Actions. It allows building workflows to run builds, tests and other stuff automatically based on the predefined rules straight out of a Github repository without any additional dependencies. For example it is possible to run the workflows after every commit to certain branches or pull requests targeting those branches. All this can be done with one or more simple .yml or .yaml files. We started looking how easy it would be to build a workflow to run tests automatically for our upcoming QAAVA QGIS plugin.
How to build a suitable environment for QGIS plugins
Many people familiar with QGIS plugin development have probably noticed that quite a lot of dependencies are needed to build a working development environment. Even the most simple unit tests might require initializing QGIS applications programmatically. Additionally some of the integration tests in plugins might require working PostGIS database connection. Is it possible to set up a similar environment using building blocks of Github Actions? Yes it is!
Familiar operating systems, such as Ubuntu can be used within workflows. One could in theory install all needed dependencies directly using package manager within OS. Luckily we don’t have to do that because docker and docker-compose can be utilized inside workflow! QGIS itself alongside with needed development dependencies can be found from QGIS docker image. With QGIS container, it is possible to mount the plugin code inside it and run commands with it. There are many ways to start PostgreSQL with PostGIS in a workflow. One alternative is to use custom made postgis-action. In our case we wanted to start PostGIS using docker-compose because we wanted to initialize fixtures to the database and use the same docker-compose.yml file to start PostGIS for local tests using pytest-docker.
With QGIS and PostGIS handled the last (but not the least) step was to actually get the tests running inside the environment. QGIS image does not have pytest installed but it can be installed with pip. After mounting the development directory inside QGIS container and stariting the tests the result was: … nothing. After quite a bit of debugging we found that when tests are initializing QGIS application, some kind of display is expected to be found. Otherwise the initialization and eventually the tests fail (without any message obviously). How on earth can a display be used within a virtual environment where CI workflow is running? Of course by mocking it. The command to run test with mocked display was xvfb-run -s ‘+extension GLX -screen 0 1024x768x24’ pytest.
Below is the whole tests.yml file that was needed to configure automatic tests. All that was required was to put the file inside .github/workflows directory in the root of the repository.
# workflow name
name: Tests
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the wanted branches
on:
push:
branches:
- master
pull_request:
branches:
- master
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
test:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Pull qgis image
run: docker pull qgis/qgis:final-3_12_3
- name: Pull PostGIS image
run: docker pull kartoza/postgis:12.0
- name: Set up PostGIS databases
run: docker-compose -f Qaava/test/docker-compose.yml up -d
# Runs all tests
- name: Run tests
run: docker run --rm --net=host --volume `pwd`/Qaava:/app -w=/app qgis/qgis:final-3_12_3 sh -c "pip3 install -q pytest && xvfb-run -s '+extension GLX -screen 0 1024x768x24' pytest -v"
After commits and pull requests to the master branch, the workflow is automatically started and the result can be seen from the actions page or from little green (or red) tick next to the commit. You can also integrate little badge on README.md to show to users that test are passing.
What about building, translating and deploying to QGIS Plugin repository?
Many other kinds of workflows can be configured to achieve specific goals. These can be written by hand with the help of CI tools or you can utilize great python library qgis-plugin-ci designed to handle building, translating, deploying and releasing for you! It can be even integrated to other CI environments such as Github Actions to support custom workflows.
Extra tip!
Did you know you can use the same QGIS docker image to run QGIS using docker? This is useful for example when testing compatibility of the plugin with multiple QGIS versions. In Ubuntu this would work with following command that starts QGIS and mounts QGIS python plugin folder and home folder to be used in QGIS:
xhost +
docker run --rm --name qgis_final-3_12_3 \
-it \
-e DISPLAY=unix$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v ${HOME}/.local/share/QGIS/QGIS3/profiles/default/python:/root/.local/share/QGIS/QGIS3/profiles/default/python \
-v ${HOME}:/home/${USER} \
qgis/qgis:final-3_12_3
