Application backend

k3s kubernetes

All applications are hosted on a k3s cluster running on a single Hostup VPC. There is no GitOps setup, instead I use a simple bash script to set everything up. The setup installs helm for release management, sets up the model repository and registers a worker with GitLab so that all further code changes can be deployed with its CI/CD. TLS is done through Let's Encrypt.

Model updates

Since I opted do the training on Google Colab, I have stuck to manually running and copying new model versions directly to the model repository. To still have some valuable automation and control, I utilize Gitlab's CI/CD pipelines. Models are swapped by updating the embedded applications to point to new model versions. This results in a controlled model rollout, the ability to rollback, and one could also run extra steps such as dataset validation in the pipeline.

Applications

The model widgets are gradio applications, which are super simple to get up and running, which was valuable when building this showcase. The library seems to allow for a lot of customization and design if needed. The web frontend is a simple Next.js application. Reasonably, if this was a production setup, one would probably build the interface directly in React, talking to a model backend, possibly through a web socket.