CI/CD for Model Training in ML

This repository holds files and scripts for incorporating simple CI/CD practices for model training in ML. Following resources have been used to prepare the files under the build directory:

• Vertex AI Training with TFX and Vertex Pipelines
• MLOps with Vertex AI by GCP

Another accompanying repository for doing CI/CD for model training can be found here: deep-diver/Model-Training-as-a-CI-CD-System.

This repository acts as a sister repository to deep-diver/Model-Training-as-a-CI-CD-System. It's recommended that one goes through this repository with the accompanying blog post from Google Cloud: Model training as a CI/CD system: Part II. The figures below schematically present what this project implements:

We first get a compiled pipeline specification for model re-training capable of accepting hyperparameter values as RuntimeParameters:

Then we schedule and trigger model training jobs:

Acknowledgements

ML-GDE program for providing GCP credits. Thanks to Karl Weinmeister for providing review feedback on this project.