Home
Softono
ab-testing-in-ml

ab-testing-in-ml

Open source Apache-2.0 Python
GitHub
40
Stars
9
Forks
0
Issues
3
Watchers
1 year
Last Commit
ML Frameworks DevOps & Infrastructure

About ab-testing-in-ml

Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.

Platforms

Web Self-hosted Kubernetes

Languages

Python

Links

Source Code
d
Published by
data-max-hq
Visit View Profile

README.md

View on GitHub

A/B Testing for ML applications

Deploy A/B testing infrastructure in a containerized microservice architecture for Machine Learning applications.

Requirements

This repository uses Kubernetes, helm, ambassador, seldon-core, and seldon-core-analytics.

Getting started

Install prerequisites

  1. Install helm

     brew install helm

  2. Create a Kubernetes cluster: AKS, EKS, GKE or local cluster. For local clusters, one can use minikube, kind, or k3s. For instance:

    1. GKE
      gcloud container clusters create demo-cluster-ab-test \
--zone=europe-west3-a \
--disk-size=30GB \
--cluster-version=1.24.12-gke.1000 \
--machine-type=e2-highcpu-4

Train model

Prepare the model artifacts:

make train

Build container images

One can build the images locally, or use Cloud Submit:

Locally

docker build -t ab-test:a -f Dockerfile.a .
docker build -t ab-test:b -f Dockerfile.b .
docker build -t streamlit-app:v1.0.0 -f Dockerfile.streamlit .

Cloud Submit

sh build-modela.sh
sh build-modelb.sh
sh build-streamlit.sh

Deploy required components

  • emissary-ingress

    make emissary

  • Prometheus

    make prometheus

  • Grafana

    make grafana

  • seldon-core

    make seldon-core

  • podmonitor

    make podmonitor

Deployment

make abtest

Start Streamlit App

make streamlit

Port-forward Grafana

make port-grafana

Contact

Sadik Bakiu (sadik [at] data-max.io)

Developed with ❤ at Data Max

References