ML Platform
A practical analysis of evolving a Machine Learning platform from a lightweight Proof-of-Concept into a resilient, automated and production-oriented architecture.
The Problem
Machine Learning systems rarely fail because somebody forgot how to train a model.
They usually become difficult when the model leaves the notebook and enters the real world: multiple services, multiple versions, multiple environments, automated deployments, security checks, monitoring, scaling and—eventually—someone asking, “Who deployed this?”
The purpose of this architecture is therefore not simply to deploy an ML model. It is to establish a repeatable path from data ingestion and training through validation, packaging, deployment and inference.
POC Architecture
Optimized for experimentation, fast iteration and lightweight validation. A single-node Kubernetes environment is sufficient for proving that the system works.
Production Architecture
Optimized for availability, automated delivery, security validation, horizontal scaling and observable model-serving workloads.
POC vs Production
The architectural difference is less about replacing one technology with another and more about introducing operational guarantees.
POC Architecture
The Proof-of-Concept architecture focuses on one important objective: prove the complete ML lifecycle without building an entire cloud empire first.
Production Direction
Once the POC proves the workflow, production architecture introduces controls around reliability, security and repeatability.
The production system should make the correct deployment path the easiest deployment path.
That means automated tests, security scans, artifact promotion, declarative infrastructure and Git-based deployment state.
Instead of manually applying Kubernetes manifests, the desired state is stored in Git and reconciled by Argo CD. Instead of assuming a model is healthy because its container is running, the system introduces application and model-level observability.
Repository Setup
The following workflow runs the ML serving project locally using Kubernetes or K3s.
The Terminal
A production architecture is impressive. A working one is better. The following is the deliberately unglamorous part: commands.
# Clone the repository
git clone https://github.com/Mainak23/project1o1.git
cd main
# Check Kubernetes
sudo systemctl status k3s
kubectl get nodes
# Create namespace
kubectl create namespace ml-serving
# Configure GHCR
kubectl create secret docker-registry ghcr-secret \
--docker-server=ghcr.io \
--docker-username=<GITHUB_USERNAME> \
--docker-password='<GITHUB_TOKEN>' \
-n ml-serving
# Validate deployment
kubectl apply \
--dry-run=client \
-f model-deployment.yaml
# Deploy
kubectl apply \
-f model-deployment.yaml
# Inspect workload
kubectl get deployment -n ml-serving
kubectl get pods -n ml-serving
# Check rollout
kubectl rollout status \
deployment/ml-serving2 \
-n ml-serving
Replace the container image version in model-deployment.yaml with the intended GHCR image version before deployment.
Inspect Before You Panic
Kubernetes gives us several useful places to look before declaring the entire platform deceased.
# Deployment
kubectl get deployment \
-n ml-serving
# Pods
kubectl get pods \
-n ml-serving
# Logs
kubectl logs deployment/ml-serving2 \
-n ml-serving \
--tail=100
# Follow logs
kubectl logs deployment/ml-serving2 \
-n ml-serving \
-f
The expected pod state is straightforward:
1/1 Running
This is one of the few places in engineering where “Running” is a genuinely pleasant thing to see.
Test the API
Forward the Kubernetes service to the local machine.
kubectl port-forward \
svc/ml-serving2 \
8081:8080 \
-n ml-serving
Health check:
curl http://127.0.0.1:8081/health
{"status":"healthy"}
Once the service responds, the prediction endpoint can be tested using the expected 30-feature input vector.
curl -X POST http://127.0.0.1:8081/predict \
-H "Content-Type: application/json" \
-d '{
"data": [[
17.99,10.38,122.8,1001.0,0.1184,
0.2776,0.3001,0.1471,0.2419,0.07871,
1.095,0.9053,8.589,153.4,0.006399,
0.04904,0.05373,0.01587,0.03003,0.006193,
25.38,17.33,184.6,2019.0,0.1622,
0.6656,0.7119,0.2654,0.4601,0.1189
]]
}'
Expected response:
{"prediction":[0]}
Production Quality Gates
The next architectural step is to move from “Can we deploy this?” toward “Should this be allowed to deploy?”
No comments:
Post a Comment