Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Preparing Machine Learning Models for Production Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Managing versioning and storage requirements
Serving Models on Kubernetes
- An introduction to inference servers
- Deploying TensorFlow Serving and TorchServe
- Configuring model endpoints
Optimizing Inference Performance
- Implementing effective batching strategies
- Managing concurrent request processing
- Tuning for optimal latency and throughput
Autoscaling Machine Learning Workloads
- Utilizing the Horizontal Pod Autoscaler (HPA)
- Leveraging the Vertical Pod Autoscaler (VPA)
- Implementing Kubernetes Event-Driven Autoscaling (KEDA)
Provisioning GPUs and Managing Resources
- Setting up GPU-enabled nodes
- An overview of the NVIDIA device plugin
- Defining resource requests and limits for ML workloads
Strategies for Model Rollout and Release
- Executing blue/green deployments
- Applying canary rollout patterns
- Conducting A/B testing for model evaluation
Monitoring and Observability for Production ML
- Tracking key metrics for inference workloads
- Establishing best practices for logging and tracing
- Creating dashboards and setting up alerting mechanisms
Security and Reliability Considerations
- Securing model endpoints
- Implementing network policies and access control
- Safeguarding high availability
Conclusion and Future Directions
Requirements
- A solid grasp of containerized application workflows
- Practical experience with Python-based machine learning models
- Basic familiarity with Kubernetes core concepts
Target Audience
- ML engineers
- DevOps engineers
- Platform engineering teams
Testimonials (1)
I like the Kubernetes part of the training as I had some tasks with K8s pods and ports and it was nice to get a better understanding now. Thank you!