Contents Menu Expand Light mode Dark mode Auto light/dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
Home
Home
  • Past Experiences
    • Anomaly Detection in Time Series IoT Data
    • Energy Demand Forecasting in Time Series IoT Data
    • ADAS: Data Engine
      • Business Challenge and Goals
      • ML Problem Framing
      • Planning, Operational Strategy
      • Workflows, Team, Roles
      • Testing Strategy
      • Data Ingestion Workflows
      • Scene Understanding & Data Mining
      • Model Training & Experimentation
      • Packaging, Evaluation & Promotion Workflows
      • Deployment & Serving
      • Monitoring & Continual Learning
      • Cost, Lifecycle, Compliance
      • Reliability, Capacity, Maps
    • Customer Lifetime Value
    • Real-Time Purchase Intent Scoring
    • Reviews Summarisation
    • RAG-Based Product Discovery
  • Projects
    • Natural Language Processing
      • Airbnb Listing description based Semantic Search
    • Computer Vision
      • Image Segmentation for Ecommerce Products
    • Machine Learning
      • Predictive Price Modeling for Airbnb listings
  • Patents, Papers, Thesis
  • AI Agents: A Lead Engineer’s Handbook
    • Agent Fundamentals: What, Why, and When?
    • Agentic Patterns
    • Context Engineering for AI Agents
    • The State of the Industry: Insights from the Field
    • Conclusion: The Lead Engineer’s Mental Model for Building Agents
    • Cost Optimization
    • Data Management and Knowledge Integration
    • Deployment and Scaling
    • Guardrails
    • Human-in-the-Loop (HITL)
    • Latency Optimization
    • LLM – Prompts, Goals, and Persona
    • Managing Agent Memory (Short-Term and Long-Term)
    • Monitoring and Observability
    • Orchestration and Task Decomposition
    • Production Challenges and Best Practices
    • Securing AI Agents and Preventing Abuse
    • Tool Use and Integration Management
    • Building Trustworthy and Ethical AI Agents
  • MLOps
    • ML Problem framing
    • The MLOps Blueprint & Operational Strategy
    • ML Platforms
      • ML Platforms: How to
      • Uber Michelangelo
      • LinkedIn DARWIN
      • Netflix
      • Shopify Merlin
      • Zomato: Real-time ML
      • Coveo: MLOPs at reasonable scale
      • Monzo ML Stack
      • Didact AI
    • Project Planning
      • Project Requirements Document
      • Tech Stack
      • Config Management
      • Pipeline Design
      • Environment Strategy
      • CI/CD Strategy and Branching Model
      • Directory Structure
      • Environments, Branching, CI/CD, and Deployments Explained
      • Project Management for MLOps
    • Data Sourcing, Discovery
      • Data Sourcing, Discovery & Understanding
      • Project-Trending Now: Implementing Web Scraping, Ingestion
      • Data Discovery Platforms: Industry Case Studies
      • Facebook: Nemo
      • Netflix Metacat
      • Uber Databook
      • LinkedIn Datahub
    • Data Engineering, Pipelines
      • Data Engineering for Reliable ML Pipelines
      • Data Engineering & Pipelines: A Lead’s Compendium
      • Real-Time & Streaming Data Pipelines: Challenges, Solutions
      • Netflix Keystone
      • Doordash Riviera
    • Feature Engineering, Feature Stores
      • Feature Engineering and Feature Stores
      • Feature Engineering for MLOps
      • Feature Stores for MLOps
      • Point-in-Time Correctness & Time Travel in ML Data Pipelines
      • Feast Feature Store
        • Feast Architecture: A Technical Deep Dive for MLOps
        • Feast Concepts
        • Feast Components
        • Feast Use Cases
        • Running Feast with AWS
        • Running Feast in Production
        • Validating Historical Features with Great Expectations
        • Adding or Reusing Tests in Feast
    • Model Development, Tuning, Selection, Ensembles, Calibration
      • Chapter 7: Model Development
      • How to train DL Models
      • Model Development
      • Model Development: Lessons from production systems
      • Model Ensembles
      • Model Selection
      • Hyperparameter Optimization
      • ML Expt tracking, Data Lineage, Model Registry
      • Model Calibration
    • ML Training Pipelines
    • Testing in ML Systems
      • Testing in ML Systems
      • Data Testing & Validation in Production
      • Testing ML Systems: Ensuring Reliability from Code to Production
    • Model Deployment & Serving
      • Chapter 10: Deployment & Serving
      • Guide: Model Deployment & Serving
      • Deep Dive: Inference Stack
    • Monitoring, Observability, Drift, Interpretability
      • Chapter 11: Monitoring, Observability, Drifts
      • Guide: ML System Failures, Data Distribution Shifts, Monitoring, and Observability
      • Interpretability, SHAP, LIME
      • Prometheus + Grafana and ELK Stacks
    • Continual learning, Retraining, A/B Testing
      • Chapter 12: Continual Learning & Production Testing
      • Continual Learning & Model Retraining
      • A/B Testing
      • A/B Testing & Experimentation: Industry lessons
      • Guide: Production Testing & Experimentation
      • Deep Research: Production Testing & Experimentation
    • Governance, Ethics & The Human Element
  • PyTorch
    • General
    • state_dict
    • Mixed Precision
    • Distributed Data Parallel
    • DDP: Under the Hood
    • DP vs DDP
    • FSDP
    • Tensor parallelism
    • Pipeline Parallelism
    • Device Mesh
  • Low Level Design
    • Parking Lot
  • Data Visualization Projects
Back to top
View this page

Testing in ML Systems¶

¶

  • Chapter 8: The Crucible – Comprehensive Testing in ML Systems

  • Guide: Data Validation

  • Guide: ML Testing

Next
Testing in ML Systems
Previous
ML Training Pipelines
Copyright © 2024, Deepak Karkala
Made with Sphinx and @pradyunsg's Furo
On this page
  • Testing in ML Systems