Evals in Data Science
🔥 Building models is fun… but here’s the real test: is your model actually any good, or just pretending? 👀
Evaluations, or evals, are our model’s report card. They tell us:
- For a spam filter: Do we catch all spam (recall) without misclassifying grandma’s emails as junk (precision)?
- For price prediction: How close are our predictions on average (RMSE)?
But evals aren’t just about numbers - they influence trust, fairness, and real-world usefulness of our models.
Discussion prompts:
- What’s your go-to evaluation metric and why?
- Seen a model that looked great on paper but flopped in reality?
- Should fairness & usability be considered first-class evaluation metrics alongside accuracy?
Free book to dive deeper:
- Fairness and Machine Learning: rigorous, practical guide to evaluating models for fairness: https://fairmlbook.org/
Drop your thoughts below ⬇️
🔥 Building models is fun… but here’s the real test: is your model actually any good, or just pretending? 👀
Evaluations, or evals, are our model’s report card. They tell us:
- For a spam filter: Do we catch all spam (recall) without misclassifying grandma’s emails as junk (precision)?
- For price prediction: How close are our predictions on average (RMSE)?
But evals aren’t just about numbers - they influence trust, fairness, and real-world usefulness of our models.
Discussion prompts:
- What’s your go-to evaluation metric and why?
- Seen a model that looked great on paper but flopped in reality?
- Should fairness & usability be considered first-class evaluation metrics alongside accuracy?
Free book to dive deeper:
- Fairness and Machine Learning: rigorous, practical guide to evaluating models for fairness: https://fairmlbook.org/
Drop your thoughts below ⬇️
❤4
AI vs ML vs Deep Learning 🤖
You’ve probably seen these 3 terms thrown around like they’re the same thing. They’re not.
AI (Artificial Intelligence): the big umbrella. Anything that makes machines “smart.” Could be rules, could be learning.
ML (Machine Learning): a subset of AI. Machines learn patterns from data instead of being explicitly programmed.
Deep Learning: a subset of ML. Uses neural networks with many layers (deep) powering things like ChatGPT, image recognition, etc.
Think of it this way:
AI = Science
ML = A chapter in the science
Deep Learning = A paragraph in that chapter.
You’ve probably seen these 3 terms thrown around like they’re the same thing. They’re not.
AI (Artificial Intelligence): the big umbrella. Anything that makes machines “smart.” Could be rules, could be learning.
ML (Machine Learning): a subset of AI. Machines learn patterns from data instead of being explicitly programmed.
Deep Learning: a subset of ML. Uses neural networks with many layers (deep) powering things like ChatGPT, image recognition, etc.
Think of it this way:
AI = Science
ML = A chapter in the science
Deep Learning = A paragraph in that chapter.
❤9👏3
Overfitting vs Underfitting 🎯
Why do ML models fail? Usually because of one of these two villains:
Overfitting: The model memorizes training data but fails on new data. (Like a student who memorizes past exam questions but can’t handle a new one.)
Underfitting: The model is too simple to capture patterns. (Like using a straight line to fit a curve.)
The sweet spot? A model that generalizes well.
Note: Regularization, cross-validation, and more data usually help fight these problems.
Why do ML models fail? Usually because of one of these two villains:
Overfitting: The model memorizes training data but fails on new data. (Like a student who memorizes past exam questions but can’t handle a new one.)
Underfitting: The model is too simple to capture patterns. (Like using a straight line to fit a curve.)
The sweet spot? A model that generalizes well.
Note: Regularization, cross-validation, and more data usually help fight these problems.
❤12