Day-12 – Scikit-Learn Pipeline + End-to-End ML Workflow

 Until now:

• You manually cleaned data 
• Trained models separately

Today you learn:

How professional ML workflows are built 🚀

🎯 Goal of Day-12

You will:

✅ Understand ML pipeline
✅ Automate preprocessing + model training
✅ Build cleaner production-ready workflow

🧠 What is a Pipeline?

Simple meaning:

A sequence of ML steps connected together.

Example:

Raw Data

     ↓

Cleaning

     ↓

Feature Scaling

     ↓

Model Training

     ↓

Prediction 

Instead of writing separate code every time.

🧠 Why Pipelines Matter

Without pipeline:

❌ Messy code
❌ Repeated logic
❌ Easy mistakes

With pipeline:

✅ Clean workflow
✅ Reusable
✅ Production-ready

🚀 Part 1 – Import Libraries

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

🚀 Part 2 – Create Dataset

data = {
"Hours": [1,2,3,4,5,6,7,8],
"Marks": [40,45,50,55,70,80,90,95],
"Pass": [0,0,0,0,1,1,1,1]
}

df = pd.DataFrame(data)

🚀 Part 3 – Features & Target

X = df[["Hours", "Marks"]]
y = df["Pass"]

🚀 Part 4 – Train/Test Split

X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)

🚀 Part 5 – Create Pipeline

pipeline = Pipeline([
("imputer", SimpleImputer(strategy="mean")),
("scaler", StandardScaler()),
("model", LogisticRegression())
])

🧠 What Happens Here?

Step 1:

SimpleImputer

Handles missing values.

Step 2:

StandardScaler

Normalizes data.

Step 3:

LogisticRegression

Trains model.

🚀 Part 6 – Train Pipeline

pipeline.fit(X_train, y_train)

🚀 Part 7 – Predict

y_pred = pipeline.predict(X_test)

print(y_pred)

🚀 Part 8 – Accuracy

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy:", accuracy)

🧠 What is Feature Scaling?

Example:

Feature                       Value

-------------------------------------------

Salary                         500000

Age                             25

Problem:

• Salary dominates ML model.

Solution:

• Scale values to similar range.

🚀 Part 9 – Predict New Data

new_data = pd.DataFrame({
"Hours": [6],
"Marks": [85]
})

prediction = pipeline.predict(new_data)

print("Prediction:", prediction)

🧠 Real AI Insight

Pipelines are used in:

• Production ML systems
• MLOps workflows
• Enterprise AI platforms

👉 This is VERY important for interviews.

⚠ Important Interview Question

Q:

Why use pipeline?

Answer:

To automate preprocessing and modeling steps consistently and avoid data leakage.

🎯 End of Day-12 Goals

You now:

✅ Understand ML pipelines
✅ Automate preprocessing
✅ Build structured ML workflow

github link: https://github.com/dotnetfullstackdeveloper/ai-engineer-journey/blob/main/Week-02-Machine-Learning/Day-12%20%E2%80%93%20Scikit-Learn%20Pipeline%20%2B%20End-to-End%20ML%20Workflow