← Back to course index Module 1 · Getting Started

AI vs ML vs DL

Detailed study notes on Artificial Intelligence, Machine Learning, and Deep Learning — how the three fields relate, why Symbolic AI failed on real-world problems, and how Deep Learning solved Machine Learning's biggest bottleneck.

⏱️ 18 min read 🎯 Beginner Friendly

On This Page

🎯The AI / ML / DL Hierarchy
🤖AI & Early Symbolic Systems
📊Machine Learning
🧠Deep Learning
📋Comparison Matrix
🐍Python Code Examples
🔑Key Concepts
📝Summary & Practice
⚡Quick Revision

🎯

1. The AI / ML / DL Hierarchy

A common technical interview question asks candidates to differentiate between Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL). Visually, these three fields form a nested set of concentric circles.

Artificial Intelligence

Any system that shows human-like intelligence

Machine Learning

Learns rules directly from data

Deep Learning

Multi-layer neural networks

Artificial Intelligence (AI): The outermost umbrella covering any system that demonstrates human-like intelligence.
Machine Learning (ML): A specific subset within AI focused on learning rules and patterns directly from data.
Deep Learning (DL): A specialized subset within ML that uses multi-layer artificial neural networks to learn representations from complex data.

Tip: Think of it as three nested circles — the biggest circle is AI, inside it sits ML, and inside ML sits DL. Every DL model is an ML model, and every ML model is a form of AI — but not the reverse.

🤖

2. Artificial Intelligence (AI) & Early Symbolic Systems

What is Intelligence?

Human intelligence is complex and multifaceted. It comprises diverse capabilities, including quantitative reasoning, logical coding, puzzle-solving, spatial awareness, emotional intelligence, creativity, and imagination.

General AI vs. Specific (Narrow) AI

Artificial General Intelligence (AGI): The long-term goal of building machines capable of replicating the full spectrum of human intellect, including emotions and creative imagination.
Narrow AI (Current Reality): Existing AI applications focus on specific, quantifiable tasks with clear mathematical objectives (e.g., playing chess, performing calculations, or filtering spam). We focus on task-specific AI because subjective qualities like human creativity cannot yet be formulated into quantitative equations.

Symbolic AI & Expert Systems (1950s Era)

In the early days of AI (starting around the 1950s), researchers attempted to build artificial intelligence using Symbolic AI (Rule-Based Systems). The classic Expert System architecture looked like this:

Symbolic AI & Expert Systems (1950s Era)

Rule-Based Approach to Building Artificial Intelligence

🧑‍⚕️

Human Expert (Domain Specialist)

Provides knowledge and expertise in a specific domain.

↓ Knowledge is extracted (interviewed)

🧑‍💻

Knowledge Engineer (Knowledge Acquisition)

Interviews the expert and converts their knowledge into formal rules (IF-THEN statements).

↓ Rules are stored

📚

Knowledge Base (Rule Repository)

Stores the extracted knowledge as a large set of conditional rules (IF-THEN statements).

⇅ Uses rules for reasoning

⚙️

Inference Engine (Reasoning Mechanism)

Applies logical reasoning over the knowledge base to derive conclusions and answer queries.

⇅ Presents questions and results

🖥️

User Interface (Interaction Layer)

Allows ordinary users to interact with the system using a simple interface (e.g., text or menu).

⇅ Asks questions, gets answers

🙋

User (Non-Expert)

An ordinary user who asks questions and receives expert-level answers.

Human Expert → Knowledge Engineer: A domain expert's knowledge is interviewed and extracted by a knowledge engineer.
Knowledge Base: The extracted knowledge is hardcoded into a vast database of conditional IF-THEN statements.
Inference Engine: Applies logical reasoning over the knowledge base to answer queries.
User Interface: Lets an ordinary (non-expert) user interact with the system and get expert-level answers.
Example: Chess-playing computers where every valid move and strategy was programmed as an explicit rule.

Limitations of Symbolic AI & Expert Systems

Expert systems excel at rigid, closed-world problems like chess or mathematical logic. However, they fail on fuzzy logic and unstructured real-world tasks:

The Image Classification Problem: Identifying whether an image contains a Dog or Not a Dog.
Why Hardcoding Fails: Dog breeds vary endlessly in size, color, ear shape, and tail length. Lighting, angles, and background variations create infinite combinations. A developer cannot manually write IF-ELSE rules for every possible visual feature of a dog. Similar limitations arise in speech recognition and natural language processing.

Limitations of Symbolic AI & Expert Systems: works well for rigid, well-defined problems but fails on fuzzy, real-world tasks. Left panel: Works for Rigid, Closed-World Problems, example Chess with fixed IF-THEN rules. Right panel: Fails for Fuzzy, Real-World Problems, example Image Classification (Dog or Not a Dog) showing diverse dog photos and impossible-to-write hardcoded rules. Bottom left: Why Hardcoding Fails (Variety of Breeds, Different Angles, Lighting Conditions, Background Variations). Bottom right: Other Real-World Challenges (Speech Recognition, Natural Language Processing). Key takeaway: Symbolic AI works well when rules are clear and finite, but real-world data is messy, varied, and infinite, making it impractical to hardcode rules for everything.

📊

3. Machine Learning (ML) — The Shift to Data-Driven Rules

Definition: Machine Learning is a branch of computer science that applies statistical and mathematical techniques to discover underlying patterns in data without being explicitly programmed.

The Paradigm Shift

Machine Learning completely flips the traditional programming paradigm:

Traditional Programming vs Machine Learning Paradigm. Traditional Programming: you give the rules to the computer — Data plus Human-Written Rules produces Output (example: IF age >= 18 THEN eligible = Yes ELSE eligible = No). Machine Learning Paradigm: you give the examples, the computer learns the rules — Data plus Correct Outputs/Labels (e.g. dog and cat images labeled Dog / Not a Dog) produces Machine-Discovered Rules/Patterns automatically, with no explicit IF-ELSE rules written. Key Difference: traditional programming uses human knowledge to write rules, machine learning uses data and correct outputs to learn the rules automatically.

Example: Dog Classification via ML

Instead of writing explicit rules, you feed the ML model 10,000 images labeled "Dog" and "Not Dog". The algorithm analyzes the numerical data and automatically derives mathematical representations that distinguish dogs from non-dogs. This mirrors how human children learn from examples rather than textbook definitions.

Mathematical Foundation

Machine Learning is fundamentally applied statistics, linear algebra, and calculus packaged into practical engineering algorithms. ML gained massive industry momentum over the last 20–30 years due to the availability of large datasets and increased computing power.

🧠

4. Deep Learning (DL) — Automated Feature Extraction & Neural Networks

Definition: Deep Learning is a specialized subfield of Machine Learning that uses multi-layer Artificial Neural Networks (ANNs) to process complex datasets.

Deep Learning (DL) — Automated Feature Extraction & Neural Networks

Deep Learning is a specialized subfield of Machine Learning that uses multi-layer Artificial Neural Networks (ANNs) to process complex datasets.

Biological Inspiration: comparison of a biological neuron and an artificial neuron, showing dendrites, axon, synapses on the biological side and inputs, weighted sum, activation function, output on the artificial side, with a key takeaway that Deep Learning is inspired by but not the same as a real brain.

Why Do We Need Deep Learning?

Deep Learning addresses two primary limitations of traditional Machine Learning.

1Automated Feature Extraction

Traditional ML Limitation

Requires domain experts to manually define features.
e.g. CGPA, certifications (student placement).
e.g. edge ratios, color histograms (vision).

Deep Learning Advantage

You supply raw, unstructured data directly.
The network automatically learns hierarchical features.
No manual feature engineering required.

2Performance Scaling with Data

Deep Learning — continues to improve as data & depth increase.

Traditional ML — gains plateau, even with more data.

Layer-wise Feature Visualization for Digit 4: shows the input image of a handwritten 4, then Layer 1 edge detectors, Layer 2 strokes and textures, Layer 3 parts of the digit, Layer 4 full digit representation, and the final output softmax classification with digit 4 predicted at 0.89 probability.

📋

5. Side-by-Side Comparison Matrix

Feature / Aspect	Artificial Intelligence (AI)	Machine Learning (ML)	Deep Learning (DL)
Scope	Overarching field aimed at creating intelligent machines.	Subset of AI focused on learning rules from data.	Subset of ML using multi-layer neural networks.
Core Approach	Rule-based systems, expert systems, and heuristics.	Statistical & mathematical pattern recognition.	Multi-layer Artificial Neural Networks (ANNs).
Feature Engineering	Manually crafted rules by domain experts.	Requires manual human feature extraction.	Automated feature extraction across network layers.
Data Requirements	Operates on formal logic/rules without big data.	Works well on small to medium structured datasets.	Requires large volumes of unstructured data.
Primary Data Type	Symbolic logic, structured decision trees.	Structured / Tabular data (CSVs, Databases).	Unstructured data (Images, Video, Text, Audio).
Hardware Dependency	Low compute requirements.	Runs efficiently on standard CPUs.	Requires heavy parallel compute (GPUs / TPUs).
Industry Preference	Legacy systems, rigid rule engines.	Banking, Finance, Insurance tabular models.	Computer Vision, LLMs, Speech Recognition, Autonomous Systems.

🐍

6. Python Code Examples & Step-by-Step Breakdown

To solidify the conceptual explanations, here are two practical Python scripts illustrating the differences between Symbolic Rule-Based Logic and Data-Driven Machine Learning.

Example 1: Symbolic Rule-Based System vs. Machine Learning Classifier

Scenario: Predicting Student Job Placement Eligibility.

Approach 1: Symbolic / Rule-Based Expert System (Traditional AI). Human experts write explicit IF-ELSE rules. Diagram shows How It Works (Domain Expert writes rules into a Knowledge Base, an Inference Engine applies the rules to produce a Placed/Not Placed prediction), the Python code for expert_system_placement_check(), a worked example predicting Placement for a student with CGPA 8.2, and Key Characteristics vs Limitations, ending in a Key Takeaway that rule-based expert systems cannot handle the complexity and variability of real-world data.

Approach 2: Machine Learning (Data-Driven Pattern Learning). The system receives historical data and labels, then automatically learns decision boundaries. Diagram shows four steps: Historical Training Data (a table of CGPA, Certifications, Soft Skills Score and Placed label), Model Training (a Machine Learning Algorithm such as a Decision Tree, Random Forest or SVM learns patterns), Learned Model (an example decision tree splitting on CGPA >= 7.5, Soft Skills >= 80, Certifications >= 2 to output Placed or Not Placed), and Make Prediction (a new student with CGPA 7.1, Certifications 2, Soft Skills 84.0 is fed into the trained model producing Prediction Output: Placed). Key takeaway: in Machine Learning we provide data and correct outputs, and the algorithm automatically learns the patterns and decision boundaries to make predictions.

Note: The y_train array was empty/unreadable in the source notes — filled in above with a plausible label set ([1, 0, 1, 0, 1, 1]) matching the six training rows so the example runs. Verify against the original if exact labels matter to you.

Step-by-Step Code Explanation:

Approach 1 (Rule-Based): Demonstrates early Symbolic AI. The programmer explicitly decides thresholds (e.g., cgpa >= 8.0). If a student falls outside these hardcoded conditions, the system cannot adapt.
Approach 2 (Machine Learning): Uses scikit-learn's DecisionTreeClassifier. We pass feature matrix X_train and target labels y_train. The algorithm automatically determines the optimal decision boundaries from the data without manual rule coding.

Example 2: Manual Feature Engineering (ML) vs. Multi-Layer Hierarchical Representation (DL Concept)

Feature Extraction: ML (manual) vs. DL (automatic). Traditional ML requires humans to design features, while Deep Learning learns features automatically from raw data. Left: Traditional ML Pipeline — Raw Image, Manual Feature Extraction (human-designed: mean intensity, edge count, other handcrafted features like HOG/SIFT/shape), Feature Vector, ML Model (e.g. SVM, Random Forest) producing Digit = 4. Right: Deep Learning Pipeline — Raw Image fed directly into a Neural Network with multiple convolution layers that automatically learn hierarchical features (edges to strokes to parts to full digit), then a Fully Connected and Output layer producing a probability distribution with Digit = 4 predicted at 0.89. Takeaway: Traditional ML = manual feature engineering, Deep Learning = automatic feature learning.

🔑

7. Key Concepts for Quick Revision

🔑 Key Concepts

Concentric Hierarchy: AI is the overarching domain, ML is a subset of AI, and DL is a subset of ML (AI ⊃ ML ⊃ DL).
Symbolic AI (Expert Systems): Rule-based systems using IF-THEN statements, built via Human Expert → Knowledge Engineer → Knowledge Base → Inference Engine → User Interface; successful for chess and logic, but failed on unstructured fuzzy data like image/speech recognition.
Machine Learning Paradigm: Replaces manual rule-writing by accepting data and labels to discover mathematical rules automatically.
Feature Engineering: The bottleneck in traditional ML where humans must manually extract input features.
Automated Feature Extraction: The core breakthrough of Deep Learning where multi-layer neural networks learn feature representations directly from raw data (edges → shapes → parts → full object/output).
Performance Scaling: ML model performance plateaus on large datasets, while DL model accuracy scales with more data and deeper network layers.
Industry Usage: ML remains dominant for structured tabular data in banking and insurance due to lower data volumes and CPU efficiency, while DL powers Computer Vision and NLP applications.

📝

8. Summary & Practice Questions

Summary Takeaways

AI focuses on making machines intelligent, initially through explicit rules (Symbolic AI / Expert Systems).
ML introduced a paradigm shift by letting algorithms learn patterns directly from data and target labels.
DL solved ML's manual feature engineering bottleneck by using multi-layer artificial neural networks that automatically extract features from raw unstructured data.
Choose ML for structured tabular data with moderate volume, and DL for large unstructured datasets (images, text, audio).

Practice Questions

Conceptual Questions

1Why did early Symbolic AI and Expert Systems fail to solve real-world problems like identifying a dog in an image?

Hint: Think about fuzzy logic and infinite variations in physical attributes.

2What is the fundamental difference between the traditional programming paradigm and the Machine Learning paradigm?

Hint: Compare what inputs are provided and what output the system generates.

3Explain the concept of "Automated Feature Extraction" in Deep Learning using an image recognition example.

Hint: Describe what lower, middle, and deeper network layers learn.

Scenario-Based Question

4A local bank wants to build a credit card fraud detection system using structured transaction records in a SQL database (100,000 rows). Would you recommend a traditional Machine Learning algorithm or a Deep Learning model? Justify your choice.

Hint: Consider data type (structured tabular), volume, compute cost, and explainability.

⚡

Quick Revision (1–2 min read)

⚡ 30-Second Recap

AI → ML → DL — three nested circles, each a specialization of the one before it.
Symbolic AI (1950s): hardcoded IF-THEN rules via an expert system pipeline (expert → knowledge engineer → knowledge base → inference engine → UI). Great for chess/logic, terrible for fuzzy real-world tasks (e.g., "is this a dog?").
ML: flips the paradigm — feed Data + Labels, the algorithm outputs the rules, instead of a human writing the rules. Good for structured/tabular data, works on CPUs, but needs manual feature engineering and plateaus with more data.
DL: subset of ML using multi-layer neural networks; automatically learns features layer-by-layer (edges → shapes → parts → object) directly from raw data (images/text/audio); needs GPUs and lots of data, but keeps improving as data/depth grow (no plateau).
When to use what: small/structured data & need explainability → ML; large/unstructured data (images, video, text, audio) & enough compute → DL.