Preface x
Acknowledgment xi
Biography xii
About the Companion Websitexiii
1 Introduction 1
1.1 Machine Learning to Deep Learning 1
1.2 Modern Embedded Systems 2
1.3 Deep Transfer Learning in Embedded System 4
1.4 Deep Learning Frameworks: An Overview 5
1.4.1 PyTorch 5
1.4.2 TensorFlow 6
1.4.3 Other Major Frameworks 6
1.5 Deep Learning and AI: Big Data and the Road Ahead 7
1.5.1 Advances in Computational Hardware 7
1.5.2 Adoptions in Industry and Business 8
1.5.3 Ethical, Legal, and Privacy Concerns 8
1.5.4 Future Challenges 9
2 Fundamentals of Deep Learning 15
2.1 Neural Networks Overview 15
2.2 Basic Concepts and Terminologies 15
2.2.1 Neurons and Layers 15
2.2.2 Role of Activation Functions 17
2.2.3 Learning Types 18
2.3 Training on a Network 18
2.3.1 Forward and Backward Propagation 18
2.3.2 Loss Functions 19
2.4 Gradient Descent Algorithm 19
2.5 Weight Initialization and Regularization 21
2.6 Hyperparameter Tuning 22
2.6.1 Architecture-based Hyperparameters 23
2.6.2 Training-based Hyperparameters 23
2.7 Overview of Common Hyperparameters 23
2.7.1 Kernel Size 23
2.7.2 Learning Rate 23
2.7.3 Weight Decay 24
2.7.4 Dropout 24
2.7.5 Batch, Iteration, and Epochs 24
2.8 Challenges and Best Practices for Efficient Tuning 25
3 Convolutional and Recurrent Neural Network 27
3.1 Introduction 27
3.2 Historical Background 27
3.3 Convolutional Neural Networks 28
3.3.1 CNN Framework 28
3.3.2 Convolutional Operations 28
3.3.3 Padding and Stride 30
3.3.4 Pooling Layers 31
3.3.5 Fully Connected Layers 32
3.3.6 Output Layers 32
3.3.7 Overall Architecture 32
3.4 Recurrent Neural Networks 33
3.4.1 RNN Framework 33
3.4.2 Basic Architectures 33
3.4.3 Backpropagation Through Time 34
3.4.4 Long Short-term Memory Networks 35
3.4.5 Gated Recurrent Units 35
3.4.6 Bidirectional RNNs 35
3.5 Applications 36
3.6 Conclusion 37
4 Deep Learning Using PyTorch 41
4.1 Introduction to PyTorch 41
4.2 Anaconda and PyTorch for Windows System 41
4.2.1 Anaconda Installation and Environment Setup 42
4.2.2 Setting up the PyTorch Framework 44
4.2.2.1 Visual Studio Professional 47
4.2.2.2 CUDA and cuDNN Installation 48
4.3 Other Essential Packages for Deep Learning 50
4.4 Introduction to Tensor 52
4.4.1 Defining a Tensor 52
4.4.2 Tensors in PyTorch 53
4.5 Basic Torch Operations in PyTorch 53
4.6 Gradient Calculation in PyTorch 55
4.7 Exercise Problems 57
5 Introduction to Jetson Nano and Setup 59
5.1 Introduction to Jetson Embedded Devices 59
5.1.1 Jetson Nano 60
5.1.2 Hardware and Power Requirements 61
5.2 Jetpack Installation 62
5.3 Direct Setup 65
5.3.1 Increase Root Partition Size 65
5.3.2 Other Settings 68
5.3.3 Wi-Fi Driver 70
5.4 Configure Visual Studio Code on Jetson 71
5.5 OpenCV and PyTorch in Jetson 75
5.6 Setting up Jetson Inference 76
5.7 OpenCV Library and Test Video Capture Functionality 79
5.8 Conclusion 81
5.9 Exercise Problems 81
6 Linux Terminal Overview 85
6.1 Introduction 85
6.2 Basic Terminal Commands and Syntax 86
6.3 Overview of File System 87
6.4 Navigating Files and Directories 87
6.5 Create, Edit, and Delete 90
6.5.1 Copy and Move Operations 92
6.6 Create and Execute Python Code from Terminal 93
6.7 Common Wildcard Characters 95
6.8 Find, View, and Get Information 96
6.9 Permission and Ownership 99
6.9.1 Permissions Using Octal and Symbolic Notations 100
6.9.2 Update Permission Using Operators 101
6.10 Install and Uninstall Packages Using "sudo" 104
6.11 Conclusion 105
7 Docker Engine Setup 107
7.1 Introduction to Docker Engine 107
7.2 Docker in Embedded Devices 107
7.3 Jetson Inference Docker 108
7.3.1 Download and Test Pre-trained Models 109
7.4 Using Host Files in Docker Environment 110
7.4.1 Create Executable Python 111
7.4.2 Attach Host Directory to Root System and Testing 113
7.5 Building a Docker Image 115
7.5.1 Runtime Image for MediaPipe Ecosystem 115
7.5.2 Install Docker Extension 116
7.5.3 Create Docker File and Build Image 116
7.6 Run Python Through Docker Container 118
7.7 Exercise Problems 119
8 Dataset Development 121
8.1 Introduction and Requirements 121
8.2 Types of Datasets 121
8.3 Manual Dataset Creation 122
8.3.1 Classification and Detection Dataset 122
8.3.2 LabelImg Setup 123
8.4 Automatic Image Collection Using Embedded Device 127
8.5 Automatic Data Labeling 128
8.6 Data Preprocessing and Cleaning 130
8.7 Exercise Problem 131
9 Training Model for Image Classification 133
9.1 Problem Statement 133
9.2 Default Configurations and Libraries 134
9.3 Setup Data Frame Using Annotations 134
9.4 Dataset Class and Methods 136
9.5 Data Loader and Model Configuration 137
9.6 Model Training 140
9.7 Testing and Inference 141
9.8 Fine-tuning 142
9.8.1 Using a Different Model 144
9.8.2 Using Different Optimizers and Schedulers 145
9.9 Application in Embedded System 146
9.10 Exercise Problems 147
10 Object Detection with Classification 149
10.1 Introduction 149
10.2 Import Modules and Libraries 149
10.3 Default Configurations and Random Seeds 151
10.4 Create Data Frame and Process Labels 151
10.5 Training and Validation of Transformers 152
10.6 Dataset Class and Methods 153
10.7 Data Loader and Classification Backbone 153
10.8 Training and Validation Approach 155
10.9 Run Multiple Epochs and Save the Best 156
10.10 Model Inference 157
10.10.1 Preprocessing Functions 158
10.10.2 Inference on Test Images 160
10.11 Multiple Object Detection with Classification 161
10.12 Model Inference for Multiple Object Detection 163
10.13 Conclusion 165
10.14 Exercise Problems 167
11 Deploy Deep Learning Models on Jetson Nano 169
11.1 Introduction 169
11.2 Pre-trained Models 169
11.3 Inference on an Image File 170
11.4 ONNX Model 172
11.4.1 Convert PyTorch Model to ONNX 172
11.5 Inference on Live Video Stream 173
11.6 Conclusion 175
11.7 Exercise Problem 176
12 Trained PyTorch Model: From Desktop PC to Jetson Nano 177
12.1 Introduction 177
12.2 Model Training on a PC 177
12.3 ONNX Model Inference 177
12.3.1 PyTorch's.pt to ONNX Conversion 177
12.3.2 Image Classification Using ONNX 179
12.3.3 PyTorch's.pth to ONNX Conversion 181
12.3.4 Object Detection Using ONNX 181
12.4 Conclusion 186
12.5 Exercise Problems 186
13 Setting up Raspberry Pi 187
13.1 Introduction to Raspberry Pi 187
13.2 Hardware and Power Requirements 188
13.2.1 Direct Setup 190
13.3 Operating System Setup 190
13.3.1 Flashing the OS Image 191
13.3.2 Setting up OS 194
13.4 Create Virtual Environment 197
13.5 PyTorch and OpenCV Installation 198
13.6 Other Essential Packages 201
13.7 Conclusion 206
13.8 Exercise Problem 207
14 Deploy Deep Learning Models on Raspberry Pi 209
14.1 Introduction 209
14.2 Face Detection and Recognition in Video Feeds 209
14.2.1 Face Detection Using HOG 209
14.2.2 Face Recognition Using Transfer Learning 211
14.3 Real-time Object Detection 214
14.3.1 Object Detection Using YoloV3 214
14.4 Real-time Classification 217
14.4.1 Classification Using PyTorch's Quantized Model 217
14.5 Real-time Segmentation 219
14.5.1 Segmentation Using k-means Clustering 219
14.6 Exercise Problems 221
15 Trained PyTorch Model: From Desktop PC to Raspberry Pi 225
15.1 Introduction 225
15.2 Model Training on a Desktop PC 225
15.3 PyTorch's.pth to ONNX 225
15.3.1 ONNX Runtime 225
15.3.2 Model Conversion in Raspberry Pi 5 226
15.4 ONNX Model Inference on Raspberry Pi 5 228
15.4.1 Import Libraries and Define Configurations 228
15.4.2 Image Preprocessing 228
15.4.3 Model Inference with Bounding Boxes 229
15.5 Conclusion 231
15.6 Exercise Problems 232
Index 235
