Neural Networks & Deep Learning — Coursework Repository

Institution: Aristotle University of Thessaloniki (AUTH) — School of Electrical and Computer Engineering (ECE)
Course: Neural Networks & Deep Learning

This repository contains the code and reports for the three assignments of the Neural Networks & Deep Learning university course.

---

Assignment 1 — Convolutional Neural Networks for Image Classification on CIFAR-10

The first assignment focuses on building and evaluating Convolutional Neural Network (CNN) architectures for multiclass image classification on the CIFAR-10 dataset, with comparative analysis against classical baseline classifiers.

Project Overview

The task is to build and compare different classification models on the CIFAR-10 dataset across 10 categories. The notebook covers data loading, preprocessing, CNN architecture design, training, and performance evaluation. Results are compared against K-Nearest Neighbors (KNN) and Nearest Centroid (NC) classifiers.

The general workflow is as follows:

1. Load Data: The CIFAR-10 dataset is loaded via keras.datasets.
2. Preprocess Data: Images are normalized and labels are one-hot encoded.
3. Define Model: CNN architectures are defined using the Keras Sequential API, with convolutional, pooling, and dense layers.
4. Compile and Train: Models are compiled using the Adam optimizer and categorical cross-entropy loss.
5. Evaluate: Trained models are evaluated on the test set for accuracy and loss.
6. Visualize Results: Training and validation curves are plotted to assess learning progress.
7. Baseline Comparison: KNN and Nearest Centroid classifiers are applied on flattened image vectors for comparison.

Libraries and Frameworks

• Keras / TensorFlow: Used for building, compiling, and training the CNN models.
• NumPy: Numerical operations and array manipulation.
• Scikit-learn: Implementation of KNN and Nearest Centroid baseline classifiers.
• Matplotlib: Visualization of training curves and results.

Code Files

File	Description
train_cnn_cifar10.ipynb	Full pipeline: CNN architecture, training, evaluation, and comparison with KNN and NC baselines.
report_cnn_cifar10.pdf	Technical report documenting methodology, experiments, and results.

---

Assignment 2 — Support Vector Machines for Image Classification on CIFAR-10

The second assignment shifts focus from deep learning to classical machine learning methods. It implements and evaluates Support Vector Machines (SVMs) for binary and multiclass classification on CIFAR-10, with comparisons against a Multilayer Perceptron (MLP), KNN, and Nearest Centroid.

Project Overview

The workflow includes intensive preprocessing steps such as PCA for dimensionality reduction and hyperparameter tuning via grid search.

1. Load Data: CIFAR-10 dataset is loaded via keras.datasets.
2. Preprocess Data: Images are flattened, normalized, standardized using StandardScaler, and reduced in dimensionality using PCA.
3. Binary Classification: An SVM is trained on two selected CIFAR-10 classes.
4. Multiclass Classification: SVM is extended to the full 10-class problem using one-vs-one strategy.
5. Hyperparameter Tuning: GridSearchCV is used to find optimal kernel type and regularization parameters.
6. MLP Comparison: A simple Multilayer Perceptron is built using tensorflow.keras as an additional comparison model.
7. Baseline Comparison: KNN and Nearest Centroid are applied as additional baselines.

Libraries and Frameworks

• Keras / TensorFlow: Used for loading the CIFAR-10 dataset and building the MLP comparison model.
• Scikit-learn: SVM (SVC), GridSearchCV, StandardScaler, PCA, accuracy_score, hinge_loss, KNN, Nearest Centroid.
• NumPy: Numerical operations.
• Matplotlib: Visualization of results.
• Time: Tracking training and inference time.

Code Files

File	Description
svm_classifier.ipynb	Full pipeline: binary SVM, multiclass SVM, MLP, KNN, and NC classifiers with hyperparameter tuning and performance comparison.
report_svm_cifar10.pdf	Technical report documenting methodology, experiments, and results.

---

Assignment 3 — Radial Basis Function Neural Networks for Classification

The third assignment implements a Radial Basis Function Neural Network (RBFNN) from the ground up for classification tasks on the CIFAR-10 dataset, covering both binary and multiclass scenarios, and comparing against KNN and Nearest Centroid baselines.

Project Overview

Unlike the previous assignments that rely on Keras layers for the full model, this assignment involves a more hands-on implementation of the RBF layer itself.

1. Load Data: CIFAR-10 is loaded via tensorflow.keras.datasets. Classes can be selected for binary or multiclass experiments.
2. Preprocess Data: Images are converted to grayscale, flattened, standardized, and their dimensionality is reduced using PCA.
3. RBF Center Selection: K-Means clustering is applied to the training data to determine the RBF network centers.
4. Model Training: Hidden layer activations are computed using Gaussian radial basis functions, and output layer weights are trained.
5. Evaluation: Model performance is assessed using accuracy score and confusion matrix on the test set.
6. Parameter Analysis: The effect of the number of hidden centers and spread parameter is analyzed.
7. Baseline Comparison: KNN and Nearest Centroid classifiers are applied for direct comparison.

Libraries and Frameworks

• TensorFlow / Keras: Used for loading the CIFAR-10 dataset.
• Scikit-learn: KMeans for center selection, PCA, StandardScaler, KNN, Nearest Centroid, accuracy_score, confusion matrix.
• NumPy: Core numerical operations for RBF activation computation.
• Matplotlib: Visualization of results, confusion matrices, and performance curves.

Code Files

File	Description
train_rbfnn_classifier.ipynb	Full pipeline: data preprocessing, K-Means center selection, RBF network training, evaluation, and baseline comparisons.
report_rbfnn_classification.pdf	Technical report documenting methodology, experiments, and results.

---

How to Run

Clone the repository and navigate to the desired project folder:

git clone https://github.com/eleanazeri/Neural-Networks-Deep-Learning.git
cd Neural-Networks-Deep-Learning

Open the notebook for the assignment of interest:

# Assignment 1 — CNN
cd 01-CNN-Image-Classification
jupyter notebook train_cnn_cifar10.ipynb

# Assignment 2 — SVM
cd 02-SVM-Image-Classification
jupyter notebook svm_classifier.ipynb

# Assignment 3 — RBFNN
cd 03-RBFNN-Classification
jupyter notebook train_rbfnn_classifier.ipynb

Install required dependencies via:

pip install numpy scikit-learn matplotlib tensorflow

---

License

This repository is submitted as academic coursework for the Neural Networks & Deep Learning course at the Aristotle University of Thessaloniki (ECE AUTH). All code and reports are the original work of the author. Redistribution or reuse for academic submission purposes is not permitted.