Implementing Neural Networks from scratch using Python. The first 3 assignments included implementing neural networks with different architecture requirements, different activation and loss functions, dropout, batchnorm, and learning rate decay.
The second 3 assignments included implementing different variations of CNN, GAN, and DANN using PyTorch.
-
python 3.7 (Anaconda)
-
pytorch >= 1.0
-
torchvision >= 0.2.2
-
SVHN(Source dataset), MNIST(Target dataset)
-
Download from torchvision
-
DANN.ipynb : DANN model and training algorithm
-
NN.ipynb : Baseline model and training algorithm to compare with DANN
-
The models were trained by 20~30 epochs respectively
-
You can see the graphs of loss and accuracy in ipynb
-
Implementation and Result
- GRL (Gradient Reversal Layer)
- 3 Modules (Feature extractor, Classifier, Discriminator)
- Data preprocessing (Resize 28x28, gray scale, normalization [-1, 1], )
- 2 Datasets (SVHN, MNIST)
- Test Accuracy (on MNIST test set)
- DANN: 80.81 %
- Baseline model (Source only model): 56.47 %