Section2 - Chapter5:- Machine Learning for Cyber Security

Section2 - Chapter5:- Machine Learning for Cyber Security

* Introduction :- Intro to Machine Learning and Cyber Security
* Section1-Chapter1 :- ML Development environment using Docker, Conda, Jupyter Notebook , Kubernetes, Intellij etc.
* Section1-Chapter2 :- Feature Extraction, Dimensionality Reduction, Data Labelling using Unsupervised Learning .
* Section1-Chapter3 :- Supervised Learning (Malware Classification), Classification model comparison.
* Section1-Chapter4 :- AutoML Frameworks (H2O autoML)
* Section2-Chapter1 :- Machine Learning with Apache Spark on Kubernetes .
* Section2-Chapter2 :- End to End ML Pipeline using Apache Spark using Jupyter Notebook(s)
* Section2-Chapter3 :- End to End ML Pipeline using Apache Spark using CI/CD approach
* Section2-Chapter4 :- Integration of XGBoost and H2O autoML with Spark.
* Section2-Chapter5 :- GPU accelerated training using RAPIDS (a cuda framework based software)
* Section2-Chapter6 :- Deep Learning using Tensorflow/Keras on GPU
* Section2-Chapter7 :- TBD.

GPU vs Cluster Computing
GPU vs Cluster Computing cost and performance
RAPIDS, cuDF, cuML, cuGrapgh, cuDNN

AWS GPU installation
CUDA and RAPIDS installatoon on GPU
NVIDIA_Docker installation

Contact Me
LinkedIn :- https://www.linkedin.com/in/shaileshp...
Gmail :- [email protected]

Links discussed in the tutorial
GitHub :- https://github.com/ez-ml/ML-For-Cyber-Security
Docker Desktop :- https://docs.docker.com/desktop/
Kubernetes Kubectl :- https://kubernetes.io/docs/tasks/tools/install-kubectl/
IntelliJ IDEA :- https://www.jetbrains.com/idea/download
Docker Hub :- https://hub.docker.com/
Docker best practises :- https://docs.docker.com/develop/develop-images/dockerfile_best-practices #dockerfile-instructions

1) Installation of cuda framework on the host
curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
sudo add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
sudo apt update -y
sudo apt install cuda -y

2) Verify GPU and cuda
nvidia-smi

3) Install nvidia docker

docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo pkill -SIGHUP dockerd
sudo nvidia-container-cli --load-kmods info

4) Verify nvidia docker

sudo docker run --runtime=nvidia -it nvidia/cuda nvidia-smi


5) Run docker image with RAPIDS
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 rapidsai/rapidsai

6) Download dataset
https://drive.google.com/file/d/1Fkyk1GdgDibTC7if1cGyBA5V9HaHh_GO
Note :- Please comment in an underlying video with you email address, and i will share the given dataset with you .