EMAIL SPAM FILTERING USING PYTHON 3 & ML - A PROJECT BY PUSHPENDER SINGH

We are going to make use of NLTK for processing the messages, WordCloud and matplotlib for visualization and pandas for loading data, NumPy for generating random probabilities for train-test split.

Spam filtering is a beginner’s example of document classification task which involves classifying an email as spam or non-spam (a.k.a. ham) mail. Spam box in your Gmail account is the best example of this. So lets get started in building a spam filter on a publicly available mail corpus. I have extracted equal number of spam and non-spam emails from Ling-spam corpus. The extracted subset on which we will be working can be downloaded from here.

We will walk through the following steps to build this application :

1. Preparing the text data.
2. Creating word dictionary.
3. Feature extraction process
4. Training the classifier
You might be thinking about the mathematical techniques behind the used models like Naive Bayes and SVM. SVM is mathematically complex model where as Naive bayes is relatively easy to understand. You are encouraged to study about these models from online sources. Apart from that, there can be a lot of experiments that can be done in order to find the effect of various parameters like

a) Amount of training data
b) Dictionary size
c) Variants of the ML techniques used (GaussianNB, BernoulliNB, SVC)
d) Fine tuning of parameters of SVM models
e) Improving the dictionary by eliminating insignificant words (may be manually)
f) Some other feature (look for td-idf)