How to do Data Cleaning in Machine Learning with an Example and Coding in Python || Lesson 15 ||

#machinelearning#learningmonkey

In this class, we will discuss How to do Data Cleaning in Machine Learning with an Example and Coding in Python.In our next classes will take a real-time data set and apply data cleaning techniques on the data set.In machine learning.1) collect the data2)  we have to apply a model on this data3)  Then using this model we have to predict our future data.These are the steps involved in machine learning.Coming here after collecting the data what we have to do is we have to clean the data.After cleaning the data we have to apply data pre-processing techniques after completion of data pre-processing then we have to apply a model on this preprocessed data.we will discuss data cleaning techniques in this class than will take a real-time data set and will apply data cleaning techniques using Python and then we go for data preprocessing.Coming to data cleaning methods these are some of the data cleaning methods which we discuss in this class.The important point you have to remember here is there is no standard data cleaning technique.Based on the data and domain knowledge we have to identify our data cleaning techniques.What is meant by domain knowledge? we understand this at the end of the class.These are some of the data cleaning methods which we discuss. these are frequently used data cleaning methods.don't think that these are the standard methods.1) Deleting the duplicate values if you have any duplicate values in a data set removes duplicate values.2) Missing dataWe are working on a cancer prediction data set. in this cancer prediction, dataset PID means patient ID.Cancer patients will be tested with different tests. one of the tests is blood tests. this will check the red blood cell and white blood cells count.For suppose if you are having missing data what we have to do here.It depends on the situation based on the situation we do two thingsHere whether we have to drop this missing data.Replace this missing data.In what situations we have to choose to drop.let's take an example in the cancer data set we collected 2 lakh data points.This means it's a large data set. in machine learning algorithms really we need large data sets then only will get better predictions.Here only 1000 data points have missing values. so that I can drop the missing data. because I am having lots of data because of 2 lakh data. I can eliminate 1000.If I have only a thousand data points. in that 200 missing data points are there. can I eliminate 200 data points? no, because the dataset is small.We have to choose a method to replace that missing data value.1) Replace with the mean value. take the mean value of all this column value then replace it with the mean value.2) Replace with median value.3) Replace with mode value.Coming to the next data cleaning method we have to eliminate unrelated data.For example, here we are predicting whether a person is having cancer or not?In this data set is its phone number is important?We predict based on the test values.We can drop the column phone number. this is what unrelated data.Coming to the next method. typo mistakes.Taking an example of this gender value some people may write it as f for females.Some write female as fully.what we have to do during the data cleaning process we have to convert the female to f. to choose one method which feature is flexible for us.Coming to the next one is type conversion.What do you mean by type conversion?Take an example of white blood cells count 1 lakh.What that means is a numeric data.But in the data set will have it as 1 l means 1 lakh.so remove this make it as a numeric value.we have to do type conversion string type to numeric values.Coming to domain knowledge what is meant by domain knowledge?Let's assume that I am working in a real estate business from the last 10 years this is what domain means.In this domain, I have some knowledge because I am working in it for the last 10 years to my expertise in related business.Let's take the color of the house. we have to keep this column or not?Really customers give importance to the color of the house if yes we have to keep it. if no remove it.They are not giving importance to the color of the house. they are giving importance to only which location it is . how many bedrooms. how many bathrooms.Then who will tell that the color of the house is not giving importance?The expert will tell you because he is having some experience in this field.We have to meet the main expert and we have to get an understanding of the data.we do python coding in our next session. install python is given here.In this session we learn How to do Data Cleaning in Machine Learning with an Example and Coding in PythonLink for our website: https://learningmonkey.in

Follow us on Facebook @ https://www.facebook.com/learningmonkey

Follow us on Instagram @ https://www.instagram.com/learningmonkey1/

Follow us on Twitter @ https://twitter.com/_learningmonkey

Mail us @ [email protected]