12 1 Maximum Likelihood | Machine Learning

*APPROACHES TO DATA MODELING*
Our approaches to modeling data thus far have been either probabilistic or
non-probabilistic in motivation.
I Probabilistic models: Probability distributions defined on data, e.g.,
1. Bayes classifiers
2. Logistic regression
3. Least squares and ridge regression (using ML and MAP interpretation)
4. Bayesian linear regression
I Non-probabilistic models: No probability distributions involved, e.g.,
1. Perceptron
2. Support vector machine
3. Decision trees
4. K-means
In every case, we have some objective function we are trying to optimize
(greedily vs non-greedily, locally vs globally).

*MAXIMUM LIKELIHOOD*
We’ve discussed maximum likelihood for a few models, e.g., least squares
linear regression and the Bayes classifier.
Both of these models were “nice” because we could find their respective θML
analytically by writing an equation and plugging in data to solve.
Gaussian with unknown mean and covariance
In the first lecture, we saw if xi ∼iid N(µ; Σ), where θ = fµ; Σg, then
rθ ln
nYi=1
p(xijθ) = 0
gives the following maximum likelihood values for µ and Σ:
µML =
1 n
nXi=1
xi; ΣML = 1
n
nXi=1
(xi − µML)(xi − µML)

*COORDINATE ASCENT AND MAXIMUM LIKELIHOOD*
In more complicated models, we might split the parameters into groups
θ1; θ2 and try to maximize the likelihood over both of these,
θ1;ML; θ2;ML = arg max
θ1;θ2
nXi=1
ln p(xijθ1; θ2);
Although we can solve one given the other, we can’t solve it simultaneously.
Coordinate ascent (probabilistic version)
We saw how K-means presented a similar situation, and that we could
optimize using coordinate ascent. This technique is generalizable.
Algorithm: For iteration t = 1; 2; : : : ;
1. Optimize θ1(t) = arg maxθ1 Pn i=1 ln p(xijθ1; θ2(t−1))
2. Optimize θ2(t) = arg maxθ2 Pn i=1 ln p(xijθ1(t); θ2)

*COORDINATE ASCENT AND MAXIMUM LIKELIHOOD*
There is a third (subtly) different situation, where we really want to find
θ1;ML = arg max
θ1
nXi=1
ln p(xijθ1):
Except this function is “tricky” to optimize directly. However, we figure out
that we can add a second variable θ2 such that
nXi=1
ln p(xi; θ2jθ1) (Function 2)
is easier to work with. We’ll make this clearer later.
I Notice in this second case that θ2 is on the left side of the conditioning
bar. This implies a prior on θ2, (whatever “θ2” turns out to be).
I We will next discuss a fundamental technique called the EM algorithm
for finding θ1;ML by using Function 2 instead.

*Find videos about :-*

#ArtificialIntelligence #AI #DataScience #MachineLearning #DeepLearning #NeuralNetworks #ArtificialNeuralNetwork #ann #ConvolutionalNeuralNetwork #cnn #RecurrentNeuralNetwork #rnn #LongShortTermMemory #lstm #GatedRecurrentUnit #gru #ComputerVision #NaturalLanguageProcessing #nlp #Nltk #Spacy #Tensorflow #LinearRegression #LogisticRregression #KNearestNeighbour #knn #DecisionTree #RandomForest
#SupportVectorMachine #svm #Sklearn #Python #Django #DjangoRestFramework