Maximum Likelihood Estimate by Automatic Differentiation | Directed Graphical Models

In this video, we will use the Maximum Likelihood Estimate to fit the parameters of a DGM with two random variables. Both are observable and are modelled as Bernoulli distributions. We take advantage of TensorFlow's Automatic Differentiation to solve the optimization problem using a gradient-based algorithm.

Here are the notes: https://raw.githubusercontent.com/Ceyron/machine-learning-and-simulation/main/english/probabilistic_machine_learning/directed_graphical_models_maximum_likelihood_estimate_automatic_differentiation.pdf

TensorFlow Probability is a great toolbox for modelling joint distributions by Directed Graphical Models. Calculating the log likelihood creates a computational graph which can be used by TensorFlow to acquire the gradients needed for an ADAM-Optimizer. It works magically and removes the barrier of finding the derivatives.

-------

? : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation

? : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler

? : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim

-------

⚙️ My Gear:

(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)

- ?️ Microphone: Blue Yeti: https://amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: https://amzn.to/3JhEtwp
- ? Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): https://amzn.to/37katmf
- ? Laptop Charger: https://amzn.to/3ja0imP
- ? My Laptop (generally I like the Dell XPS series): https://amzn.to/38xrABL
- ? My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): https://amzn.to/3Jr4ZmV

If I had to purchase these items again, I would probably change the following:

- ?️ Rode NT: https://amzn.to/3NUIGtw
- ? Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): https://frame.work

As an Amazon Associate I earn from qualifying purchases.

-------

Timestamps:
0:00 Opening
0:27 The Directed Graphical Model
02:19 The Task of Fitting Parameters
05:05 Deriving the Likelihood
07:51 The Log-Likelihood
09:32 The Optimization Problem & Automatic Differentiation
11:40 TFP - Defining the Model
13.48 TFP - Generating (artificial) data
16:05 TFP - Defining the (transformed) variables
19:52 TFP - Solving the optimization problem
22:22 TFP - Investigating the optimal variables & Discussion