Multivariate Normal | Intuition, Introduction & Visualization | TensorFlow Probability

More than one random variable is normally distributed. So they can be jointly distributed. For this we need covariances. Here are the notes: https://raw.githubusercontent.com/Ceyron/machine-learning-and-simulation/main/english/essential_pmf_pdf/multivariate_normal_intro.pdf

The Normal distribution is ubiquitous in Machine Learning and Statistics. In naturally arises in so many application scenarios. But that is not due to the univariate but due to the Multivariate Normal, i.e., a Normal that is defined over more than just one axis. Multivariate Normal Distributions with more than 1000 axes (=dimensions) are common practice.

But we need some basic linear algebra in order to understand. Still, a visual introduction is the best way to start.

Here you can find an interactive web plot for the Multivariate Normal: https://share.streamlit.io/ceyron/numeric-notes/main/english/essential_pmf_pdf/multivariate_normal_interactive_chart.py

In this video, we will introduce the concepts of covariance and the covariance matrix. We will see that we need the Cholesky decomposition to find the analogy to the standard deviation in order to evaluate the Normal distribution.

The last part of the video will be on how the Multivariate Normal is implemented in TensorFlow Probability. Here, we will also see what it means if the Cholesky Decomposition fails.

Do you want to take a look at the Python code I wrote for the Web Application? Here it is: https://github.com/Ceyron/numeric-notes/blob/main/english/essential_pmf_pdf/multivariate_normal_interactive_chart.py

-------

? : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation

? : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler

? : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim

-------

⚙️ My Gear:

(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)

- ?️ Microphone: Blue Yeti: https://amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: https://amzn.to/3JhEtwp
- ? Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): https://amzn.to/37katmf
- ? Laptop Charger: https://amzn.to/3ja0imP
- ? My Laptop (generally I like the Dell XPS series): https://amzn.to/38xrABL
- ? My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): https://amzn.to/3Jr4ZmV

If I had to purchase these items again, I would probably change the following:

- ?️ Rode NT: https://amzn.to/3NUIGtw
- ? Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): https://frame.work

As an Amazon Associate I earn from qualifying purchases.

-------

Timestamps:
00:00 Introduction
00:47 Two Normally Distributed Random Variables
01:06 Parameters for univariate Normal Distributions
01:54 Interaction by Covariances
02:56 Random Vector
03:18 Proportional PDF
06:01 Parameters of the Multivariate Normal
07:23 A 3D Surface Plot
09:10 Going into higher dimensions
09:43 The Normalization Constant
11:20 Requirements on the Parameters
12:05 Symmetric Positive Definiteness
12:31 Cholesky Decomposition
14:28 The Precision
16:50 Plot: Intro
17:47 Plot: Shifting/Moving
18:06 Plot: Changing Variance
18:41 Plot: Changing Covariance
19:32 Plot: Symmetric Positive Definiteness
20:26 TFP: Defining the Parameters
21:12 TFP: Cholesky Decomposition
21:50 TFP: when Cholesky fails
22:51 TFP: Cholesky and Standard Deviation
23:51 TFP: Defining Multivariate Normal
24:37 TFP: Sampling
24:52 TFP: The Mode
25:03 TFP: Querying (Log-) Probability
25:45 TFP: Lazy Defining
26:04 Outro