Multiprocessing in Python: Pool

This video is sponsored by Oxylabs. Oxylabs provides market-leading web scraping solutions for large-scale public data gathering. You can receive data in JSON or CSV format and pay only per successful request. At the moment, Oxylabs offers a free trial.
https://oxylabs.io/?utm_source=lucidprogramming&utm_medium=cpc&utm_campaign=lucidprogramming_youtube_sponsor&adgroupid=20220228

In this video, we will be continuing our treatment of the multiprocessing module in Python. Specifically, we will be taking a look at the "Pool" class, and how we can go about using the Pool class to instantiate tasks that are run across multiple processors on our machine.

According to the official documentation (https://docs.python.org/2/library/multiprocessing.html):

"multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows."

Software from this video:
https://github.com/vprusso/youtube_tutorials/blob/master/multiprocessing_and_threading/multiprocessing/multiprocessing_pool.py

For more videos on multiprocessing:
http://bit.ly/lp_multiprocessing

Do you like the development environment I'm using in this video? It's a customized version of vim that's enhanced for Python development. If you want to see how I set up my vim, I have a series on this here:
http://bit.ly/lp_vim

If you've found this video helpful and want to stay up-to-date with the latest videos posted on this channel, please subscribe:
http://bit.ly/lp_subscribe