Building your own programming language in Python3 - Part 1: LEXER

In this video we will be creating a Lexer, known to be the first component needed towards the creation of an interpreter (Lexer-Parser-Interpreter).

The video attempts to re-explain the work of David Callanan (https://github.com/davidcallanan/py-myopl-code) in a manner that's better suited for the context of our local Structure of Programming Languages course.

00:31 Theory (what is a Lexer?)
3:28 The Lexer class diagram
13:02 Running the finished program to show what it does
15:24 Creating the driver file/program
15:43 Behold the neithan++ programming language (haha!)
18:09 Coding the Token Class
26:26 A designated practice area for your code
29:53 Coding the Lexer Class
40:45 Coding the Runner/Driver function
44:41 Tokenizing numbers
55:11 Demonstrating our lexer version 1

We first try to look at the theory behind the behavior of a lexer (assigning meaning to numbers and operators through token creation). We then proceed with coding the Token and Lexer classes and use a driver program to demonstrate it's utility.

In succeeding videos, we will expand the custom language to support the use of variables, conditionals and functions.

This video is created specifically for the CSCI70 class 1st Semester A/Y 2022-2023. I've prepared a public semester-long problem set for the class that you may also try solving here: https://assessment.hackerearth.com/challenges/college/midnightsnack12/leaderboard/

References:
David Callanan loosely based his tutorials on this resource:
https://ruslanspivak.com/lsbasi-part1/
David Callanan's playlist: https://www.youtube.com/watch?v=Eythq9848Fg&list=PLZQftyCk7_SdoVexSmwy_tBgs7P0b97yD&index=1