StructType & StructField | Create DataFrame | Nested Schema|Spark Interview question| Spark tutoria

We are making a complete collection of 2019 Spark interview questions and Apache Spark tutorial.This video is an addition to the collection. In this video we talk about Apache Spark StructType and StructField.

StructType is a built in data type and useful for creating customzed schema,nested schema and eliminaring order dependencies in dataframe transformation.

Struct is the field in StructType with parameters
Name
Data Type
Nullable flag
Metadata.

In this video we talk about below points:

StructType

StructField

Add structfields to StructType
a) by passing seq of StructField and b) using various add methods, c) From existing StructType

Compare two schemas

Create dataframe with own customized schema

Create nested schema

Flatten nested schema.

toDF() vs createDataFrame() : https://youtu.be/MDMiPZbI37M

About channel
---------------------
We are trying to make video series on technical interview questions on various skills for experienced and freshers. This is based on the industry experience. We are trying to compile all the

possible questions with answers and give you the core concepts of the topic.

We will present a series on tutorial on various skill sets as well. Our first tutorial will be on Apache Spark and we will present it soon.

Our aim is to make a difference in your learning.

Please subscribe to our channel and stay connected.

Thank you so much.

e-mail:- [email protected]


Our video links
________________

Spark StructType & StructField: https://youtu.be/PTJGwViFpak

Spark application's life cycle - 2 : https://youtu.be/vE9NQ7b1MCU
------------------------------------

Spark application's life cycle - 1 : https://youtu.be/5rOOdIbihHA
------------------------------------

Parquet with Spark : https://youtu.be/I4lxC_ShB_M
--------------------------------

Type safety in Dataframe vs Dataset - Part 2 : https://youtu.be/hKMaB6-bakA
----------------------------------------------------------------------

Type safety in Dataframe vs Dataset - Part 1 : https://youtu.be/JRJzxdrj7YU
----------------------------------------------------------------------

Broadcast variable example : https://youtu.be/JgoxJmOA-e8
---------------------------------------------

Broadcast variable vs Normal variable : https://youtu.be/U2hYRKb7pZ8
-------------------------------------------------------------

RDD vs Dataframe vs Dataset : https://youtu.be/wVVRSwzkBfE
-----------------------------------------------

toDF() vs createDataFrame() : https://youtu.be/MDMiPZbI37M
----------------------------------------------

User defined function Part -1 : https://youtu.be/SHiZzB4n46A
---------------------------------------------

User defined function Part - 2 : https://youtu.be/lpk8shmjRdE
----------------------------------------------

Dataframe : https://youtu.be/qbA8lt8g36g
-----------------

What is Apache Spark : https://youtu.be/nMPBZX_Y7R4
------------------------------------