EduByte.tech
Start learning
Apache Spark·Apache Spark essentials

DataFrames

Spark DataFrames are distributed tables with lazy transformations.

Advanced 10 min read Lesson 1 of 150 Challenge included
What you will be able to do
  • Create a SparkSession.
  • Read files into DataFrames.
  • Actions trigger execution.

In plain English

Spark DataFrames are distributed tables with lazy transformations.

  • Create a SparkSession.
  • Read files into DataFrames.
  • Actions trigger execution.
spark.pyPython
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName('edubyte').getOrCreate()
df = spark.read.csv('lessons.csv', header=True, inferSchema=True)
df.show(5)
Your turn

Explain it back

Easy

Answer from memory: When does Spark execute transformations?

TEXT · starter
Your answer…
Check your understanding before moving on.
Knowledge checkQuestion 1

When does Spark execute transformations?

Select an answer to continue
Study tools

Make it yours.

Keep a private note for this lesson, then turn the key ideas into flashcards you can revisit.

Sign in to save notes and flashcards.Both are private to your account and stay with your progress.
Sign in
Finished reading?

Lock in what you just learned.

Mark this lesson complete to save your place and move one step closer to the certificate.

Next: Transformations
Lesson 1 of 150 · 0% of course