The interesting part?
Transformations don't execute immediately. Spark builds the execution plan first and only starts computing when an Action is triggered.
Small concepts, but they make Spark's execution model much clearer.
#PySpark#Spark#DataEngineering
106K Followers 3K FollowingCloudera is the only data and AI platform company that brings AI to data anywhere: in clouds, data centers, and at the edge.
107K Followers 6K FollowingWe're on a mission to help unleash the potential of every team.
Maker of collaboration tools like @Jira, @Confluence, @Loom, @Trello, and more.