Skip to content

File Reader #3

Open
Open
@deusalexmachina

Description

@deusalexmachina

Read in every dataset in the directory into Python

  • Use os module in Python std lib
  • os.path
  • Convert to rdd for Spark

Reference Code (from hw):

spark = SparkSession.builder.getOrCreate()
park_violations = spark.read.csv(sys.argv[1], header=True, inferSchema=True)

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions