Hi! Today we’re going to look at Datastax Spark Cassandra Connector. Topics that are covered in this video: Generating a test CSV dataset using Python; Creating a schema in Cassandra; Preparing Jupyter workbench; Reading CSV into a DataFrame; Writing the DataFrame to the Cassandra;
Continue reading...Apache Spark
Literally a silver bullet for sbt merge strategies in projects using Spark structured streaming and Kafka
Recently it was an intolerable pain for me to build a so-called fat jar‘s for my Spark jobs in Scala using sbt. Well, what’s was happening there? (NOTE: if you’re don’t use fat jars to submit Spark jobs and —this is a better way to deal with Spark though– then...
Continue reading...