Spark Kinesis Example – Moving Beyond Word Count
If you are looking for Spark with Kinesis example, you are in the right place. This Spark Kinesis tutorial intends to help you become better at integrating the two. In this post, I’m going to provide a custom Spark Kinesis code example and a screencast of running it. We’re going to cover running, configuring, sending […]
Spark Performance Monitoring Tools – A List of Options
Which spark performance monitoring tools are available to monitor the performance of your Spark cluster? Let’s find out. Before we address this question, I assume we already know Spark includes monitoring through the Spark UI. In addition, Spark includes support for monitoring and performance debugging through the History Server as well as support for the […]
Spark FAIR Scheduler Example
Scheduling in Spark can be a confusing topic. When someone says “scheduling” in spark, do they mean scheduling applications running on the same cluster? Or, do they mean the internal scheduling of Spark tasks within the Spark application? So, before we cover an example of utilizing the Spark FAIR Scheduler, let’s make sure we’re on the […]
Spark Tutorial – Performance Monitoring with History Server
In this Apache Spark tutorial, we will explore the performance monitoring benefits when using the Spark History server. This Spark tutorial will review a simple Spark application without history server and then revisit the same Spark app with history server. We will explore all the necessary steps to configure Spark History server for measuring performance […]
Apache Spark Thrift Server Load Testing Example
Wondering how to do perform stress tests with Apache Spark Thrift Server? This post will describe one way to do it. What is Apache Spark Thrift Server? Apache Spark Thrift Server is based on the Apache HiveServer2 which was created to allow JDBC/ODBC clients to execute SQL queries using a Spark Cluster. From my […]
Spark Thrift Server with Cassandra Example
Want to use joins with Cassandra? Or, help people already familiar with SQL leverage your Spark infrastructure? They can use their existing SQL based tools they already know such as Tableau or even MS Excel. Maybe both? This post describes how to provide answers. It describes how to configure the Apache Spark Thrift Server with […]
Spark Streaming with Kafka Example
Spark Streaming with Kafka is becoming so common in data pipelines these days, it’s difficult to find one without the other. This tutorial will present an example of streaming Kafka from Spark. In this example, we’ll be feeding weather data into Kafka and then processing this data from Spark Streaming in Scala. As the data […]
Recent Posts
- Spark Kinesis Example – Moving Beyond Word Count October 20, 2017
- Spark Performance Monitoring Tools – A List of Options September 18, 2017
- Spark FAIR Scheduler Example September 15, 2017
Most Commented