Want to learn how Fyber built a spark pipeline on Kubernetes using Hashicorp Terraform and Consul?

In the video below, you can watch a detailed walkthrough of how Fyber built an epic-scale, stream-ingested data platform using Spark, Airflow, K8s and Spotinst Ocean to maintain costs and keep operational overheads extremely low.

Our data pipeline solution: “Spark On-Demand”

An overview of Fyber’s On-Demand Spark architecture and how it leverages Kubernetes and Spotinst Ocean, playing together over a single AirFlow workflow. 

Outline of the webinar

• Introduction to Spotinst Ocean: A serverless container engine
• Introduction to HashiCorp Terraform and Consul
• Introduction to Fyber: An on-demand big data architecture using Apache Spark, Kafka, Kubernetes, AirFlow, Spotinst Ocean, and more.
• Live Demo: A working example of running a Spark workload on the Fyber platform (Kubernetes and Spotinst Ocean-based)

Click here, to view Hashicorp’s web page of the joint webinar.

Fyber’s big data architecture

Below is a meet-up recording conducted on the same topic in Israel (Hebrew):