Apache-Spark-Structured-Streaming-Via-Docker-Compose
A docker-compose based project that builds a structured streaming pipeline using Apache Spark, Kafka, ROS, and Python. The pipeline starts with a ROS robotic simulation that publishes odometry data representing vehicle position and orientation. A Python script captures this data and forwards it to a Kafka broker running inside a Docker container. Apache Spark Structured Streaming, also deployed via Docker, consumes the Kafka topic in real time to process the incoming data. The project provides installation instructions for ROS Noetic on Ubuntu 20.04, Docker, and the kafka-python library, along with scripts to set up the ROS odometry publisher and verify the published topics. It demonstrates an end-to-end streaming architecture that integrates robotic simulation, message brokering, and distributed stream processing, making it useful for learning how to combine ROS, Kafka, Spark, and Docker into a unified real-time data pipeline.