Stream Processing Architectures for Real-Time Internet of Things Data

Abstract

This study provides a comprehensive comparative review of stream processing architectures for real-time Internet of Things (IoT) data, with particular emphasis on their applicability to Indonesia’s growing digital ecosystem. Three dominant models—Lambda, Kappa, and Edge Processing—were systematically analyzed based on performance metrics such as latency, scalability, and fault tolerance. The research integrates qualitative literature review and quantitative benchmarking using established frameworks, including Apache Kafka, Spark Streaming, and Apache Flink. Results indicate that the Lambda Architecture demonstrates the highest accuracy and resilience but involves complex deployment, while the Kappa Architecture offers simplified scalability through a continuous streaming paradigm. Edge Processing achieves the lowest latency but presents security and data integrity challenges. Considering Indonesia’s diverse network infrastructure, a hybrid edge-cloud model emerges as the most effective architecture for national IoT implementation. This approach combines real-time responsiveness with centralized reliability, making it well-suited for critical domains such as transportation, healthcare, and smart cities. Future research should focus on large-scale pilot testing of hybrid architectures, including AI-based anomaly detection and security protocols, to enhance operational reliability and ensure sustainable IoT deployment.