Learn how to configure content-based routing for the YugabyteDB Change Data Capture (CDC) connector. Instead of streaming all change data capture events from a table into a single Kafka Topic, you can reroute an event to other Kafka Topics based on its content. An example use case for this is complying with GDPR regulations, where data needs to be geolocated based on an event’s content.
In this blog, we will explore how YugabyteDB Change Data Capture (CDC) and open table formats like Apache Iceberg can be used to build data lakes in Amazon S3 using a single copy of the data and achieve low data ingestion latency while avoiding costly rewrites to support updates/deletes and schema evolution.
In this blog, we explore how to stream data from YugabyteDB’s Change Data Capture feature to Snowflake using SnowflakeSinkConnector on Confluent cloud via Kafka connect.
Kafka Connect is a popular tool for scaling and reliably streaming data between Apache Kafka and other data systems. It ships with a JDBC Sink which is used to insert data from Kafka to a database. Although the default JDBC Sink is good for many popular RDBMS it isn’t optimized for distributed SQL databases that provide linear scalability and high availability like YugabyteDB.
In our earlier blog introducing YugabyteDB 2.7,
…
For customers that run Kafka for their streaming data platform, the Kafka Connect Sink plugin handles delivery of specific topic data to a YugabyteDB instance. As soon as new messages are published, the Sink manages forwarding and automatic addition to a destination table.
YugabyteDB is a high-performance, distributed SQL database built on a scalable and fault-tolerant design inspired by Google Spanner. Yugabyte’s SQL API (YSQL) supports most of PostgreSQL’s functionality and is wire-protocol compatible with PostgreSQL drivers.
…
In our previous post “Develop IoT Apps with Confluent Kafka, KSQL, Spring Boot & Distributed SQL”, we highlighted how Confluent Kafka, KSQL, Spring Boot and YugabyteDB can be integrated to develop an application responsible for managing Internet-of-Things (IoT) sensor data. In this post, we will review the challenges and best practices associated with deploying such a stateful streaming application on Kubernetes.
Streaming apps are inherently stateful in nature given the large volume of data managed and that too continuously.
…
In our previous post “5 Reasons Why Apache Kafka Needs a Distributed SQL Database”, we highlighted why Kafka-based data services need a distributed SQL database like YugabyteDB as their highly scalable, long-term persistent data store. In this post, we show how Confluent Kafka, KSQL, Spring Boot and YugabyteDB can be integrated to develop an application for managing Internet-of-Things (IoT) sensor data.
A trucking company wants to track its fleet of IoT-enabled vehicles that are delivering shipments across the country.
…
Modern enterprise applications must be super-elastic, adaptable, and running 24/7. However, traditional request-driven architectures entail a tight coupling of applications. For example, App 1 asks for some information from App 2 and waits. App 2 then sends the requested information to App 1. This sort of app-to-app coupling hinders development agility and blocks rapid scaling.
In event-driven architectures, applications publish events to a message broker asynchronously. They trust the broker to route the message to the right application,
…
