Scaling Mindgate’s Payment Infrastructure with YugabyteDB
April 25, 2023
Nikhil Mahawar is the Head of Product Engineering and Innovation at Mindgate, where he has worked as a technical architect for the past 10 years. Nikhail has been instrumental in driving Mindgate’s success, which has become a leading player in the payments industry since its founding in 2008, with almost one thousand payment professionals and over 50 global customers in the Middle East and Southeast Asia. Mindgate processes close to 3 billion digital payments monthly at a transaction speed of 20,000+ TPS per second. They process 50% of all UPI* transactions, and their payments platforms are utilized by 5000+ merchants, and some of the world’s largest banks.
NOTE: Unified Payments Interface (UPI) is a system that merges multiple bank accounts into one mobile application, merging several features, seamless fund routing, merchant payments into one interface. It also supports “Peer to Peer” collect requests that can be scheduled and paid conveniently.
At a recent Distributed SQL Summit, Nikhil discussed some of the operational challenges Mindgate faced with their legacy RDBMS systems and how that impacted system performance and caused longer wait times. He also shared how migrating to YugabyteDB has resulted in better scalability, performance, and reliability, helping Mindgate to achieve their growth objectives.
Continue reading the excerpt of this discussion, or watch Nikhil’s entire presentation.
Real-Time Payments RDBMS Use Cases
Over the last eight years, we [Mindgate] have been using RDBMS to support our applications, but there are some operational challenges when it comes to deploying real-time payments on traditional, monolithic RDBMS systems. The growth of Unified Payments Interface (UPI)-based real-time payments in India has stretched the limits of our traditional RDBMS infrastructure and led us to more modern databases like YugabyteDB.
Why? Real-time payment apps require high scalability, predictable performance, and extremely reliable data, much like other modern applications across different industries.
With the growing number of transactions, we are now processing close to 50 million transactions per day, just for our largest customer. So, we have begun moving our payments system to YugabyteDB’s distributed SQL database to deliver the scale and enable the active-active setups required for critical applications like UPI.
Challenges with Legacy RDBMS
The key challenges we face are both real-time and operational. In terms of real-time challenges, we experience a large pile-up of active sessions during peak periods, reaching up to 5,000 at times.
During these periods, we saw a slow response time from the database. Typical good response times are under 50 milliseconds; however, when large volumes of transactions are coming in, response times can be as high as 200 to 300 milliseconds. This can cause our system to choke and impact our applications’ high throughput, resulting in declining transactions.
Additionally, we saw index contentions in large transaction data tables during surges such as IPL (Indian Premier League) matches or blast sales, which the database struggles to handle.
We also encountered high interconnect latency between multiple nodes, caused by errors with the data synchronization between nodes. We also experience high I/O latency on storage disks. These high latencies cause longer wait times, which impacts our system’s overall performance.
Other operational challenges include:
- High cross-site replication times
- RTO/RPO that took 30-60 minutes on business continuity planning (BCP) drills due to database replication dependencies
- Manual interventions required during backups and archivals
Migration to YugabyteDB
There were some key factors that guided our migration strategies to YugabyteDB.
The Yugabyte database offers low-latency reads and high-throughput writes, making it a preferred option. The active-active deployment topology—with synchronous replication of data in clusters dispersed across data centers and regions—was also a crucial factor for us. Additionally, the fact that Yugabyte leverages the Raft Consensus Algorithm to achieve high availability across sites, with excellent performance, was also a deciding factor.
YugabyteDB also enables bi-derctional replication of data between clusters, so that both clusters can perform reads and writes.
In addition, the built-in automatic sharding mechanism helps distribute data across multiple database shards, which can be stored on multiple machines. This increases the total storage capacity of the system, avoiding a pile up of transactions on a single data table.
Moreover, YugabyteDB can support SQL and NoSQL in the same cluster, enabling us to run applications with full distributed ACID (atomicity, consistency, isolation, and durability) support for our typical real-time payments scenarios. This ensures database transaction validity in the event of system crashes or failures.
Benefits of YugabyteDB
The scalability of YugabyteDB’s core architecture has provided many benefits. It is horizontally scalable and includes automatic failover and native repair mechanisms. All nodes are equal, eliminating any single point of failure. Although there is a leader and a follower, all nodes work together seamlessly from a data perspective.
YugabyteDB includes a proven DocDB document store, which is found across YSQL and YCQL APIs. The database was built using a custom integration of Raft replication logic, full support for distributed ACID transactions across multiple rows, shards and, nodes, and the RocksDB storage engine (which has already been proven in large RDBMS setups).
The main advantages we’ve observed with YugabyteDB are rooted in its highly scalable core architecture. It boasts linear scalability, and an automatic failover and native repair mechanism, resulting in no single point of failure with support for hybrid/multi-cloud deployments.
—Nikhil Mahawar, Head of Product Engineering and Innovation, Mindgate
Objectives Achieved with YugabyteDB
YugabyteDB has effortlessly supported our exponential growth with hyperscalability, performance efficiency, and reliability. It gives us cloud agility with security as well, helping us migrate from an on-premises setup to the cloud.
Today, we have many successful cloud-based products running on YugabyteDB that give us some provisional economic business value now compared to other RDBMS systems.
YugabyteDB is interoperable with a flexible design, so our applications can use multiple database features. There’s also a key feature called change data capture (CDC). CDC provides technology to ensure any changes in data due to operations such as inserts, updates, and deletions are identified, captured, and automatically applied to another data repository instance, or made available for consumption by applications and other tools. We use CDC to to sync up with any of the data warehouse or data lake systems that our banking customers want us to work with.
