Advancing the Power of Digital Marketing with Massive Scalability and Advanced Monitoring

This interactive marketing and advertising agency helps clients grow with modern, data-centric services. Specializing in email marketing, they recently embarked on a journey to upgrade their database infrastructure to handle transactional batch data more efficiently.

The agency caters to its clients, including major American retailers, by sending personalized emails. To do so, they needed a robust system capable of managing and analyzing large amounts of data.

They combine data-driven analytics with market trend insights to provide impactful marketing solutions. Their core services include:

• Email Marketing: Create targeted, personalized email campaigns to re-engage current customers and attract new ones.
• Digital Marketing: Maximize customer ROI through SEO, PPC, and paid media to better position their clients’ businesses online.
• Ecommerce: Convert clients’ website traffic into profitable leads with user-friendly website designs and digitized product catalogs that support a smooth e-commerce experience.
• Data Technology: Integrate customer data into a streamlined technology stack, for agile, informed business decisions that boost engagement and revenue.

Technical Requirements for the Highly Scalable Application

The digital marketing agency needed a database solution for transactional batch data in their email marketing services, with the following technical requirements for their scalable application:

• Deployment: Single-region, multiple availability zones (Multi-AZ) setup
• Primary Database Requirements:  Massive horizontal scalability was of primary importance. The agency had transactional batch data coming in from different marketing agencies and providers across their growing portfolio. They were looking for a database to scale and perform effectively to support ongoing email marketing campaigns and recommendations.
• Cloud Used: Amazon Web Services (AWS)
• Cluster Size: Three nodes, each with 16 instances (3×16 configuration)
• Workload Characterization: Read heavy, write heavy, 50/50 mix. This included the operational needs of data processing and the analytical requirements of marketing campaign assessments.
• Replication Factor: 3
• Additional Database Requirements 
  • Advanced Monitoring: Required a fully managed DBaaS with advanced monitoring to streamline operations for their small DBA team.
  • Data Handling Features: Essential features included packed rows and incremental data loading via the native COPY command.
  • Search Capability: Needed GIN indexing on JSONB columns for text search requirements.
  • Sharding Techniques: Utilized YugabyteDB’s sharding methods for efficient large dataset management.
  • Parallel Queries: Needed ability to run parallel queries with multi-threaded options at the application level.
  • Query Performance: Needed enhanced query performance compared to the previous MySQL setup, particularly for large table export and import.
  • Bulk loading with YugabyteDB Voyager: Required bulk loading of 120 million records with YugabyteDB Voyager (demonstrating its incremental INSERT scenario).

Challenges with Existing MySQL Implementation

The firm faced challenges with MySQL’s table-level locking, lack of traceability, and the complexity of managing multiple servers. They also encountered issues achieving optimal query performance with the MySQL setup, particularly for large table export and import.

The issues with MySQL that required resolution included:

• Table-level Locking: Significantly increased processing times from file to database and often left the application without the correct dataset available for the marketing team.
• Lack of Traceability: Since changes were not tracked in the database, it was difficult to identify which files had been processed and whether a restart was necessary.
• Suboptimal query performance: Query performance was suboptimal and was particularly noticeable during the export and import of large tables. 
• Complex database structure: The management of a complex database structure, which consisted of two master and two replica instances, was difficult. The setup made parallel data loading possible but added complexity, especially when maintaining four MySQL servers during periods of intense data load and access.

These four issues led the agency to migrate its existing MySQL setup to YugabyteDB Aeon (formerly YugabyteDB Managed). The following architecture outlines the campaign data flow from MySQL to YugabyteDB Aeon.

Building a Highly Scalable and Performant Email Marketing Solution

1. Highly Scalable and Performant Clusters

   The agency’s adoption of YugabyteDB Aeon for scaling its email marketing solution highlights their strategic approach to database management, especially when addressing large (and growing) dynamic workloads and efficient system performance. They used horizontal and vertical scaling to achieve optimal performance and scalability to meet application needs and business objectives.

   Here’s how they utilized YugabyteDB Aeon’s scalability features:

   Horizontal Scalability

   The agency took advantage of YugabyteDB Aeon’s capability to scale horizontally (or out). They were able to add more nodes to the database cluster to handle increased load and data. This method is particularly beneficial for handling:

   • More transactions per second
   • Large numbers of concurrent client connections
   • Large datasets

   Key Features of Horizontal Scaling in YugabyteDB

   • Tablet Sharding and Distribution: YugabyteDB Aeon automatically splits user tables into multiple shards, known as tablets. These tablets are then distributed across the nodes in the cluster.
   • Dynamic Tablet Splitting: As more nodes are added, YugabyteDB Aeon dynamically splits and relocates some tablets to the new nodes. Therefore, each node manages fewer tablets, enhancing the cluster’s ability to handle more transactions and queries in parallel.

   Advantages of Horizontal Scaling

   • Improved Performance: The distribution of workloads across more nodes enables parallel processing of requests, thus reducing response times.
   • Cost-Effectiveness: Commodity hardware can typically be used for horizontal scaling and is generally more affordable than high-end servers.
   • Elastic Scaling: Nodes can be added or removed based on demand. For instance, during high-traffic events like Black Friday/Cyber Monday, additional nodes can be added temporarily and then removed.

   Vertical Scaling

   The agency also capitalized on the ability to vertically scale (or scale up) easily to upgrade existing node hardware. This involved upgrading the CPU, memory, storage, etc.. This simplified operations by keeping the number of nodes the same.

   

2. Advanced Performance Monitoring Capabilities

   With a small database team, the digital marketing agency utilizes YugabyteDB Aeon for advanced, real-time database monitoring, ensuring optimal performance, quick issue resolution, and informed decision-making about scaling and optimizations. They also leverage advanced monitoring capabilities, including:

   • Alerts: Automate the notifications of potential problems by enabling alerts for cluster, database, and billing criteria.
   • Performance metrics:
   • Live queries: The Live Queries tab shows the queries that are currently “in-flight” on the cluster.
   • Slow queries: The YSQL Slow Queries tab shows queries run on the cluster, sorted by running time. The agency can also evaluate the slowest-running YSQL queries that have been run on the cluster.
   • Performance Advisor: Scan clusters for performance optimizations, including index and schema changes, and detect potentially hot nodes.
   • Metrics export: Export metrics to third-party monitoring tools such as Datadog and Grafana

In Summary…

The marketing agency selected YugabyteDB Aeon over MySQL Heatwave, CockroachDB, and Oracle Cloud for its email marketing platform. They needed a scalable database capable of handling transactional batch data for large-scale email campaigns, and YugabyteDB Aeon fulfilled and surpassed their requirements.

The main drivers for choosing YugabyteDB Aeon included advanced monitoring capabilities to support their small DBA team, efficient data handling features like packed rows and incremental loading, and superior query performance with features like GIN indexing, sharding, and parallel queries. These capabilities, coupled with the challenges they faced with MySQL’s table-level locking, lack of traceability, and complex management, prompted their shift to YugabyteDB Aeon.

Learn more about YugabyteDB Aeon and the features that this interactive digital marketing agency is leveraging through the following resources:

1. Attend a Thursday demo to see geo-partitioning in action
2. Sign up for a full-featured, free trial of YugabyteDB Aeon to give geo-partitioning a try
3. Visit our Docs and Blog space to discover how you can build a global application with YugabyteDB