YugabyteDB Community Update, Tricks and Tips – Dec 13, 2019

Welcome to this week’s community update where we recap a few interesting questions that have popped up in the last week or so on the YugabyteDB Slack channel, Forum, GitHub or Stackoverflow. We’ll also review upcoming events, new blogs and documentation. Ok, let’s dive right in:

How best to configure clusters across deployment types

Ava over on StackOverflow asked how to best setup configurations for different deployment models like single AZ, multi-AZ, multi-region and multi-cloud.

For documentation on configuring clusters on GCP, Azure and AWS, check out: “Deploying YugabyteDB on Public Clouds.” The documentation includes a discussion about prerequisites, plus how to configure data drives, file systems, networking, YugabyteDB specific configuration files and more.

On multi-AZ, multi-region and multi-cloud deployments we offer a variety of geo distribution configurations for tservers and masters to help you best optimize your deployment given your requirements. A simple optimization on these clusters is to put all leaders in the same availability zone so write transactions don’t have to travel across multiple zones. You can do this by specifying set_preferred_zones via the yb-admin tool.

Wait, what’s a tserver?

The YB-TServer (short for Yugabyte Tablet Server) is the process that does the actual IO for end user requests. Data for a table is split/sharded into tablets. Each tablet is composed of one or more tablet-peers depending on the replication factor. And each YB-TServer hosts one or more tablet-peers.

What’s a master server?

The YB-Master is the keeper of system meta-data/records, such as what tables exist in the system, where their tablets live, what users/roles exist, the permissions associated with them, and so on. It is also responsible for coordinating background operations (such as load-balancing or initiating re-replication of under-replicated data) and performing a variety of administrative operations such as create/alter/drop of a table. Note: that the YB-Master is highly available as it forms a Raft group with its peers, and it is not in the critical path of IO against user tables.

How encryption works in YugabyteDB

James over on StackOverflow asks how encryption at rest works in YugabyteDB and if it can be enabled on the fly.

Yes, encryption at rest can be enabled after the fact on an existing YugabyteDB cluster. New incoming data will start getting encrypted right away. Old data will get encrypted lazily – when scheduled compactions occur. As a general rule, for write heavy workloads, you can expect the overhead to be under 5%. For read heavy workloads, especially those where the hot data set is in cache, the impact should be negligible.

The yb-tserver(s), if restarted, will learn of the current key from yb-master(s) using RPC calls. To protect this exchange from an attack, you can use the on-the-wire encryption feature, which makes sure that all server-to-server communication happens securely. If a yb-master is restarted, it’ll learn of the current key from other yb-masters. However, if all the masters go down at once, none of the masters will have this in memory state, and you will have to use yb-admin tool to give the key to all the masters.

YSQL & YCQL API compatibility

Ali over on stackoverflow asks how compatible are YSQL (fully-relational) & YCQL (semi-relational) with each other.

Today, the YugabyteDB APIs are independent from one another. This means that the data inserted or managed by one API cannot be queried by the other API. Additionally, there is no common way to access the data across the APIs (external frameworks such as Presto can help for simple cases.) In the future, we will allow YCQL tables to be “accessed” from YSQL (which is fully PostgreSQL compatible) as a foreign table using foreign data wrappers (FDW).

How encryption works in an asynchronous Two Data Center (2DC) deployment

Sergiy over on Slack asks how encryption works in a 2DC deployment.

Currently, we support encryption in transit (between datacenters) and encryption at rest for data on disk. We can also use encryption in transit to secure communication from client servers. Using this technique you can have clusters in a multi-datacenter asynchronous replication fully secured.

New Documentation, Blogs, Tutorials, and Videos

New Blogs

• YugabyteDb Engineering Update – Dec 9, 2019
• Getting Started with PostgreSQL’s Row-Level Security in YugabyteDB
• Using Stored Procedures in Distributed SQL Databases
• Announcing YugabyteDB on Crossplane, the Open Source Multicloud Control Plane
• What is Distributed SQL?
• How Plume Handled Billions of Operations Per Day Despite an AWS Zone Outage

New Videos

• Getting Started with PostgreSQL Authentication and Authorization in YugabyteDB
• Getting Started with PostgreSQL’s Row-Level Security in YugabyteDB
• Designing a Change Data Capture and Two Data Center Architecture for a Distributed SQL Database
• Distributed Databases Deconstructed: CockroachDB, TiDB and YugabyteDB
• We have uploaded over a dozen videos from this year’s Distributed SQL Summit. You can links to the presentations, slides and a recap of the highlights by checking out, “2019 Distributed SQL Summit Recap and Highlights.”

New Docs

• Encryption at rest
• Configuring encryption at rest with the yb-admin tool
• Using PostgreSQL’s JDBC Driver
• YugabyteDB configuration reference documentation for TServer, Master nodes and port assignments
• Two data center (2DC) deployment instructions

Upcoming Meetups and Conferences

PostgreSQL Meetups

• Jan 16: Silicon Valley PostgreSQL Meetup

Distributed SQL Webinars

• Dec 18: Distributed SQL Database Administration 101
• Jan 15: Developing Cloud-Native Spring Microservices with a Distributed SQL Backend

Get Started

Ready to start exploring YugabyteDB features? Getting up and running locally on your laptop is fast. Visit our quickstart page to get started.

What’s Next?

• Compare YugabyteDB in depth to databases like CockroachDB, Google Cloud Spanner and MongoDB.
• Get started with YugabyteDB on macOS, Linux, Docker, and Kubernetes.
• Contact us to learn more about licensing, pricing or to schedule a technical overview.