Yugabyte Database Engineering Update – August 20, 2018

Time for another update from the engineering team at Yugabyte! It has been a month since the last update, so let’s dive right in.

Community News

Recent Events

On the conference front, Yugabyte was at Google Next towards the end of July. YugabyteDB was already very well integrated into the Google Cloud Compute ecosystem, we additionally announced support for Google Container Engine (GKE). You can also read about how YugabyteDB compares with the various Google Cloud databases.

There have been a number of questions about how YugabyteDB compares to Google Spanner. While both are geo-distributed databases that support distributed transactions, YugabyteDB is:

• Redis, Cassandra and PostgreSQL compatible APIs for reading and writing data
• Open source under Apache 2.0 license
• Supports Google Cloud, AWS, Azure, Kubernetes and private data centers

We are Hiring!

YugaByte is looking for a passionate Developer Advocate! Are you excited about becoming the voice of our users? Do you love experimenting with new technologies and presenting it at conferences, meetups and workshops? We would love to talk to you. Check out our list of open positions.

Release Updates

We recently released YugabyteDB 1.0.5. This release has the following key features on top of the previous YugabyteDB 1.0.4 version.

Cassandra-compatible YCQL Features

• Added support for “broadcast address” to enable public and private IP support
• Ability to return fine-grained errors in CAS (compare and set) and batch statements
• Support for weak secondary indexes for higher ingest performance

Redis-compatible YEDIS Features

• Authentication support using passwords, including support for multiple active passwords to enable online rotation of passwords. Read more.
• Safety flags to disable dropping data inadvertently, for example using commands such as FLUSHALL
• Monitor support

Ecosystem and Deployments

• Basic helm chart support to deploy Kubernetes clusters with ease.

Roadmap

YugabyteDB 1.1 is right around the corner! We are going to be announcing a number of critical features including:

• General availability of Distributed ACID transactions, consistent secondary indexes and a native JSON data type.
• On the security front, authentication of users in the Cassandra and Redis APIs, as well as support for TLS encryption.
• EE features such as read replica support and support for managed Kubernetes environments such as GKE and PKS are going to preview as a beta features

To view a list of all items being worked, browse to our GitHub projects page. And stay tuned for an update on our Jepsen testing soon.

Documentation, Blogs, Tutorials

We have recently added docs on how to use YugabyteDB with client drivers in C++, C# and Go in addition to the existing languages such as NodeJS and Java. Happy app-building!

Check out how to use the MONITOR command in the YEDIS API that is a great debugging tool to see all requests being processed by a server.

We have been busy on the blogging front with a bunch of tech deep-dive posts.

• Read about how we tried to torture our own database by running a high density workload on YugabyteDB. We loaded 18TB of data on a 4 node AWS cluster, ran a brutal read workload on it and finally added a node to scale it out!
• Want to use consistent secondary indexes with a NoSQL DB? No problem, this post will walk you through using secondary indexes in YugabyteDB.
• There is a lot of talk about how Raft is great. Now you can take a look under the hood and understand how Raft is really used in YugabyteDB.
• Looking to understand how to run a distributed database on Kubernetes? Our post on understanding how YugabyteDB runs on Kubernetes walks you through all the details including instructions on how to run your own YugabyteDB cluster on minikube.
• You can also read all about the architecture of Apache Cassandra, and some core architectural differences when comparing with YugabyteDB.

Enhancement Requests

Once again, there has been no dearth of enhancement requests. There are too many to list, but here are three useful enhancements that we have recently implemented.

Broadcast address support in Cassandra API

In modern cloud-based deployments, servers could have public IP address or DNS names that they are not aware of. A cloud provider could provide public ip address functionality via port forwarding, for example. Also clouds such as GCP and AWS have different billing strategies when traffic goes through public IP vs private IP. In many of these scenarios, the services on these servers know nothing about the DNS names (or public IP addresses).

As an example, consider machines instances on most public clouds. They usually have a public IP address that should be used from outside the cloud region and a private IP address that is preferred inside. This feature allows YugabyteDB to span regions or cloud providers by allowing the appropriate address to be used for inter-server communication.

Read more about the design of the broadcast address feature.

Multiple databases in the Redis API

In many scenarios, it is very desirable to allow applications to interact with only their data (and not mix data across apps together). This feature enables this by allowing creation of multiple “databases” that can be used by different applications. Open-source Redis already allows for multiple databases, but this is generally not the recommended practice due to the single-threaded nature of the server. However, with YugabyteDB offering a persistent and elastic Redis database across multiple nodes, this feature was highly requested by many of our users.

In YugabyteDB, each redis database is modeled as a separate DocDB table. The database name can be any arbitrary string (it has to be an integer in open-source Redis). New commands have been added to the YEDIS API – CreateDB, ListDB, DeleteDB and Select to enable working with databases. You can read more details about the design of databases in Yedis.

Authentication support with multiple passwords in Redis API

Security is a critical part of databases. To secure the Redis API on YugabyteDB deployments, we enhanced the Redis AUTH command to support authentication for client connections by storing credentials in a centralized location in the DB. We got an enhancement request to support authentication of clients against multiple passwords simultaneously. This enables the app tier to be upgraded in a rolling fashion, because during the upgrades, some clients would use the old password while others would use the new one. This feature enables a zero-downtime upgrade of the application tier while not compromising on security.

You can set multiple passwords using the CONFIG SET command. Read more about the design of Redis authentication.

What’s Next?

• Using YugabyteDB at your company? Tell us about it and we’ll send you a hoodie as a thank you!
• Star us on Github.
• Get Started with YugabyteDB!