Xline
  • Docs
    • What's new
    • Get started
    • Deploy
    • Develop
    • reference
  • Deep dive
    • Consensus
    • KV engine
    • RPC
    • Test
  • Community
    • Chat
    • Contribute
    • 2024 GSoC
    • Community Meeting
  • Resources
    • Blog
    • Video
icon

github

  • Docs

    • What's new
    • Get started
    • Deploy
    • Develop
    • reference
  • Deep dive

    • Consensus
    • KV engine
    • RPC
    • Test
  • Community

    • Chat
    • Contribute
    • 2024 GSoC
    • Community Meeting
  • Resources

    • Blog
    • Video
Xline

A High-Performance Geo-distributed Metadata Management System

Image

Data isolation and data fragmentation resulting from cloud barriers have become impediments to business growth. The biggest challenge of multi-data center architecture is how to maintain strong data consistency and ensure high performance. Traditional single-data center solutions cannot meet the availability, performance, and consistency requirements of multi-data center scenarios.

  • Xline is a distributed KV storage for metadata management. Xline makes it possible to manage metadata, such as indexes, permissions, and configurations across multiple clusters.
  • Xline can achieve high-performance data access and strong consistency in cross data center scenarios.
  • Xline is compatible with the ETCD interface, so that existing ETCD users can seamlessly switch to Xline and get high-performance metadata management across multiple clusters.

Key Features of Xline

Image
icon

A distributed KV storage engine for multi-cluster

Unified data management for multi-cluster scenarios, making mutual access, discovery, and modification simple and convenient.

icon

A unified metadata management system

Data access across clouds by caching hot data in memory, and provide unified data management to automate data migration and backup.

icon

A high-performance multi-datacenter consensus protocol

It is the first geo distributed consistency management service based on the WAN consensus protocol. It addresses the challenges of convergence and consistency across clouds.

icon

Compatible with the ETCD interface

Provide KV interface, Multi-Version Concurrency Control and is compatible with K8S.

Innovation
image

Cross-datacenter network latency is the most important factor that impacts the performance of geo-distributed systems, especially when a consensus protocol is used. As is well known, consensus protocols are frequently used to achieve high availability. For instance, Etcd uses the Raft protocol, which is quite popular in recently developed systems.

Although Raft is stable and easy to implement, it takes 2 RTTs to complete a consensus request from the view of a client. One RTT takes place between the client and the leader server, and the leader server takes another RTT to broadcast the message to the follower servers. In a geo-distributed environment where an RTT can run from tens to hundreds of milliseconds, two RTTs would be too expensive.

We adopt a new consensus protocol named CURP to resolve the above issue. Please refer to the paper for a detailed description. The main benefit of the protocol is reducing 1 RTT when contention is not too high. As far as we know, Xline is the first product to use CURP.