Fix transaction contains range are not serializable

al8n · Apr 27, 2024 · 2fc2012 · 2fc2012
1 parent 9f3400e
commit 2fc2012
Show file tree

Hide file tree

Showing 41 changed files with 4,206 additions and 1,011 deletions.
diff --git a/Cargo.toml b/Cargo.toml
@@ -11,7 +11,7 @@ members = [
 resolver = "2"
 
 [workspace.package]
-version = "0.1.2"
+version = "0.1.3"
 edition = "2021"
 rust-version = "1.75.0"
 repository = "https://github.com/al8n/skipdb"

diff --git a/README.md b/README.md
@@ -10,7 +10,7 @@
 This repository the implementation:
 
 1. Generic optimistic transaction manger, which supports concurrent execution of transactions, providing serializable snapshot isolation, avoiding write skews.
-2. An ACID, MVCC and WASM friendly in-memory database based on the transaction manager.
+2. An atomicity, consistency, isolation, MVCC and WASM friendly in-memory database based on the transaction manager.
 
 [<img alt="github" src="https://img.shields.io/badge/github-al8n/skipdb-8da0cb?style=for-the-badge&logo=Github" height="22">][Github-url]
 <img alt="LoC" src="https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2Fal8n%2F327b2a8aef9003246e45c6e47fe63937%2Fraw%2Fskipdb-total" height="22">
@@ -49,9 +49,29 @@ English | [简体中文][zh-cn-url]
 
 </div>
 
-## Features
+## Introduction
 
-- ACID, MVCC, serializable snapshot isolation, concurrent safe and almost lock-free.
+This repository contains a transaction framework (`async-txn` and `txn`) based on optimistic concurrency control, which is inspired by [`foundationdb`'s paper](https://www.foundationdb.org/files/fdb-paper.pdf) and [`badger`](https://github.com/dgraph-io/badger).
+
+This repository contains two kinds of in-memory key-value database which supports both async (`async-skipdb`) and sync (`skipdb`):
+
+1. `SerializableDb`
+
+   Supports both concurrent execution of full serializable snapshot isolation transactions and optimistic concurrency control transactions.
+
+   Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.
+
+   Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
+2. `OptimisticDb`
+
+   Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.
+
+   All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
+### Features
+
+- Atomicity, Consistency, Isolation, MVCC, concurrent safe and almost lock-free.
 - No extra allocation and copy, there is no `Arc` wrapper for both key and value stored in the database, which means that users provide `K` and `V`, and database store `K` and `V` directly.
 - Zero-copy and in-place compaction, which means there is no copy, no extra allocation when compacting.
 - Concurrent execution of transactions, providing serializable snapshot isolation, avoiding write skews.
@@ -61,6 +81,7 @@ English | [简体中文][zh-cn-url]
 - Async version is runtime agnostic, `tokio`, `async-std`, `smol`, `wasm-bindgen-futures` and any other async runtime.
 - 100% safe, sets `[forbid(unsafe_code)]`.
 
+
 ## Installation
 
 - For sync
@@ -103,14 +124,6 @@ English | [简体中文][zh-cn-url]
 
 Please see [skipdb](./skipdb/) or [async-skipdb](./async-skipdb).
 
-## Crates
-
-- `skipdb`: sync version skipdb
-- `async-skipdb`: async version skipdb
-- `txn`: sync version serializable snapshot isolation and concurrent execution of transactions.
-- `async-txn`: async version serializable snapshot isolation and concurrent execution of transactions.
-- `wmark`: watermark
-
 #### License
 
 `skipdb` is under the terms of both the MIT license and the

diff --git a/async-skipdb/Cargo.toml b/async-skipdb/Cargo.toml
@@ -1,12 +1,12 @@
 [package]
 name = "async-skipdb"
-version = "0.1.5"
+version = "0.2.0"
 rust-version.workspace = true
 edition.workspace = true
 repository.workspace = true
 homepage.workspace = true
 documentation = "https://docs.rs/async-skipdb"
-description = "An embedded, in-memory, zero-copy, ACID, MVCC, almost lock-free and serializable snapshot isolation database engine."
+description = "An embedded, in-memory, zero-copy, atomicity, consistency, MVCC, almost lock-free and serializable snapshot isolation database engine."
 license.workspace = true
 keywords = ["database", "key-value-store", "memdb", "embedded-database", "memodb"]
 categories = ["database-implementations", "concurrency", "data-structures", "asynchronous"]

diff --git a/async-skipdb/README-zh_CN.md b/async-skipdb/README-zh_CN.md
@@ -3,9 +3,7 @@
 </div>
 <div align="center">
 
-Blazing fast ACID and MVCC in memory database based on lock-free skiplist.
-
-`async-skipdb` uses the same SSI (Serializable Snapshot Isolation) transaction model used in [`badger`](https://github.com/dgraph-io/badger).
+An embedded, in-memory, zero-copy, atomicity, consistency, MVCC, almost lock-free and serializable snapshot isolation database engine.
 
 [<img alt="github" src="https://img.shields.io/badge/github-al8n/async--skipdb-8da0cb?style=for-the-badge&logo=Github" height="22">][Github-url]
 [<img alt="Build" src="https://img.shields.io/github/actions/workflow/status/al8n/skipdb/ci.yml?logo=Github-Actions&style=for-the-badge" height="22">][CI-url]
@@ -24,23 +22,39 @@ English | [简体中文][zh-cn-url]
 
 ## Introduction
 
-Blazing fast ACID and MVCC in memory database based on lock-free skiplist.
+An embedded, in-memory, zero-copy, MVCC, almost lock-free and serializable snapshot isolation database engine.
 
-`async-skipdb` uses the same SSI (Serializable Snapshot Isolation) transaction model used in [`badger`](https://github.com/dgraph-io/badger).
+`async-skipdb`'s SSI (Serializable Snapshot Isolation) transaction model is referenced to [`foundationdb`'s paper](https://www.foundationdb.org/files/fdb-paper.pdf) and [`badger`](https://github.com/dgraph-io/badger).
 
 For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
+This crate contains two kinds of in-memory key-value database:
+
+1. `SerializableDb`
+
+   Supports both concurrent execution of full serializable snapshot isolation transactions and optimistic concurrency control transactions.
+
+   Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.
+
+   Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
+2. `OptimisticDb`
+
+   Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.
+
+   All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
 ## Features
 
-- ACID, MVCC, serializable snapshot isolation, concurrent safe and almost lock-free.
+- Atomicity, Consistency, Isolation, MVCC, concurrent safe and almost lock-free.
 - No extra allocation and copy, there is no `Arc` wrapper for both key and value stored in the database, which means that users provide `K` and `V`, and database store `K` and `V` directly.
 - Zero-copy and in-place compaction, which means there is no copy, no extra allocation when compacting.
 - Concurrent execution of transactions, providing serializable snapshot isolation, avoiding write skews.
 - Both read transaction and write transaction are `Send + Sync + 'static`, which means you do not need to handle annoying lifetime problem anymore.
 - Lock-free and concurrent safe read transaction: the read transaction is totally concurrent safe and can be shared in multiple threads, there is no lock in read transaction.
 - `BTreeMap` like user friendly API and all iterators implement `Iterator` trait, which means users use Rust powerful conbinators when iterating over the database.
 - Runtime agnostic, `tokio`, `async-std`, `smol`, `wasm-bindgen-futures` and any other async runtime.
-- 100% safe, skipdb sets `[forbid(unsafe_code)]`.
+- 100% safe, sets `[forbid(unsafe_code)]`.
 
 ## Installation
 
@@ -74,57 +88,8 @@ For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
 ## Example
 
-- If your `K` implement `Hash`.
-
-  ```rust
-  use async_skipdb::equivalent::TokioEquivalentDb;
-
-  #[derive(Debug)]
-  struct Person {
-    hobby: String,
-    age: u8,
-  }
-
-  #[tokio::main]
-  async fn main() {
-    let db: TokioEquivalentDb<String, Person> = TokioEquivalentDb::new().await;
-
-    {
-      let alice = Person { hobby: "swim".to_string(), age: 20 };
-      let bob = Person { hobby: "run".to_string(), age: 30 };
-
-      let mut txn = db.write().await;
-      txn.insert("Alice".to_string(), alice).unwrap();
-      txn.insert("Bob".to_string(), bob).unwrap();
-
-      {
-        let alice = txn.get("Alice").unwrap().unwrap();
-        assert_eq!(alice.value().age, 20);
-        assert_eq!(alice.value().hobby, "swim");
-      }
-
-      txn.commit().await.unwrap();
-    }
-
-    {
-      let txn = db.read().await;
-      let alice = txn.get("Alice").unwrap();
-      assert_eq!(alice.value().age, 20);
-      assert_eq!(alice.value().hobby, "swim");
-
-      let bob = txn.get("Bob").unwrap();
-      assert_eq!(bob.value().age, 30);
-      assert_eq!(bob.value().hobby, "run"); 
-    }
-  }
-  ```
-
-- If your key cannot implement `Hash` for some reasons.
-
-  Then you can use `ComparableDb`, but this will require `K: CheapClone + Ord`, you can see [`cheap_clone`](https://crates.io/crates/cheap-clone) trait for more details.
-
   ```rust
-  use async_skipdb::comparable::TokioComparableDb;
+  use async_skipdb::serializable::TokioSerializableDb;
 
   #[derive(Debug)]
   struct Person {
@@ -135,13 +100,13 @@ For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
   #[tokio::main]
   async fn main() {
-    let db: TokioComparableDb<u64, Person> = TokioComparableDb::new().await;
+    let db: TokioSerializableDb<u64, Person> = TokioSerializableDb::new().await;
 
     {
       let alice = Person { name: "Alice".to_string(), hobby: "swim".to_string(), age: 20 };
       let bob = Person { name: "Bob".to_string(), hobby: "run".to_string(), age: 30 };
 
-      let mut txn = db.write().await;
+      let mut txn = db.serializable_write().await;
       txn.insert(1, alice).unwrap();
       txn.insert(2, bob).unwrap();
 

diff --git a/async-skipdb/README.md b/async-skipdb/README.md
@@ -3,9 +3,7 @@
 </div>
 <div align="center">
 
-An embedded, in-memory, zero-copy, ACID, MVCC, almost lock-free and serializable snapshot isolation database engine.
-
-`async-skipdb` uses the same SSI (Serializable Snapshot Isolation) transaction model used in [`badger`](https://github.com/dgraph-io/badger).
+An embedded, in-memory, zero-copy, atomicity, consistency, MVCC, almost lock-free and serializable snapshot isolation database engine.
 
 [<img alt="github" src="https://img.shields.io/badge/github-al8n/async--skipdb-8da0cb?style=for-the-badge&logo=Github" height="22">][Github-url]
 [<img alt="Build" src="https://img.shields.io/github/actions/workflow/status/al8n/skipdb/ci.yml?logo=Github-Actions&style=for-the-badge" height="22">][CI-url]
@@ -24,15 +22,31 @@ English | [简体中文][zh-cn-url]
 
 ## Introduction
 
-An embedded, in-memory, zero-copy, ACID, MVCC, almost lock-free and serializable snapshot isolation database engine.
+An embedded, in-memory, zero-copy, MVCC, almost lock-free and serializable snapshot isolation database engine.
 
-`async-skipdb` uses the same SSI (Serializable Snapshot Isolation) transaction model used in [`badger`](https://github.com/dgraph-io/badger).
+`async-skipdb`'s SSI (Serializable Snapshot Isolation) transaction model is referenced to [`foundationdb`'s paper](https://www.foundationdb.org/files/fdb-paper.pdf) and [`badger`](https://github.com/dgraph-io/badger).
 
 For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
+This crate contains two kinds of in-memory key-value database:
+
+1. `SerializableDb`
+
+   Supports both concurrent execution of full serializable snapshot isolation transactions and optimistic concurrency control transactions.
+
+   Transactions are created by `SerializableDb::serializable_write` can handle all kinds of write skew correctly.
+
+   Transactions are created by `SerializableDb::optimistic_write` can handle all kinds of direct dependent write skew, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
+2. `OptimisticDb`
+
+   Only support oncurrent execution of optimistic concurrency control, which means the write transaction cannot detect all kinds of write skew.
+
+   All kinds of direct dependent write skew can be handled correctly, but cannot handle all kinds of indirect dependent write skew e.g. range intersection between two concurrent transactions (see unit tests `write_skew_intersecting_data2` and `write_skew_intersecting_data3` for more details).
+
 ## Features
 
-- ACID, MVCC, serializable snapshot isolation, concurrent safe and almost lock-free.
+- Atomicity, Consistency, Isolation, MVCC, concurrent safe and almost lock-free.
 - No extra allocation and copy, there is no `Arc` wrapper for both key and value stored in the database, which means that users provide `K` and `V`, and database store `K` and `V` directly.
 - Zero-copy and in-place compaction, which means there is no copy, no extra allocation when compacting.
 - Concurrent execution of transactions, providing serializable snapshot isolation, avoiding write skews.
@@ -74,57 +88,8 @@ For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
 ## Example
 
-- If your `K` implement `Hash`.
-
-  ```rust
-  use async_skipdb::equivalent::TokioEquivalentDb;
-
-  #[derive(Debug)]
-  struct Person {
-    hobby: String,
-    age: u8,
-  }
-
-  #[tokio::main]
-  async fn main() {
-    let db: TokioEquivalentDb<String, Person> = TokioEquivalentDb::new().await;
-
-    {
-      let alice = Person { hobby: "swim".to_string(), age: 20 };
-      let bob = Person { hobby: "run".to_string(), age: 30 };
-
-      let mut txn = db.write().await;
-      txn.insert("Alice".to_string(), alice).unwrap();
-      txn.insert("Bob".to_string(), bob).unwrap();
-
-      {
-        let alice = txn.get("Alice").unwrap().unwrap();
-        assert_eq!(alice.value().age, 20);
-        assert_eq!(alice.value().hobby, "swim");
-      }
-
-      txn.commit().await.unwrap();
-    }
-
-    {
-      let txn = db.read().await;
-      let alice = txn.get("Alice").unwrap();
-      assert_eq!(alice.value().age, 20);
-      assert_eq!(alice.value().hobby, "swim");
-
-      let bob = txn.get("Bob").unwrap();
-      assert_eq!(bob.value().age, 30);
-      assert_eq!(bob.value().hobby, "run"); 
-    }
-  }
-  ```
-
-- If your key cannot implement `Hash` for some reasons.
-
-  Then you can use `ComparableDb`, but this will require `K: CheapClone + Ord`, you can see [`cheap_clone`](https://crates.io/crates/cheap-clone) trait for more details.
-
   ```rust
-  use async_skipdb::comparable::TokioComparableDb;
+  use async_skipdb::serializable::TokioSerializableDb;
 
   #[derive(Debug)]
   struct Person {
@@ -135,13 +100,13 @@ For sync version, please see [`skipdb`](https://crates.io/crates/skipdb).
 
   #[tokio::main]
   async fn main() {
-    let db: TokioComparableDb<u64, Person> = TokioComparableDb::new().await;
+    let db: TokioSerializableDb<u64, Person> = TokioSerializableDb::new().await;
 
     {
       let alice = Person { name: "Alice".to_string(), hobby: "swim".to_string(), age: 20 };
       let bob = Person { name: "Bob".to_string(), hobby: "run".to_string(), age: 30 };
 
-      let mut txn = db.write().await;
+      let mut txn = db.serializable_write().await;
       txn.insert(1, alice).unwrap();
       txn.insert(2, bob).unwrap();
 

diff --git a/async-skipdb/examples/equivalent.rs → async-skipdb/examples/optimistic.rs b/async-skipdb/examples/equivalent.rs → async-skipdb/examples/optimistic.rs
@@ -1,4 +1,4 @@
-use async_skipdb::equivalent::TokioEquivalentDb;
+use async_skipdb::optimistic::TokioOptimisticDb;
 
 #[derive(Debug)]
 struct Person {
@@ -8,7 +8,7 @@ struct Person {
 
 #[tokio::main]
 async fn main() {
-  let db: TokioEquivalentDb<String, Person> = TokioEquivalentDb::new().await;
+  let db: TokioOptimisticDb<String, Person> = TokioOptimisticDb::new().await;
 
   {
     let alice = Person {

diff --git a/async-skipdb/examples/comparable.rs → async-skipdb/examples/serializable.rs b/async-skipdb/examples/comparable.rs → async-skipdb/examples/serializable.rs
@@ -1,4 +1,4 @@
-use async_skipdb::comparable::TokioComparableDb;
+use async_skipdb::serializable::TokioSerializableDb;
 
 #[derive(Debug)]
 struct Person {
@@ -9,7 +9,7 @@ struct Person {
 
 #[tokio::main]
 async fn main() {
-  let db: TokioComparableDb<u64, Person> = TokioComparableDb::new().await;
+  let db: TokioSerializableDb<u64, Person> = TokioSerializableDb::new().await;
 
   {
     let alice = Person {
@@ -23,7 +23,7 @@ async fn main() {
       age: 30,
     };
 
-    let mut txn = db.write().await;
+    let mut txn = db.serializable_write().await;
     txn.insert(1, alice).unwrap();
     txn.insert(2, bob).unwrap();