Skip to contents

storr.tiledb

Overview

A storr driver using TileDB Embedded storage engine.

What is storr?

A key-value store with content addressable storage for R objects with optional in-memory caching layer.

The interface is provided by storr package and defines a set of common operations (set, get, del methods) across a range of different storage drivers (DBI, LMDB, Redis, RDS, environment). The storr package is written by Rich FitzJohn.

What is storr.tiledb?

The storr.tiledb contributes a new storr compliant driver using TileDB Engine for backend.

It defines a new storr R6 class that leverages the advantages of TileDB and provides new features such as adding notes and expiration time-stamps along with key-namespace pairs.

[!WARNING]
The package is in experimental status. Currently, the driver is complete and fully functional in that it passes the auto test specification defined by storr package. More testing is needed and additional features to be implemented before it is moved to mature state. Feedback is welcomed.

Key features

  • Key-value operations use driver’s methods for speed and efficiency

  • Key-value metadata such as notes and timestamps for data expiration (time-to-live, TTL)

  • Asynchronous key-value operations or in parallel through mirai framework

  • Hash tables (hashtab) for in memory caching layers instead of environments

  • Cloud storage (S3, Azure, GSC)

  • Data versioning (time-travelling)

  • Encryption

Installation

Development version from GitHub:

# pak
pak::pkg_install("cgiachalis/storr.tiledb")

# remotes
remotes::install_github("cgiachalis/storr.tiledb")

Quick start 

library(storr.tiledb)

# Temp URI path
uri <- tempfile()

# Set up storr
sto <- storr_tiledb(uri, init = TRUE)

# Set
sto$set("mykey1", list(a = 1))
sto$set("mykey2", "abc")

# Get
sto$get("mykey2")
 [1] "abc"
 
# List all keys
sto$list()
[1] "mykey1" "mykey2"

# Del
sto$del("mykey1")

Other storr drivers