Concepts
How UniKVS combines Transformers and Storages, Value types, and variables.
Architecture
UniKVS uses the builder pattern. Input passes through Transformers to Storages.
There are two roles:
- Transformers transparently encode/decode data.
- Storages are persistence destinations. With multiple Storages, writes go to all of them in parallel.
Each Storage uses the Transformers added up to its registration. On reads, the upstream pipeline of the found Storage is applied in reverse to decode the data.
Config builder
Create a builder with UniKvs.config() in the following order. Passing schema enables input and output validation with Valibot schemas, and also infers the key-to-value mapping type automatically.
import { PlainValue, StreamValue, UniKvs } from "unikvs";
import * as v from "valibot";
const kvs = UniKvs.config({
schema: {
message: PlainValue(v.string()),
logs: StreamValue(v.instance(Uint8Array)),
},
})
.appendStorage(storage)
.create();
Install valibot separately when using Valibot schemas. Omitting schema falls back to specifying the mapping with a type parameter. In that case only type checking applies, with no runtime validation.
setVariables(vars)
Sets runtime variables (optional).
appendTransformer(transformer)
Adds a Transformer (optional).
appendStorage(storage)
Adds a Storage (required, multiple allowed).
create()
Creates the KVS client.
Client operations
| Method | Description |
|---|---|
open() |
Initializes Storages and Transformers. |
close() |
Closes all Storages and Transformers. |
set(key, value) |
Saves a Value under a key. |
get(key) |
Gets the Value for a key. |
stream(key) |
Gets a stream for a key. |
has(key) |
Checks whether a key exists. |
delete(key) |
Deletes a key. |
clear() |
Deletes all data. |
All operations support cancellation via AbortSignal and passing of runtime variables.
Value types
Value, PlainValue, and StreamValue are both types and functions. Used as types, they decide the available methods per key at the type level; used as functions, they infer types from Valibot schemas and validate values at runtime.
| Type | Write | Read | Stream read |
|---|---|---|---|
PlainValue<T> |
set(key, T) |
get(key): T |
Not available. |
StreamValue<T> |
set(key, T | ReadableStream<T>) |
Not available. | stream(key): ValueStream<T> |
Value<T> |
set(key, T | ReadableStream<T>) |
get(key): T |
stream(key): ValueStream<T> |
Value<T> is sugar for both, equivalent to PlainValue<T> | StreamValue<T>.
Use for storing and getting a single value. stream() is a type error.
import { UniKvs, type PlainValue } from "unikvs";
const kvs = UniKvs.config<{
message: PlainValue<string>;
}>()
.appendStorage(storage)
.create();
await kvs.set("message", "hello");
const msg = await kvs.get("message");Use for sequential processing of large data. get() is a type error.
import { UniKvs, type StreamValue } from "unikvs";
const kvs = UniKvs.config<{
logs: StreamValue<Uint8Array>;
}>()
.appendStorage(storage)
.create();
await kvs.set("logs", new Uint8Array([0x01]));
const valueStream = await kvs.stream("logs");Sugar syntax for both, equivalent to PlainValue<T> | StreamValue<T>.
import { UniKvs, type Value } from "unikvs";
const kvs = UniKvs.config<{
blob: Value<Uint8Array>;
}>()
.appendStorage(storage)
.create();
await kvs.set("blob", new Uint8Array([1, 2, 3]));
const all = await kvs.get("blob");
const valueStream = await kvs.stream("blob");Schema definition
Passing schemas to UniKvs.config({ schema }) infers the mapping type from Valibot schemas and validates input and output values at runtime.
Keys are used as-is. Use for fixed keys.
import { PlainValue, StreamValue, UniKvs } from "unikvs";
import * as v from "valibot";
const kvs = UniKvs.config({
schema: {
message: PlainValue(v.pipe(v.string(), v.minLength(1))),
logs: StreamValue(v.instance(Uint8Array)),
},
})
.appendStorage(storage)
.create();
await kvs.set("message", "hello");
const msg = await kvs.get("message");
// msg is typed as stringList pairs of key and value schemas. Use for dynamic keys. The first matching definition wins.
import { PlainValue, StreamValue, UniKvs } from "unikvs";
import * as v from "valibot";
const kvs = UniKvs.config({
schema: [
[v.pipe(v.string(), v.regex(/^msg-.+/)), PlainValue(v.string())],
[v.pipe(v.string(), v.regex(/^img-.+/)), StreamValue(v.instance(Uint8Array))],
],
})
.appendStorage(storage)
.create();
await kvs.set("msg-1", "hello");Validation timing
| Operation | What is validated | Error on failure |
|---|---|---|
set |
Validates the input value. | InvalidInputError |
get |
Validates the output value. | InvalidOutputError |
stream |
Validates each chunk. | InvalidOutputError |
has / delete |
Only in the array form, validates the key. | InvalidInputError |
Runtime variables
Variables are a runtime context that switches operation behavior. Set initial values with setVariables(), and override them per operation with vars.
const kvs = UniKvs.config<{ foo: Value<Uint8Array> }>()
.setVariables({ region: "ap-northeast-1" })
.appendStorage(storage)
.create();
await kvs.set("foo", new Uint8Array([1]), {
vars: { region: "us-east-1" },
});
Plugin kinds
| Kind | Package | Description |
|---|---|---|
| Transformer | @unikvs/compression |
Compresses and decompresses with gzip, deflate, and deflate-raw. |
| Transformer | @unikvs/checksum |
Validates MD5, SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512. |
| Transformer | @unikvs/debug |
Logs the operations, keys, and debug information of data read and written. |
| Transformer | @unikvs/passthrough |
Passes data through without transforming it. |
| Transformer | @unikvs/json |
Serializes values to JSON, or to JSON Lines in streams. |
| Transformer | @unikvs/superjson |
Serializes values with SuperJSON while preserving Date, Map, Set, BigInt, and undefined. |
| Transformer | @unikvs/cbor |
Serializes values to CBOR, or to a CBOR sequence in streams. |
| Storage | @unikvs/memory |
Stores in memory. Works in all environments. |
| Storage | @unikvs/fs.node |
Stores in the local filesystem. Node.js only. |
| Storage | @unikvs/fs.bun |
Stores in the local filesystem. Bun only. |
| Storage | @unikvs/redis.bun |
Stores in Redis. Bun only. |
| Storage | @unikvs/s3.node |
Stores in S3-compatible object storage. Node.js only. |
| Storage | @unikvs/s3.bun |
Stores in S3-compatible object storage. Bun only. |
| Storage | @unikvs/indexeddb |
Stores in the browser IndexedDB. |
| Storage | @unikvs/opfs |
Stores in the browser OPFS. |
| Storage | @unikvs/writeonly |
Turns an existing storage into write-only storage. |