@unikvs/v8-serde

Explains how to use the Transformer plugin that serializes values fast with Node.js v8.serialize.

Overview

@unikvs/v8-serde is a Transformer plugin that converts JavaScript values to byte sequences with Node.js v8.serialize. Transformer Node.js only

It converts values that structured clone can handle, fast with the native implementation. Writes convert to bytes with encode and reads restore the original value with decode, so you can read and write without thinking about serialization details.

The storage format is V8-specific and is not compatible with other runtimes or other languages. It fits snapshots and caches shared between Node.js peers.

This transformer is always open. No explicit open or close is needed; isOpen always returns true and name is always "V8Serde".

Install it as follows.

npm install @unikvs/v8-serde

pnpm add @unikvs/v8-serde

yarn add @unikvs/v8-serde

bun add @unikvs/v8-serde

nub add @unikvs/v8-serde

aube add @unikvs/v8-serde

Supported Values

The values you can convert are as follows.

The limitations are as follows.

• Functions, symbols, WeakMap, WeakSet, Promise, and SharedArrayBuffer cannot be encoded. Passing one at the root or nested throws V8SerdeEncodeError.
• Properties keyed by a symbol are removed together with their values.
• Class instances become plain objects and lose their prototype.

Usage

The V8Serde constructor takes no arguments.

import { V8Serde } from "@unikvs/v8-serde";

const v8serde = new V8Serde();

Its main members are as follows. encode, decode, getEncodable, and getDecodable are all async, so call them with await.

import { V8Serde } from "@unikvs/v8-serde";

const v8serde = new V8Serde();

const bytes = await v8serde.encode({ data: { message: "hello" } });
const value = await v8serde.decode({ data: bytes });

Register it with UniKvs via appendTransformer. It becomes the upstream of subsequently registered storages.

import { V8Serde } from "@unikvs/v8-serde";
import UniKvs from "unikvs";

const kvs = UniKvs.config()
  .appendTransformer(new V8Serde())
  .appendStorage(storage)
  .create();

await kvs.open();

await kvs.set("data", { message: "hello" });

const output = await kvs.get("data");

set converts to bytes for storage, and get converts them back to the original value automatically.

Streams

It supports both single values and streams.

• Single values are converted with encode and decode.
• Stream values are converted with the TransformStream from getEncodable and getDecodable. Obtaining them is async, so call them with await.

getEncodable encodes each input chunk as one independent value and emits its bytes in order. The output is a stream-only format.

getDecodable decodes bytes incrementally across arbitrary chunk boundaries.

• As soon as one value completes, its value is emitted immediately. If a chunk contains multiple values, the completed ones are emitted in order.
• A value cut off mid-way is buffered internally until it completes. It waits for the next chunk, so partial data is never emitted as a value by mistake.
• An empty stream and zero-length chunks produce zero values and no error.
• If incomplete data remains when the stream ends, it throws V8SerdeDecodeError. Completed values are emitted beforehand.
• If the input is not valid for this package’s format, it throws V8SerdeDecodeError at that point.

import { V8Serde } from "@unikvs/v8-serde";

const v8serde = new V8Serde();

const source = new ReadableStream<unknown>({
  start(controller) {
    controller.enqueue(new Date());
    controller.enqueue(new Map([["a", 1]]));
    controller.close();
  },
});

const decoded = source
  .pipeThrough(await v8serde.getEncodable())
  .pipeThrough(await v8serde.getDecodable());

for await (const value of decoded) {
  console.log(value);
}

Errors

Conversion failures in encode and getEncodable become V8SerdeEncodeError, and those in decode and getDecodable become V8SerdeDecodeError. Both keep the original error in cause. Messages support Japanese display. On a runtime other than Node.js, all four methods throw UnsupportedRuntimeError.

import { V8Serde, V8SerdeDecodeError, V8SerdeEncodeError } from "@unikvs/v8-serde";

const v8serde = new V8Serde();

try {
  await v8serde.encode({ data: () => 1 });
} catch (error) {
  if (error instanceof V8SerdeEncodeError) {
    console.log(error.cause);
  } else {
    throw error;
  }
}

try {
  await v8serde.decode({ data: new Uint8Array(0) });
} catch (error) {
  if (error instanceof V8SerdeDecodeError) {
    console.log(error.cause);
  } else {
    throw error;
  }
}

Examples

An example with Memory storage. Date, Map, Set, bigint, and undefined round-trip with their types intact.

import { V8Serde } from "@unikvs/v8-serde";
import { Memory } from "@unikvs/memory";
import UniKvs, { type PlainValue } from "unikvs";

type Snapshot = {
  createdAt: Date;
  counts: Map<string, number>;
  flags: Set<string>;
  revision: bigint;
  note: string | undefined;
};

const kvs = UniKvs.config<{ snapshot: PlainValue<Snapshot> }>()
  .appendTransformer(new V8Serde())
  .appendStorage(new Memory())
  .create();

await kvs.open();

const snapshot: Snapshot = {
  createdAt: new Date("2024-01-02T03:04:05.678Z"),
  counts: new Map([
    ["read", 1],
    ["write", 2],
  ]),
  flags: new Set(["dirty"]),
  revision: 1n,
  note: undefined,
};

await kvs.set("snapshot", snapshot);

const restored = await kvs.get("snapshot");
console.log(restored.createdAt instanceof Date);
console.log(restored.counts instanceof Map);
console.log(restored.flags instanceof Set);
console.log(typeof restored.revision);
console.log("note" in restored);

await kvs.close();