@unikvs/compression

Explains how to use the Transformer plugin that transparently compresses and decompresses byte sequences with gzip / deflate / deflate-raw.

Overview

@unikvs/compression is a Transformer plugin that transparently compresses and decompresses byte sequences. Transformer It wraps the Web standard CompressionStream and DecompressionStream to fit ITransformer.

Writes compress with encode and reads expand with decode, so you can read and write without thinking about compression.

This transformer is always open. No explicit open or close is needed; isOpen always returns true and name is always "Compression".

Install it as follows.

npm install @unikvs/compression

pnpm add @unikvs/compression

yarn add @unikvs/compression

bun add @unikvs/compression

nub add @unikvs/compression

aube add @unikvs/compression

Supported Formats

The formats you can specify in the constructor are the CompressionFormat type. It is limited to strings accepted by both CompressionStream and DecompressionStream. The three tested values are as follows.

Choose "gzip" for compatibility. If you have an agreement with another system, match that format.

import { Compression } from "@unikvs/compression";

const compression = new Compression("gzip");

Use a single format per KVS. Mixing formats fails to decompress.

If you specify a format the runtime does not support, a TypeError is thrown when constructing the CompressionStream or DecompressionStream.

Usage

The Compression constructor takes exactly one format. There are no options.

import { Compression } from "@unikvs/compression";

const compression = new Compression("gzip");

Its main members are as follows.

Register it with UniKvs via appendTransformer. It becomes the upstream of subsequently registered storages.

import { Compression } from "@unikvs/compression";
import UniKvs from "unikvs";

const kvs = UniKvs.config()
  .appendTransformer(new Compression("gzip"))
  .appendStorage(storage)
  .create();

await kvs.open();

await kvs.set("data", input);

const output = await kvs.get("data");

set stores compressed data, and get decompresses it automatically.

Streams and Byte Sequences

@unikvs/compression supports both single values and streams.

• Single values are converted with encode and decode. An empty Uint8Array or a ~1 MiB sequence round-trips as is.
• Stream values are converted with the TransformStream from getEncodable and getDecodable. Even uneven chunks restore the original after a round-trip.

The limitations are as follows.

• Passing non-compressed bytes to decode is rejected.
• If the compression and decompression formats differ, decompression fails. Round-trip with the same format.
• For data with no room for compression, such as random bytes, the size may not shrink. Decompression still restores the original.

Notes

Repetitive text tends to shrink well, while already compressed or random-like data shrinks poorly. Even without shrinking, round-trip accuracy is not impaired.

Examples

A minimal example with Memory storage. Written values are stored compressed and automatically decompressed on reads.

import { Compression } from "@unikvs/compression";
import { Memory } from "@unikvs/memory";
import UniKvs, { type PlainValue } from "unikvs";

const kvs = UniKvs.config<{ data: PlainValue<Uint8Array<ArrayBuffer>> }>()
  .appendTransformer(new Compression("gzip"))
  .appendStorage(new Memory())
  .create();

await kvs.open();

const input = new TextEncoder().encode(
  "Repeat this text multiple times to ensure compression efficiency. ".repeat(100),
);
await kvs.set("data", input);

const output = await kvs.get("data");
console.log(output.length === input.length);

await kvs.close();