@unikvs/s3.node

Learn how to use @unikvs/s3.node to store data in S3-compatible object storage from Node.js.

Overview

@unikvs/s3.node is a Node.js-only storage plugin that saves byte arrays to S3-compatible object storage. Node.js only It depends on AWS SDK for JavaScript v3, using @aws-sdk/client-s3 for object operations and Upload from @aws-sdk/lib-storage for stream uploads.

Install it as follows.

npm install @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

pnpm add @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

yarn add @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

bun add @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

nub add @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

aube add @unikvs/s3.node @aws-sdk/client-s3 @aws-sdk/lib-storage

Usage

The storage class is S3.

import { S3 } from "@unikvs/s3.node";

new S3(bucket: string, config: S3ClientConfig = {}, options: S3StorageOptions = {})

There is no argument for a pre-built S3Client or a prefix. For key mapping or bucket separation, include it in the key or create instances per bucket.

The lifecycle is as follows. Right after instantiation, isOpen is false. Calling open() creates an internal S3Client and sets true. Calling close() destroys it and resets to false. open() and close() take no arguments.

import { UniKvs } from "unikvs";
import { S3 } from "@unikvs/s3.node";

const storage = new S3("my-bucket", {
  region: "ap-northeast-1",
});

const kvs = UniKvs.config()
  .appendStorage(storage)
  .create();

Object Layout

Keys map directly to object keys. No prefixing or escaping is applied.

Single writes use PutObjectCommand, reads use GetObjectCommand, existence checks use HeadObjectCommand, and deletions use DeleteObjectCommand.

clear deletes all objects in the bucket. No prefix filtering is applied. It paginates with ListObjectsV2Command and deletes with DeleteObjectsCommand. Use a dedicated bucket for UniKVS. Shared buckets lose other apps’ data too.

Streams and Multipart

A single write saves with PutObjectCommand, and a single read converts the GetObjectCommand result with transformToByteArray.

Writable streams use Upload from @aws-sdk/lib-storage, so large payloads benefit from multipart upload. Obtain one as follows.

const writable = storage.getWritable({ key, vars, signal });

The arguments are vars, key, and signal. The part size is read from vars. The @unikvs/s3.node:partSize key takes precedence, falling back to @unikvs/s3:partSize. Specify a positive integer in bytes. When omitted, the Upload defaults apply.

If signal is already aborted, it throws StorageAbortedError without starting. If signal aborts mid-write, the reason is forwarded to the internal AbortController. close() waits for completion, while abort() attempts cancellation.

Obtain a readable stream as follows.

const readable = await storage.getReadable({ key, signal });

Internally it sends a GetObjectCommand and converts the result with transformToWebStream. Large objects can be read in multiple chunks.

Errors

• read on a missing key fails with an error. Use exists beforehand when needed.
• delete on a missing key does not fail.

Examples

A minimal example against S3-compatible storage. The endpoint and credentials are dummy values for common S3-compatible storage (such as MinIO). Create the bucket beforehand.

import { S3 } from "@unikvs/s3.node";

const storage = new S3("my-bucket", {
  endpoint: "http://127.0.0.1:9000",
  region: "ap-northeast-1",
  credentials: {
    accessKeyId: "minioadmin",
    secretAccessKey: "minioadmin",
  },
  forcePathStyle: true,
});

storage.open();

const signal = AbortSignal.timeout(30_000);
const key = "hello.txt";
const data = new TextEncoder().encode("Hello S3");

await storage.write({ key, data, signal });
const loaded = await storage.read({ key, signal });
console.log(new TextDecoder().decode(loaded));

storage.close();