@unikvs/http

Learn how to read and write with @unikvs/http, the generic HTTP storage for UniKVS, using only fetch.

Overview

@unikvs/http is a storage plugin that uses any HTTP endpoint as KV storage. It reads and writes with only fetch.

• Runs on minimal runtimes such as Edge, Workers, and Lambda. It uses no runtime-specific APIs.
• Handles byte sequences only. write accepts a Uint8Array<ArrayBuffer>, and read returns a Uint8Array<ArrayBuffer>.
• Maps keys under baseUrl. Auth headers and tokens can be attached.
• Suitable for sharing between microservices and for saving through a gateway.

Install it as follows.

npm install @unikvs/http

pnpm add @unikvs/http

yarn add @unikvs/http

bun add @unikvs/http

nub add @unikvs/http

aube add @unikvs/http

The main dependency is @unikvs/core.

Usage

The central class is Http.

import { Http } from "@unikvs/http";

public constructor(baseUrl: string, options?: HttpOptions)

The replacement type for fetch is the following interface.

export interface IFetch {
  (request: Request): Promise<Response>;
}

baseUrl is the base location for storage. The normalization rules are as follows.

• Only absolute URLs are accepted. Relative URLs are rejected.
• URLs with a query or fragment are rejected.
• A trailing / is removed.
• Inputs that become empty after removal, such as / alone, are rejected.
• Empty strings and strings with leading or trailing whitespace are rejected.

options is optional. The arguments are as follows.

Request headers are resolved with three layers of precedence.

1. vars["@unikvs/http:headers"] and vars["@unikvs/http:token"]
2. vars["@unikvs/fetch:headers"] and vars["@unikvs/fetch:token"]
3. The constructor headers and token

When a token is present and Authorization is missing, Bearer ${token} is added automatically. An explicit Authorization header takes precedence. baseUrl, keyPrefix, and fetch cannot be overridden with vars.

The main members are as follows.

• name is "Http".
• isOpen is always true.
• open() and close() only check for interruption. They create no connection.
• write saves with PUT.
• read fetches with GET.
• exists checks with HEAD.
• delete removes with DELETE.

import { Http } from "@unikvs/http";

const storage = new Http("https://kv.example.com/store");
const signal = AbortSignal.timeout(5_000);
const vars = {};

await storage.open({ signal });

await storage.write({
  key: "hello.bin",
  data: new TextEncoder().encode("hello"),
  signal,
  vars,
});

const exists = await storage.exists({ key: "hello.bin", signal, vars });

if (exists) {
  const data = await storage.read({ key: "hello.bin", signal, vars });
  console.log(new TextDecoder().decode(data));
}

await storage.delete({ key: "hello.bin", signal, vars });
await storage.close({ signal });

Keys

Keys map to a single segment directly under baseUrl.

https://kv.example.com/store/unikvs%3Ahello%2Ebin

• The mapping is `${baseUrl}/${encoded}`, where encoded is the encoded form of `${keyPrefix}${key}`.
• . is converted to %2E. Keys containing dots still fit in a single segment.
• clear() only uses DELETE ?prefix=. An example follows.

DELETE https://kv.example.com/store/?prefix=unikvs%3A

Server

This is the contract the server side provides. The client sends the following requests directly under baseUrl.

PUT https://kv.example.com/store/unikvs%3Ahello%2Ebin
GET https://kv.example.com/store/unikvs%3Ahello%2Ebin
HEAD https://kv.example.com/store/unikvs%3Ahello%2Ebin
DELETE https://kv.example.com/store/unikvs%3Ahello%2Ebin
DELETE https://kv.example.com/store/?prefix=unikvs%3A

• All paths are directly under baseUrl. The key part is encoded as a single segment. . is converted to %2E.
• Implement the server assuming request headers arrive as-is. When Authorization: Bearer ... is present, validate it.
• Status codes map as follows.

• PUT /{encoded} stores a value. It is sent in bulk on write and when a writable stream closes. The byte sequence arrives with Content-Type: application/octet-stream. Return 2xx on success.
• GET /{encoded} reads a value. On success, return the byte sequence with 2xx and application/octet-stream. Return 404 or 410 for missing keys.
• HEAD /{encoded} checks existence. Return 404 or 410 for missing keys. It may be unsupported, in which case return 405 or 501. The client retries only in that case with a GET carrying Range: bytes=0-0, so that GET should be supported.
• DELETE /{encoded} deletes a value. Implement it idempotently. Since 404 and 410 for missing keys are treated as success, return them as-is for already deleted keys.
• DELETE /?prefix={encodedPrefix} is for bulk deletion only. Delete only data matching the prefix. When unsupported, return one of 404, 405, or 501.

openapi: 3.0.3
info:
  title: UniKVS HTTP Storage API
  version: 1.0.0
  description: 'Contract for requests that `@unikvs/http` sends directly under baseUrl. Forward request headers as-is, and validate Authorization: Bearer when present.'
servers:
  - url: '{baseUrl}'
    variables:
      baseUrl:
        default: https://kv.example.com/store
        description: Base location for storage. An absolute URL without a trailing /.
paths:
  /{encoded}:
    parameters:
      - $ref: '#/components/parameters/Encoded'
    put:
      summary: Store a value
      operationId: putValue
      security:
        - bearer: []
        - {}
      requestBody:
        required: true
        content:
          application/octet-stream:
            schema:
              type: string
              format: binary
              description: The byte sequence to store.
      responses:
        '2XX':
          description: Stored successfully.
        '5XX':
          description: Save failed. The client treats it as HttpResponseError.
        default:
          description: Any non-2xx status is a save failure. The client treats it as HttpResponseError.
    get:
      summary: Read a value
      operationId: getValue
      security:
        - bearer: []
        - {}
      parameters:
        - name: Range
          in: header
          required: false
          schema:
            type: string
          description: Sent as bytes=0-0 when used as an existence-check fallback. Optional.
      responses:
        '2XX':
          description: Retrieved successfully.
          content:
            application/octet-stream:
              schema:
                type: string
                format: binary
                description: The stored byte sequence.
        '404':
          description: Not found. The client treats it as missing.
        '410':
          description: Not found. The client treats it as missing.
        '5XX':
          description: Fetch failed. The client treats it as HttpResponseError.
        default:
          description: Any status other than 2xx, 404, or 410 is a fetch failure. The client treats it as HttpResponseError.
    head:
      summary: Check existence
      operationId: hasValue
      security:
        - bearer: []
        - {}
      responses:
        '2XX':
          description: Exists.
        '404':
          description: Does not exist.
        '410':
          description: Does not exist.
        '405':
          description: Not supported. The client retries with a Range GET.
        '501':
          description: Not supported. The client retries with a Range GET.
        '5XX':
          description: Existence check failed. The client treats it as HttpResponseError.
        default:
          description: Any non-2xx, non-404, non-410 status other than 405 or 501 is a check failure. The client treats it as HttpResponseError.
    delete:
      summary: Delete a value
      operationId: deleteValue
      security:
        - bearer: []
        - {}
      responses:
        '2XX':
          description: Deleted successfully. Idempotent.
        '404':
          description: Treated as already deleted and successful. Idempotent.
        '410':
          description: Treated as already deleted and successful. Idempotent.
        '5XX':
          description: Delete failed. The client treats it as HttpResponseError.
        default:
          description: Any status other than 2xx, 404, or 410 is a delete failure. The client treats it as HttpResponseError.
  /:
    delete:
      summary: Delete values matching a prefix
      operationId: clearByPrefix
      security:
        - bearer: []
        - {}
      parameters:
        - $ref: '#/components/parameters/Prefix'
      responses:
        '2XX':
          description: Bulk delete succeeded.
        '404':
          description: Not supported. The client treats it as ClearNotSupported.
        '405':
          description: Not supported. The client treats it as ClearNotSupported.
        '501':
          description: Not supported. The client treats it as ClearNotSupported.
        '5XX':
          description: Bulk delete failed. The client treats it as HttpResponseError.
        default:
          description: Any status other than 2xx, 404, 405, or 501 is a bulk delete failure. The client treats it as HttpResponseError.
components:
  parameters:
    Encoded:
      name: encoded
      in: path
      required: true
      schema:
        type: string
      description: A single segment encoding keyPrefix concatenated with the key. . is converted to %2E.
    Prefix:
      name: prefix
      in: query
      required: true
      schema:
        type: string
      description: The encoded keyPrefix. Only matching prefixes are deleted.
  securitySchemes:
    bearer:
      type: http
      scheme: bearer
      description: 'Sent as Authorization: Bearer. Optional. An explicit Authorization header takes precedence.'

Streams

Both readable and writable streams are supported.

Obtain a writable stream with the following method.

public getWritable(
  args: Pick<IStorage.GetWritableArgs, "key" | "signal" | "vars">,
): WritableStream<Uint8Array<ArrayBuffer>>

Chunks are buffered, then sent in bulk with PUT on close. On abort, buffered chunks are discarded. No streaming delivery to the server is required.

Obtain a readable stream with the following method.

public async getReadable(
  args: Pick<IStorage.GetReadableArgs, "key" | "signal" | "vars">,
): Promise<ReadableStream<Uint8Array<ArrayBuffer>>>

The response body is delegated as-is. Responses without a body return an empty stream.

Notes

• Runtime-independent. It works wherever fetch, Request, and Response are available.
• No built-in timeout. Timeouts are delegated to AbortSignal.
• exists() falls back to GET when HEAD is unsupported. It falls back only on 405 or 501.
• read fails for missing keys. 404 and 410 are treated as missing.
• delete is idempotent. Missing keys are treated as success.
• clear() with an empty keyPrefix is rejected before sending. Pass allowClearWithoutPrefix: true to allow it.
• An unsupported clear() becomes ClearNotSupportedError. A 404 may also mean an incorrect baseUrl.
• Interruption is forwarded as-is. It is not wrapped in HttpNetworkError. Decide whether to retry by whether interruption caused the failure, not by the type.

Errors

import { Http, KeyNotFoundError } from "@unikvs/http";

const storage = new Http("https://kv.example.com/store");
const signal = AbortSignal.timeout(5_000);
const vars = {};

try {
  await storage.open({ signal });
  await storage.read({ key: "missing.bin", signal, vars });
} catch (error) {
  if (error instanceof KeyNotFoundError) {
    console.error(`Key ${error.meta.key} was not found`);
  } else {
    throw error;
  }
} finally {
  await storage.close({ signal });
}

Examples

import { Http } from "@unikvs/http";

const storage = new Http("https://kv.example.com/store");
const signal = AbortSignal.timeout(5_000);
const vars = {};
await storage.open({ signal });

const key = "greeting.bin";
await storage.write({
  key,
  data: new TextEncoder().encode("hello, Http"),
  signal,
  vars,
});

if (await storage.exists({ key, signal, vars })) {
  const data = await storage.read({ key, signal, vars });
  console.log(new TextDecoder().decode(data));
}

await storage.delete({ key, signal, vars });
await storage.clear({ signal, vars });
await storage.close({ signal });

import { Http, type IFetch } from "@unikvs/http";

const fetchImpl: IFetch = async (request: Request) => {
  return globalThis.fetch(request);
};

const storage = new Http("https://kv.example.com/store", {
  fetch: fetchImpl,
  headers: { "X-App": "myapp" },
  token: "secret",
});
const signal = AbortSignal.timeout(5_000);
const vars = {};
await storage.open({ signal });

const key = "greeting.bin";
await storage.write({
  key,
  data: new TextEncoder().encode("hello, Http"),
  signal,
  vars,
});

if (await storage.exists({ key, signal, vars })) {
  const data = await storage.read({ key, signal, vars });
  console.log(new TextDecoder().decode(data));
}

await storage.delete({ key, signal, vars });
await storage.clear({ signal, vars });
await storage.close({ signal });