Skip to content

Services · Reference

File Extraction Service

A configurable Effect service that safely extracts bounded Markdown text from in-memory documents using Xberg.

Last updated October 8, 2026

Overview

service-file-extraction extracts Markdown text from document bytes through Xberg. It provides bounded input and output sizes, UTF-8-safe truncation, concurrency control, extraction timeouts, archive security limits, and typed Effect failures.

Install

shell
bunx shadcn@latest add @krak-stack/service-file-extraction

Example: Extract an S3 Document to Markdown

Read the object through S3Service, then pass its bytes and stored metadata to the extraction service. The returned content is Markdown. Install @krak-stack/service-s3 separately when using this integration.

ts
import { Effect, Layer } from "effect";
import { FileExtractionService } from "@/services/file-extraction";
import { S3Service } from "@/services/s3";

const objectToMarkdown = ({
  objectKey,
  filename,
  mimeType,
}: {
  objectKey: string;
  filename: string;
  mimeType: string;
}) =>
  Effect.gen(function* () {
    const s3 = yield* S3Service;
    const extraction = yield* FileExtractionService;
    const bytes = yield* s3.bytes(objectKey);
    const result = yield* extraction.markdown({ bytes, filename, mimeType });
    return result.content;
  });

const layer = Layer.merge(S3Service.layer, FileExtractionService.layer);
const markdown = await Effect.runPromise(
  objectToMarkdown({
    objectKey: "uploads/document.pdf",
    filename: "document.pdf",
    mimeType: "application/pdf",
  }).pipe(Effect.provide(layer)),
);

Example: Extract In-Memory Bytes

Use the extraction service directly when bytes have already been loaded from another source.

ts
const result = await Effect.runPromise(
  Effect.gen(function* () {
    const extraction = yield* FileExtractionService;
    return yield* extraction.markdown({
      bytes,
      filename: "document.pdf",
      mimeType: "application/pdf",
    });
  }).pipe(Effect.provide(FileExtractionService.layer)),
);

API

ts
type FileExtractionInput = {
  bytes: Uint8Array;
  filename: string;
  mimeType: string;
};

type ExtractFileInput = FileExtractionInput & {
  outputFormat: OutputFormat;
};

type FileExtractedText = {
  content: string;
  contentByteSize: number;
  truncated: boolean;
};

extraction.markdown(input: FileExtractionInput)
extraction.text(input: FileExtractionInput)
extraction.html(input: FileExtractionInput)
extraction.extract(input: ExtractFileInput)
  • markdown renders OutputFormat.Markdown.
  • text renders OutputFormat.Plain.
  • html renders OutputFormat.Html.
  • extract requires any Xberg OutputFormat, including Djot, Json, Structured, and Custom.

Output Formats

Use markdown, text, or html when the desired format is known. Use extract with a required Xberg OutputFormat for lower-level access to any supported format.

ts
import { OutputFormat } from "@/services/file-extraction";

const markdown = yield* extraction.markdown(input);
const text = yield* extraction.text(input);
const html = yield* extraction.html(input);
const djot = yield* extraction.extract({
  ...input,
  outputFormat: OutputFormat.Djot,
});

All methods return the same bounded result shape with content, contentByteSize, and truncated.

Configuration

The default layer accepts inputs up to 20 MB, returns up to 512 KB of Markdown, runs two extractions concurrently, and applies a 30-second timeout. Use layerWith to override operational limits while retaining secure extraction defaults.

ts
const layer = FileExtractionService.layerWith({
  maxInputBytes: 10 * 1024 * 1024,
  maxOutputBytes: 256 * 1024,
  maxConcurrentExtractions: 4,
  timeoutSeconds: 20,
});
Dependencies
Packages and shadcn components required by this registry item.

Dependencies