Chat Service
A composable Effect AI chat service with streaming text, tool execution, approvals, bounded agent rounds, and serializable history.
bunx --bun shadcn@latest add https://krakstack.net/r/chat-service.jsonOverview
chat-service installs a generic Effect AI streaming service and its request/event schemas. It handles serializable chat history, streamed text, tool execution, approval responses, bounded agent rounds, and client-facing event conversion. Authentication, model selection, toolkit composition, persistence, and HTTP transport stay application-owned.
Install
bunx shadcn@latest add @krak-stack/chat-serviceThe item creates services/chat/index.ts and services/chat/schema.ts.
Configure A Model And Toolkit
Build the model and toolkit in your application layer or request handler. The model is provided through Effect context; the toolkit and system prompt are passed per conversation.
const chat = yield* ChatService;
const events = yield* chat.stream({
action: request.action,
history: request.history,
systemPrompt,
toolkit,
}).pipe(Effect.provide(model));events is an Effect Stream of ChatEvent. Forward it through SSE, NDJSON, a server function, or another streaming transport.
Add An HTTP API Endpoint
Define an endpoint with ChatRequest as its payload and ChatEvent as its streaming success type.
export const ChatApiGroup = HttpApiGroup.make("chat").add(
HttpApiEndpoint.post("stream", "/chat", {
payload: ChatRequest,
success: HttpApiSchema.StreamSse({ data: ChatEvent }),
}),
);In the handler, authenticate first, build request-scoped tool handlers, then return the stream from ChatService. Provide ChatService.layer where the API layer is assembled.
const chatHandler = HttpApiBuilder.group(AppApi, "chat", (handlers) =>
handlers.handle("stream", ({ payload }) =>
Effect.gen(function* () {
const chat = yield* ChatService;
return yield* chat.stream({
action: payload.action,
history: payload.history,
systemPrompt,
toolkit,
}).pipe(Effect.provide(model));
}),
),
).pipe(Layer.provide(ChatService.layer));Conversation History
Each completed response emits a history event containing the serialized Effect AI chat. Send that value back in the next ChatRequest. Validate ownership before accepting persisted history, and enforce an application-appropriate size limit.
Tool Approvals
Tools marked with needsApproval emit an approval-required event. Resume the conversation with an approval action containing the emitted approvalId. The service rejects missing, duplicate, or already-resolved approvals as invalid history.
Agent Rounds
The service continues after resolved tool calls until the model returns text, requests approval, or reaches the built-in five-round limit. A round-limit error event tells clients to ask a more specific question.