Skip to content

feat: complete Ai interface compatibility in jack-ai proxy wrapper #12

@hellno

Description

@hellno

Context

Our createJackAI() wrapper (jack-ai.ts) currently only implements .run() from the full Cloudflare Ai interface. This works for all current templates since workers-ai-provider only calls .run(), but users calling env.AI methods directly will hit missing functionality.

Current State

// What we return
{ run: (model, inputs, options?) => Promise<T | ReadableStream> }

// Full Ai interface (from @cloudflare/workers-types)
abstract class Ai {
  aiGatewayLogId: string | null;
  gateway(gatewayId: string): AiGateway;
  autorag(autoragId: string): AutoRAG;
  run(model, inputs, options?): Promise<...>;
  models(params?): Promise<AiModelsSearchObject[]>;
  toMarkdown(): ToMarkdownService;
  toMarkdown(files, options?): Promise<ConversionResponse[]>;
}

Missing Methods

Method Priority Notes
gateway(id) P1 AI Gateway for caching, rate limiting, analytics. Power users need this for production
toMarkdown() P2 Document-to-markdown conversion (PDFs, images). Useful for document processing templates
models(params?) P3 List available models. Nice-to-have for model pickers in agent UIs
autorag(id) P3 AutoRAG binding. Newer API, lower demand
aiGatewayLogId P3 Only relevant once gateway() is implemented

Implementation Approach

Each method needs a corresponding route in the binding-proxy worker (apps/binding-proxy-worker/):

  • /ai/gateway → forward to env.AI.gateway(id) with metering
  • /ai/models → forward to env.AI.models(params) with metering
  • /ai/toMarkdown → forward to env.AI.toMarkdown(files, options) with metering

The jack-ai.ts wrapper would add methods that call these proxy routes, same pattern as .run().

Why Not Just Use env.AI Directly?

Jack cloud is multi-tenant. The proxy provides:

  • Per-project quota enforcement (1000 req/day default)
  • Usage metering via Analytics Engine
  • Rate limiting (100 req/10s burst protection)

Direct env.AI is not available in jack cloud deployments — only the __AI_PROXY service binding is injected.

Acceptance Criteria

  • createJackAI() return type passes satisfies Ai (or close to it)
  • Each proxied method has metering + quota enforcement
  • Existing .run() behavior unchanged
  • Templates can use gateway() for caching in production

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions