Server

OpenAI-compatible HTTP server wrapping the batched engine: v1/models, v1/chat/completions, v1/completions, and healthz, with both streaming and non-streaming responses.

anllm.server.app.create_app(*, model_path=None, model_factory=None, model=None, tokenizer=None, model_id='anllm', batch_size=8, max_seq_len=2048, prefill_step=256)

Create the OpenAI-compatible app.

Pass either model_path (load from disk) or model_factory plus tokenizer (build at startup). MLX binds lazy graphs to the thread that created them, so the factory runs on the engine's worker thread; handing a pre-built model instance only works when that instance was created on the same thread that will pump the engine.

anllm.server.app.TokenizerAdapter

Uniform tokenizer surface over transformers-style tokenizers.