Model¶
The shared, family-agnostic causal language model: ForCausalLM, one
TransformerBlock, Attention, and MLP that serve both Qwen3 and Llama.
Family differences (per-head QK-norm, head_dim, MoE expert naming) are detected
automatically. Qwen3ForCausalLM etc. remain backward-compatible aliases.
Loaders¶
anllm.core.model.load_model(model_path, features=None)
¶
Load any supported model (Qwen3, Llama 3.x, Mistral) from a path.
Dispatch is driven by model_type read from the checkpoint's
config.json, so adding a new family (or a future Qwen3.x variant)
is purely a config-time change — the architecture is shared.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_path
|
str | Path
|
Local directory containing |
required |
features
|
ModelFeatures | None
|
Optional kernel/dequantization feature flags. |
None
|
Returns:
| Type | Description |
|---|---|
ForCausalLM
|
A fully assembled :class: |
anllm.core.model.load_qwen3(model_path, features=None)
¶
Load a Qwen3 (or compatibly-structured) model from a checkpoint path.
ForCausalLM¶
anllm.core.model.ForCausalLM
¶
Dense (and hybrid-MoE) causal language model shared across families.
Supports Qwen3 and Llama through a single, family-dispatched architecture:
- head_dim and GQA are normalized by :class:ModelConfig,
- per-head QK-norm is auto-detected from the checkpoint,
- layer-local MoE is resolved from config.is_moe_layer.
Qwen3ForCausalLM remains a thin alias for backward compatibility.
TransformerBlock¶
anllm.core.model.TransformerBlock
¶
One transformer block. Attention + (dense | MoE) FFN with residuals.
A block is an MoE block only when the model family and this specific
layer declare experts. For dense families (Llama 3.x ≤8B) the FFN is a
plain :class:MLP; for MoE families (Qwen3 30B+, Llama 4+) the hybrid FFN
is used on the sparse layers. Expert weight names are looked up per family
through the _EXPERT_* maps so a single MoE kernel serves all of them.
Attention¶
anllm.core.model.Attention
¶
Multi-head / grouped-query attention shared by Qwen3 and Llama.
The two families differ only in
- whether per-head QK-norm is applied (Qwen3 yes, Llama no), and
- whether
head_dimcomes from config (Qwen3) or is derived ashidden_size // num_attention_heads(Llama).
Both are handled here: QK-norm is enabled only when the checkpoint ships
q_norm/k_norm weights (or the config explicitly asks for it), and
head_dim was already normalized by :class:ModelConfig.
MLP¶
anllm.core.model.MLP
¶
Gated SiLU feed-forward network shared by Qwen3 and Llama dense layers.