Mixture of Experts (MoE)¶
Hybrid-MoE routing and grouped expert computation, used by MoE variants (Qwen3 large builds). Dense families (Llama 3.x ≤8B) do not use this.
Hybrid-MoE routing and grouped expert computation, used by MoE variants (Qwen3 large builds). Dense families (Llama 3.x ≤8B) do not use this.
anllm.core.moe.Moe
¶anllm.core.moe.route_topk(x, w_router, top_k, norm_topk_prob=False)
¶anllm.core.moe.grouped_expert_linear(x, w_experts, expert_ids)
¶