Kimi K3
Notes on Kimi K3, a 2.8T-parameter, 104B-active multimodal MoE that combines hybrid KDA-MLA attention, Attention Residuals, Stable LatentMoE, multi-effort agentic RL, and a one-million-token context window.
Notes on Kimi K3, a 2.8T-parameter, 104B-active multimodal MoE that combines hybrid KDA-MLA attention, Attention Residuals, Stable LatentMoE, multi-effort agentic RL, and a one-million-token context window.