Rotary Position Embeddings (RoPE)

RoPE applies position information via rotating the Q/K vectors. Used by the shared Attention block.

anllm.core.rope.RoPE