Repository navigation

OLMoE3: log the OLMo-core hero's metrics, fix z-loss under GA, and add OLMo-core's perplexity evals - #5880

Open

AkshitaB wants to merge 6 commits into

AI-Hypercomputer:agagik-olmo35from

AkshitaB:akshitab/olmo35-hero-metrics

Open

OLMoE3: log the OLMo-core hero's metrics, fix z-loss under GA, and add OLMo-core's perplexity evals#5880
AkshitaB wants to merge 6 commits into
AI-Hypercomputer:agagik-olmo35from
AkshitaB:akshitab/olmo35-hero-metrics

Commits on Oct 20, 2026
- Cut vocab tiles along the sequence; add an opt-in GA loop barrier
  
  authored and
  Akshita Bhagia
  committed
  bb28536
- Read ga_params_loop_barrier with a default, and test that it leaves GA unchanged
  
  authored and
  Akshita Bhagia
  committed
  4bffa26
- Add moe_global_load_balancing so OLMoE3's load-balancing loss spans the whole batch
  
  Akshita Bhagia
  and
  claude
  committed
  ba83dec
Commits on Oct 21, 2026
- Log OLMo-core's training metrics for OLMoE3: per-layer router stats, CE, masked fraction, throughput
  
  Akshita Bhagia
  and
  claude
  committed
  be14764
- Log z_loss under gradient accumulation as the batch's per-token mean, not the sum over micro-batches
  
  Akshita Bhagia
  and
  claude
  committed
  02e2120
- Run OLMo-core's in-loop perplexity evals (eval/lm/<set>/CE loss and PPL) in the training loop
  
  Akshita Bhagia
  and
  claude
  committed
  b05d15d