You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
GPU profiling harness: per-stage time and peak-memory breakdown (GKE first, then A40) #31
Every CUDA-level item on the roadmap (#32, #33, #34, #35, #36) should be chosen and sized from profiles, not guesses. The single-L4 sweep showed admission limits scaling with cells (rows × features), not rows², which suggests per-cell activations in TabICL's column-embedding and row-interaction stages dominate the fit peak more than attention. That's a hypothesis to confirm.
Tasks
Profiling harness (benchmarks/profile_fit_predict.py): torch profiler (CUDA time, memory timeline per stage: col_embedder, row_interactor, icl_predictor) plus an Nsight Systems recipe, over the sweep's shapes (10–1,000 features, up to the admission limit) for TabICLv2 and TabPFN-3.5, in kv and repr modes.
Record what TabICL's own InferenceManager already does (stage batching from its activation model, CPU/disk offload, async copies) and how it interacts with tabctx's kv mode.
Run on a GKE GPU node first (L4 and A100), then on the prod A40 once tabctx is deployed (sm_86: no FA3, no FP8).
Write up: top kernels by time, peak-memory breakdown by stage and tensor, and a ranked target list for the kernel issues.
Activity
bench: profiling harness (#31), copy bandwidth (#39), GKE GPU devbox
perf(tabicl): torch.compile the attention blocks (1.7-2.2x GPU half)
feat: multi-GPU replicas with workload-aware placement (0.12.0) (#38)
perf(tabicl): torch.compile the attention blocks (1.7-2.2x GPU half)
perf: worker processes, compiled TabICL, GPU preprocessing, int8 KV, …