iCreatAI News
Skip to content
Trending storyWatching

Hugging Face previews fine-tuning duration prediction research

1 reports1 sources2 days agoUpdated

Understand this story

AI overview

Hugging Face has released a paper preview proposing a method to predict the end-to-end fine-tuning time of generative language models before task dispatch. The study is based on a comparison of 272 real SFT production tasks (approximately 7,382 GPU-hours, NVIDIA H200): traditional FLOP estimation could barely explain actual duration (R²=0.31, MAE 12.41 hours); a CatBoost regressor relying only on pre-dispatch information achieved R²=0.98, MAE 1.71 hours, and a median error of 0.51 hours in 5-fold cross-validation. These are results disclosed in the paper preview and await validation by the formal paper and more scenarios.

AI overview based on source reports · Updated 2 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 19
  1. Hugging Face Blog
    Why your fine-tuning job takes longer than the FLOPs say

    Using 272 production supervised fine-tuning runs on NVIDIA H200 GPUs (about 7,382 GPU-hours), the authors compared analytical FLOP-based duration estimates against learned predictors. The standard FLOP estimate explained little of the observed time (R² = 0.31, MAE 12.41 h), while a CatBoost regressor using only pre-dispatch configuration features reached R² = 0.98 with a mean absolute error of 1.71 h and a median error of 0.51 h in 5-fold cross-validation. The authors attribute the gap to training-specific signals such as evaluation cadence, checkpoint policy, per-GPU load and fine-tuning regime, and note the model is limited to SFT on H200s with few runs above 60 hours.

Activity for this story

Current comparable activity
3
Peak comparable activity
10Nov 19 04:00
Change in the past 24 hours
-51%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.