ShinkaEvolve 修复 Slurm 与本地评估的任务生命周期问题(#226)
fix: job lifecycle for Slurm and local evaluations (status, hang limit, process groups, shutdown) (#226) Job lifecycle fixes from the codebase review. - Leave the time limit of Slurm jobs to Slurm; accept every sbatch --time format and validate it at config time. - Treat a purged Slurm job id as finished, batch squeue polling, keep jobs tracked through a status outage, warn about jobs that stay queued. - Derive the local hang limit from the longest successful evaluation, with a capped provisional limit before the first success. - Run local evaluators in their own process group and kill the group on timeout, cancel, shutdown and fatal signals. - Cancel running evaluations on shutdown after flushing charges and closing the database. Slurm paths were not exercised on a real cluster.
来源与原文
SakanaAI/ShinkaEvolve · 提交动态
作者:Robert Tjarko Lange
发布时间:(北京时间)
阅读原始来源 · github.com ↗完整链接与内容说明
https://github.com/SakanaAI/ShinkaEvolve/commit/170204be56d2945269c468c4ac36bd6ac533cf41
本站目前仅有 AI 导读与摘要,未展示原文正文;技术细节、代码和完整结论请查看原始来源。
ShinkaEvolve 合并 #226,修复 Slurm 与本地评估的任务生命周期问题:Slurm 任务时限交由 Slurm 管理并接受所有 sbatch --time 格式、在配置阶段校验,被清除的任务 id 视为已完成并批量轮询 squeue。本地挂起上限改为按最长成功评估推导,首次成功前使用有上限的临时值;本地评估器在独立进程组中运行,超时、取消、关闭和致命信号时杀掉整个进程组。关闭时先结算费用并关闭数据库,再取消运行中的评估;Slurm 相关路径未在真实集群上验证。
来源:SakanaAI/ShinkaEvolve · 提交动态 · github.com