Skip to content

fix(agent): normalize Worker CUDA device requests - #156

Merged
PigeonDan1 merged 2 commits into
X-LANCE:harness-tui-agentfrom
xiangshengyun:fix/sure-infer-worker-device-20261029
Oct 30, 2026
Merged

PigeonDan1 merged 2 commits into
X-LANCE:harness-tui-agentfrom
xiangshengyun:fix/sure-infer-worker-device-20261029

Conversation

@xiangshengyun

Copy link
Copy Markdown
Contributor

Summary

Fix multi-GPU inference Workers receiving an unsupported model-facing device value such as cuda:0.

Problem

Each Worker is isolated to one physical GPU through CUDA_VISIBLE_DEVICES. Inside that Worker, the assigned GPU is already exposed as the local device
cuda:0. Passing DEVICE=cuda:0 to some model runtimes is not supported and can cause inference to fail with:

unsupported device request: cuda:0

## Changes

- Normalize Worker model-facing device requests:
    - cuda:N → cuda
    - cuda → cuda
    - auto → auto
    - reject unsupported Worker device values such as cpu

- Keep physical GPU isolation through CUDA_VISIBLE_DEVICES.
- Apply the normalized device value to DEVICE and SURE_EVAL_DEVICE_ACTUAL.
- Update smoke execution and parallel Worker environment handling.
- Add regression coverage for Worker device normalization.

## Validation

- Worker smoke inference completed successfully.
- Multi-GPU inference completed all 24,624 samples without the cuda:0 device error.
- Updated Worker device values are compatible with the model runtime.

@PigeonDan1
PigeonDan1 merged commit ac420ab into X-LANCE:harness-tui-agent Oct 30, 2026
2 checks passed
@xiangshengyun
xiangshengyun deleted the fix/sure-infer-worker-device-20261029 branch November 13, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants