Skip to content

Parallelize detection and map head execution using CUDA streams#1

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/execute-map-detection-head-in-parallel
Draft

Parallelize detection and map head execution using CUDA streams#1
Copilot wants to merge 2 commits intomainfrom
copilot/execute-map-detection-head-in-parallel

Conversation

Copy link

Copilot AI commented Nov 27, 2025

The detection and map heads in SparseDriveHead.forward() were executing sequentially. This change runs them in parallel using CUDA streams when both are enabled.

Changes

  • Parallel execution: det_head and map_head now run on separate CUDA streams concurrently
  • Device-aware: Checks feature_maps.is_cuda to only use streams when tensors are on GPU
  • Fallback: Sequential execution preserved for CPU or when only one head is enabled
if with_det and with_map and is_cuda:
    det_stream = torch.cuda.Stream()
    map_stream = torch.cuda.Stream()
    
    with torch.cuda.stream(det_stream):
        det_output = self.det_head(feature_maps, metas)
    
    with torch.cuda.stream(map_stream):
        map_output = self.map_head(feature_maps, metas)
    
    torch.cuda.current_stream().wait_stream(det_stream)
    torch.cuda.current_stream().wait_stream(map_stream)
Original prompt

I want to execute the map and detection head in parallel. Currently it seems that they run sequential. Dont use multiprocessing for that but rather something like torch cuda stream


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI self-assigned this Nov 27, 2025
Co-authored-by: alexx-km <60221685+alexx-km@users.noreply.github.com>
Copilot AI changed the title [WIP] Enable parallel execution of map and detection head Parallelize detection and map head execution using CUDA streams Nov 27, 2025
Copilot AI requested a review from alexx-km November 27, 2025 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants