Skip to content

Distributed solution - new madengine CLI#11

Open
coketaste wants to merge 155 commits intomainfrom
coketaste/refactor
Open

Distributed solution - new madengine CLI#11
coketaste wants to merge 155 commits intomainfrom
coketaste/refactor

Conversation

@coketaste
Copy link
Contributor

@coketaste coketaste commented Jul 3, 2025

DoD:
This solution splits the madengine run_models.py workflow into separate 'build' and 'run' phases to enable distributed execution scenarios.

@coketaste coketaste self-assigned this Jul 3, 2025
coketaste added 28 commits July 3, 2025 15:43
@coketaste
Fixed the test cases for distributed solution
ea2dc0c
@coketaste
Updated the interface and fix the issue due to updating
bb64b73
@coketaste
Reorganize the cli interface of distributed solution
86d1790
@coketaste
Updated the interface of distributed solution and refine the code wit…
dd71dfa 
…h test coverage
@coketaste
Added setup.py for installation with dev
8236a7b
@coketaste
Fix the test case of distributed cli
0c42bbf
@coketaste
Fixed the flow of manifest and run_phase to work properly
f942a45
@coketaste
Updated setup.py for the cases of modern and legacy installation
08ff29b
@coketaste
Fixed and enhanced live log in build and run phases
d82d78e
@coketaste
Fixed the log generate for different phase, and correct log name
c848419
@coketaste
Fix the perf.csv generation in distributed execution
3e2a44c
@coketaste
Fixed the data which update to perf.csv
8a359ac
@coketaste
Fixed the columns in perf.csv due to parsing issue
ac32cbe
@coketaste
Fix the incorrect regex escaping in the container runner that prevent…
bb6d3fc 
…ed proper performance metric extraction
@coketaste
Update the patterns of performance and metric
a255c50
@coketaste
Fixed the issue of docker_image column in perf.csv
8988508
@coketaste
Improve the interface and reduce erro in registry flow
f0a10a7
@coketaste
Updated the flow of run phase, fix the docker pull, fix the creds ver…
8caae5c 
…ify before docker login
@coketaste
updated the tagged name for docker image and add a docker_image_tagge…
942e666 
…d field to the build manifest
@coketaste
Updated the sequence of operations in build phas
a7baa17
@coketaste
Fixed the registry_image
2e613ca
@coketaste
Update the registry_image
b4e7d22
@coketaste
Updated the process of run phase
d1ecb97
@coketaste
Refactored the file structure of package for distributed_cli
799cce7
@coketaste
Fixed the errors in unit tests
9875fda
@coketaste
Fix the error in unit test of distributed cli
168ffe5
@coketaste
Refactored constants to make design as best practices
756d82a
@coketaste
Cleanup:
9431d7f 
Simplified - No confusing multiple configuration files
Modern - Follows current Python packaging standards (PEP 621)
Maintainable - Single source of truth
Compatible - Works with all modern Python tools
@coketaste coketaste changed the title Distributed solution refactoring current run_model flow Distributed solution - new madengine CLI Aug 5, 2025
coketaste and others added 22 commits August 5, 2025 17:26
@coketaste
Updated README
72982f8
@coketaste
Added discover command to mad_cli
b6b79ca
@coketaste
Implemented CLI detect MAD_CONTAINER_IMAGE in additional context, pro…
00f4a5e 
…duction-ready and maintains full backward compatibility with existing madengine workflows
@coketaste
Merge pull request #31 from ROCm/coketaste/refactor-interface
ee5740d 
Ensure Backward Compatibility in New madengine-cli
@coketaste
Implemented the core multi-GPU architectures support for docker image…
364bef4 
… building
@coketaste
Implemented unit tests for the feature of multi-gpu arch
156bcfe
@coketaste
Debug and fix the unit test of multi gpu arch
8457257
@coketaste
Debug the issue of display results table
3a0b4c7
@coketaste
Enhanced the results table, and improved the flow of handle gpu arch …
682bec2 
…surfix at docker image name
@coketaste
Creates architecture-specific images with proper naming and metadata,…
89784ca 
… regardless of the underlying Dockerfile configuration.
@coketaste
Fixed the syntax error
23bbf57
@coketaste
Merge pull request #32 from ROCm/coketaste/refactor-multi-gpu-archs
4e61147 
Feature to build docker images for multigpu archs
@Boss2002n
ported changes from coketaste/amd-smi
5444a67
@Boss2002n
Revert "ported changes from coketaste/amd-smi"
9dfe5d8 
This reverts commit 5444a67.
@coketaste
Resolved merging conflicts
d5c3402
@coketaste
Fixed the tools for distributed mode
e9202c2
@coketaste
Fixed the cleanup
b49ed4b
@coketaste
Merge pull request #52 from ROCm/coketaste/refactor-tools
0ac1855 
Sync up amd-smi and update tools
@coketaste
Fixed the table of resutls
15cbeaa
@coketaste
Fixed the GPU Product Name
026fec3
@coketaste
Fixed the issue in selftest
9b7b347
@coketaste
Enhanced unit tests and cleanup
eca075a
@coketaste coketaste marked this pull request as ready for review November 28, 2025 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@coketaste coketaste

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@coketaste @Boss2002n