Skip to content

Updated workflow templating#656

Draft
mranst wants to merge 247 commits intodevelopfrom
se/mranst/workflow_templating_redux2
Draft

Updated workflow templating#656
mranst wants to merge 247 commits intodevelopfrom
se/mranst/workflow_templating_redux2

Conversation

@mranst
Copy link
Collaborator

@mranst mranst commented Nov 10, 2025

This is the most recent version of the concept of updating the workflow templating to use python. This approach should be more straightforward and easier to understand than the previous version, as it uses the same jinja templating in the graph section to make it easier to read (though it doesn't necessarily have to). This pr also merges the task_runtimes and task_questions objects used by the previous pr into one object for simplicity. I'm aiming for this approach to allow for greater independence of the tasks from the suite creation system, to allow generating configs for single tasks and having mock values for code tests for each task, but that's not ready to show off yet.

I still need to merge develop into this and update the docs, but I wanted to open this up as a draft

@mranst
Copy link
Collaborator Author

mranst commented Mar 3, 2026

I refactored some machinery with the configs I've come to dislike recently, it's implemented similarly to the task_attributes in this PR in the sense that it requires registry for the lists, but it's more dynamic and doesn't depend as much on parsing filenames and the general naming of things. Recent commits also add easy support for setting suite-dependent slurm parameters

This is ready for testing, and all tier1 tests are passing for this. I'd like to get this in soon if possible, as merging develop into this can be an extensive process due to how many files this touches, but we can wait for PRs like #716 and #677 to merge before shaking things up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants