DAOS-17928 rebuild: migrate object processing from system xstream to main xstream #17311
Conversation
…main xstream (#17031) To enhance object migration efficiency, we recommend processing OIDs directly in main xstreams rather than routing them through system xstreams. Currently, the workflow involves main xstreams scanning and gathering OIDs before distributing them to corresponding ranks' system xstreams for processing. However, this approach introduces significant overhead from B+ Tree operations. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
* DAOS-17928 rebuild: refine migration fixes - Rename ds_pool_migrate_arg fields with pma_ prefix for better grep - Remove duplicate pool check in ds_migrate_end_ult - Make sp_rebuilding atomic using ATOMIC - Simplify sp_rebuilding increment logic after lookup. - Use deep stack size for iv ult. - Fix to return real error if migration failed. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
…reate() In migrate_pool_tls_create(), error handling could result in both migrate_pool_tls_destroy() and migrate_pool_tls_put() being called on the same pool_tls object, leading to a possible double free/use-after-free. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
|
Ticket title is 'Migrate object processing from system xstream to main xstream' |
Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17311/1/testReport/ |
|
Test stage Test RPMs on EL 8.6 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17311/1/display/redirect |
|
Test stage Functional on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17311/1/display/redirect |
1 similar comment
|
Test stage Functional on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-17311/1/display/redirect |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17311/2/testReport/ |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17311/4/testReport/ |
|
Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17311/6/execution/node/702/log |
|
Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17311/6/execution/node/721/log |
|
Test stage Functional Hardware Medium completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17311/7/execution/node/482/log |
|
Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17311/7/execution/node/492/log |
To enhance object migration efficiency, we recommend processing OIDs directly in main
xstreams rather than routing them through system xstreams. Currently, the workflow involves
main xstreams scanning and gathering OIDs before distributing them to corresponding ranks'
system xstreams for processing. However, this approach introduces significant overhead
from B+ Tree operations.
Steps for the author:
After all prior steps are complete: