DAOS-18367 vos: properly evict object for failed transaction #17325
Conversation
|
Ticket title is 'Enhance dtx_act_ent_cleanup() to only evict self-created object when transaction failure' |
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17325/1/testReport/ |
|
Test stage Functional Hardware Large MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17325/1/execution/node/1282/log |
|
Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17325/1/execution/node/1367/log |
Nasf-Fan
force-pushed
the
Nasf-Fan/DAOS-18367_1
branch
from
0747c8a to
6413f26
Compare
Nasf-Fan
changed the title
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17325/2/testReport/ |
|
Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17325/2/execution/node/1345/log |
|
Test stage Functional Hardware Medium Verbs Provider MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17325/2/execution/node/1365/log |
dmg_telemetry_io_basic.py failed for DAOS-18388, not related with the patch, to be retested. |
Currently, if a transaction failed for some reason, the cleanup logic will try to evict related vos object from cache to avoid leaving stable information in cache. Such logic works well for the system with PMEM. But under md-on-ssd mode, the eviction may cause trouble. Because one vos modification may hold the same object multiple times, and there is CPU yield during these object hold actions. That creates race windows for other concurrent operations against the same object. This patch changes the logic: when the transaction changes some vos object(s), it will record related oid(s), if such transaction failed in subsequent process, it will only evict these modified objects. The others in cache will not be affected during transaction cleanup. On the other hand, under md-on-ssd mode, CPU may yield during backend TX start, the object that is held by current modification maybe marked as evicted in such race windows. So add logic to check whether related object is evicted or not after backend TX started, if yes, then restart current transaction. Signed-off-by: Fan Yong <fan.yong@hpe.com>
Nasf-Fan
force-pushed
the
Nasf-Fan/DAOS-18367_1
branch
from
6413f26 to
96e5af4
Compare
|
Test stage NLT on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17325/3/testReport/ |
|
Test stage Functional Hardware Medium Verbs Provider MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-17325/3/execution/node/1364/log |
|
Test stage Functional Hardware Medium MD on SSD completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-17325/3/testReport/ |
Currently, if a transaction failed for some reason, the cleanup logic
will try to evict related vos object from cache to avoid leaving stable
information in cache. Such logic works well for the system with PMEM.
But under md-on-ssd mode, the eviction may cause trouble. Because one
vos modification may hold the same object multiple times, and there is
CPU yield during these object hold actions. That creates race windows
for other concurrent operations against the same object.
This patch changes the logic: when the transaction changes some vos
object(s), it will record related oid(s), if such transaction failed
in subsequent process, it will only evict these modified objects. The
others in cache will not be affected during transaction cleanup.
On the other hand, under md-on-ssd mode, CPU may yield during backend
TX start, the object that is held by current modification maybe marked
as evicted in such race windows. So add logic to check whether related
object is evicted or not after backend TX started, if yes, then restart
current transaction.
Signed-off-by: Fan Yong fan.yong@hpe.com
Steps for the author:
After all prior steps are complete: