Skip to content

tests: failpoint_tests::txn_cleanup_2pc_locks is flaky #516

New issue
New issue
Open
Open
tests: failpoint_tests::txn_cleanup_2pc_locks is flaky#516
@pingyu

Description

@pingyu
pingyu
opened on Jan 14, 2026

CI link: https://github.com/tikv/client-rust/actions/runs/20982618570/job/60310631494?pr=515

PR: #515

Nextest run ID 1784fdea-b79b-49f5-ae6b-ac103c96cca0 with nextest profile: ci
    Starting 26 tests across 4 binaries (55 tests skipped)
        SLOW [> 60.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>120.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>180.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>240.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>300.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>360.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>420.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>480.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        SLOW [>540.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
 TERMINATING [>600.000s] (─────) tikv-client::failpoint_tests txn_cleanup_2pc_locks
     TIMEOUT [ 601.103s] ( 1/26) tikv-client::failpoint_tests txn_cleanup_2pc_locks
        PASS [   9.742s] ( 2/26) tikv-client::failpoint_tests txn_cleanup_async_commit_locks
        PASS [   1.373s] ( 3/26) tikv-client::failpoint_tests txn_cleanup_locks_batch_size
        PASS [   5.370s] ( 4/26) tikv-client::failpoint_tests txn_cleanup_range_async_commit_locks
        PASS [   6.477s] ( 5/26) tikv-client::failpoint_tests txn_optimistic_heartbeat
        PASS [   1.349s] ( 6/26) tikv-client::integration_tests txn_bank_transfer
        PASS [   0.524s] ( 7/26) tikv-client::integration_tests txn_batch_mutate_optimistic
        PASS [   4.546s] ( 8/26) tikv-client::integration_tests txn_batch_mutate_pessimistic
        PASS [   0.492s] ( 9/26) tikv-client::integration_tests txn_crud
        PASS [   0.522s] (10/26) tikv-client::integration_tests txn_get_for_update
        PASS [   0.026s] (11/26) tikv-client::integration_tests txn_get_timestamp
        PASS [   0.485s] (12/26) tikv-client::integration_tests txn_insert_duplicate_keys
        PASS [   0.503s] (13/26) tikv-client::integration_tests txn_key_exists
        PASS [  13.040s] (14/26) tikv-client::integration_tests txn_lock_keys
        PASS [  13.055s] (15/26) tikv-client::integration_tests txn_lock_keys_error_handle
        PASS [   0.492s] (16/26) tikv-client::integration_tests txn_pessimistic
        PASS [   0.518s] (17/26) tikv-client::integration_tests txn_pessimistic_delete
        PASS [  23.512s] (18/26) tikv-client::integration_tests txn_pessimistic_heartbeat
        PASS [   1.152s] (19/26) tikv-client::integration_tests txn_pessimistic_rollback
        PASS [   1.636s] (20/26) tikv-client::integration_tests txn_read
        PASS [   0.487s] (21/26) tikv-client::integration_tests txn_scan
        PASS [   0.485s] (22/26) tikv-client::integration_tests txn_scan_reverse
        PASS [   0.802s] (23/26) tikv-client::integration_tests txn_scan_reverse_multi_regions
        PASS [   9.626s] (24/26) tikv-client::integration_tests txn_split_batch
        PASS [   0.573s] (25/26) tikv-client::integration_tests txn_unsafe_destroy_range
        PASS [   1.018s] (26/26) tikv-client::integration_tests txn_update_safepoint
────────────
     Summary [ 698.921s] 26 tests run: 25 passed, 1 timed out, 55 skipped
     TIMEOUT [ 601.103s] ( 1/26) tikv-client::failpoint_tests txn_cleanup_2pc_locks
  stdout ───

    running 1 test
    init finish with 105 regions
    test txn_cleanup_2pc_locks has been running for over 60 seconds
  stderr ───
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:37Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:37Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:37Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:38Z INFO  failpoint_tests::common] splitting regions...
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:39Z INFO  failpoint_tests] test no commit
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:39Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:39Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  failpoint_tests] test all committed
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:2379"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::cluster] All PD endpoints are consistent: ["127.0.0.1:2379"]
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "http://127.0.0.1:2379/"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20161"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20160"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::pd::client] connect to tikv endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:40Z INFO  tikv_client::common::security] connect to rpc server at endpoint: "127.0.0.1:20162"
    [2026-01-14T04:45:47Z INFO  tikv_client::pd::timestamp] TSO stream terminated
    [2026-01-14T04:45:49Z INFO  tikv_client::pd::timestamp] TSO stream terminated

    (test timed out)

error: test run failed
make: *** [Makefile:32: integration-test-txn] Error 100

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions