Skip to content

[Feature] Add time travel ability#28

Open
seawenc wants to merge 6 commits intoapache:mainfrom
seawenc:main
Open

[Feature] Add time travel ability#28
seawenc wants to merge 6 commits intoapache:mainfrom
seawenc:main

Conversation

@seawenc
Copy link

@seawenc seawenc commented Dec 1, 2023

Add time travel ability

Purpose
Linked issue: [Feature] Add time travel ability (#27)

test success:

chengsheng@:presto$ ./client/client.sh
## 方式一:使用日期时间
presto:default> SET SESSION paimon.scan_datetime='2023-11-29 14-55'; # 错误格式
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
Query 20231201_120136_00019_f6snk failed: Time travel param format not supported:2023-11-29 14-55,Supported formats include:yyyy-MM-dd,yyyy-MM-dd HH,yyyy-MM-dd HH:mm,yyyy-MM-dd HH:mm:ss,yyyy-MM-dd HH:mm:ss.SSS
 
presto:default> SET SESSION paimon.scan_datetime='2023-11-29';
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v5 | 2023-11-24 14:56:55.449
 2 | v1 | 2023-11-24 14:49:08.319
(2 rows)
 
Query 20231201_081819_00007_j7n8h, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:02 [0 rows, 0B] [0 rows/s, 0B/s]
 
 
presto:default> SET SESSION paimon.scan_datetime='2023-11-29 14:55';
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v5 | 2023-11-24 14:56:55.449
 2 | v1 | 2023-11-24 14:49:08.319
(2 rows)
 
Query 20231201_120146_00021_f6snk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]
 
presto:default> exit;
 
## 需要退出清空session参数 否则两种类型的参数会相互影响
 
chengsheng@:presto$ ./client/client.sh
## 方式二:使用时间戳
presto:default> SET SESSION paimon.scan_timestamp_millis=1700805905000;
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v5 | 2023-11-24 14:56:55.449
 2 | v1 | 2023-11-24 14:49:08.319
(2 rows)
 
Query 20231201_115739_00009_f6snk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:03 [0 rows, 0B] [0 rows/s, 0B/s]
 
presto:default> SET SESSION paimon.scan_timestamp_millis=1700809010000;
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v5 | 2023-11-24 14:56:55.449
 2 | v1 | 2023-11-24 14:49:08.319
(2 rows)
 
Query 20231201_115834_00011_f6snk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]
 
presto:default> exit;
 
## 需要退出清空session参数 否则两种类型的参数会相互影响
 
## 方式三:指定tag版本
chengsheng@:presto$ ./client/client.sh
presto:default> set session paimon.scan_snapshot_id=1;
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v1 | 2023-11-24 14:49:01.633
(1 row)
 
Query 20231201_121340_00027_f6snk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]
 
presto:default> set session paimon.scan_snapshot_id=2;
SET SESSION
presto:default> select * from paimon.default.paimon_hive_time_travel;
 a | b  |       update_time      
---+----+-------------------------
 1 | v1 | 2023-11-24 14:49:01.633
 2 | v1 | 2023-11-24 14:49:08.319
(2 rows)
 
Query 20231201_121350_00029_f6snk, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

public class PrestoTableHandle implements ConnectorTableHandle {

public static final String SCAN_TIMESTAMP = "scan_timestamp_millis";
public static final String SCAN_DATETIME = "scan_datetime";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove this option, if it is required, it should be done in paimon-core.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants