Details
-
Improvement
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
2023-2-Storage, 2023-3-Storage
Description
对比balance合并调度策略优化后版本与优化前版本的合并性能,优化后版本慢80%:
只有合并,无其他操作,数据50%乱序。
测试环境:192.168.130.1 1C1D 16CPU 32G内存
1.关合并,生成数据备份
备份路径在:/data1/iotdb/m_0227_74c950d/data_close_comp_1partition
CN :MAX_HEAP_SIZE="2G"
DN:MAX_HEAP_SIZE="20G"
MAX_DIRECT_MEMORY_SIZE="6G"
COMMON :
time_partition_interval=6048000000
max_waiting_time_when_insert_blocked=3600000
enable_seq_space_compaction=false
enable_unseq_space_compaction=false
enable_cross_space_compaction=false
benchmark配置见附件,运行脚本:run_2_comp.sh
2. copy 同一份数据,对比2个版本的合并性能
2个版本的显式设置的参数:
CN :
MAX_HEAP_SIZE="2G"
DN :
MAX_HEAP_SIZE="20G"
MAX_DIRECT_MEMORY_SIZE="6G"
COMMON:
time_partition_interval=6048000000
query_timeout_threshold=36000000
max_waiting_time_when_insert_blocked=3600000
enable_seq_space_compaction=true
enable_unseq_space_compaction=true
enable_cross_space_compaction=true
compaction_write_throughput_mb_per_sec=1024
2个版本的参数对比
优化版多了一个参数:
opt_0302_33cd749/conf/iotdb-common.properties
*优化版
max_number_of_points_in_page=10000
优化前的版本此参数
max_number_of_points_in_page=1048576*
3.两个 版本日志路径
/data1/iotdb/bef_opt_854e4c8/logs
/data1/iotdb/opt_0302_33cd749/logs