[KYLIN-4916] ShardingReadRDD's partition number was set to shardNum，causing empty spark tasks - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Won't Fix
Affects Version/s: None
Fix Version/s: v4.0.0-beta
Component/s: None
Labels:
None

Description

when creating ShardingReadRDD, the created FileScanRDD's partition number was set to shard number, causing too many empty tasks when shard number is big.

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Shengjun Zheng

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 25/Feb/21 02:57

Updated:: 26/Feb/21 02:55

Resolved:: 25/Feb/21 11:03