SP1 - Details for Query 3513

Submitted Time: 2026/08/26 10:30:17
Duration: 0.7 s
Succeeded Jobs: 3945 3946 3947 3948 3949 3950

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 2 ms"; 2 [labelType="html" label=" Project "]; 3 [labelType="html" label="Filter number of output rows: 161"]; } 4 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster5 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 53 ms"; 6 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 21.7 KiB number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 583 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 583 local bytes read: 20.9 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 55.9 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 20.9 KiB"]; 9 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster10 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: 82 ms"; 11 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 12 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 20.0 KiB number of coalesced partitions: 1"]; 13 [labelType="html" label="Exchange shuffle records written: 583 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 583 local bytes read: 19.3 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 51.4 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 19.3 KiB"]; subgraph cluster14 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 47 ms"; 15 [labelType="html" label=" Project "]; 16 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 17 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 9.9 KiB number of coalesced partitions: 1"]; 18 [labelType="html" label="Exchange shuffle records written: 583 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 583 local bytes read: 9.3 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 27.3 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 9.3 KiB"]; subgraph cluster19 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n18 ms (6 ms, 12 ms, 12 ms (stage 5885.0: task 5452))"; 20 [labelType="html" label=" Project "]; 21 [labelType="html" label="Filter number of output rows: 1,166"]; 22 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 2 ms (1 ms, 1 ms, 1 ms (stage 5887.0: task 5453)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 5887.0: task 5453)) number of output rows: 1,282 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 5887.0: task 5453))"]; } 23 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 15.2 KiB number of coalesced partitions: 1"]; 24 [labelType="html" label="Exchange shuffle records written: 641 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 10 ms (1 ms, 2 ms, 4 ms (stage 5883.0: task 5450)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,282 local bytes read total (min, med, max (stageId: taskId)) 28.8 KiB (14.4 KiB, 14.4 KiB, 14.4 KiB (stage 5887.0: task 5453)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 5887.0: task 5453)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 8 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 30.0 KiB (4.1 KiB, 9.7 KiB, 10.5 KiB (stage 5883.0: task 5449)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 14.4 KiB (2.5 KiB, 4.3 KiB, 4.5 KiB (stage 5883.0: task 5448))"]; subgraph cluster25 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n147 ms (24 ms, 44 ms, 51 ms (stage 5883.0: task 5449))"; 26 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 119 ms (17 ms, 36 ms, 45 ms (stage 5883.0: task 5449)) peak memory total (min, med, max (stageId: taskId)) 1024.0 KiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 5883.0: task 5451)) number of output rows: 641 number of sort fallback tasks: 0 avg hash probes per key: 0"]; 27 [labelType="html" label=" Project "]; 28 [labelType="html" label="Filter number of output rows: 5,316"]; 29 [labelType="html" label="ColumnarToRow number of output rows: 7,020 number of input batches: 4"]; } 30 [labelType="html" label="Scan parquet number of files read: 4 scan time total (min, med, max (stageId: taskId)) 37 ms (5 ms, 13 ms, 13 ms (stage 5883.0: task 5450)) metadata time: 0 ms size of files read: 1959.2 KiB number of output rows: 7,020"]; 2->0; 3->2; 4->3; 6->4; 7->6; 8->7; 9->8; 11->9; 12->11; 13->12; 15->13; 16->15; 17->16; 18->17; 20->18; 21->20; 22->21; 23->22; 24->23; 26->24; 27->26; 28->27; 29->28; 30->29; }

AdaptiveSparkPlan isFinalPlan=true

Project [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L, str_extract_by_str(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, -, RIGHT) AS COL_57480ACF_8179_499F_96AC_3401AC57500F#870826, str_extract_by_str(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, -, RIGHT) AS COL_322B6FFF_D8D0_4D93_9B52_097B7A1C927B#871092]

Filter (NOT (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <=> 1) AND ((COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L >= 0) AND (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <= 20)))

WholeStageCodegen (6)

Window [count(1) windowspecdefinition(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_4970D721_9719_49FC_9562_5E633E81A831#870788L], [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752], [1 DESC NULLS LAST]

Sort [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (5)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 10), ENSURE_REQUIREMENTS, [plan_id=115938]

Window [count(1) windowspecdefinition(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L], [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748], [1 DESC NULLS LAST]

Sort [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 10), ENSURE_REQUIREMENTS, [plan_id=115885]

Project [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, > , LEFT), ALL) AS COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, > , RIGHT), ALL) AS COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]

Sort [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST], true, 0

WholeStageCodegen (3)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=115814]

Project [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560 AS COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L AS COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]

Filter NOT (COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L <=> 1)

HashAggregate(keys=[COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396], functions=[count(1)])

WholeStageCodegen (2)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, 10), ENSURE_REQUIREMENTS, [plan_id=115718]

HashAggregate(keys=[COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396], functions=[partial_count(1)])

Project [str_concat(array(str_concat(array(1-, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621, )), > , str_concat(array(2-, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, )))) AS COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]

Filter (COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545 <=> 1)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545,COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620,COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621] Batched: true, DataFilters: [(COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545 <=> 1)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/RHA_2021_56618], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB,1)], ReadSchema: struct<COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB:string,COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (48)
+- == Final Plan ==
   * Project (28)
   +- * Filter (27)
      +- Window (26)
         +- * Sort (25)
            +- AQEShuffleRead (24)
               +- ShuffleQueryStage (23), Statistics(sizeInBytes=55.9 KiB, rowCount=583)
                  +- Exchange (22)
                     +- Window (21)
                        +- * Sort (20)
                           +- AQEShuffleRead (19)
                              +- ShuffleQueryStage (18), Statistics(sizeInBytes=51.4 KiB, rowCount=583)
                                 +- Exchange (17)
                                    +- * Project (16)
                                       +- * Sort (15)
                                          +- AQEShuffleRead (14)
                                             +- ShuffleQueryStage (13), Statistics(sizeInBytes=27.3 KiB, rowCount=583)
                                                +- Exchange (12)
                                                   +- * Project (11)
                                                      +- * Filter (10)
                                                         +- * HashAggregate (9)
                                                            +- AQEShuffleRead (8)
                                                               +- ShuffleQueryStage (7), Statistics(sizeInBytes=30.0 KiB, rowCount=641)
                                                                  +- Exchange (6)
                                                                     +- * HashAggregate (5)
                                                                        +- * Project (4)
                                                                           +- * Filter (3)
                                                                              +- * ColumnarToRow (2)
                                                                                 +- Scan parquet  (1)
+- == Initial Plan ==
   CollectLimit (47)
   +- Project (46)
      +- Filter (45)
         +- Window (44)
            +- Sort (43)
               +- Exchange (42)
                  +- Window (41)
                     +- Sort (40)
                        +- Exchange (39)
                           +- Project (38)
                              +- Sort (37)
                                 +- Exchange (36)
                                    +- Project (35)
                                       +- Filter (34)
                                          +- HashAggregate (33)
                                             +- Exchange (32)
                                                +- HashAggregate (31)
                                                   +- Project (30)
                                                      +- Filter (29)
                                                         +- Scan parquet  (1)


(1) Scan parquet 
Output [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/RHA_2021_56618]
PushedFilters: [EqualNullSafe(COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB,1)]
ReadSchema: struct<COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB:string,COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0:string,COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB:string>

(2) ColumnarToRow [codegen id : 1]
Input [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]

(3) Filter [codegen id : 1]
Input [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]
Condition : (COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545 <=> 1)

(4) Project [codegen id : 1]
Output [1]: [str_concat(array(str_concat(array(1-, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621, )),  > , str_concat(array(2-, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, )))) AS COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Input [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]

(5) HashAggregate [codegen id : 1]
Input [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Keys [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#871117L]
Results [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]

(6) Exchange
Input [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Arguments: hashpartitioning(COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, 10), ENSURE_REQUIREMENTS, [plan_id=115718]

(7) ShuffleQueryStage
Output [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Arguments: 0

(8) AQEShuffleRead
Input [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Arguments: coalesced

(9) HashAggregate [codegen id : 2]
Input [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Keys [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#870694L]
Results [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396 AS COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, count(1)#870694L AS COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]

(10) Filter [codegen id : 2]
Input [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]
Condition : NOT (COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L <=> 1)

(11) Project [codegen id : 2]
Output [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560 AS COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L AS COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Input [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]

(12) Exchange
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: rangepartitioning(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=115814]

(13) ShuffleQueryStage
Output [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: 1

(14) AQEShuffleRead
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: coalesced

(15) Sort [codegen id : 3]
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST], true, 0

(16) Project [codegen id : 3]
Output [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730,  > , LEFT), ALL) AS COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730,  > , RIGHT), ALL) AS COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]

(17) Exchange
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: hashpartitioning(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 10), ENSURE_REQUIREMENTS, [plan_id=115885]

(18) ShuffleQueryStage
Output [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: 2

(19) AQEShuffleRead
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: coalesced

(20) Sort [codegen id : 4]
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(21) Window
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: [count(1) windowspecdefinition(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L], [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748], [1 DESC NULLS LAST]

(22) Exchange
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: hashpartitioning(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 10), ENSURE_REQUIREMENTS, [plan_id=115938]

(23) ShuffleQueryStage
Output [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: 3

(24) AQEShuffleRead
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: coalesced

(25) Sort [codegen id : 5]
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(26) Window
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: [count(1) windowspecdefinition(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_4970D721_9719_49FC_9562_5E633E81A831#870788L], [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752], [1 DESC NULLS LAST]

(27) Filter [codegen id : 6]
Input [6]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L]
Condition : (NOT (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <=> 1) AND ((COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L >= 0) AND (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <= 20)))

(28) Project [codegen id : 6]
Output [8]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L, str_extract_by_str(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, -, RIGHT) AS COL_57480ACF_8179_499F_96AC_3401AC57500F#870826, str_extract_by_str(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, -, RIGHT) AS COL_322B6FFF_D8D0_4D93_9B52_097B7A1C927B#871092]
Input [6]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L]

(29) Filter
Input [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]
Condition : (COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545 <=> 1)

(30) Project
Output [1]: [str_concat(array(str_concat(array(1-, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621, )),  > , str_concat(array(2-, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, )))) AS COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Input [3]: [COL_CCB5E7AD_BAFE_4446_A939_2E32C86714FB#866545, COL_4CBA6864_47F6_40CD_A569_4F56B8796DB0#866620, COL_ACE9795F_5E2E_41FD_B27F_0D80DB7F16AB#866621]

(31) HashAggregate
Input [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Keys [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#871117L]
Results [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]

(32) Exchange
Input [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Arguments: hashpartitioning(COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, 10), ENSURE_REQUIREMENTS, [plan_id=115659]

(33) HashAggregate
Input [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396, count#871118L]
Keys [1]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#870694L]
Results [2]: [COL_EA9E7AA6_6C11_4882_BB4C_FD589CEF8217#870396 AS COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, count(1)#870694L AS COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]

(34) Filter
Input [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]
Condition : NOT (COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L <=> 1)

(35) Project
Output [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560 AS COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L AS COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Input [2]: [COL_BDF9B5F5_65A9_4A3E_815D_127BCB42EC3C#870560, COL_12D827DC_6AC7_4440_B6F1_0FC786E02191#870696L]

(36) Exchange
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: rangepartitioning(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=115664]

(37) Sort
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]
Arguments: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730 ASC NULLS FIRST, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L ASC NULLS FIRST], true, 0

(38) Project
Output [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730,  > , LEFT), ALL) AS COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, str_remove_space(str_extract_by_str(COL_B89E4889_C640_4CF0_9034_F637866563CE#870730,  > , RIGHT), ALL) AS COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Input [2]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L]

(39) Exchange
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: hashpartitioning(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 10), ENSURE_REQUIREMENTS, [plan_id=115668]

(40) Sort
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(41) Window
Input [4]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752]
Arguments: [count(1) windowspecdefinition(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L], [COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748], [1 DESC NULLS LAST]

(42) Exchange
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: hashpartitioning(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 10), ENSURE_REQUIREMENTS, [plan_id=115672]

(43) Sort
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(44) Window
Input [5]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L]
Arguments: [count(1) windowspecdefinition(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_4970D721_9719_49FC_9562_5E633E81A831#870788L], [COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752], [1 DESC NULLS LAST]

(45) Filter
Input [6]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L]
Condition : (NOT (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <=> 1) AND ((COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L >= 0) AND (COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L <= 20)))

(46) Project
Output [8]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L, str_extract_by_str(COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, -, RIGHT) AS COL_57480ACF_8179_499F_96AC_3401AC57500F#870826, str_extract_by_str(COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, -, RIGHT) AS COL_322B6FFF_D8D0_4D93_9B52_097B7A1C927B#871092]
Input [6]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L]

(47) CollectLimit
Input [8]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L, COL_57480ACF_8179_499F_96AC_3401AC57500F#870826, COL_322B6FFF_D8D0_4D93_9B52_097B7A1C927B#871092]
Arguments: 500000

(48) AdaptiveSparkPlan
Output [8]: [COL_B89E4889_C640_4CF0_9034_F637866563CE#870730, COL_49EC2519_6A4A_4037_85C8_F96C097349FD#870731L, COL_A35E8058_686F_4985_98EC_9A61100D01E0#870748, COL_23A3E6E9_4B8E_4C7A_A36B_951A124936A9#870752, COL_9B9B87EE_0C7F_42EA_9F6D_A9199224BCFF#870766L, COL_4970D721_9719_49FC_9562_5E633E81A831#870788L, COL_57480ACF_8179_499F_96AC_3401AC57500F#870826, COL_322B6FFF_D8D0_4D93_9B52_097B7A1C927B#871092]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet