SP1 - Details for Query 128

Submitted Time: 2026/06/25 10:31:34
Duration: 1 s
Succeeded Jobs: 152 153 154 155 156 157 158 159 160 161 162 163 164

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 4 ms (0 ms, 2 ms, 2 ms (stage 259.0: task 233)) number of written files: 3 job commit time: 14 ms number of output rows: 569 number of dynamic part: 0 written output: 11.6 KiB"]; 2 [labelType="html" label=" WriteFiles "]; 3 [labelType="html" label=" Union "]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (13)\n \nduration: 59 ms"; 5 [labelType="html" label=" Project "]; 6 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 919.0 B number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 17 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 17 local bytes read: 890.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 680.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 890.0 B"]; subgraph cluster9 { isCluster="true"; label="WholeStageCodegen (8)\n \nduration: total (min, med, max (stageId: taskId))\n5 ms (2 ms, 3 ms, 3 ms (stage 235.0: task 226))"; 10 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 235.0: task 226)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 235.0: task 226)) number of output rows: 34 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 235.0: task 226))"]; } 11 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 784.0 B number of coalesced partitions: 1"]; 12 [labelType="html" label="Exchange shuffle records written: 17 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 34 local bytes read total (min, med, max (stageId: taskId)) 1508.0 B (754.0 B, 754.0 B, 754.0 B (stage 235.0: task 226)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 235.0: task 226)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 680.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 754.0 B"]; subgraph cluster13 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 17 ms"; 14 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 13 ms peak memory: 256.0 KiB number of output rows: 17 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 15 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster16 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: 20 ms"; 17 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.5 MiB spill size: 0.0 B"]; } 18 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 34.9 KiB number of coalesced partitions: 1"]; 19 [labelType="html" label="Exchange shuffle records written: 18,504 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 18,504 local bytes read: 32.4 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 433.7 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 32.4 KiB"]; subgraph cluster20 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n361 ms (4 ms, 6 ms, 325 ms (stage 223.0: task 205))"; 21 [labelType="html" label=" Project "]; 22 [labelType="html" label="Filter number of output rows: 18,504"]; 23 [labelType="html" label="ColumnarToRow number of output rows: 509,703 number of input batches: 125"]; } 24 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 209 ms (3 ms, 5 ms, 177 ms (stage 223.0: task 205)) metadata time: 0 ms size of files read: 25.4 MiB number of output rows: 509,703"]; subgraph cluster25 { isCluster="true"; label="WholeStageCodegen (14)\n \nduration: 55 ms"; 26 [labelType="html" label=" Project "]; 27 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 28 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2.0 KiB number of coalesced partitions: 1"]; 29 [labelType="html" label="Exchange shuffle records written: 105 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 105 local bytes read: 1968.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 4.1 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1968.0 B"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (12)\n \nduration: total (min, med, max (stageId: taskId))\n5 ms (2 ms, 3 ms, 3 ms (stage 249.0: task 231))"; 31 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 249.0: task 231)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 249.0: task 231)) number of output rows: 210 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 249.0: task 231))"]; } 32 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2.1 KiB number of coalesced partitions: 1"]; 33 [labelType="html" label="Exchange shuffle records written: 105 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 210 local bytes read total (min, med, max (stageId: taskId)) 3.9 KiB (1989.0 B, 1989.0 B, 1989.0 B (stage 249.0: task 231)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 249.0: task 231)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 4.1 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1989.0 B"]; subgraph cluster34 { isCluster="true"; label="WholeStageCodegen (10)\n \nduration: 135 ms"; 35 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 125 ms peak memory: 256.0 KiB number of output rows: 105 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 36 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster37 { isCluster="true"; label="WholeStageCodegen (9)\n \nduration: 179 ms"; 38 [labelType="html" label="Sort sort time: 4 ms peak memory: 24.0 MiB spill size: 0.0 B"]; } 39 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 737.0 KiB number of coalesced partitions: 1"]; 40 [labelType="html" label="Exchange shuffle records written: 248,813 local merged chunks fetched: 0 shuffle write time: 4 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 248,813 local bytes read: 708.5 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 5.7 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 708.5 KiB"]; subgraph cluster41 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n582 ms (4 ms, 5 ms, 548 ms (stage 224.0: task 212))"; 42 [labelType="html" label=" Project "]; 43 [labelType="html" label="Filter number of output rows: 248,813"]; 44 [labelType="html" label="ColumnarToRow number of output rows: 509,703 number of input batches: 125"]; } 45 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 245 ms (3 ms, 4 ms, 224 ms (stage 224.0: task 212)) metadata time: 0 ms size of files read: 25.4 MiB number of output rows: 509,703"]; subgraph cluster46 { isCluster="true"; label="WholeStageCodegen (15)\n \nduration: 52 ms"; 47 [labelType="html" label=" Project "]; 48 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 49 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 5.6 KiB number of coalesced partitions: 1"]; 50 [labelType="html" label="Exchange shuffle records written: 447 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 447 local bytes read: 5.4 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 21.0 KiB local merged bytes read: 0.0 B number of partitions: 9 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 5.4 KiB"]; subgraph cluster51 { isCluster="true"; label="WholeStageCodegen (11)\n \nduration: total (min, med, max (stageId: taskId))\n5 ms (2 ms, 3 ms, 3 ms (stage 243.0: task 229))"; 52 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 243.0: task 229)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 243.0: task 229)) number of output rows: 894 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 243.0: task 229))"]; } 53 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 6.8 KiB number of coalesced partitions: 1"]; 54 [labelType="html" label="Exchange shuffle records written: 447 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 894 local bytes read total (min, med, max (stageId: taskId)) 12.8 KiB (6.4 KiB, 6.4 KiB, 6.4 KiB (stage 243.0: task 229)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 243.0: task 229)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 21.0 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 6.4 KiB"]; subgraph cluster55 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: 53 ms"; 56 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 48 ms peak memory: 256.0 KiB number of output rows: 447 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 57 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster58 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 46 ms"; 59 [labelType="html" label="Sort sort time: 1 ms peak memory: 18.0 MiB spill size: 0.0 B"]; } 60 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 284.1 KiB number of coalesced partitions: 1"]; 61 [labelType="html" label="Exchange shuffle records written: 64,798 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 64,798 local bytes read: 267.2 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 2024.9 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 267.2 KiB"]; subgraph cluster62 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n403 ms (5 ms, 7 ms, 361 ms (stage 225.0: task 219))"; 63 [labelType="html" label=" Project "]; 64 [labelType="html" label="Filter number of output rows: 64,798"]; 65 [labelType="html" label="ColumnarToRow number of output rows: 509,703 number of input batches: 125"]; } 66 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 204 ms (4 ms, 5 ms, 170 ms (stage 225.0: task 219)) metadata time: 0 ms size of files read: 25.4 MiB number of output rows: 509,703"]; 1->0; 2->1; 3->2; 5->3; 6->5; 7->6; 8->7; 10->8; 11->10; 12->11; 14->12; 15->14; 17->15; 18->17; 19->18; 21->19; 22->21; 23->22; 24->23; 26->3; 27->26; 28->27; 29->28; 31->29; 32->31; 33->32; 35->33; 36->35; 38->36; 39->38; 40->39; 42->40; 43->42; 44->43; 45->44; 47->3; 48->47; 49->48; 50->49; 52->50; 53->52; 54->53; 56->54; 57->56; 59->57; 60->59; 61->60; 63->61; 64->63; 65->64; 66->65; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand file:/data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3, false, Parquet, [path=file:///data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

WriteFiles

Union

Project [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32083, COL_2E7579DF_241F_4969_A954_4FA96050480C#31918 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32084, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32085, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32086, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32087]

Sort [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST], true, 0

WholeStageCodegen (13)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7213]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L], functions=[count(1)])

WholeStageCodegen (8)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, 10), ENSURE_REQUIREMENTS, [plan_id=6970]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L], functions=[partial_count(1)])

WholeStageCodegen (5)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6749]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650 <=> B)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650 <=> B)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32128, COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32129, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32130, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32131, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32132]

Sort [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST], true, 0

WholeStageCodegen (14)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7485]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L], functions=[count(1)])

WholeStageCodegen (12)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, 10), ENSURE_REQUIREMENTS, [plan_id=7321]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L], functions=[partial_count(1)])

WholeStageCodegen (10)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (9)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6793]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045 <=> C)

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045 <=> C)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32133, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32134, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32135, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32136, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32137]

Sort [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST], true, 0

WholeStageCodegen (15)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7347]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L], functions=[count(1)])

WholeStageCodegen (11)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, 10), ENSURE_REQUIREMENTS, [plan_id=7125]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L], functions=[partial_count(1)])

WholeStageCodegen (7)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (6)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6837]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075 <=> M)

ColumnarToRow

WholeStageCodegen (3)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075 <=> M)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (97)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (60)
   +- WriteFiles (59)
      +- Union (58)
         :- * Project (19)
         :  +- * Sort (18)
         :     +- AQEShuffleRead (17)
         :        +- ShuffleQueryStage (16), Statistics(sizeInBytes=680.0 B, rowCount=17)
         :           +- Exchange (15)
         :              +- * HashAggregate (14)
         :                 +- AQEShuffleRead (13)
         :                    +- ShuffleQueryStage (12), Statistics(sizeInBytes=680.0 B, rowCount=17)
         :                       +- Exchange (11)
         :                          +- * HashAggregate (10)
         :                             +- Window (9)
         :                                +- * Sort (8)
         :                                   +- AQEShuffleRead (7)
         :                                      +- ShuffleQueryStage (6), Statistics(sizeInBytes=433.7 KiB, rowCount=1.85E+4)
         :                                         +- Exchange (5)
         :                                            +- * Project (4)
         :                                               +- * Filter (3)
         :                                                  +- * ColumnarToRow (2)
         :                                                     +- Scan parquet  (1)
         :- * Project (38)
         :  +- * Sort (37)
         :     +- AQEShuffleRead (36)
         :        +- ShuffleQueryStage (35), Statistics(sizeInBytes=4.1 KiB, rowCount=105)
         :           +- Exchange (34)
         :              +- * HashAggregate (33)
         :                 +- AQEShuffleRead (32)
         :                    +- ShuffleQueryStage (31), Statistics(sizeInBytes=4.1 KiB, rowCount=105)
         :                       +- Exchange (30)
         :                          +- * HashAggregate (29)
         :                             +- Window (28)
         :                                +- * Sort (27)
         :                                   +- AQEShuffleRead (26)
         :                                      +- ShuffleQueryStage (25), Statistics(sizeInBytes=5.7 MiB, rowCount=2.49E+5)
         :                                         +- Exchange (24)
         :                                            +- * Project (23)
         :                                               +- * Filter (22)
         :                                                  +- * ColumnarToRow (21)
         :                                                     +- Scan parquet  (20)
         +- * Project (57)
            +- * Sort (56)
               +- AQEShuffleRead (55)
                  +- ShuffleQueryStage (54), Statistics(sizeInBytes=21.0 KiB, rowCount=447)
                     +- Exchange (53)
                        +- * HashAggregate (52)
                           +- AQEShuffleRead (51)
                              +- ShuffleQueryStage (50), Statistics(sizeInBytes=21.0 KiB, rowCount=447)
                                 +- Exchange (49)
                                    +- * HashAggregate (48)
                                       +- Window (47)
                                          +- * Sort (46)
                                             +- AQEShuffleRead (45)
                                                +- ShuffleQueryStage (44), Statistics(sizeInBytes=2024.9 KiB, rowCount=6.48E+4)
                                                   +- Exchange (43)
                                                      +- * Project (42)
                                                         +- * Filter (41)
                                                            +- * ColumnarToRow (40)
                                                               +- Scan parquet  (39)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (96)
   +- WriteFiles (95)
      +- Union (94)
         :- Project (71)
         :  +- Sort (70)
         :     +- Exchange (69)
         :        +- HashAggregate (68)
         :           +- Exchange (67)
         :              +- HashAggregate (66)
         :                 +- Window (65)
         :                    +- Sort (64)
         :                       +- Exchange (63)
         :                          +- Project (62)
         :                             +- Filter (61)
         :                                +- Scan parquet  (1)
         :- Project (82)
         :  +- Sort (81)
         :     +- Exchange (80)
         :        +- HashAggregate (79)
         :           +- Exchange (78)
         :              +- HashAggregate (77)
         :                 +- Window (76)
         :                    +- Sort (75)
         :                       +- Exchange (74)
         :                          +- Project (73)
         :                             +- Filter (72)
         :                                +- Scan parquet  (20)
         +- Project (93)
            +- Sort (92)
               +- Exchange (91)
                  +- HashAggregate (90)
                     +- Exchange (89)
                        +- HashAggregate (88)
                           +- Window (87)
                              +- Sort (86)
                                 +- Exchange (85)
                                    +- Project (84)
                                       +- Filter (83)
                                          +- Scan parquet  (39)


(1) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(2) ColumnarToRow [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]

(3) Filter [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650 <=> B)

(4) Project [codegen id : 1]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]

(5) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6749]

(6) ShuffleQueryStage
Output [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: 0

(7) AQEShuffleRead
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: coalesced

(8) Sort [codegen id : 4]
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(9) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L], [1], [1 DESC NULLS LAST]

(10) HashAggregate [codegen id : 5]
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32109L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]

(11) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, 10), ENSURE_REQUIREMENTS, [plan_id=6970]

(12) ShuffleQueryStage
Output [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Arguments: 3

(13) AQEShuffleRead
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Arguments: coalesced

(14) HashAggregate [codegen id : 8]
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#31923L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, count(1)#31923L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]

(15) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7213]

(16) ShuffleQueryStage
Output [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: 5

(17) AQEShuffleRead
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: coalesced

(18) Sort [codegen id : 13]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST], true, 0

(19) Project [codegen id : 13]
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32083, COL_2E7579DF_241F_4969_A954_4FA96050480C#31918 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32084, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32085, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32086, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32087]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]

(20) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(21) ColumnarToRow [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]

(22) Filter [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045 <=> C)

(23) Project [codegen id : 2]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]

(24) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6793]

(25) ShuffleQueryStage
Output [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: 1

(26) AQEShuffleRead
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: coalesced

(27) Sort [codegen id : 9]
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(28) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L], [1], [1 DESC NULLS LAST]

(29) HashAggregate [codegen id : 10]
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32111L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]

(30) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, 10), ENSURE_REQUIREMENTS, [plan_id=7321]

(31) ShuffleQueryStage
Output [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Arguments: 6

(32) AQEShuffleRead
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Arguments: coalesced

(33) HashAggregate [codegen id : 12]
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#30889L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, count(1)#30889L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]

(34) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7485]

(35) ShuffleQueryStage
Output [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: 8

(36) AQEShuffleRead
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: coalesced

(37) Sort [codegen id : 14]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST], true, 0

(38) Project [codegen id : 14]
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32128, COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32129, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32130, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32131, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32132]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]

(39) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411360]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(40) ColumnarToRow [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]

(41) Filter [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075 <=> M)

(42) Project [codegen id : 3]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]

(43) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6837]

(44) ShuffleQueryStage
Output [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: 2

(45) AQEShuffleRead
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: coalesced

(46) Sort [codegen id : 6]
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(47) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L], [1], [1 DESC NULLS LAST]

(48) HashAggregate [codegen id : 7]
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32113L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]

(49) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, 10), ENSURE_REQUIREMENTS, [plan_id=7125]

(50) ShuffleQueryStage
Output [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Arguments: 4

(51) AQEShuffleRead
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Arguments: coalesced

(52) HashAggregate [codegen id : 11]
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#27913L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, count(1)#27913L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]

(53) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=7347]

(54) ShuffleQueryStage
Output [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: 7

(55) AQEShuffleRead
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: coalesced

(56) Sort [codegen id : 15]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST], true, 0

(57) Project [codegen id : 15]
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32133, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32134, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32135, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32136, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32137]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]

(58) Union

(59) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32083, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32084, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32085, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32086, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32087]

(60) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3, false, Parquet, [path=file:///data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(61) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650 <=> B)

(62) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#23648, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#23650]

(63) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6687]

(64) Sort
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(65) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L], [1], [1 DESC NULLS LAST]

(66) HashAggregate
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32109L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]

(67) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, 10), ENSURE_REQUIREMENTS, [plan_id=6692]

(68) HashAggregate
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L, count#32110L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#31923L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#31889 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#31898L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, count(1)#31923L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]

(69) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=6695]

(70) Sort
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L DESC NULLS LAST], true, 0

(71) Project
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32083, COL_2E7579DF_241F_4969_A954_4FA96050480C#31918 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32084, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32085, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32086, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32087]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#31918, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#31919L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#31925L]

(72) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045 <=> C)

(73) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32043, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32045]

(74) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6698]

(75) Sort
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(76) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L], [1], [1 DESC NULLS LAST]

(77) HashAggregate
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32111L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]

(78) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, 10), ENSURE_REQUIREMENTS, [plan_id=6703]

(79) HashAggregate
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L, count#32112L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#30889L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#30790 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#30799L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, count(1)#30889L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]

(80) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=6706]

(81) Sort
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L DESC NULLS LAST], true, 0

(82) Project
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32128, COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32129, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32130, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32131, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32132]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#30884, COL_D84E9517_7132_491A_A4B5_BF75639391C2#30885L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#30891L]

(83) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075 <=> M)

(84) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#32073, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#32075]

(85) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=6709]

(86) Sort
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(87) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L], [1], [1 DESC NULLS LAST]

(88) HashAggregate
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#32113L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]

(89) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, 10), ENSURE_REQUIREMENTS, [plan_id=6714]

(90) HashAggregate
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L, count#32114L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#27913L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#27369 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_20E75411_F375_4DC1_B499_5E84E85FE997#27509L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, count(1)#27913L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]

(91) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=6717]

(92) Sort
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L DESC NULLS LAST], true, 0

(93) Project
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32133, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32134, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32135, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32136, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32137]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#27907, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#27908L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#27915L]

(94) Union

(95) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#32083, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#32084, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#32085, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#32086, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#32087]

(96) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3, false, Parquet, [path=file:///data/output/cache/parquet/datablock/3645e300_05c7_487b_ab32_6f748c9e16c3], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(97) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet