SP1 - Details for Query 42

Submitted Time: 2026/08/26 14:00:34
Duration: 4 s
Succeeded Jobs: 35 36 37 38 39 40 41 42 43 44 45 46 47

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 3 ms (1 ms, 1 ms, 1 ms (stage 87.0: task 137)) number of written files: 3 job commit time: 16 ms number of output rows: 934 number of dynamic part: 0 written output: 14.2 KiB"]; 2 [labelType="html" label=" WriteFiles "]; 3 [labelType="html" label=" Union "]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (13)\n \nduration: 130 ms"; 5 [labelType="html" label=" Project "]; 6 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 868.0 B number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 14 local merged chunks fetched: 0 shuffle write time: 3 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 14 local bytes read: 856.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 560.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 856.0 B"]; subgraph cluster9 { isCluster="true"; label="WholeStageCodegen (8)\n \nduration: total (min, med, max (stageId: taskId))\n8 ms (4 ms, 4 ms, 4 ms (stage 60.0: task 130))"; 10 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 60.0: task 130)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 60.0: task 130)) number of output rows: 28 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 60.0: task 130))"]; } 11 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 685.0 B number of coalesced partitions: 1"]; 12 [labelType="html" label="Exchange shuffle records written: 14 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 28 local bytes read total (min, med, max (stageId: taskId)) 1322.0 B (661.0 B, 661.0 B, 661.0 B (stage 60.0: task 130)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 60.0: task 130)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 560.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 661.0 B"]; subgraph cluster13 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 49 ms"; 14 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 44 ms peak memory: 256.0 KiB number of output rows: 14 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 15 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster16 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: 55 ms"; 17 [labelType="html" label="Sort sort time: 1 ms peak memory: 18.0 MiB spill size: 0.0 B"]; } 18 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 145.8 KiB number of coalesced partitions: 1"]; 19 [labelType="html" label="Exchange shuffle records written: 84,725 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 84,725 local bytes read: 136.7 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 1985.7 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 136.7 KiB"]; subgraph cluster20 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n1.5 s (6 ms, 27 ms, 1.3 s (stage 51.0: task 102))"; 21 [labelType="html" label=" Project "]; 22 [labelType="html" label="Filter number of output rows: 84,725"]; 23 [labelType="html" label="ColumnarToRow number of output rows: 2,401,930 number of input batches: 587"]; } 24 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 946 ms (4 ms, 25 ms, 749 ms (stage 51.0: task 102)) metadata time: 0 ms size of files read: 91.4 MiB number of output rows: 2,401,930"]; subgraph cluster25 { isCluster="true"; label="WholeStageCodegen (14)\n \nduration: 135 ms"; 26 [labelType="html" label=" Project "]; 27 [labelType="html" label="Sort sort time: 4 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 28 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1902.0 B number of coalesced partitions: 1"]; 29 [labelType="html" label="Exchange shuffle records written: 93 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 93 local bytes read: 1795.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 3.6 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1795.0 B"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (12)\n \nduration: total (min, med, max (stageId: taskId))\n8 ms (3 ms, 5 ms, 5 ms (stage 77.0: task 136))"; 31 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 74.0: task 135)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 74.0: task 135)) number of output rows: 186 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 74.0: task 135))"]; } 32 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1945.0 B number of coalesced partitions: 1"]; 33 [labelType="html" label="Exchange shuffle records written: 93 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 186 local bytes read total (min, med, max (stageId: taskId)) 3.6 KiB (1840.0 B, 1840.0 B, 1840.0 B (stage 74.0: task 135)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 74.0: task 135)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 3.6 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1840.0 B"]; subgraph cluster34 { isCluster="true"; label="WholeStageCodegen (11)\n \nduration: 567 ms"; 35 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 551 ms peak memory: 256.0 KiB number of output rows: 93 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 36 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster37 { isCluster="true"; label="WholeStageCodegen (10)\n \nduration: 606 ms"; 38 [labelType="html" label="Sort sort time: 21 ms peak memory: 64.0 MiB spill size: 0.0 B"]; } 39 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2.3 MiB number of coalesced partitions: 1"]; 40 [labelType="html" label="Exchange shuffle records written: 1,181,464 local merged chunks fetched: 0 shuffle write time: 10 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,181,464 local bytes read: 2.2 MiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 27.0 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 2.2 MiB"]; subgraph cluster41 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n2.1 s (5 ms, 9 ms, 2.0 s (stage 52.0: task 112))"; 42 [labelType="html" label=" Project "]; 43 [labelType="html" label="Filter number of output rows: 1,181,464"]; 44 [labelType="html" label="ColumnarToRow number of output rows: 2,401,930 number of input batches: 587"]; } 45 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 762 ms (4 ms, 8 ms, 694 ms (stage 52.0: task 112)) metadata time: 0 ms size of files read: 91.4 MiB number of output rows: 2,401,930"]; subgraph cluster46 { isCluster="true"; label="WholeStageCodegen (15)\n \nduration: 154 ms"; 47 [labelType="html" label=" Project "]; 48 [labelType="html" label="Sort sort time: 3 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 49 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 9.8 KiB number of coalesced partitions: 1"]; 50 [labelType="html" label="Exchange shuffle records written: 827 local merged chunks fetched: 0 shuffle write time: 4 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 827 local bytes read: 9.2 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 38.8 KiB local merged bytes read: 0.0 B number of partitions: 9 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 9.2 KiB"]; subgraph cluster51 { isCluster="true"; label="WholeStageCodegen (9)\n \nduration: total (min, med, max (stageId: taskId))\n51 ms (5 ms, 46 ms, 46 ms (stage 69.0: task 133))"; 52 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 2 ms (1 ms, 1 ms, 1 ms (stage 69.0: task 133)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 69.0: task 133)) number of output rows: 1,654 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 69.0: task 133))"]; } 53 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 11.1 KiB number of coalesced partitions: 1"]; 54 [labelType="html" label="Exchange shuffle records written: 827 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,654 local bytes read total (min, med, max (stageId: taskId)) 21.4 KiB (10.7 KiB, 10.7 KiB, 10.7 KiB (stage 69.0: task 133)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 69.0: task 133)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 38.8 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 10.7 KiB"]; subgraph cluster55 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: 91 ms"; 56 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 83 ms peak memory: 256.0 KiB number of output rows: 827 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 57 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster58 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 96 ms"; 59 [labelType="html" label="Sort sort time: 2 ms peak memory: 20.0 MiB spill size: 0.0 B"]; } 60 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 670.0 KiB number of coalesced partitions: 1"]; 61 [labelType="html" label="Exchange shuffle records written: 158,502 local merged chunks fetched: 0 shuffle write time: 3 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 158,502 local bytes read: 649.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 4.8 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 649.6 KiB"]; subgraph cluster62 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n1.4 s (5 ms, 12 ms, 1.3 s (stage 53.0: task 122))"; 63 [labelType="html" label=" Project "]; 64 [labelType="html" label="Filter number of output rows: 158,502"]; 65 [labelType="html" label="ColumnarToRow number of output rows: 2,381,930 number of input batches: 582"]; } 66 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 750 ms (3 ms, 10 ms, 666 ms (stage 53.0: task 122)) metadata time: 0 ms size of files read: 91.4 MiB number of output rows: 2,381,930"]; 1->0; 2->1; 3->2; 5->3; 6->5; 7->6; 8->7; 10->8; 11->10; 12->11; 14->12; 15->14; 17->15; 18->17; 19->18; 21->19; 22->21; 23->22; 24->23; 26->3; 27->26; 28->27; 29->28; 31->29; 32->31; 33->32; 35->33; 36->35; 38->36; 39->38; 40->39; 42->40; 43->42; 44->43; 45->44; 47->3; 48->47; 49->48; 50->49; 52->50; 53->52; 54->53; 56->54; 57->56; 59->57; 60->59; 61->60; 63->61; 64->63; 65->64; 66->65; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand file:/data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23, false, Parquet, [path=file:///data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

WriteFiles

Union

Project [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17734, COL_2E7579DF_241F_4969_A954_4FA96050480C#11088 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17735, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17736, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17737, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17738]

Sort [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST], true, 0

WholeStageCodegen (13)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2257]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L], functions=[count(1)])

WholeStageCodegen (8)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, 10), ENSURE_REQUIREMENTS, [plan_id=2014]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L], functions=[partial_count(1)])

WholeStageCodegen (5)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1793]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301 <=> B)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301 <=> B)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17779, COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17780, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17781, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17782, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17783]

Sort [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST], true, 0

WholeStageCodegen (14)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2626]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L], functions=[count(1)])

WholeStageCodegen (12)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, 10), ENSURE_REQUIREMENTS, [plan_id=2557]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L], functions=[partial_count(1)])

WholeStageCodegen (11)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (10)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1837]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680 <=> C)

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680 <=> C)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17784, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17785, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17786, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17787, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17788]

Sort [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST], true, 0

WholeStageCodegen (15)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2361]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L], functions=[count(1)])

WholeStageCodegen (9)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, 10), ENSURE_REQUIREMENTS, [plan_id=2169]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L], functions=[partial_count(1)])

WholeStageCodegen (7)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (6)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1881]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726 <=> M)

ColumnarToRow

WholeStageCodegen (3)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726 <=> M)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (97)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (60)
   +- WriteFiles (59)
      +- Union (58)
         :- * Project (19)
         :  +- * Sort (18)
         :     +- AQEShuffleRead (17)
         :        +- ShuffleQueryStage (16), Statistics(sizeInBytes=560.0 B, rowCount=14)
         :           +- Exchange (15)
         :              +- * HashAggregate (14)
         :                 +- AQEShuffleRead (13)
         :                    +- ShuffleQueryStage (12), Statistics(sizeInBytes=560.0 B, rowCount=14)
         :                       +- Exchange (11)
         :                          +- * HashAggregate (10)
         :                             +- Window (9)
         :                                +- * Sort (8)
         :                                   +- AQEShuffleRead (7)
         :                                      +- ShuffleQueryStage (6), Statistics(sizeInBytes=1985.7 KiB, rowCount=8.47E+4)
         :                                         +- Exchange (5)
         :                                            +- * Project (4)
         :                                               +- * Filter (3)
         :                                                  +- * ColumnarToRow (2)
         :                                                     +- Scan parquet  (1)
         :- * Project (38)
         :  +- * Sort (37)
         :     +- AQEShuffleRead (36)
         :        +- ShuffleQueryStage (35), Statistics(sizeInBytes=3.6 KiB, rowCount=93)
         :           +- Exchange (34)
         :              +- * HashAggregate (33)
         :                 +- AQEShuffleRead (32)
         :                    +- ShuffleQueryStage (31), Statistics(sizeInBytes=3.6 KiB, rowCount=93)
         :                       +- Exchange (30)
         :                          +- * HashAggregate (29)
         :                             +- Window (28)
         :                                +- * Sort (27)
         :                                   +- AQEShuffleRead (26)
         :                                      +- ShuffleQueryStage (25), Statistics(sizeInBytes=27.0 MiB, rowCount=1.18E+6)
         :                                         +- Exchange (24)
         :                                            +- * Project (23)
         :                                               +- * Filter (22)
         :                                                  +- * ColumnarToRow (21)
         :                                                     +- Scan parquet  (20)
         +- * Project (57)
            +- * Sort (56)
               +- AQEShuffleRead (55)
                  +- ShuffleQueryStage (54), Statistics(sizeInBytes=38.8 KiB, rowCount=827)
                     +- Exchange (53)
                        +- * HashAggregate (52)
                           +- AQEShuffleRead (51)
                              +- ShuffleQueryStage (50), Statistics(sizeInBytes=38.8 KiB, rowCount=827)
                                 +- Exchange (49)
                                    +- * HashAggregate (48)
                                       +- Window (47)
                                          +- * Sort (46)
                                             +- AQEShuffleRead (45)
                                                +- ShuffleQueryStage (44), Statistics(sizeInBytes=4.8 MiB, rowCount=1.59E+5)
                                                   +- Exchange (43)
                                                      +- * Project (42)
                                                         +- * Filter (41)
                                                            +- * ColumnarToRow (40)
                                                               +- Scan parquet  (39)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (96)
   +- WriteFiles (95)
      +- Union (94)
         :- Project (71)
         :  +- Sort (70)
         :     +- Exchange (69)
         :        +- HashAggregate (68)
         :           +- Exchange (67)
         :              +- HashAggregate (66)
         :                 +- Window (65)
         :                    +- Sort (64)
         :                       +- Exchange (63)
         :                          +- Project (62)
         :                             +- Filter (61)
         :                                +- Scan parquet  (1)
         :- Project (82)
         :  +- Sort (81)
         :     +- Exchange (80)
         :        +- HashAggregate (79)
         :           +- Exchange (78)
         :              +- HashAggregate (77)
         :                 +- Window (76)
         :                    +- Sort (75)
         :                       +- Exchange (74)
         :                          +- Project (73)
         :                             +- Filter (72)
         :                                +- Scan parquet  (20)
         +- Project (93)
            +- Sort (92)
               +- Exchange (91)
                  +- HashAggregate (90)
                     +- Exchange (89)
                        +- HashAggregate (88)
                           +- Window (87)
                              +- Sort (86)
                                 +- Exchange (85)
                                    +- Project (84)
                                       +- Filter (83)
                                          +- Scan parquet  (39)


(1) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(2) ColumnarToRow [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]

(3) Filter [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301 <=> B)

(4) Project [codegen id : 1]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]

(5) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1793]

(6) ShuffleQueryStage
Output [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: 0

(7) AQEShuffleRead
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: coalesced

(8) Sort [codegen id : 4]
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(9) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L], [1], [1 DESC NULLS LAST]

(10) HashAggregate [codegen id : 5]
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17760L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]

(11) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, 10), ENSURE_REQUIREMENTS, [plan_id=2014]

(12) ShuffleQueryStage
Output [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Arguments: 3

(13) AQEShuffleRead
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Arguments: coalesced

(14) HashAggregate [codegen id : 8]
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#11134L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, count(1)#11134L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]

(15) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2257]

(16) ShuffleQueryStage
Output [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: 5

(17) AQEShuffleRead
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: coalesced

(18) Sort [codegen id : 13]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST], true, 0

(19) Project [codegen id : 13]
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17734, COL_2E7579DF_241F_4969_A954_4FA96050480C#11088 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17735, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17736, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17737, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17738]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]

(20) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(21) ColumnarToRow [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]

(22) Filter [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680 <=> C)

(23) Project [codegen id : 2]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]

(24) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1837]

(25) ShuffleQueryStage
Output [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: 1

(26) AQEShuffleRead
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: coalesced

(27) Sort [codegen id : 10]
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(28) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L], [1], [1 DESC NULLS LAST]

(29) HashAggregate [codegen id : 11]
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17762L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]

(30) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, 10), ENSURE_REQUIREMENTS, [plan_id=2557]

(31) ShuffleQueryStage
Output [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Arguments: 7

(32) AQEShuffleRead
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Arguments: coalesced

(33) HashAggregate [codegen id : 12]
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#17567L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, count(1)#17567L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]

(34) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2626]

(35) ShuffleQueryStage
Output [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: 8

(36) AQEShuffleRead
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: coalesced

(37) Sort [codegen id : 14]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST], true, 0

(38) Project [codegen id : 14]
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17779, COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17780, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17781, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17782, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17783]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]

(39) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_425007]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(40) ColumnarToRow [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]

(41) Filter [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726 <=> M)

(42) Project [codegen id : 3]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]

(43) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1881]

(44) ShuffleQueryStage
Output [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: 2

(45) AQEShuffleRead
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: coalesced

(46) Sort [codegen id : 6]
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(47) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L], [1], [1 DESC NULLS LAST]

(48) HashAggregate [codegen id : 7]
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17764L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]

(49) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, 10), ENSURE_REQUIREMENTS, [plan_id=2169]

(50) ShuffleQueryStage
Output [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Arguments: 4

(51) AQEShuffleRead
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Arguments: coalesced

(52) HashAggregate [codegen id : 9]
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#15766L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, count(1)#15766L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]

(53) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=2361]

(54) ShuffleQueryStage
Output [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: 6

(55) AQEShuffleRead
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: coalesced

(56) Sort [codegen id : 15]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST], true, 0

(57) Project [codegen id : 15]
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17784, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17785, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17786, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17787, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17788]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]

(58) Union

(59) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17734, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17735, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17736, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17737, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17738]

(60) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23, false, Parquet, [path=file:///data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(61) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301 <=> B)

(62) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#9299, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#9301]

(63) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1731]

(64) Sort
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(65) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L], [1], [1 DESC NULLS LAST]

(66) HashAggregate
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17760L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]

(67) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, 10), ENSURE_REQUIREMENTS, [plan_id=1736]

(68) HashAggregate
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L, count#17761L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#11134L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#10834 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#10950L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, count(1)#11134L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]

(69) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=1739]

(70) Sort
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L DESC NULLS LAST], true, 0

(71) Project
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17734, COL_2E7579DF_241F_4969_A954_4FA96050480C#11088 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17735, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17736, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17737, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17738]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#11088, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#11089L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#11136L]

(72) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680 <=> C)

(73) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17678, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17680]

(74) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1742]

(75) Sort
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(76) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L], [1], [1 DESC NULLS LAST]

(77) HashAggregate
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17762L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]

(78) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, 10), ENSURE_REQUIREMENTS, [plan_id=1747]

(79) HashAggregate
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L, count#17763L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#17567L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#17525 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#17536L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, count(1)#17567L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]

(80) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=1750]

(81) Sort
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L DESC NULLS LAST], true, 0

(82) Project
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17779, COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17780, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17781, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17782, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17783]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#17562, COL_D84E9517_7132_491A_A4B5_BF75639391C2#17563L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#17569L]

(83) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726 <=> M)

(84) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#17724, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#17726]

(85) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=1753]

(86) Sort
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(87) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L], [1], [1 DESC NULLS LAST]

(88) HashAggregate
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#17764L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]

(89) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, 10), ENSURE_REQUIREMENTS, [plan_id=1758]

(90) HashAggregate
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L, count#17765L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#15766L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#15510 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_20E75411_F375_4DC1_B499_5E84E85FE997#15627L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, count(1)#15766L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]

(91) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=1761]

(92) Sort
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L DESC NULLS LAST], true, 0

(93) Project
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17784, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17785, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17786, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17787, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17788]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#15761, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#15762L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#15768L]

(94) Union

(95) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#17734, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#17735, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#17736, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#17737, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#17738]

(96) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23, false, Parquet, [path=file:///data/output/cache/parquet/datablock/0bf8c8a9_37a0_4244_a983_6144e1cbbe23], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(97) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet