SP1 - Details for Query 162

Submitted Time: 2026/06/25 12:33:47
Duration: 3 s
Succeeded Jobs: 207 208 209 210 211 212 213 214 215 216 217 218 219

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 3 ms (1 ms, 1 ms, 1 ms (stage 362.0: task 386)) number of written files: 3 job commit time: 13 ms number of output rows: 1,069 number of dynamic part: 0 written output: 15.3 KiB"]; 2 [labelType="html" label=" WriteFiles "]; 3 [labelType="html" label=" Union "]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (13)\n \nduration: 25 ms"; 5 [labelType="html" label=" Project "]; 6 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 7 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 902.0 B number of coalesced partitions: 1"]; 8 [labelType="html" label="Exchange shuffle records written: 16 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 16 local bytes read: 887.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 640.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 887.0 B"]; subgraph cluster9 { isCluster="true"; label="WholeStageCodegen (8)\n \nduration: total (min, med, max (stageId: taskId))\n5 ms (2 ms, 3 ms, 3 ms (stage 338.0: task 379))"; 10 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 335.0: task 378)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 335.0: task 378)) number of output rows: 32 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 335.0: task 378))"]; } 11 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 782.0 B number of coalesced partitions: 1"]; 12 [labelType="html" label="Exchange shuffle records written: 16 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 32 local bytes read total (min, med, max (stageId: taskId)) 1512.0 B (756.0 B, 756.0 B, 756.0 B (stage 335.0: task 378)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 335.0: task 378)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 640.0 B local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 756.0 B"]; subgraph cluster13 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 57 ms"; 14 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 53 ms peak memory: 256.0 KiB number of output rows: 16 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 15 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster16 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: 61 ms"; 17 [labelType="html" label="Sort sort time: 2 ms peak memory: 20.0 MiB spill size: 0.0 B"]; } 18 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 284.1 KiB number of coalesced partitions: 1"]; 19 [labelType="html" label="Exchange shuffle records written: 129,711 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 129,711 local bytes read: 273.2 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 3.0 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 273.2 KiB"]; subgraph cluster20 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n1.3 s (4 ms, 6 ms, 1.3 s (stage 326.0: task 350))"; 21 [labelType="html" label=" Project "]; 22 [labelType="html" label="Filter number of output rows: 129,711"]; 23 [labelType="html" label="ColumnarToRow number of output rows: 1,904,713 number of input batches: 466"]; } 24 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 694 ms (3 ms, 5 ms, 647 ms (stage 326.0: task 350)) metadata time: 0 ms size of files read: 84.0 MiB number of output rows: 1,904,713"]; subgraph cluster25 { isCluster="true"; label="WholeStageCodegen (14)\n \nduration: 57 ms"; 26 [labelType="html" label=" Project "]; 27 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 28 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1962.0 B number of coalesced partitions: 1"]; 29 [labelType="html" label="Exchange shuffle records written: 99 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 99 local bytes read: 1898.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 3.9 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1898.0 B"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (12)\n \nduration: total (min, med, max (stageId: taskId))\n6 ms (2 ms, 4 ms, 4 ms (stage 352.0: task 384))"; 31 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 352.0: task 384)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 352.0: task 384)) number of output rows: 198 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 352.0: task 384))"]; } 32 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2010.0 B number of coalesced partitions: 1"]; 33 [labelType="html" label="Exchange shuffle records written: 99 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 198 local bytes read total (min, med, max (stageId: taskId)) 3.7 KiB (1895.0 B, 1895.0 B, 1895.0 B (stage 352.0: task 384)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 352.0: task 384)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 3.9 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 1895.0 B"]; subgraph cluster34 { isCluster="true"; label="WholeStageCodegen (11)\n \nduration: 349 ms"; 35 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 344 ms peak memory: 256.0 KiB number of output rows: 99 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 36 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster37 { isCluster="true"; label="WholeStageCodegen (10)\n \nduration: 368 ms"; 38 [labelType="html" label="Sort sort time: 15 ms peak memory: 64.0 MiB spill size: 0.0 B"]; } 39 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2.1 MiB number of coalesced partitions: 1"]; 40 [labelType="html" label="Exchange shuffle records written: 870,333 local merged chunks fetched: 0 shuffle write time: 12 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 870,333 local bytes read: 2.0 MiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 19.9 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 2.0 MiB"]; subgraph cluster41 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n1.8 s (5 ms, 8 ms, 1.8 s (stage 327.0: task 360))"; 42 [labelType="html" label=" Project "]; 43 [labelType="html" label="Filter number of output rows: 870,333"]; 44 [labelType="html" label="ColumnarToRow number of output rows: 1,904,713 number of input batches: 466"]; } 45 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 654 ms (4 ms, 7 ms, 604 ms (stage 327.0: task 360)) metadata time: 0 ms size of files read: 84.0 MiB number of output rows: 1,904,713"]; subgraph cluster46 { isCluster="true"; label="WholeStageCodegen (15)\n \nduration: 38 ms"; 47 [labelType="html" label=" Project "]; 48 [labelType="html" label="Sort sort time: 0 ms peak memory: 16.1 MiB spill size: 0.0 B"]; } 49 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 11.2 KiB number of coalesced partitions: 1"]; 50 [labelType="html" label="Exchange shuffle records written: 954 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 954 local bytes read: 10.7 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 44.7 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 10.7 KiB"]; subgraph cluster51 { isCluster="true"; label="WholeStageCodegen (9)\n \nduration: total (min, med, max (stageId: taskId))\n7 ms (3 ms, 4 ms, 4 ms (stage 344.0: task 381))"; 52 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 2 ms (1 ms, 1 ms, 1 ms (stage 344.0: task 381)) peak memory total (min, med, max (stageId: taskId)) 32.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 344.0: task 381)) number of output rows: 1,908 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 344.0: task 381))"]; } 53 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 12.7 KiB number of coalesced partitions: 1"]; 54 [labelType="html" label="Exchange shuffle records written: 954 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,908 local bytes read total (min, med, max (stageId: taskId)) 24.2 KiB (12.1 KiB, 12.1 KiB, 12.1 KiB (stage 344.0: task 381)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 344.0: task 381)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size: 44.7 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 12.1 KiB"]; subgraph cluster55 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: 92 ms"; 56 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 87 ms peak memory: 256.0 KiB number of output rows: 954 number of sort fallback tasks: 0 avg hash probes per key: 0"]; } 57 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster58 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 95 ms"; 59 [labelType="html" label="Sort sort time: 3 ms peak memory: 24.0 MiB spill size: 0.0 B"]; } 60 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1079.0 KiB number of coalesced partitions: 1"]; 61 [labelType="html" label="Exchange shuffle records written: 205,403 local merged chunks fetched: 0 shuffle write time: 31 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 205,403 local bytes read: 990.0 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 6.3 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 990.0 KiB"]; subgraph cluster62 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n1.2 s (3 ms, 5 ms, 1.2 s (stage 328.0: task 370))"; 63 [labelType="html" label=" Project "]; 64 [labelType="html" label="Filter number of output rows: 205,403"]; 65 [labelType="html" label="ColumnarToRow number of output rows: 1,904,713 number of input batches: 466"]; } 66 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 662 ms (2 ms, 4 ms, 624 ms (stage 328.0: task 370)) metadata time: 0 ms size of files read: 84.0 MiB number of output rows: 1,904,713"]; 1->0; 2->1; 3->2; 5->3; 6->5; 7->6; 8->7; 10->8; 11->10; 12->11; 14->12; 15->14; 17->15; 18->17; 19->18; 21->19; 22->21; 23->22; 24->23; 26->3; 27->26; 28->27; 29->28; 31->29; 32->31; 33->32; 35->33; 36->35; 38->36; 39->38; 40->39; 42->40; 43->42; 44->43; 45->44; 47->3; 48->47; 49->48; 50->49; 52->50; 53->52; 54->53; 56->54; 57->56; 59->57; 60->59; 61->60; 63->61; 64->63; 65->64; 66->65; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand file:/data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d, false, Parquet, [path=file:///data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

WriteFiles

Union

Project [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47422, COL_2E7579DF_241F_4969_A954_4FA96050480C#47238 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47423, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47424, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47425, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47426]

Sort [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST], true, 0

WholeStageCodegen (13)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10230]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L], functions=[count(1)])

WholeStageCodegen (8)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, 10), ENSURE_REQUIREMENTS, [plan_id=10088]

HashAggregate(keys=[COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L], functions=[partial_count(1)])

WholeStageCodegen (5)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9867]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989 <=> B)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989 <=> B)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47467, COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47468, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47469, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47470, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47471]

Sort [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST], true, 0

WholeStageCodegen (14)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10599]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L], functions=[count(1)])

WholeStageCodegen (12)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, 10), ENSURE_REQUIREMENTS, [plan_id=10530]

HashAggregate(keys=[COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L], functions=[partial_count(1)])

WholeStageCodegen (11)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (10)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9911]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384 <=> C)

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384 <=> C)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Project [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47472, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47473, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47474, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47475, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47476]

Sort [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST], true, 0

WholeStageCodegen (15)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10334]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L], functions=[count(1)])

WholeStageCodegen (9)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, 10), ENSURE_REQUIREMENTS, [plan_id=10140]

HashAggregate(keys=[COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L], functions=[partial_count(1)])

WholeStageCodegen (7)

Window [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L], [1], [1 DESC NULLS LAST]

Sort [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (6)

AQEShuffleRead coalesced

Exchange hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9955]

Project [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]

Filter (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414 <=> M)

ColumnarToRow

WholeStageCodegen (3)

FileScan parquet [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414] Batched: true, DataFilters: [(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414 <=> M)], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692], PartitionFilters: [], PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)], ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (97)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (60)
   +- WriteFiles (59)
      +- Union (58)
         :- * Project (19)
         :  +- * Sort (18)
         :     +- AQEShuffleRead (17)
         :        +- ShuffleQueryStage (16), Statistics(sizeInBytes=640.0 B, rowCount=16)
         :           +- Exchange (15)
         :              +- * HashAggregate (14)
         :                 +- AQEShuffleRead (13)
         :                    +- ShuffleQueryStage (12), Statistics(sizeInBytes=640.0 B, rowCount=16)
         :                       +- Exchange (11)
         :                          +- * HashAggregate (10)
         :                             +- Window (9)
         :                                +- * Sort (8)
         :                                   +- AQEShuffleRead (7)
         :                                      +- ShuffleQueryStage (6), Statistics(sizeInBytes=3.0 MiB, rowCount=1.30E+5)
         :                                         +- Exchange (5)
         :                                            +- * Project (4)
         :                                               +- * Filter (3)
         :                                                  +- * ColumnarToRow (2)
         :                                                     +- Scan parquet  (1)
         :- * Project (38)
         :  +- * Sort (37)
         :     +- AQEShuffleRead (36)
         :        +- ShuffleQueryStage (35), Statistics(sizeInBytes=3.9 KiB, rowCount=99)
         :           +- Exchange (34)
         :              +- * HashAggregate (33)
         :                 +- AQEShuffleRead (32)
         :                    +- ShuffleQueryStage (31), Statistics(sizeInBytes=3.9 KiB, rowCount=99)
         :                       +- Exchange (30)
         :                          +- * HashAggregate (29)
         :                             +- Window (28)
         :                                +- * Sort (27)
         :                                   +- AQEShuffleRead (26)
         :                                      +- ShuffleQueryStage (25), Statistics(sizeInBytes=19.9 MiB, rowCount=8.70E+5)
         :                                         +- Exchange (24)
         :                                            +- * Project (23)
         :                                               +- * Filter (22)
         :                                                  +- * ColumnarToRow (21)
         :                                                     +- Scan parquet  (20)
         +- * Project (57)
            +- * Sort (56)
               +- AQEShuffleRead (55)
                  +- ShuffleQueryStage (54), Statistics(sizeInBytes=44.7 KiB, rowCount=954)
                     +- Exchange (53)
                        +- * HashAggregate (52)
                           +- AQEShuffleRead (51)
                              +- ShuffleQueryStage (50), Statistics(sizeInBytes=44.7 KiB, rowCount=954)
                                 +- Exchange (49)
                                    +- * HashAggregate (48)
                                       +- Window (47)
                                          +- * Sort (46)
                                             +- AQEShuffleRead (45)
                                                +- ShuffleQueryStage (44), Statistics(sizeInBytes=6.3 MiB, rowCount=2.05E+5)
                                                   +- Exchange (43)
                                                      +- * Project (42)
                                                         +- * Filter (41)
                                                            +- * ColumnarToRow (40)
                                                               +- Scan parquet  (39)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (96)
   +- WriteFiles (95)
      +- Union (94)
         :- Project (71)
         :  +- Sort (70)
         :     +- Exchange (69)
         :        +- HashAggregate (68)
         :           +- Exchange (67)
         :              +- HashAggregate (66)
         :                 +- Window (65)
         :                    +- Sort (64)
         :                       +- Exchange (63)
         :                          +- Project (62)
         :                             +- Filter (61)
         :                                +- Scan parquet  (1)
         :- Project (82)
         :  +- Sort (81)
         :     +- Exchange (80)
         :        +- HashAggregate (79)
         :           +- Exchange (78)
         :              +- HashAggregate (77)
         :                 +- Window (76)
         :                    +- Sort (75)
         :                       +- Exchange (74)
         :                          +- Project (73)
         :                             +- Filter (72)
         :                                +- Scan parquet  (20)
         +- Project (93)
            +- Sort (92)
               +- Exchange (91)
                  +- HashAggregate (90)
                     +- Exchange (89)
                        +- HashAggregate (88)
                           +- Window (87)
                              +- Sort (86)
                                 +- Exchange (85)
                                    +- Project (84)
                                       +- Filter (83)
                                          +- Scan parquet  (39)


(1) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,B)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(2) ColumnarToRow [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]

(3) Filter [codegen id : 1]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989 <=> B)

(4) Project [codegen id : 1]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]

(5) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9867]

(6) ShuffleQueryStage
Output [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: 0

(7) AQEShuffleRead
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: coalesced

(8) Sort [codegen id : 4]
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(9) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L], [1], [1 DESC NULLS LAST]

(10) HashAggregate [codegen id : 5]
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47448L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]

(11) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, 10), ENSURE_REQUIREMENTS, [plan_id=10088]

(12) ShuffleQueryStage
Output [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Arguments: 3

(13) AQEShuffleRead
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Arguments: coalesced

(14) HashAggregate [codegen id : 8]
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#47243L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, count(1)#47243L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]

(15) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10230]

(16) ShuffleQueryStage
Output [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: 5

(17) AQEShuffleRead
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: coalesced

(18) Sort [codegen id : 13]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST], true, 0

(19) Project [codegen id : 13]
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47422, COL_2E7579DF_241F_4969_A954_4FA96050480C#47238 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47423, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47424, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47425, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47426]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]

(20) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,C)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(21) ColumnarToRow [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]

(22) Filter [codegen id : 2]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384 <=> C)

(23) Project [codegen id : 2]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]

(24) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9911]

(25) ShuffleQueryStage
Output [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: 1

(26) AQEShuffleRead
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: coalesced

(27) Sort [codegen id : 10]
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(28) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L], [1], [1 DESC NULLS LAST]

(29) HashAggregate [codegen id : 11]
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47450L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]

(30) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, 10), ENSURE_REQUIREMENTS, [plan_id=10530]

(31) ShuffleQueryStage
Output [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Arguments: 7

(32) AQEShuffleRead
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Arguments: coalesced

(33) HashAggregate [codegen id : 12]
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#45338L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, count(1)#45338L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]

(34) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10599]

(35) ShuffleQueryStage
Output [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: 8

(36) AQEShuffleRead
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: coalesced

(37) Sort [codegen id : 14]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST], true, 0

(38) Project [codegen id : 14]
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47467, COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47468, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47469, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47470, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47471]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]

(39) Scan parquet 
Output [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/01_A_PRE_DECOUP_411692]
PushedFilters: [EqualNullSafe(COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9,M)]
ReadSchema: struct<COL_30863531_1D7E_428B_B583_E41A16C72ABB:string,COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9:string>

(40) ColumnarToRow [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]

(41) Filter [codegen id : 3]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414 <=> M)

(42) Project [codegen id : 3]
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]

(43) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9955]

(44) ShuffleQueryStage
Output [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: 2

(45) AQEShuffleRead
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: coalesced

(46) Sort [codegen id : 6]
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(47) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L], [1], [1 DESC NULLS LAST]

(48) HashAggregate [codegen id : 7]
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47452L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]

(49) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, 10), ENSURE_REQUIREMENTS, [plan_id=10140]

(50) ShuffleQueryStage
Output [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Arguments: 4

(51) AQEShuffleRead
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Arguments: coalesced

(52) HashAggregate [codegen id : 9]
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#42111L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, count(1)#42111L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]

(53) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=10334]

(54) ShuffleQueryStage
Output [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: 6

(55) AQEShuffleRead
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: coalesced

(56) Sort [codegen id : 15]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST], true, 0

(57) Project [codegen id : 15]
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47472, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47473, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47474, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47475, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47476]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]

(58) Union

(59) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47422, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47423, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47424, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47425, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47426]

(60) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d, false, Parquet, [path=file:///data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(61) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989 <=> B)

(62) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, 78, 5) AS COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#38987, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#38989]

(63) Exchange
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9805]

(64) Sort
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(65) Window
Input [1]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L], [1], [1 DESC NULLS LAST]

(66) HashAggregate
Input [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47448L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]

(67) Exchange
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Arguments: hashpartitioning(COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, 10), ENSURE_REQUIREMENTS, [plan_id=9810]

(68) HashAggregate
Input [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L, count#47449L]
Keys [2]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#47243L]
Results [3]: [COL_0637F7DB_5B0A_4048_B0E7_FA5F86111DA0#47188 AS COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_CC0697E6_0ABE_43A9_98FB_483BDDE9693F#47197L AS COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, count(1)#47243L AS COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]

(69) Exchange
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: rangepartitioning(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=9813]

(70) Sort
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]
Arguments: [COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L DESC NULLS LAST], true, 0

(71) Project
Output [5]: [str_create(B) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47422, COL_2E7579DF_241F_4969_A954_4FA96050480C#47238 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47423, bigint_to_int(COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47424, bigint_to_int(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47425, dec_round(dec_mult(array(dec_div(array(COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47426]
Input [3]: [COL_2E7579DF_241F_4969_A954_4FA96050480C#47238, COL_5901774E_F141_4B02_B8D0_E26F7CD976E3#47239L, COL_D0B45913_B192_4B01_B30D_F23C89D4A063#47245L]

(72) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384 <=> C)

(73) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, 78, 5) AS COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47382, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47384]

(74) Exchange
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9816]

(75) Sort
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(76) Window
Input [1]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L], [1], [1 DESC NULLS LAST]

(77) HashAggregate
Input [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47450L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]

(78) Exchange
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Arguments: hashpartitioning(COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, 10), ENSURE_REQUIREMENTS, [plan_id=9821]

(79) HashAggregate
Input [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L, count#47451L]
Keys [2]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#45338L]
Results [3]: [COL_06C05362_D5B7_4AD2_9BFE_13DDC9412182#45139 AS COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_5AD217E3_4EE9_41A9_BE2B_C0C7733B9C0B#45202L AS COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, count(1)#45338L AS COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]

(80) Exchange
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: rangepartitioning(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=9824]

(81) Sort
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]
Arguments: [COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L DESC NULLS LAST], true, 0

(82) Project
Output [5]: [str_create(C) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47467, COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47468, bigint_to_int(COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47469, bigint_to_int(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47470, dec_round(dec_mult(array(dec_div(array(COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47471]
Input [3]: [COL_F7441094_A4C6_4C98_B699_87BB860420B2#45333, COL_D84E9517_7132_491A_A4B5_BF75639391C2#45334L, COL_FD70A560_00A9_4445_8195_2226C4AC9C5B#45340L]

(83) Filter
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]
Condition : (COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414 <=> M)

(84) Project
Output [1]: [str_extract_middle(COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, 75, 13) AS COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Input [2]: [COL_30863531_1D7E_428B_B583_E41A16C72ABB#47412, COL_17B43B85_6EF2_4511_9F9E_84EDB43F71E9#47414]

(85) Exchange
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: hashpartitioning(1, 10), ENSURE_REQUIREMENTS, [plan_id=9827]

(86) Sort
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: [1 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(87) Window
Input [1]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615]
Arguments: [count(1) windowspecdefinition(1, 1 DESC NULLS LAST, specifiedwindowframe(RangeFrame, unboundedpreceding$(), currentrow$())) AS COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L], [1], [1 DESC NULLS LAST]

(88) HashAggregate
Input [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#47452L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]

(89) Exchange
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Arguments: hashpartitioning(COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, 10), ENSURE_REQUIREMENTS, [plan_id=9832]

(90) HashAggregate
Input [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L, count#47453L]
Keys [2]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L]
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#42111L]
Results [3]: [COL_6240343E_CFDC_4E36_9CD1_5A3084A7C58A#41615 AS COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_20E75411_F375_4DC1_B499_5E84E85FE997#41745L AS COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, count(1)#42111L AS COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]

(91) Exchange
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: rangepartitioning(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST, 10), ENSURE_REQUIREMENTS, [plan_id=9835]

(92) Sort
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]
Arguments: [COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L DESC NULLS LAST], true, 0

(93) Project
Output [5]: [str_create(M) AS COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47472, COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081 AS COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47473, bigint_to_int(COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L) AS COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47474, bigint_to_int(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L) AS COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47475, dec_round(dec_mult(array(dec_div(array(COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L)), 100.0)), 2) AS COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47476]
Input [3]: [COL_FF2D4BEA_6FF9_46E0_A33E_6FE7BF79A99E#42081, COL_3B1BCF7F_CA7A_46A9_904F_75CC8FEDA69D#42082L, COL_69FF673D_992E_4EAB_90C0_47F38FB9B849#42113L]

(94) Union

(95) WriteFiles
Input [5]: [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E#47422, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96#47423, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4#47424, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180#47425, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89#47426]

(96) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d, false, Parquet, [path=file:///data/output/cache/parquet/datablock/f6397906_c2ed_44e2_b2d5_73244ab40a0d], Overwrite, [COL_78E31252_3023_4EB2_ADA8_410E9DB2EF6E, COL_B0041BD2_F1BC_421C_8A27_69D4188CFA96, COL_208AF172_F470_4E6D_9AF7_76456CAD6BC4, COL_679DEF33_EC1E_425F_B5ED_F7BF189AD180, COL_47AA0BFF_1285_4526_8DC9_8A0B620BAF89]

(97) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet