hdh-spark-job - Details for Query 1188

Submitted Time: 2026/10/27 14:53:47
Duration: 0.6 s
Succeeded Jobs: 1119 1120 1121 1122 1124 1125 1126 1127

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (14)\n \nduration: 0 ms"; 2 [labelType="html" label="HashAggregate time in aggregation build: 0 ms number of output rows: 1"]; } 3 [labelType="html" label="Exchange shuffle records written: 1 local merged chunks fetched: 0 shuffle write time: 0 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 59.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 16.0 B local merged bytes read: 0.0 B number of partitions: 1 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 59.0 B"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (13)\n \nduration: 26 ms"; 5 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 26 ms peak memory: 0.0 B number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0"]; 6 [labelType="html" label=" Project "]; 7 [labelType="html" label="SortMergeJoin number of output rows: 1,191 spill size: 0.0 B"]; } subgraph cluster8 { isCluster="true"; label="WholeStageCodegen (11)\n \nduration: 34 ms"; 9 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 10 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 7.8 KiB number of coalesced partitions: 1"]; 11 [labelType="html" label="Exchange shuffle records written: 895 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 895 local bytes read: 7.5 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 35.0 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 7.5 KiB"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (10)\n \nduration: 30 ms"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label="Filter number of output rows: 895"]; 15 [labelType="html" label="SortMergeJoin number of output rows: 21,131 spill size: 0.0 B"]; } subgraph cluster16 { isCluster="true"; label="WholeStageCodegen (8)\n \nduration: 51 ms"; 17 [labelType="html" label="Sort sort time: 5 ms peak memory: 64.5 MiB spill size: 0.0 B"]; } 18 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 359.5 KiB number of coalesced partitions: 1"]; 19 [labelType="html" label="Exchange shuffle records written: 21,127 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 21,127 local bytes read: 342.8 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 2.4 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 342.8 KiB"]; subgraph cluster20 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: 53 ms"; 21 [labelType="html" label=" Project "]; 22 [labelType="html" label="SortMergeJoin number of output rows: 21,127 spill size: 0.0 B"]; } subgraph cluster23 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 166 ms"; 24 [labelType="html" label="Sort sort time: 12 ms peak memory: 64.5 MiB spill size: 0.0 B"]; } 25 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 416.5 KiB number of coalesced partitions: 1"]; 26 [labelType="html" label="Exchange shuffle records written: 21,121 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 5 ms (1 ms, 1 ms, 2 ms (stage 1715.0: task 1459)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 21,121 local bytes read: 398.4 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 3 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 2.1 MiB (67.3 KiB, 1038.9 KiB, 1038.9 KiB (stage 1715.0: task 1459)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 398.4 KiB (13.9 KiB, 192.2 KiB, 192.3 KiB (stage 1715.0: task 1459))"]; subgraph cluster27 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n302 ms (10 ms, 140 ms, 152 ms (stage 1715.0: task 1458))"; 28 [labelType="html" label=" Project "]; } 29 [labelType="html" label="Scan csv number of output rows: 21,121 number of files read: 1 metadata time: 0 ms size of files read: 8.3 MiB"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 1.4 m"; 31 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 32 [labelType="html" label="ObjectHashAggregate number of output rows: 445 time in aggregation build: 10 ms spill size: 0.0 B number of sort fallback tasks: 1"]; 33 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 18.6 KiB number of coalesced partitions: 1"]; 34 [labelType="html" label="Exchange shuffle records written: 445 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 445 local bytes read: 17.7 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 53.8 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 17.7 KiB"]; 35 [labelType="html" label="ObjectHashAggregate number of output rows: 445 time in aggregation build: 17 ms spill size: 0.0 B number of sort fallback tasks: 1"]; subgraph cluster36 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 16 ms"; 37 [labelType="html" label=" Project "]; } 38 [labelType="html" label="Scan csv number of output rows: 1,216 number of files read: 1 metadata time: 0 ms size of files read: 111.6 KiB"]; subgraph cluster39 { isCluster="true"; label="WholeStageCodegen (9)\n \nduration: 78 ms"; 40 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 41 [labelType="html" label="ObjectHashAggregate number of output rows: 464 time in aggregation build: 6 ms spill size: 0.0 B number of sort fallback tasks: 1"]; 42 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 9.6 KiB number of coalesced partitions: 1"]; 43 [labelType="html" label="Exchange shuffle records written: 464 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 464 local bytes read: 9.1 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 43.8 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 9.1 KiB"]; 44 [labelType="html" label="ObjectHashAggregate number of output rows: 464 time in aggregation build: 11 ms spill size: 0.0 B number of sort fallback tasks: 1"]; subgraph cluster45 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 11 ms"; 46 [labelType="html" label=" Project "]; } 47 [labelType="html" label="Scan csv number of output rows: 486 number of files read: 1 metadata time: 0 ms size of files read: 50.8 KiB"]; subgraph cluster48 { isCluster="true"; label="WholeStageCodegen (12)\n \nduration: 0 ms"; 49 [labelType="html" label="Sort sort time: 9 ms peak memory: 65.0 MiB spill size: 0.0 B"]; } 50 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 154.7 KiB number of coalesced partitions: 1"]; 51 [labelType="html" label="Exchange shuffle records written: 26,044 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 4 ms (1 ms, 1 ms, 1 ms (stage 1718.0: task 1464)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 26,044 local bytes read: 147.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 3 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 1017.3 KiB (7.0 KiB, 492.0 KiB, 518.3 KiB (stage 1718.0: task 1464)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 147.6 KiB (1883.0 B, 70.6 KiB, 75.1 KiB (stage 1718.0: task 1464))"]; subgraph cluster52 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n194 ms (5 ms, 93 ms, 96 ms (stage 1718.0: task 1464))"; 53 [labelType="html" label=" Project "]; 54 [labelType="html" label="Filter number of output rows: 26,044"]; } 55 [labelType="html" label="Scan csv number of output rows: 26,044 number of files read: 1 metadata time: 0 ms size of files read: 8.0 MiB"]; 2->0; 3->2; 5->3; 6->5; 7->6; 9->7; 10->9; 11->10; 13->11; 14->13; 15->14; 17->15; 18->17; 19->18; 21->19; 22->21; 24->22; 25->24; 26->25; 28->26; 29->28; 31->22; 32->31; 33->32; 34->33; 35->34; 37->35; 38->37; 40->15; 41->40; 42->41; 43->42; 44->43; 46->44; 47->46; 49->7; 50->49; 51->50; 53->51; 54->53; 55->54; }

AdaptiveSparkPlan isFinalPlan=true

HashAggregate(keys=[], functions=[count(1)])

WholeStageCodegen (14)

Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=42087]

HashAggregate(keys=[], functions=[partial_count(1)])

Project

SortMergeJoin [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420], [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836], LeftOuter

WholeStageCodegen (13)

Sort [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420 ASC NULLS FIRST], false, 0

WholeStageCodegen (11)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, 10), ENSURE_REQUIREMENTS, [plan_id=42002]

Project [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]

Filter (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#428548)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727)) AND (((COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426 <=> 0.0) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725 <=> 0.0)) OR ((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427 <=> 0.0) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545 <=> 0.0))))

SortMergeJoin [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429], [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724], FullOuter

WholeStageCodegen (10)

Sort [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

WholeStageCodegen (8)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41835]

Project [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448 AS COL_059961D5_87D2_4745_B270_05C29151B074#428548]

SortMergeJoin [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429], [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445], FullOuter

WholeStageCodegen (7)

Sort [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

WholeStageCodegen (5)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41524]

Project [FINESS#424514 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, NO_RSS#424515 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, str_to_dec(MNT_MON#424656, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, str_to_dec(MNT_DMI#424657, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, str_remove_space(NO_ADMIN#424516, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]

WholeStageCodegen (1)

FileScan csv [FINESS#424514,NO_RSS#424515,NO_ADMIN#424516,MNT_MON#424656,MNT_DMI#424657] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/VISUALVALOSEJOU_440300/420780652.2025.1..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<FINESS:string,NO_RSS:string,NO_ADMIN:string,MNT_MON:string,MNT_DMI:string>

Sort [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445 ASC NULLS FIRST], false, 0

WholeStageCodegen (6)

ObjectHashAggregate(keys=[COL_C898C1FE_14C4_493A_878B_315EB757695B#426032], functions=[sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, 10), ENSURE_REQUIREMENTS, [plan_id=41537]

ObjectHashAggregate(keys=[COL_C898C1FE_14C4_493A_878B_315EB757695B#426032], functions=[partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)])

Project [str_remove_space(str_extract_middle(_c0#425927, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, str_remove_space(str_extract_middle(_c0#425927, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, round(str_to_dec(str_join(array(str_extract_middle(_c0#425927, 83, 7), str_extract_middle(_c0#425927, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038]

WholeStageCodegen (2)

FileScan csv [_c0#425927] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/DMI_M12_2025_440461/DMI-M12-2025 - BIS...., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Sort [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724 ASC NULLS FIRST], false, 0

WholeStageCodegen (9)

ObjectHashAggregate(keys=[COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248], functions=[sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, 10), ENSURE_REQUIREMENTS, [plan_id=41557]

ObjectHashAggregate(keys=[COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248], functions=[partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)])

Project [str_remove_space(str_extract_middle(_c0#426114, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, str_remove_space(str_extract_middle(_c0#426114, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, str_to_dec(str_join(array(str_extract_middle(_c0#426114, 65, 7), str_extract_middle(_c0#426114, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255]

WholeStageCodegen (3)

FileScan csv [_c0#426114] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/MOLECULES_ONERE_440457/MO-M12-2025.txt], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Sort [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836 ASC NULLS FIRST], false, 0

WholeStageCodegen (12)

AQEShuffleRead coalesced

Exchange hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836, 10), ENSURE_REQUIREMENTS, [plan_id=41578]

Project [str_extract_middle(_c0#426336, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]

Filter isnotnull(str_extract_middle(_c0#426336, 28, 20))

WholeStageCodegen (4)

FileScan csv [_c0#426336] Batched: false, DataFilters: [isnotnull(str_extract_middle(_c0#426336, 28, 20))], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/RSS_M12_2025_440462/RSS- 420780652.txt], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Details

== Physical Plan ==
AdaptiveSparkPlan (80)
+- == Final Plan ==
   * HashAggregate (48)
   +- ShuffleQueryStage (47), Statistics(sizeInBytes=16.0 B, rowCount=1)
      +- Exchange (46)
         +- * HashAggregate (45)
            +- * Project (44)
               +- * SortMergeJoin LeftOuter (43)
                  :- * Sort (35)
                  :  +- AQEShuffleRead (34)
                  :     +- ShuffleQueryStage (33), Statistics(sizeInBytes=35.0 KiB, rowCount=895)
                  :        +- Exchange (32)
                  :           +- * Project (31)
                  :              +- * Filter (30)
                  :                 +- * SortMergeJoin FullOuter (29)
                  :                    :- * Sort (20)
                  :                    :  +- AQEShuffleRead (19)
                  :                    :     +- ShuffleQueryStage (18), Statistics(sizeInBytes=2.4 MiB, rowCount=2.11E+4)
                  :                    :        +- Exchange (17)
                  :                    :           +- * Project (16)
                  :                    :              +- * SortMergeJoin FullOuter (15)
                  :                    :                 :- * Sort (6)
                  :                    :                 :  +- AQEShuffleRead (5)
                  :                    :                 :     +- ShuffleQueryStage (4), Statistics(sizeInBytes=2.1 MiB, rowCount=2.11E+4)
                  :                    :                 :        +- Exchange (3)
                  :                    :                 :           +- * Project (2)
                  :                    :                 :              +- Scan csv  (1)
                  :                    :                 +- * Sort (14)
                  :                    :                    +- ObjectHashAggregate (13)
                  :                    :                       +- AQEShuffleRead (12)
                  :                    :                          +- ShuffleQueryStage (11), Statistics(sizeInBytes=53.8 KiB, rowCount=445)
                  :                    :                             +- Exchange (10)
                  :                    :                                +- ObjectHashAggregate (9)
                  :                    :                                   +- * Project (8)
                  :                    :                                      +- Scan csv  (7)
                  :                    +- * Sort (28)
                  :                       +- ObjectHashAggregate (27)
                  :                          +- AQEShuffleRead (26)
                  :                             +- ShuffleQueryStage (25), Statistics(sizeInBytes=43.8 KiB, rowCount=464)
                  :                                +- Exchange (24)
                  :                                   +- ObjectHashAggregate (23)
                  :                                      +- * Project (22)
                  :                                         +- Scan csv  (21)
                  +- * Sort (42)
                     +- AQEShuffleRead (41)
                        +- ShuffleQueryStage (40), Statistics(sizeInBytes=1017.3 KiB, rowCount=2.60E+4)
                           +- Exchange (39)
                              +- * Project (38)
                                 +- * Filter (37)
                                    +- Scan csv  (36)
+- == Initial Plan ==
   HashAggregate (79)
   +- Exchange (78)
      +- HashAggregate (77)
         +- Project (76)
            +- SortMergeJoin LeftOuter (75)
               :- Sort (70)
               :  +- Exchange (69)
               :     +- Project (68)
               :        +- Filter (67)
               :           +- SortMergeJoin FullOuter (66)
               :              :- Sort (60)
               :              :  +- Exchange (59)
               :              :     +- Project (58)
               :              :        +- SortMergeJoin FullOuter (57)
               :              :           :- Sort (51)
               :              :           :  +- Exchange (50)
               :              :           :     +- Project (49)
               :              :           :        +- Scan csv  (1)
               :              :           +- Sort (56)
               :              :              +- ObjectHashAggregate (55)
               :              :                 +- Exchange (54)
               :              :                    +- ObjectHashAggregate (53)
               :              :                       +- Project (52)
               :              :                          +- Scan csv  (7)
               :              +- Sort (65)
               :                 +- ObjectHashAggregate (64)
               :                    +- Exchange (63)
               :                       +- ObjectHashAggregate (62)
               :                          +- Project (61)
               :                             +- Scan csv  (21)
               +- Sort (74)
                  +- Exchange (73)
                     +- Project (72)
                        +- Filter (71)
                           +- Scan csv  (36)


(1) Scan csv 
Output [5]: [FINESS#424514, NO_RSS#424515, NO_ADMIN#424516, MNT_MON#424656, MNT_DMI#424657]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/VISUALVALOSEJOU_440300/420780652.2025.12.MCO.VisualValoSejours.csv]
ReadSchema: struct<FINESS:string,NO_RSS:string,NO_ADMIN:string,MNT_MON:string,MNT_DMI:string>

(2) Project [codegen id : 1]
Output [5]: [FINESS#424514 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, NO_RSS#424515 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, str_to_dec(MNT_MON#424656, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, str_to_dec(MNT_DMI#424657, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, str_remove_space(NO_ADMIN#424516, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Input [5]: [FINESS#424514, NO_RSS#424515, NO_ADMIN#424516, MNT_MON#424656, MNT_DMI#424657]

(3) Exchange
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41524]

(4) ShuffleQueryStage
Output [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: 0

(5) AQEShuffleRead
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: coalesced

(6) Sort [codegen id : 5]
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

(7) Scan csv 
Output [1]: [_c0#425927]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/DMI_M12_2025_440461/DMI-M12-2025 - BIS.txt]
ReadSchema: struct<_c0:string>

(8) Project [codegen id : 2]
Output [3]: [str_remove_space(str_extract_middle(_c0#425927, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, str_remove_space(str_extract_middle(_c0#425927, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, round(str_to_dec(str_join(array(str_extract_middle(_c0#425927, 83, 7), str_extract_middle(_c0#425927, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038]
Input [1]: [_c0#425927]

(9) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032]
Functions [2]: [partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)]
Aggregate Attributes [2]: [sum#430940, buf#430941]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]

(10) Exchange
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Arguments: hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, 10), ENSURE_REQUIREMENTS, [plan_id=41537]

(11) ShuffleQueryStage
Output [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Arguments: 1

(12) AQEShuffleRead
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Arguments: coalesced

(13) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032]
Functions [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)]
Aggregate Attributes [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038)#426060, collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)#426062]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032 AS 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, round(sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038)#426060, 2) AS 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, size(collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)#426062, true) AS 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]

(14) Sort [codegen id : 6]
Input [3]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]
Arguments: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445 ASC NULLS FIRST], false, 0

(15) SortMergeJoin [codegen id : 7]
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Right keys [1]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445]
Join type: FullOuter
Join condition: None

(16) Project [codegen id : 7]
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448 AS COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Input [8]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]

(17) Exchange
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41835]

(18) ShuffleQueryStage
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: 4

(19) AQEShuffleRead
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: coalesced

(20) Sort [codegen id : 8]
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

(21) Scan csv 
Output [1]: [_c0#426114]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/MOLECULES_ONERE_440457/MO-M12-2025.txt]
ReadSchema: struct<_c0:string>

(22) Project [codegen id : 3]
Output [3]: [str_remove_space(str_extract_middle(_c0#426114, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, str_remove_space(str_extract_middle(_c0#426114, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, str_to_dec(str_join(array(str_extract_middle(_c0#426114, 65, 7), str_extract_middle(_c0#426114, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255]
Input [1]: [_c0#426114]

(23) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248]
Functions [2]: [partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)]
Aggregate Attributes [2]: [sum#430944, buf#430945]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]

(24) Exchange
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Arguments: hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, 10), ENSURE_REQUIREMENTS, [plan_id=41557]

(25) ShuffleQueryStage
Output [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Arguments: 2

(26) AQEShuffleRead
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Arguments: coalesced

(27) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248]
Functions [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)]
Aggregate Attributes [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255)#426282, collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)#426284]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248 AS MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, round(sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255)#426282, 2) AS MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, size(collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)#426284, true) AS MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]

(28) Sort [codegen id : 9]
Input [3]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]
Arguments: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724 ASC NULLS FIRST], false, 0

(29) SortMergeJoin [codegen id : 10]
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Right keys [1]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724]
Join type: FullOuter
Join condition: None

(30) Filter [codegen id : 10]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]
Condition : (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#428548)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727)) AND (((COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426 <=> 0.0) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725 <=> 0.0)) OR ((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427 <=> 0.0) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545 <=> 0.0))))

(31) Project [codegen id : 10]
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]

(32) Exchange
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, 10), ENSURE_REQUIREMENTS, [plan_id=42002]

(33) ShuffleQueryStage
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: 5

(34) AQEShuffleRead
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: coalesced

(35) Sort [codegen id : 11]
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420 ASC NULLS FIRST], false, 0

(36) Scan csv 
Output [1]: [_c0#426336]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/RSS_M12_2025_440462/RSS- 420780652.txt]
ReadSchema: struct<_c0:string>

(37) Filter [codegen id : 4]
Input [1]: [_c0#426336]
Condition : isnotnull(str_extract_middle(_c0#426336, 28, 20))

(38) Project [codegen id : 4]
Output [1]: [str_extract_middle(_c0#426336, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Input [1]: [_c0#426336]

(39) Exchange
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836, 10), ENSURE_REQUIREMENTS, [plan_id=41578]

(40) ShuffleQueryStage
Output [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: 3

(41) AQEShuffleRead
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: coalesced

(42) Sort [codegen id : 12]
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836 ASC NULLS FIRST], false, 0

(43) SortMergeJoin [codegen id : 13]
Left keys [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Right keys [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Join type: LeftOuter
Join condition: None

(44) Project [codegen id : 13]
Output: []
Input [2]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]

(45) HashAggregate [codegen id : 13]
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#430936L]
Results [1]: [count#430937L]

(46) Exchange
Input [1]: [count#430937L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=42087]

(47) ShuffleQueryStage
Output [1]: [count#430937L]
Arguments: 6

(48) HashAggregate [codegen id : 14]
Input [1]: [count#430937L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#430916L]
Results [1]: [count(1)#430916L AS count#430917L]

(49) Project
Output [5]: [FINESS#424514 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, NO_RSS#424515 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, str_to_dec(MNT_MON#424656, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, str_to_dec(MNT_DMI#424657, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, str_remove_space(NO_ADMIN#424516, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Input [5]: [FINESS#424514, NO_RSS#424515, NO_ADMIN#424516, MNT_MON#424656, MNT_DMI#424657]

(50) Exchange
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41427]

(51) Sort
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

(52) Project
Output [3]: [str_remove_space(str_extract_middle(_c0#425927, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, str_remove_space(str_extract_middle(_c0#425927, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, round(str_to_dec(str_join(array(str_extract_middle(_c0#425927, 83, 7), str_extract_middle(_c0#425927, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038]
Input [1]: [_c0#425927]

(53) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032]
Functions [2]: [partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)]
Aggregate Attributes [2]: [sum#430940, buf#430941]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]

(54) Exchange
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Arguments: hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, 10), ENSURE_REQUIREMENTS, [plan_id=41423]

(55) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032, sum#430942, buf#430943]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032]
Functions [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)]
Aggregate Attributes [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038)#426060, collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)#426062]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#426032 AS 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, round(sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#426038)#426060, 2) AS 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, size(collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#426034, 0, 0)#426062, true) AS 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]

(56) Sort
Input [3]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]
Arguments: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445 ASC NULLS FIRST], false, 0

(57) SortMergeJoin
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Right keys [1]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445]
Join type: FullOuter
Join condition: None

(58) Project
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448 AS COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Input [8]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#428445, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#428446, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#428448]

(59) Exchange
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, 10), ENSURE_REQUIREMENTS, [plan_id=41436]

(60) Sort
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429 ASC NULLS FIRST], false, 0

(61) Project
Output [3]: [str_remove_space(str_extract_middle(_c0#426114, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, str_remove_space(str_extract_middle(_c0#426114, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, str_to_dec(str_join(array(str_extract_middle(_c0#426114, 65, 7), str_extract_middle(_c0#426114, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255]
Input [1]: [_c0#426114]

(62) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248]
Functions [2]: [partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)]
Aggregate Attributes [2]: [sum#430944, buf#430945]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]

(63) Exchange
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Arguments: hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, 10), ENSURE_REQUIREMENTS, [plan_id=41432]

(64) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248, sum#430946, buf#430947]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248]
Functions [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)]
Aggregate Attributes [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255)#426282, collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)#426284]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#426248 AS MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, round(sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#426255)#426282, 2) AS MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, size(collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#426250, 0, 0)#426284, true) AS MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]

(65) Sort
Input [3]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]
Arguments: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724 ASC NULLS FIRST], false, 0

(66) SortMergeJoin
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429]
Right keys [1]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724]
Join type: FullOuter
Join condition: None

(67) Filter
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]
Condition : (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#428548)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727)) AND (((COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426 <=> 0.0) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725 <=> 0.0)) OR ((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427 <=> 0.0) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545 <=> 0.0))))

(68) Project
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#428419, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#428426, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#428427, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#428429, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#428545, COL_059961D5_87D2_4745_B270_05C29151B074#428548, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#428724, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#428725, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#428727]

(69) Exchange
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, 10), ENSURE_REQUIREMENTS, [plan_id=41446]

(70) Sort
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Arguments: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420 ASC NULLS FIRST], false, 0

(71) Filter
Input [1]: [_c0#426336]
Condition : isnotnull(str_extract_middle(_c0#426336, 28, 20))

(72) Project
Output [1]: [str_extract_middle(_c0#426336, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Input [1]: [_c0#426336]

(73) Exchange
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836, 10), ENSURE_REQUIREMENTS, [plan_id=41447]

(74) Sort
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Arguments: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836 ASC NULLS FIRST], false, 0

(75) SortMergeJoin
Left keys [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420]
Right keys [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]
Join type: LeftOuter
Join condition: None

(76) Project
Output: []
Input [2]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#428420, RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#429836]

(77) HashAggregate
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#430936L]
Results [1]: [count#430937L]

(78) Exchange
Input [1]: [count#430937L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=41454]

(79) HashAggregate
Input [1]: [count#430937L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#430916L]
Results [1]: [count(1)#430916L AS count#430917L]

(80) AdaptiveSparkPlan
Output [1]: [count#430917L]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet