hdh-spark-job - Details for Query 1208

Submitted Time: 2026/10/27 14:57:43
Duration: 0.7 s
Succeeded Jobs: 1140 1141 1142 1143 1145 1146 1148 1149

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (14)\n \nduration: 0 ms"; 2 [labelType="html" label="HashAggregate time in aggregation build: 0 ms number of output rows: 1"]; } 3 [labelType="html" label="Exchange shuffle records written: 1 local merged chunks fetched: 0 shuffle write time: 0 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 59.0 B fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 16.0 B local merged bytes read: 0.0 B number of partitions: 1 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 59.0 B"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (13)\n \nduration: 31 ms"; 5 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build: 31 ms peak memory: 0.0 B number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0"]; 6 [labelType="html" label=" Project "]; 7 [labelType="html" label="SortMergeJoin number of output rows: 248 spill size: 0.0 B"]; } subgraph cluster8 { isCluster="true"; label="WholeStageCodegen (11)\n \nduration: 33 ms"; 9 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 10 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2.8 KiB number of coalesced partitions: 1"]; 11 [labelType="html" label="Exchange shuffle records written: 248 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 248 local bytes read: 2.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 9.7 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 2.6 KiB"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (10)\n \nduration: 30 ms"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label="Filter number of output rows: 248"]; 15 [labelType="html" label="SortMergeJoin number of output rows: 21,131 spill size: 0.0 B"]; } subgraph cluster16 { isCluster="true"; label="WholeStageCodegen (8)\n \nduration: 51 ms"; 17 [labelType="html" label="Sort sort time: 4 ms peak memory: 64.5 MiB spill size: 0.0 B"]; } 18 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 359.5 KiB number of coalesced partitions: 1"]; 19 [labelType="html" label="Exchange shuffle records written: 21,127 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 21,127 local bytes read: 342.8 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 2.4 MiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 342.8 KiB"]; subgraph cluster20 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: 66 ms"; 21 [labelType="html" label=" Project "]; 22 [labelType="html" label="SortMergeJoin number of output rows: 21,127 spill size: 0.0 B"]; } subgraph cluster23 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 150 ms"; 24 [labelType="html" label="Sort sort time: 12 ms peak memory: 64.5 MiB spill size: 0.0 B"]; } 25 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 416.5 KiB number of coalesced partitions: 1"]; 26 [labelType="html" label="Exchange shuffle records written: 21,121 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 5 ms (1 ms, 1 ms, 2 ms (stage 1767.0: task 1488)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 21,121 local bytes read: 398.4 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 3 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 2.1 MiB (67.3 KiB, 1038.9 KiB, 1038.9 KiB (stage 1767.0: task 1487)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 398.4 KiB (13.9 KiB, 192.2 KiB, 192.3 KiB (stage 1767.0: task 1488))"]; subgraph cluster27 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n221 ms (14 ms, 102 ms, 105 ms (stage 1767.0: task 1488))"; 28 [labelType="html" label=" Project "]; } 29 [labelType="html" label="Scan csv number of output rows: 21,121 number of files read: 1 metadata time: 0 ms size of files read: 8.3 MiB"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 1.5 m"; 31 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 32 [labelType="html" label="ObjectHashAggregate number of output rows: 445 time in aggregation build: 6 ms spill size: 0.0 B number of sort fallback tasks: 1"]; 33 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 18.6 KiB number of coalesced partitions: 1"]; 34 [labelType="html" label="Exchange shuffle records written: 445 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 445 local bytes read: 17.7 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 53.8 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 17.7 KiB"]; 35 [labelType="html" label="ObjectHashAggregate number of output rows: 445 time in aggregation build: 74 ms spill size: 0.0 B number of sort fallback tasks: 1"]; subgraph cluster36 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 74 ms"; 37 [labelType="html" label=" Project "]; } 38 [labelType="html" label="Scan csv number of output rows: 1,216 number of files read: 1 metadata time: 0 ms size of files read: 111.6 KiB"]; subgraph cluster39 { isCluster="true"; label="WholeStageCodegen (9)\n \nduration: 82 ms"; 40 [labelType="html" label="Sort sort time: 0 ms peak memory: 64.1 MiB spill size: 0.0 B"]; } 41 [labelType="html" label="ObjectHashAggregate number of output rows: 464 time in aggregation build: 9 ms spill size: 0.0 B number of sort fallback tasks: 1"]; 42 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 9.6 KiB number of coalesced partitions: 1"]; 43 [labelType="html" label="Exchange shuffle records written: 464 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 464 local bytes read: 9.1 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 43.8 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 9.1 KiB"]; 44 [labelType="html" label="ObjectHashAggregate number of output rows: 464 time in aggregation build: 28 ms spill size: 0.0 B number of sort fallback tasks: 1"]; subgraph cluster45 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 28 ms"; 46 [labelType="html" label=" Project "]; } 47 [labelType="html" label="Scan csv number of output rows: 486 number of files read: 1 metadata time: 0 ms size of files read: 50.8 KiB"]; subgraph cluster48 { isCluster="true"; label="WholeStageCodegen (12)\n \nduration: 0 ms"; 49 [labelType="html" label="Sort sort time: 14 ms peak memory: 65.0 MiB spill size: 0.0 B"]; } 50 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 154.7 KiB number of coalesced partitions: 1"]; 51 [labelType="html" label="Exchange shuffle records written: 26,044 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 4 ms (1 ms, 1 ms, 1 ms (stage 1770.0: task 1493)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 26,044 local bytes read: 147.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 3 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 1017.3 KiB (7.0 KiB, 492.0 KiB, 518.3 KiB (stage 1770.0: task 1493)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 147.6 KiB (1883.0 B, 70.6 KiB, 75.1 KiB (stage 1770.0: task 1493))"]; subgraph cluster52 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n192 ms (5 ms, 86 ms, 101 ms (stage 1770.0: task 1493))"; 53 [labelType="html" label=" Project "]; 54 [labelType="html" label="Filter number of output rows: 26,044"]; } 55 [labelType="html" label="Scan csv number of output rows: 26,044 number of files read: 1 metadata time: 0 ms size of files read: 8.0 MiB"]; 2->0; 3->2; 5->3; 6->5; 7->6; 9->7; 10->9; 11->10; 13->11; 14->13; 15->14; 17->15; 18->17; 19->18; 21->19; 22->21; 24->22; 25->24; 26->25; 28->26; 29->28; 31->22; 32->31; 33->32; 34->33; 35->34; 37->35; 38->37; 40->15; 41->40; 42->41; 43->42; 44->43; 46->44; 47->46; 49->7; 50->49; 51->50; 53->51; 54->53; 55->54; }

AdaptiveSparkPlan isFinalPlan=true

HashAggregate(keys=[], functions=[count(1)])

WholeStageCodegen (14)

Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=43673]

HashAggregate(keys=[], functions=[partial_count(1)])

Project

SortMergeJoin [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151], [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478], LeftOuter

WholeStageCodegen (13)

Sort [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151 ASC NULLS FIRST], false, 0

WholeStageCodegen (11)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, 10), ENSURE_REQUIREMENTS, [plan_id=43572]

Project [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]

Filter (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#438277)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300)) AND (((isnotnull(MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298) AND (COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157 <=> 0.0)) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298 <=> 0.0)) OR (((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158 <=> 0.0) AND isnotnull(COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274)) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274 <=> 0.0))))

SortMergeJoin [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160], [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297], FullOuter

WholeStageCodegen (10)

Sort [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

WholeStageCodegen (8)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=43402]

Project [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179 AS COL_059961D5_87D2_4745_B270_05C29151B074#438277]

SortMergeJoin [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160], [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176], FullOuter

WholeStageCodegen (7)

Sort [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

WholeStageCodegen (5)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=43091]

Project [FINESS#434259 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, NO_RSS#434260 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, str_to_dec(MNT_MON#434401, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, str_to_dec(MNT_DMI#434402, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, str_remove_space(NO_ADMIN#434261, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]

WholeStageCodegen (1)

FileScan csv [FINESS#434259,NO_RSS#434260,NO_ADMIN#434261,MNT_MON#434401,MNT_DMI#434402] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/VISUALVALOSEJOU_440300/420780652.2025.1..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<FINESS:string,NO_RSS:string,NO_ADMIN:string,MNT_MON:string,MNT_DMI:string>

Sort [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176 ASC NULLS FIRST], false, 0

WholeStageCodegen (6)

ObjectHashAggregate(keys=[COL_C898C1FE_14C4_493A_878B_315EB757695B#435777], functions=[sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, 10), ENSURE_REQUIREMENTS, [plan_id=43104]

ObjectHashAggregate(keys=[COL_C898C1FE_14C4_493A_878B_315EB757695B#435777], functions=[partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)])

Project [str_remove_space(str_extract_middle(_c0#435672, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, str_remove_space(str_extract_middle(_c0#435672, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, round(str_to_dec(str_join(array(str_extract_middle(_c0#435672, 83, 7), str_extract_middle(_c0#435672, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783]

WholeStageCodegen (2)

FileScan csv [_c0#435672] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/DMI_M12_2025_440461/DMI-M12-2025 - BIS...., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Sort [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297 ASC NULLS FIRST], false, 0

WholeStageCodegen (9)

ObjectHashAggregate(keys=[COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993], functions=[sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, 10), ENSURE_REQUIREMENTS, [plan_id=43124]

ObjectHashAggregate(keys=[COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993], functions=[partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)])

Project [str_remove_space(str_extract_middle(_c0#435859, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, str_remove_space(str_extract_middle(_c0#435859, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, str_to_dec(str_join(array(str_extract_middle(_c0#435859, 65, 7), str_extract_middle(_c0#435859, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000]

WholeStageCodegen (3)

FileScan csv [_c0#435859] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/MOLECULES_ONERE_440457/MO-M12-2025.txt], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Sort [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478 ASC NULLS FIRST], false, 0

WholeStageCodegen (12)

AQEShuffleRead coalesced

Exchange hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478, 10), ENSURE_REQUIREMENTS, [plan_id=43145]

Project [str_extract_middle(_c0#436081, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]

Filter isnotnull(str_extract_middle(_c0#436081, 28, 20))

WholeStageCodegen (4)

FileScan csv [_c0#436081] Batched: false, DataFilters: [isnotnull(str_extract_middle(_c0#436081, 28, 20))], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/RSS_M12_2025_440462/RSS- 420780652.txt], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string>

Details

== Physical Plan ==
AdaptiveSparkPlan (80)
+- == Final Plan ==
   * HashAggregate (48)
   +- ShuffleQueryStage (47), Statistics(sizeInBytes=16.0 B, rowCount=1)
      +- Exchange (46)
         +- * HashAggregate (45)
            +- * Project (44)
               +- * SortMergeJoin LeftOuter (43)
                  :- * Sort (35)
                  :  +- AQEShuffleRead (34)
                  :     +- ShuffleQueryStage (33), Statistics(sizeInBytes=9.7 KiB, rowCount=248)
                  :        +- Exchange (32)
                  :           +- * Project (31)
                  :              +- * Filter (30)
                  :                 +- * SortMergeJoin FullOuter (29)
                  :                    :- * Sort (20)
                  :                    :  +- AQEShuffleRead (19)
                  :                    :     +- ShuffleQueryStage (18), Statistics(sizeInBytes=2.4 MiB, rowCount=2.11E+4)
                  :                    :        +- Exchange (17)
                  :                    :           +- * Project (16)
                  :                    :              +- * SortMergeJoin FullOuter (15)
                  :                    :                 :- * Sort (6)
                  :                    :                 :  +- AQEShuffleRead (5)
                  :                    :                 :     +- ShuffleQueryStage (4), Statistics(sizeInBytes=2.1 MiB, rowCount=2.11E+4)
                  :                    :                 :        +- Exchange (3)
                  :                    :                 :           +- * Project (2)
                  :                    :                 :              +- Scan csv  (1)
                  :                    :                 +- * Sort (14)
                  :                    :                    +- ObjectHashAggregate (13)
                  :                    :                       +- AQEShuffleRead (12)
                  :                    :                          +- ShuffleQueryStage (11), Statistics(sizeInBytes=53.8 KiB, rowCount=445)
                  :                    :                             +- Exchange (10)
                  :                    :                                +- ObjectHashAggregate (9)
                  :                    :                                   +- * Project (8)
                  :                    :                                      +- Scan csv  (7)
                  :                    +- * Sort (28)
                  :                       +- ObjectHashAggregate (27)
                  :                          +- AQEShuffleRead (26)
                  :                             +- ShuffleQueryStage (25), Statistics(sizeInBytes=43.8 KiB, rowCount=464)
                  :                                +- Exchange (24)
                  :                                   +- ObjectHashAggregate (23)
                  :                                      +- * Project (22)
                  :                                         +- Scan csv  (21)
                  +- * Sort (42)
                     +- AQEShuffleRead (41)
                        +- ShuffleQueryStage (40), Statistics(sizeInBytes=1017.3 KiB, rowCount=2.60E+4)
                           +- Exchange (39)
                              +- * Project (38)
                                 +- * Filter (37)
                                    +- Scan csv  (36)
+- == Initial Plan ==
   HashAggregate (79)
   +- Exchange (78)
      +- HashAggregate (77)
         +- Project (76)
            +- SortMergeJoin LeftOuter (75)
               :- Sort (70)
               :  +- Exchange (69)
               :     +- Project (68)
               :        +- Filter (67)
               :           +- SortMergeJoin FullOuter (66)
               :              :- Sort (60)
               :              :  +- Exchange (59)
               :              :     +- Project (58)
               :              :        +- SortMergeJoin FullOuter (57)
               :              :           :- Sort (51)
               :              :           :  +- Exchange (50)
               :              :           :     +- Project (49)
               :              :           :        +- Scan csv  (1)
               :              :           +- Sort (56)
               :              :              +- ObjectHashAggregate (55)
               :              :                 +- Exchange (54)
               :              :                    +- ObjectHashAggregate (53)
               :              :                       +- Project (52)
               :              :                          +- Scan csv  (7)
               :              +- Sort (65)
               :                 +- ObjectHashAggregate (64)
               :                    +- Exchange (63)
               :                       +- ObjectHashAggregate (62)
               :                          +- Project (61)
               :                             +- Scan csv  (21)
               +- Sort (74)
                  +- Exchange (73)
                     +- Project (72)
                        +- Filter (71)
                           +- Scan csv  (36)


(1) Scan csv 
Output [5]: [FINESS#434259, NO_RSS#434260, NO_ADMIN#434261, MNT_MON#434401, MNT_DMI#434402]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/VISUALVALOSEJOU_440300/420780652.2025.12.MCO.VisualValoSejours.csv]
ReadSchema: struct<FINESS:string,NO_RSS:string,NO_ADMIN:string,MNT_MON:string,MNT_DMI:string>

(2) Project [codegen id : 1]
Output [5]: [FINESS#434259 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, NO_RSS#434260 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, str_to_dec(MNT_MON#434401, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, str_to_dec(MNT_DMI#434402, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, str_remove_space(NO_ADMIN#434261, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Input [5]: [FINESS#434259, NO_RSS#434260, NO_ADMIN#434261, MNT_MON#434401, MNT_DMI#434402]

(3) Exchange
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=43091]

(4) ShuffleQueryStage
Output [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: 0

(5) AQEShuffleRead
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: coalesced

(6) Sort [codegen id : 5]
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

(7) Scan csv 
Output [1]: [_c0#435672]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/DMI_M12_2025_440461/DMI-M12-2025 - BIS.txt]
ReadSchema: struct<_c0:string>

(8) Project [codegen id : 2]
Output [3]: [str_remove_space(str_extract_middle(_c0#435672, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, str_remove_space(str_extract_middle(_c0#435672, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, round(str_to_dec(str_join(array(str_extract_middle(_c0#435672, 83, 7), str_extract_middle(_c0#435672, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783]
Input [1]: [_c0#435672]

(9) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777]
Functions [2]: [partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)]
Aggregate Attributes [2]: [sum#439859, buf#439860]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]

(10) Exchange
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Arguments: hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, 10), ENSURE_REQUIREMENTS, [plan_id=43104]

(11) ShuffleQueryStage
Output [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Arguments: 1

(12) AQEShuffleRead
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Arguments: coalesced

(13) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777]
Functions [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)]
Aggregate Attributes [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783)#435805, collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)#435807]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777 AS 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, round(sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783)#435805, 2) AS 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, size(collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)#435807, true) AS 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]

(14) Sort [codegen id : 6]
Input [3]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]
Arguments: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176 ASC NULLS FIRST], false, 0

(15) SortMergeJoin [codegen id : 7]
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Right keys [1]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176]
Join type: FullOuter
Join condition: None

(16) Project [codegen id : 7]
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179 AS COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Input [8]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]

(17) Exchange
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=43402]

(18) ShuffleQueryStage
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: 4

(19) AQEShuffleRead
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: coalesced

(20) Sort [codegen id : 8]
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

(21) Scan csv 
Output [1]: [_c0#435859]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/MOLECULES_ONERE_440457/MO-M12-2025.txt]
ReadSchema: struct<_c0:string>

(22) Project [codegen id : 3]
Output [3]: [str_remove_space(str_extract_middle(_c0#435859, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, str_remove_space(str_extract_middle(_c0#435859, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, str_to_dec(str_join(array(str_extract_middle(_c0#435859, 65, 7), str_extract_middle(_c0#435859, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000]
Input [1]: [_c0#435859]

(23) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993]
Functions [2]: [partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)]
Aggregate Attributes [2]: [sum#439863, buf#439864]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]

(24) Exchange
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Arguments: hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, 10), ENSURE_REQUIREMENTS, [plan_id=43124]

(25) ShuffleQueryStage
Output [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Arguments: 2

(26) AQEShuffleRead
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Arguments: coalesced

(27) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993]
Functions [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)]
Aggregate Attributes [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000)#436027, collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)#436029]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993 AS MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, round(sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000)#436027, 2) AS MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, size(collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)#436029, true) AS MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]

(28) Sort [codegen id : 9]
Input [3]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]
Arguments: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297 ASC NULLS FIRST], false, 0

(29) SortMergeJoin [codegen id : 10]
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Right keys [1]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297]
Join type: FullOuter
Join condition: None

(30) Filter [codegen id : 10]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]
Condition : (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#438277)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300)) AND (((isnotnull(MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298) AND (COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157 <=> 0.0)) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298 <=> 0.0)) OR (((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158 <=> 0.0) AND isnotnull(COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274)) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274 <=> 0.0))))

(31) Project [codegen id : 10]
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]

(32) Exchange
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, 10), ENSURE_REQUIREMENTS, [plan_id=43572]

(33) ShuffleQueryStage
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: 5

(34) AQEShuffleRead
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: coalesced

(35) Sort [codegen id : 11]
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151 ASC NULLS FIRST], false, 0

(36) Scan csv 
Output [1]: [_c0#436081]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/RSS_M12_2025_440462/RSS- 420780652.txt]
ReadSchema: struct<_c0:string>

(37) Filter [codegen id : 4]
Input [1]: [_c0#436081]
Condition : isnotnull(str_extract_middle(_c0#436081, 28, 20))

(38) Project [codegen id : 4]
Output [1]: [str_extract_middle(_c0#436081, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Input [1]: [_c0#436081]

(39) Exchange
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478, 10), ENSURE_REQUIREMENTS, [plan_id=43145]

(40) ShuffleQueryStage
Output [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: 3

(41) AQEShuffleRead
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: coalesced

(42) Sort [codegen id : 12]
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478 ASC NULLS FIRST], false, 0

(43) SortMergeJoin [codegen id : 13]
Left keys [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Right keys [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Join type: LeftOuter
Join condition: None

(44) Project [codegen id : 13]
Output: []
Input [2]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]

(45) HashAggregate [codegen id : 13]
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#439857L]
Results [1]: [count#439858L]

(46) Exchange
Input [1]: [count#439858L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=43673]

(47) ShuffleQueryStage
Output [1]: [count#439858L]
Arguments: 6

(48) HashAggregate [codegen id : 14]
Input [1]: [count#439858L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#439076L]
Results [1]: [count(1)#439076L AS count#439077L]

(49) Project
Output [5]: [FINESS#434259 AS COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, NO_RSS#434260 AS COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, str_to_dec(MNT_MON#434401, ##.##) AS COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, str_to_dec(MNT_DMI#434402, ##.##) AS COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, str_remove_space(NO_ADMIN#434261, ALL) AS COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Input [5]: [FINESS#434259, NO_RSS#434260, NO_ADMIN#434261, MNT_MON#434401, MNT_DMI#434402]

(50) Exchange
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=42997]

(51) Sort
Input [5]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

(52) Project
Output [3]: [str_remove_space(str_extract_middle(_c0#435672, 12, 20), ALL) AS COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, str_remove_space(str_extract_middle(_c0#435672, 58, 15), ALL) AS COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, round(str_to_dec(str_join(array(str_extract_middle(_c0#435672, 83, 7), str_extract_middle(_c0#435672, 90, 3)), .), ##.##), 2) AS COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783]
Input [1]: [_c0#435672]

(53) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777]
Functions [2]: [partial_sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), partial_collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)]
Aggregate Attributes [2]: [sum#439859, buf#439860]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]

(54) Exchange
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Arguments: hashpartitioning(COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, 10), ENSURE_REQUIREMENTS, [plan_id=42993]

(55) ObjectHashAggregate
Input [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777, sum#439861, buf#439862]
Keys [1]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777]
Functions [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783), collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)]
Aggregate Attributes [2]: [sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783)#435805, collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)#435807]
Results [3]: [COL_C898C1FE_14C4_493A_878B_315EB757695B#435777 AS 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, round(sum(COL_50B93A8F_2FAC_42DA_862D_B1ADBEF5479D#435783)#435805, 2) AS 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, size(collect_list(COL_4F5EF121_C9BA_4BCB_BC00_75827CB8EDA5#435779, 0, 0)#435807, true) AS 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]

(56) Sort
Input [3]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]
Arguments: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176 ASC NULLS FIRST], false, 0

(57) SortMergeJoin
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Right keys [1]: [01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176]
Join type: FullOuter
Join condition: None

(58) Project
Output [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177 AS COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179 AS COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Input [8]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 01_DMI_DECOUPAG_440467_COL_7CC70679_D8A9_4AFA_853C_0B83772BB57F#438176, 01_DMI_DECOUPAG_440467_COL_09C9C0C9_12D2_4B88_AA5D_EE56C4B13EED#438177, 01_DMI_DECOUPAG_440467_COL_0BE82637_88B2_4D38_AB56_F261497B0D57#438179]

(59) Exchange
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: hashpartitioning(COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, 10), ENSURE_REQUIREMENTS, [plan_id=43006]

(60) Sort
Input [7]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277]
Arguments: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160 ASC NULLS FIRST], false, 0

(61) Project
Output [3]: [str_remove_space(str_extract_middle(_c0#435859, 12, 20), ALL) AS COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, str_remove_space(str_extract_middle(_c0#435859, 40, 15), ALL) AS COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, str_to_dec(str_join(array(str_extract_middle(_c0#435859, 65, 7), str_extract_middle(_c0#435859, 72, 3)), .), ##.##) AS COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000]
Input [1]: [_c0#435859]

(62) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993]
Functions [2]: [partial_sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), partial_collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)]
Aggregate Attributes [2]: [sum#439863, buf#439864]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]

(63) Exchange
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Arguments: hashpartitioning(COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, 10), ENSURE_REQUIREMENTS, [plan_id=43002]

(64) ObjectHashAggregate
Input [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993, sum#439865, buf#439866]
Keys [1]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993]
Functions [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000), collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)]
Aggregate Attributes [2]: [sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000)#436027, collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)#436029]
Results [3]: [COL_9979D704_E827_4785_AA6A_46B46DBF4847#435993 AS MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, round(sum(COL_83950E9E_E1A2_40D0_8B87_BED2FABAA685#436000)#436027, 2) AS MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, size(collect_list(COL_88F0CE42_F8A7_4281_8763_D8933C024F46#435995, 0, 0)#436029, true) AS MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]

(65) Sort
Input [3]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]
Arguments: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297 ASC NULLS FIRST], false, 0

(66) SortMergeJoin
Left keys [1]: [COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160]
Right keys [1]: [MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297]
Join type: FullOuter
Join condition: None

(67) Filter
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]
Condition : (((isnull(COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150) OR isnotnull(COL_059961D5_87D2_4745_B270_05C29151B074#438277)) OR isnotnull(MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300)) AND (((isnotnull(MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298) AND (COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157 <=> 0.0)) AND NOT (MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298 <=> 0.0)) OR (((COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158 <=> 0.0) AND isnotnull(COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274)) AND NOT (COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274 <=> 0.0))))

(68) Project
Output [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Input [10]: [COL_439A37D8_DFEE_4DF0_A4A3_80E3EF97FAD9#438150, COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, COL_1951BB46_DAA1_41AE_95E1_5D8AF0DDC5FE#438157, COL_DB68A28C_38CD_4B37_8BFB_36C0ED5BFF25#438158, COL_AA2A4EC8_3B37_49D4_8E9E_4D9406083A09#438160, COL_9AC6AF0B_628B_42CB_AD38_5FA9E8E4EEC4#438274, COL_059961D5_87D2_4745_B270_05C29151B074#438277, MOLECULES_ONERE_440476_COL_F1D11223_61B6_416F_B9C1_A2E3B5D0A062#438297, MOLECULES_ONERE_440476_COL_7FE83AB2_FE45_4E07_ABE2_B7A44F323A8F#438298, MOLECULES_ONERE_440476_COL_0E667C51_6C84_403A_9D26_666830EFDC70#438300]

(69) Exchange
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: hashpartitioning(COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, 10), ENSURE_REQUIREMENTS, [plan_id=43016]

(70) Sort
Input [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Arguments: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151 ASC NULLS FIRST], false, 0

(71) Filter
Input [1]: [_c0#436081]
Condition : isnotnull(str_extract_middle(_c0#436081, 28, 20))

(72) Project
Output [1]: [str_extract_middle(_c0#436081, 28, 20) AS RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Input [1]: [_c0#436081]

(73) Exchange
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: hashpartitioning(RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478, 10), ENSURE_REQUIREMENTS, [plan_id=43017]

(74) Sort
Input [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Arguments: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478 ASC NULLS FIRST], false, 0

(75) SortMergeJoin
Left keys [1]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151]
Right keys [1]: [RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]
Join type: LeftOuter
Join condition: None

(76) Project
Output: []
Input [2]: [COL_B0421838_B608_4AA6_AA2E_38FAEED1F891#438151, RSS_DECOUPAGE_440487_COL_4E5BCDED_FFA3_4620_99A4_CB56AB972321#438478]

(77) HashAggregate
Input: []
Keys: []
Functions [1]: [partial_count(1)]
Aggregate Attributes [1]: [count#439857L]
Results [1]: [count#439858L]

(78) Exchange
Input [1]: [count#439858L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=43024]

(79) HashAggregate
Input [1]: [count#439858L]
Keys: []
Functions [1]: [count(1)]
Aggregate Attributes [1]: [count(1)#439076L]
Results [1]: [count(1)#439076L AS count#439077L]

(80) AdaptiveSparkPlan
Output [1]: [count#439077L]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet