SP1 - Details for Query 629

Submitted Time: 2026/08/28 12:28:33
Duration: 8 s
Succeeded Jobs: 747 748 751 752

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="ObjectHashAggregate number of output rows: 54 time in aggregation build: 22 ms spill size: 0.0 B number of sort fallback tasks: 0"]; 2 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 4.1 KiB number of coalesced partitions: 1"]; 3 [labelType="html" label="Exchange shuffle records written: 54 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 54 local bytes read: 3.9 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 7.9 KiB local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 3.9 KiB"]; 4 [labelType="html" label="ObjectHashAggregate number of output rows: 54 time in aggregation build: 161 ms spill size: 0.0 B number of sort fallback tasks: 0"]; subgraph cluster5 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: 161 ms"; 6 [labelType="html" label=" Project "]; 7 [labelType="html" label="SortMergeJoin number of output rows: 41,179 spill size: 0.0 B"]; } subgraph cluster8 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 231 ms"; 9 [labelType="html" label="Sort sort time: 8 ms peak memory: 17.0 MiB spill size: 0.0 B"]; } 10 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 476.3 KiB number of coalesced partitions: 1"]; 11 [labelType="html" label="Exchange shuffle records written: 41,179 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 6 ms (1 ms, 4 ms, 4 ms (stage 1220.0: task 1674)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 41,179 local bytes read: 453.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 1930.3 KiB (293.7 KiB, 1636.6 KiB, 1636.6 KiB (stage 1220.0: task 1674)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 453.6 KiB (69.5 KiB, 384.0 KiB, 384.0 KiB (stage 1220.0: task 1674))"]; subgraph cluster12 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n9.0 s (4 ms, 6 ms, 7.1 s (stage 1220.0: task 1674))"; 13 [labelType="html" label=" Project "]; 14 [labelType="html" label="Filter number of output rows: 41,179"]; 15 [labelType="html" label="ColumnarToRow number of output rows: 2,996,009 number of input batches: 732"]; } 16 [labelType="html" label="Scan parquet number of files read: 1 scan time total (min, med, max (stageId: taskId)) 1.0 s (2 ms, 5 ms, 807 ms (stage 1220.0: task 1674)) metadata time: 0 ms size of files read: 150.9 MiB number of output rows: 2,996,009"]; subgraph cluster17 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: 0 ms"; 18 [labelType="html" label=" Project "]; 19 [labelType="html" label="Filter number of output rows: 83,244"]; } 20 [labelType="html" label="Window spill size: 0.0 B"]; subgraph cluster21 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: 0 ms"; 22 [labelType="html" label="Sort sort time: 12 ms peak memory: 20.0 MiB spill size: 0.0 B"]; } 23 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 1113.9 KiB number of coalesced partitions: 1"]; 24 [labelType="html" label="Exchange shuffle records written: 93,005 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 6 ms (2 ms, 3 ms, 3 ms (stage 1221.0: task 1680)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 93,005 local bytes read: 1057.6 KiB fetch wait time: 0 ms remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 2 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 6.3 MiB (2.6 MiB, 3.7 MiB, 3.7 MiB (stage 1221.0: task 1680)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 1057.6 KiB (437.9 KiB, 619.8 KiB, 619.8 KiB (stage 1221.0: task 1680))"]; subgraph cluster25 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n307 ms (150 ms, 157 ms, 157 ms (stage 1221.0: task 1681))"; 26 [labelType="html" label=" Project "]; 27 [labelType="html" label="Filter number of output rows: 93,005"]; } 28 [labelType="html" label="Scan csv number of output rows: 93,005 number of files read: 1 metadata time: 0 ms size of files read: 6.4 MiB"]; 1->0; 2->1; 3->2; 4->3; 6->4; 7->6; 9->7; 10->9; 11->10; 13->11; 14->13; 15->14; 16->15; 18->7; 19->18; 20->19; 22->20; 23->22; 24->23; 26->24; 27->26; 28->27; }

AdaptiveSparkPlan isFinalPlan=true

ObjectHashAggregate(keys=[COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868], functions=[collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), count(1)])

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, 10), ENSURE_REQUIREMENTS, [plan_id=30927]

ObjectHashAggregate(keys=[COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868], functions=[partial_collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), partial_count(1)])

Project [00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965 AS COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]

SortMergeJoin [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671], [00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964], LeftOuter

WholeStageCodegen (6)

Sort [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671 ASC NULLS FIRST], false, 0

WholeStageCodegen (3)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, 10), ENSURE_REQUIREMENTS, [plan_id=30731]

Project [COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567 AS COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, str_join(array(str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE), str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 68, 2), null), ONLY_ONE)), -) AS COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]

Filter (str_if_with_rule((((str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SAS) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUF)) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUN)), 1, 0) <=> 1)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561,COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564,COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567] Batched: true, DataFilters: [(str_if_with_rule((((str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F3..., Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/00_A_LIGNES_RSF_345253], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<COL_B706A975_F287_4DC9_B631_A35735FE936F:string,COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38:s...

Project [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 AS 00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506 AS 00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965]

Filter (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521 <=> 1)

WholeStageCodegen (5)

Window [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505], [1 DESC NULLS LAST]

Sort [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 10), ENSURE_REQUIREMENTS, [plan_id=30747]

Project [Numéro Séjour#110384 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, UF#110385 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]

Filter (NOT CASE WHEN isnull(StartsWith(Numéro Séjour#110384, #)) THEN false ELSE StartsWith(Numéro Séjour#110384, #) END AND isnotnull(Numéro Séjour#110384))

WholeStageCodegen (2)

FileScan csv [Numéro Séjour#110384,UF#110385] Batched: false, DataFilters: [NOT CASE WHEN isnull(StartsWith(Numéro Séjour#110384, #)) THEN false ELSE StartsWith(Numéro Séjo..., Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/02_PASSAGE_PATI_344985/passage_pationt_..., PartitionFilters: [], PushedFilters: [IsNotNull(Numéro Séjour)], ReadSchema: struct<Numéro Séjour:string,UF:string>

Details

== Physical Plan ==
AdaptiveSparkPlan (43)
+- == Final Plan ==
   ObjectHashAggregate (25)
   +- AQEShuffleRead (24)
      +- ShuffleQueryStage (23), Statistics(sizeInBytes=7.9 KiB, rowCount=54)
         +- Exchange (22)
            +- ObjectHashAggregate (21)
               +- * Project (20)
                  +- * SortMergeJoin LeftOuter (19)
                     :- * Sort (8)
                     :  +- AQEShuffleRead (7)
                     :     +- ShuffleQueryStage (6), Statistics(sizeInBytes=1930.3 KiB, rowCount=4.12E+4)
                     :        +- Exchange (5)
                     :           +- * Project (4)
                     :              +- * Filter (3)
                     :                 +- * ColumnarToRow (2)
                     :                    +- Scan parquet  (1)
                     +- * Project (18)
                        +- * Filter (17)
                           +- Window (16)
                              +- * Sort (15)
                                 +- AQEShuffleRead (14)
                                    +- ShuffleQueryStage (13), Statistics(sizeInBytes=6.3 MiB, rowCount=9.30E+4)
                                       +- Exchange (12)
                                          +- * Project (11)
                                             +- * Filter (10)
                                                +- Scan csv  (9)
+- == Initial Plan ==
   CollectLimit (42)
   +- ObjectHashAggregate (41)
      +- Exchange (40)
         +- ObjectHashAggregate (39)
            +- Project (38)
               +- SortMergeJoin LeftOuter (37)
                  :- Sort (29)
                  :  +- Exchange (28)
                  :     +- Project (27)
                  :        +- Filter (26)
                  :           +- Scan parquet  (1)
                  +- Project (36)
                     +- Filter (35)
                        +- Window (34)
                           +- Sort (33)
                              +- Exchange (32)
                                 +- Project (31)
                                    +- Filter (30)
                                       +- Scan csv  (9)


(1) Scan parquet 
Output [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/00_A_LIGNES_RSF_345253]
ReadSchema: struct<COL_B706A975_F287_4DC9_B631_A35735FE936F:string,COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38:string,COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D:string>

(2) ColumnarToRow [codegen id : 1]
Input [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]

(3) Filter [codegen id : 1]
Input [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]
Condition : (str_if_with_rule((((str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SAS) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUF)) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUN)), 1, 0) <=> 1)

(4) Project [codegen id : 1]
Output [2]: [COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567 AS COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, str_join(array(str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE), str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 68, 2), null), ONLY_ONE)), -) AS COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Input [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]

(5) Exchange
Input [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: hashpartitioning(COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, 10), ENSURE_REQUIREMENTS, [plan_id=30731]

(6) ShuffleQueryStage
Output [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: 0

(7) AQEShuffleRead
Input [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: coalesced

(8) Sort [codegen id : 3]
Input [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671 ASC NULLS FIRST], false, 0

(9) Scan csv 
Output [2]: [Numéro Séjour#110384, UF#110385]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/02_PASSAGE_PATI_344985/passage_pationt_09122025.csv]
PushedFilters: [IsNotNull(Numéro Séjour)]
ReadSchema: struct<Numéro Séjour:string,UF:string>

(10) Filter [codegen id : 2]
Input [2]: [Numéro Séjour#110384, UF#110385]
Condition : (NOT CASE WHEN isnull(StartsWith(Numéro Séjour#110384, #)) THEN false ELSE StartsWith(Numéro Séjour#110384, #) END AND isnotnull(Numéro Séjour#110384))

(11) Project [codegen id : 2]
Output [2]: [Numéro Séjour#110384 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, UF#110385 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Input [2]: [Numéro Séjour#110384, UF#110385]

(12) Exchange
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 10), ENSURE_REQUIREMENTS, [plan_id=30747]

(13) ShuffleQueryStage
Output [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: 1

(14) AQEShuffleRead
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: coalesced

(15) Sort [codegen id : 4]
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(16) Window
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505], [1 DESC NULLS LAST]

(17) Filter [codegen id : 5]
Input [3]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521]
Condition : (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521 <=> 1)

(18) Project [codegen id : 5]
Output [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 AS 00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506 AS 00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965]
Input [3]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521]

(19) SortMergeJoin [codegen id : 6]
Left keys [1]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671]
Right keys [1]: [00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964]
Join type: LeftOuter
Join condition: None

(20) Project [codegen id : 6]
Output [2]: [00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965 AS COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Input [4]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, 00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964, 00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965]

(21) ObjectHashAggregate
Input [2]: [COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Keys [1]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Functions [2]: [partial_collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), partial_count(1)]
Aggregate Attributes [2]: [buf#111185, count#111186L]
Results [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]

(22) Exchange
Input [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Arguments: hashpartitioning(COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, 10), ENSURE_REQUIREMENTS, [plan_id=30927]

(23) ShuffleQueryStage
Output [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Arguments: 2

(24) AQEShuffleRead
Input [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Arguments: coalesced

(25) ObjectHashAggregate
Input [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Keys [1]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Functions [2]: [collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), count(1)]
Aggregate Attributes [2]: [collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0)#111166, count(1)#111168L]
Results [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868 AS COL_08B36B57_E713_433B_BB5A_9E64E1F73064#111179, collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0)#111166 AS COL_87FCA531_4352_4B9B_A83D_ED08133548A3#111180, count(1)#111168L AS COL_C7B423E7_9E6A_4FA0_A897_D3FCFABFA012#111181L]

(26) Filter
Input [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]
Condition : (str_if_with_rule((((str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SAS) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUF)) OR (str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE) <=> SUN)), 1, 0) <=> 1)

(27) Project
Output [2]: [COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567 AS COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, str_join(array(str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 78, 5), null), ONLY_ONE), str_remove_space(str_if_with_rule(((COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> B) OR (COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564 <=> C)), str_extract_middle(COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, 68, 2), null), ONLY_ONE)), -) AS COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Input [3]: [COL_B706A975_F287_4DC9_B631_A35735FE936F#110561, COL_1ADA21FB_DB08_4A1F_986E_E8E363B14F38#110564, COL_1CCAED33_3E69_435E_B522_DAE3B9E63F9D#110567]

(28) Exchange
Input [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: hashpartitioning(COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, 10), ENSURE_REQUIREMENTS, [plan_id=30681]

(29) Sort
Input [2]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Arguments: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671 ASC NULLS FIRST], false, 0

(30) Filter
Input [2]: [Numéro Séjour#110384, UF#110385]
Condition : (NOT CASE WHEN isnull(StartsWith(Numéro Séjour#110384, #)) THEN false ELSE StartsWith(Numéro Séjour#110384, #) END AND isnotnull(Numéro Séjour#110384))

(31) Project
Output [2]: [Numéro Séjour#110384 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, UF#110385 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Input [2]: [Numéro Séjour#110384, UF#110385]

(32) Exchange
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 10), ENSURE_REQUIREMENTS, [plan_id=30674]

(33) Sort
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(34) Window
Input [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506]
Arguments: [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505], [1 DESC NULLS LAST]

(35) Filter
Input [3]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521]
Condition : (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521 <=> 1)

(36) Project
Output [2]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505 AS 00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506 AS 00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965]
Input [3]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110505, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110506, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#110521]

(37) SortMergeJoin
Left keys [1]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671]
Right keys [1]: [00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964]
Join type: LeftOuter
Join condition: None

(38) Project
Output [2]: [00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965 AS COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Input [4]: [COL_136DCAA3_2545_47C9_A61E_86ED7914066D#110671, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, 00_C_PASSAGE_PA_345648_COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#110964, 00_C_PASSAGE_PA_345648_COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#110965]

(39) ObjectHashAggregate
Input [2]: [COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Keys [1]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Functions [2]: [partial_collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), partial_count(1)]
Aggregate Attributes [2]: [buf#111185, count#111186L]
Results [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]

(40) Exchange
Input [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Arguments: hashpartitioning(COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, 10), ENSURE_REQUIREMENTS, [plan_id=30687]

(41) ObjectHashAggregate
Input [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868, buf#111187, count#111188L]
Keys [1]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868]
Functions [2]: [collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0), count(1)]
Aggregate Attributes [2]: [collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0)#111166, count(1)#111168L]
Results [3]: [COL_E75FD19A_D578_4037_B330_0F4E587CAF25#110868 AS COL_08B36B57_E713_433B_BB5A_9E64E1F73064#111179, collect_set(COL_AF978AAB_D361_4C61_94AF_432EBFAA8810#111119, 0, 0)#111166 AS COL_87FCA531_4352_4B9B_A83D_ED08133548A3#111180, count(1)#111168L AS COL_C7B423E7_9E6A_4FA0_A897_D3FCFABFA012#111181L]

(42) CollectLimit
Input [3]: [COL_08B36B57_E713_433B_BB5A_9E64E1F73064#111179, COL_87FCA531_4352_4B9B_A83D_ED08133548A3#111180, COL_C7B423E7_9E6A_4FA0_A897_D3FCFABFA012#111181L]
Arguments: 1500

(43) AdaptiveSparkPlan
Output [3]: [COL_08B36B57_E713_433B_BB5A_9E64E1F73064#111179, COL_87FCA531_4352_4B9B_A83D_ED08133548A3#111180, COL_C7B423E7_9E6A_4FA0_A897_D3FCFABFA012#111181L]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet