hdh-spark-job - Details for Query 1583

Submitted Time: 2026/09/01 09:15:07
Duration: 0.1 s
Succeeded Jobs: 2043

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time: 1 ms number of written files: 1 job commit time: 5 ms number of output rows: 247 number of dynamic part: 0 written output: 7.2 KiB"]; 2 [labelType="html" label=" WriteFiles "]; 3 [labelType="html" label="InMemoryTableScan number of output rows: 247"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: 0 ms"; 5 [labelType="html" label=" Project "]; } 6 [labelType="html" label="Scan csv number of output rows: 0"]; 1->0; 2->1; 3->2; 5->3; 6->5; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand file:/data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0, false, Parquet, [path=file:///data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0], ErrorIfExists, [04F95267_1705_42D1_B45F_796778D83A05, B7BA1423_058A_4625_862B_CF74655E19FB, FD478E7F_8CFD_46D4_8174_9A79B486E028, 03DF3B94_AF38_46D7_A925_98AD85C3B15D]

WriteFiles

InMemoryTableScan [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]

Project [Pays#339770 AS 04F95267_1705_42D1_B45F_796778D83A05#339847, ISO 3166-1 alpha-2#339771 AS B7BA1423_058A_4625_862B_CF74655E19FB#339848, ISO 3166-1 alpha-3#339772 AS FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, ISO 3166-1 numérique#339773 AS 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]

WholeStageCodegen (1)

FileScan csv [Pays#339770,ISO 3166-1 alpha-2#339771,ISO 3166-1 alpha-3#339772,ISO 3166-1 numérique#339773] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/execution/515F75A7_03A7_461A_9E59_052C6B70C..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Pays:string,ISO 3166-1 alpha-2:string,ISO 3166-1 alpha-3:string,ISO 3166-1 numérique:string>

Details

== Physical Plan ==
AdaptiveSparkPlan (10)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (7)
   +- WriteFiles (6)
      +- TableCacheQueryStage (5), Statistics(sizeInBytes=8.3 KiB, rowCount=247)
         +- InMemoryTableScan (1)
               +- InMemoryRelation (2)
                     +- * Project (4)
                        +- Scan csv  (3)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (9)
   +- WriteFiles (8)
      +- InMemoryTableScan (1)
            +- InMemoryRelation (2)
                  +- * Project (4)
                     +- Scan csv  (3)


(1) InMemoryTableScan
Output [4]: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]
Arguments: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]

(2) InMemoryRelation
Arguments: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@68f4cb77,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [Pays#339770 AS 04F95267_1705_42D1_B45F_796778D83A05#339847, ISO 3166-1 alpha-2#339771 AS B7BA1423_058A_4625_862B_CF74655E19FB#339848, ISO 3166-1 alpha-3#339772 AS FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, ISO 3166-1 numérique#339773 AS 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]
+- FileScan csv [Pays#339770,ISO 3166-1 alpha-2#339771,ISO 3166-1 alpha-3#339772,ISO 3166-1 numérique#339773] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/execution/515F75A7_03A7_461A_9E59_052C6B70C..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Pays:string,ISO 3166-1 alpha-2:string,ISO 3166-1 alpha-3:string,ISO 3166-1 numérique:string>
,None)

(3) Scan csv 
Output [4]: [Pays#339770, ISO 3166-1 alpha-2#339771, ISO 3166-1 alpha-3#339772, ISO 3166-1 numérique#339773]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/execution/515F75A7_03A7_461A_9E59_052C6B70CA3C/current/pays.csv]
ReadSchema: struct<Pays:string,ISO 3166-1 alpha-2:string,ISO 3166-1 alpha-3:string,ISO 3166-1 numérique:string>

(4) Project [codegen id : 1]
Output [4]: [Pays#339770 AS 04F95267_1705_42D1_B45F_796778D83A05#339847, ISO 3166-1 alpha-2#339771 AS B7BA1423_058A_4625_862B_CF74655E19FB#339848, ISO 3166-1 alpha-3#339772 AS FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, ISO 3166-1 numérique#339773 AS 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]
Input [4]: [Pays#339770, ISO 3166-1 alpha-2#339771, ISO 3166-1 alpha-3#339772, ISO 3166-1 numérique#339773]

(5) TableCacheQueryStage
Output [4]: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]
Arguments: 0

(6) WriteFiles
Input [4]: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]

(7) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0, false, Parquet, [path=file:///data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0], ErrorIfExists, [04F95267_1705_42D1_B45F_796778D83A05, B7BA1423_058A_4625_862B_CF74655E19FB, FD478E7F_8CFD_46D4_8174_9A79B486E028, 03DF3B94_AF38_46D7_A925_98AD85C3B15D]

(8) WriteFiles
Input [4]: [04F95267_1705_42D1_B45F_796778D83A05#339847, B7BA1423_058A_4625_862B_CF74655E19FB#339848, FD478E7F_8CFD_46D4_8174_9A79B486E028#339849, 03DF3B94_AF38_46D7_A925_98AD85C3B15D#339850]

(9) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0, false, Parquet, [path=file:///data/data/subscriptions/NOMENCLATURES_A_425642/35d44bf7-7599-4ebe-a250-a94299c0aaf1/2026-09-01_09-15-07-471_LOCK_b1770ae3-eaa3-4912-9a46-b81eed3489e0], ErrorIfExists, [04F95267_1705_42D1_B45F_796778D83A05, B7BA1423_058A_4625_862B_CF74655E19FB, FD478E7F_8CFD_46D4_8174_9A79B486E028, 03DF3B94_AF38_46D7_A925_98AD85C3B15D]

(10) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet