hdh-spark-job - Details for Query 804

Submitted Time: 2026/10/22 15:41:45
Duration: 65 ms
Succeeded Jobs: 812

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: 0 ms"; 2 [labelType="html" label=" Project "]; 3 [labelType="html" label="ColumnarToRow number of output rows: 4,096 number of input batches: 1"]; } 4 [labelType="html" label="Scan parquet number of files read: 1 scan time: 9 ms metadata time: 0 ms size of files read: 910.0 KiB number of output rows: 4,096"]; 2->0; 3->2; 4->3; }

CollectLimit 1500

Project [AT_NUMDOS_438023#189458 AS COL_499EB20D_E124_400E_A51E_F020884D243F#190731, AT_STAPAT_438024#189490 AS COL_A9D76CF5_2B72_4797_8162_50D3EC4025F5#190732, AT_PREFIX_438025#189483 AS COL_2B7D9197_D824_41CE_9E49_C420C3822425#190733, AT_NOMPAT_438026#189453 AS COL_AA64E8D3_7ED0_414B_866A_8621D0D7CAAF#190734, AT_PNOPAT_438027#189479 AS COL_7FAA4F01_934E_4C63_AFB7_19B1EC986E3E#190735, AT_NJFPAT_438028#189450 AS COL_E0620654_CC4B_4BA2_B4D0_92A6E070406C#190736, AT_DATNAI_438029#189417 AS COL_CB4739E1_E2F6_48D7_9602_079F56460634#190737, AT_DATNAISS_438030#189416 AS COL_FBA511B8_2EA0_43D6_A93A_34E57C0BCFE9#190738, AT_SEXE_438031#189486 AS COL_8AAB4531_E92B_4285_8733_BB3FCA8B2F33#190739, AT_CODPOS_438041#189390 AS COL_9C4B35A3_D42F_438A_9AA2_60B2260EE33B#190740, AT_VILLE_438042#189506 AS COL_63F964BD_E655_4F23_9E40_0A336B9ABF65#190741]

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [AT_CODPOS_438041#189390,AT_DATNAISS_438030#189416,AT_DATNAI_438029#189417,AT_NJFPAT_438028#189450,AT_NOMPAT_438026#189453,AT_NUMDOS_438023#189458,AT_PNOPAT_438027#189479,AT_PREFIX_438025#189483,AT_SEXE_438031#189486,AT_STAPAT_438024#189490,AT_VILLE_438042#189506] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/uet/H_PATIEN_438022], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<AT_CODPOS_438041:string,AT_DATNAISS_438030:string,AT_DATNAI_438029:string,AT_NJFPAT_438028...

Details

== Physical Plan ==
CollectLimit (4)
+- * Project (3)
   +- * ColumnarToRow (2)
      +- Scan parquet  (1)


(1) Scan parquet 
Output [11]: [AT_CODPOS_438041#189390, AT_DATNAISS_438030#189416, AT_DATNAI_438029#189417, AT_NJFPAT_438028#189450, AT_NOMPAT_438026#189453, AT_NUMDOS_438023#189458, AT_PNOPAT_438027#189479, AT_PREFIX_438025#189483, AT_SEXE_438031#189486, AT_STAPAT_438024#189490, AT_VILLE_438042#189506]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/uet/H_PATIEN_438022]
ReadSchema: struct<AT_CODPOS_438041:string,AT_DATNAISS_438030:string,AT_DATNAI_438029:string,AT_NJFPAT_438028:string,AT_NOMPAT_438026:string,AT_NUMDOS_438023:string,AT_PNOPAT_438027:string,AT_PREFIX_438025:string,AT_SEXE_438031:string,AT_STAPAT_438024:string,AT_VILLE_438042:string>

(2) ColumnarToRow [codegen id : 1]
Input [11]: [AT_CODPOS_438041#189390, AT_DATNAISS_438030#189416, AT_DATNAI_438029#189417, AT_NJFPAT_438028#189450, AT_NOMPAT_438026#189453, AT_NUMDOS_438023#189458, AT_PNOPAT_438027#189479, AT_PREFIX_438025#189483, AT_SEXE_438031#189486, AT_STAPAT_438024#189490, AT_VILLE_438042#189506]

(3) Project [codegen id : 1]
Output [11]: [AT_NUMDOS_438023#189458 AS COL_499EB20D_E124_400E_A51E_F020884D243F#190731, AT_STAPAT_438024#189490 AS COL_A9D76CF5_2B72_4797_8162_50D3EC4025F5#190732, AT_PREFIX_438025#189483 AS COL_2B7D9197_D824_41CE_9E49_C420C3822425#190733, AT_NOMPAT_438026#189453 AS COL_AA64E8D3_7ED0_414B_866A_8621D0D7CAAF#190734, AT_PNOPAT_438027#189479 AS COL_7FAA4F01_934E_4C63_AFB7_19B1EC986E3E#190735, AT_NJFPAT_438028#189450 AS COL_E0620654_CC4B_4BA2_B4D0_92A6E070406C#190736, AT_DATNAI_438029#189417 AS COL_CB4739E1_E2F6_48D7_9602_079F56460634#190737, AT_DATNAISS_438030#189416 AS COL_FBA511B8_2EA0_43D6_A93A_34E57C0BCFE9#190738, AT_SEXE_438031#189486 AS COL_8AAB4531_E92B_4285_8733_BB3FCA8B2F33#190739, AT_CODPOS_438041#189390 AS COL_9C4B35A3_D42F_438A_9AA2_60B2260EE33B#190740, AT_VILLE_438042#189506 AS COL_63F964BD_E655_4F23_9E40_0A336B9ABF65#190741]
Input [11]: [AT_CODPOS_438041#189390, AT_DATNAISS_438030#189416, AT_DATNAI_438029#189417, AT_NJFPAT_438028#189450, AT_NOMPAT_438026#189453, AT_NUMDOS_438023#189458, AT_PNOPAT_438027#189479, AT_PREFIX_438025#189483, AT_SEXE_438031#189486, AT_STAPAT_438024#189490, AT_VILLE_438042#189506]

(4) CollectLimit
Input [11]: [COL_499EB20D_E124_400E_A51E_F020884D243F#190731, COL_A9D76CF5_2B72_4797_8162_50D3EC4025F5#190732, COL_2B7D9197_D824_41CE_9E49_C420C3822425#190733, COL_AA64E8D3_7ED0_414B_866A_8621D0D7CAAF#190734, COL_7FAA4F01_934E_4C63_AFB7_19B1EC986E3E#190735, COL_E0620654_CC4B_4BA2_B4D0_92A6E070406C#190736, COL_CB4739E1_E2F6_48D7_9602_079F56460634#190737, COL_FBA511B8_2EA0_43D6_A93A_34E57C0BCFE9#190738, COL_8AAB4531_E92B_4285_8733_BB3FCA8B2F33#190739, COL_9C4B35A3_D42F_438A_9AA2_60B2260EE33B#190740, COL_63F964BD_E655_4F23_9E40_0A336B9ABF65#190741]
Arguments: 1500

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet