SP1 - Details for Query 799

Submitted Time: 2026/02/24 09:34:34
Duration: 65 ms
Succeeded Jobs: 1027

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: 2 ms"; 2 [labelType="html" label=" Project "]; 3 [labelType="html" label="Filter number of output rows: 144"]; 4 [labelType="html" label="ColumnarToRow number of output rows: 497 number of input batches: 1"]; } 5 [labelType="html" label="Scan parquet number of files read: 1 scan time: 1 ms metadata time: 0 ms size of files read: 4.2 KiB number of output rows: 497"]; 2->0; 3->2; 4->3; 5->4; }

CollectLimit 500000

Project [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 AS COL_C8BF3A29_580E_41A9_B5CC_362A645E5F07#235233, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L AS COL_CE92616D_4249_4FBC_80EA_39B56255CEB6#235234L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217 AS COL_C0FE2A32_C61F_4B82_B6F1_006B1E2B42F5#235235]

Filter (CASE WHEN isnull(((COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 >= 18) AND (COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 <= 34))) THEN false ELSE ((COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 >= 18) AND (COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 <= 34)) END AND CASE WHEN isnull(COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L) THEN false ELSE (COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L > 40) END)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215,COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L,COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217] Batched: true, DataFilters: [CASE WHEN isnull(((COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 >= 18) AND (COL_6921B4DF_4756..., Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/TEMPERATURE_VS__254012], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<COL_6921B4DF_4756_47F0_875E_A2548FBC8003:int,COL_89E4B016_0BC7_4688_8B39_4AF369789A04:bigi...

Details

== Physical Plan ==
CollectLimit (5)
+- * Project (4)
   +- * Filter (3)
      +- * ColumnarToRow (2)
         +- Scan parquet  (1)


(1) Scan parquet 
Output [3]: [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/TEMPERATURE_VS__254012]
ReadSchema: struct<COL_6921B4DF_4756_47F0_875E_A2548FBC8003:int,COL_89E4B016_0BC7_4688_8B39_4AF369789A04:bigint,COL_173B74AF_84DB_4FB5_9923_2435F3241E6D:string>

(2) ColumnarToRow [codegen id : 1]
Input [3]: [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217]

(3) Filter [codegen id : 1]
Input [3]: [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217]
Condition : (CASE WHEN isnull(((COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 >= 18) AND (COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 <= 34))) THEN false ELSE ((COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 >= 18) AND (COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 <= 34)) END AND CASE WHEN isnull(COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L) THEN false ELSE (COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L > 40) END)

(4) Project [codegen id : 1]
Output [3]: [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215 AS COL_C8BF3A29_580E_41A9_B5CC_362A645E5F07#235233, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L AS COL_CE92616D_4249_4FBC_80EA_39B56255CEB6#235234L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217 AS COL_C0FE2A32_C61F_4B82_B6F1_006B1E2B42F5#235235]
Input [3]: [COL_6921B4DF_4756_47F0_875E_A2548FBC8003#235215, COL_89E4B016_0BC7_4688_8B39_4AF369789A04#235216L, COL_173B74AF_84DB_4FB5_9923_2435F3241E6D#235217]

(5) CollectLimit
Input [3]: [COL_C8BF3A29_580E_41A9_B5CC_362A645E5F07#235233, COL_CE92616D_4249_4FBC_80EA_39B56255CEB6#235234L, COL_C0FE2A32_C61F_4B82_B6F1_006B1E2B42F5#235235]
Arguments: 500000

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet