digraph G {
0 [labelType="html" label="<br><b>AdaptiveSparkPlan</b><br><br>"];
1 [labelType="html" label="<b>Execute InsertIntoHadoopFsRelationCommand</b><br><br>task commit time: 1 ms<br>number of written files: 1<br>job commit time: 6 ms<br>number of output rows: 47,008<br>number of dynamic part: 0<br>written output: 3.7 MiB"];
2 [labelType="html" label="<br><b>WriteFiles</b><br><br>"];
3 [labelType="html" label="<b>Exchange</b><br><br>shuffle records written: 47,008<br>local merged chunks fetched: 0<br>shuffle write time: 9 ms<br>remote merged bytes read: 0.0 B<br>local merged blocks fetched: 0<br>corrupt merged block chunks: 0<br>remote merged reqs duration: 0 ms<br>remote merged blocks fetched: 0<br>records read: 47,008<br>local bytes read: 2.9 MiB<br>fetch wait time: 0 ms<br>remote bytes read: 0.0 B<br>merged fetch fallback count: 0<br>local blocks read: 1<br>remote merged chunks fetched: 0<br>remote blocks read: 0<br>data size: 8.4 MiB<br>local merged bytes read: 0.0 B<br>number of partitions: 1<br>remote reqs duration: 0 ms<br>remote bytes read to disk: 0.0 B<br>shuffle bytes written: 2.9 MiB"];
subgraph cluster4 {
isCluster="true";
label="WholeStageCodegen (1)\n \nduration: 423 ms";
5 [labelType="html" label="<b>Scan ExistingRDD</b><br><br>number of output rows: 47,008"];
}
1->0;
2->1;
3->2;
5->3;
}
6
AdaptiveSparkPlan isFinalPlan=true
Execute InsertIntoHadoopFsRelationCommand file:/data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, false, CSV, [path=file:///data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, nullValue=, ignoreLeadingWhiteSpace=true, quoteAll=false, sep=;, quote=", emptyValue=, ignoreTrailingWhiteSpace=true, escape=\, charset=UTF-8, lineSep=
, header=true], ErrorIfExists, [at_numdos, at_stapat, at_prefix, at_nompat, at_pnopat, at_njfpat, at_datnai, at_datnaiss, at_sexe, ROW_NUMBER, NewNumDos]
WriteFiles
Exchange SinglePartition, REPARTITION_BY_NUM, [plan_id=40919]
Scan ExistingRDD[at_numdos#785105,at_stapat#785106,at_prefix#785107,at_nompat#785108,at_pnopat#785109,at_njfpat#785110,at_datnai#785111,at_datnaiss#785112,at_sexe#785113,ROW_NUMBER#785114,NewNumDos#785115]
WholeStageCodegen (1)
== Physical Plan ==
AdaptiveSparkPlan (9)
+- == Final Plan ==
Execute InsertIntoHadoopFsRelationCommand (5)
+- WriteFiles (4)
+- ShuffleQueryStage (3), Statistics(sizeInBytes=8.4 MiB, rowCount=4.70E+4)
+- Exchange (2)
+- * Scan ExistingRDD (1)
+- == Initial Plan ==
Execute InsertIntoHadoopFsRelationCommand (8)
+- WriteFiles (7)
+- Exchange (6)
+- Scan ExistingRDD (1)
(1) Scan ExistingRDD [codegen id : 1]
Output [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
Arguments: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115], MapPartitionsRDD[6522] at createDataFrame at AbsExportExecutor.java:55, ExistingRDD, UnknownPartitioning(0)
(2) Exchange
Input [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
Arguments: SinglePartition, REPARTITION_BY_NUM, [plan_id=40919]
(3) ShuffleQueryStage
Output [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
Arguments: 0
(4) WriteFiles
Input [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
(5) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, false, CSV, [path=file:///data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, nullValue=, ignoreLeadingWhiteSpace=true, quoteAll=false, sep=;, quote=", emptyValue=, ignoreTrailingWhiteSpace=true, escape=\, charset=UTF-8, lineSep=
, header=true], ErrorIfExists, [at_numdos, at_stapat, at_prefix, at_nompat, at_pnopat, at_njfpat, at_datnai, at_datnaiss, at_sexe, ROW_NUMBER, NewNumDos]
(6) Exchange
Input [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
Arguments: SinglePartition, REPARTITION_BY_NUM, [plan_id=40912]
(7) WriteFiles
Input [11]: [at_numdos#785105, at_stapat#785106, at_prefix#785107, at_nompat#785108, at_pnopat#785109, at_njfpat#785110, at_datnai#785111, at_datnaiss#785112, at_sexe#785113, ROW_NUMBER#785114, NewNumDos#785115]
(8) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, false, CSV, [path=file:///data/output/export/csv/e7f5a6ec-ca67-4917-b5a7-88489d38a1bf, nullValue=, ignoreLeadingWhiteSpace=true, quoteAll=false, sep=;, quote=", emptyValue=, ignoreTrailingWhiteSpace=true, escape=\, charset=UTF-8, lineSep=
, header=true], ErrorIfExists, [at_numdos, at_stapat, at_prefix, at_nompat, at_pnopat, at_njfpat, at_datnai, at_datnaiss, at_sexe, ROW_NUMBER, NewNumDos]
(9) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true