| Stage Id ▾ | Description | Submitted | Duration | Tasks: Succeeded/Total | Input | Output | Shuffle Read | Shuffle Write | Failure Reason |
|---|---|---|---|---|---|---|---|---|---|
| 2865 | DataFrameCommonJob_checkCSVSource csv at DataFrameCommonJob.java:312 org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:444) com.adb.hdh.spark.job.impl.DataFrameCommonJob.readCsvFile(DataFrameCommonJob.java:312) com.adb.hdh.spark.job.impl.DataFrameCommonJob.readCsvAndGetDataSet(DataFrameCommonJob.java:193) com.adb.hdh.spark.job.impl.DataFrameCommonJob.checkCsvFile(DataFrameCommonJob.java:239) com.adb.hdh.spark.job.server.JobDispatcher.dispatch(JobDispatcher.java:216) com.adb.hdh.spark.job.server.servlet.JobSubmitHandler.runJob(JobSubmitHandler.java:76) com.adb.hdh.spark.job.server.servlet.JobSubmitHandler.lambda$handle$0(JobSubmitHandler.java:59) java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) java.base/java.util.concurrent.FutureTask.run(Unknown Source) java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) java.base/java.lang.Thread.run(Unknown Source) | 2026/09/01 06:50:11 | 2 s |
0/1
(1 failed)
| 345.0 MiB | Job aborted due to stage failure: Task 0 in stage 2865.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2865.0 (TID 25549) (100d7a05236f executor driver): com.univocity.parsers.common.TextParsingException: java.lang.ArrayIndexOutOfBoundsException - nullJob aborted due to stage failure: Task 0 in stage 2865.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2865.0 (TID 25549) (100d7a05236f executor driver): com.univocity.parsers.common.TextParsingException: java.lang.ArrayIndexOutOfBoundsException - null Parser Configuration: CsvParserSettings: Auto configuration enabled=true Auto-closing enabled=true Autodetect column delimiter=false Autodetect quotes=false Column reordering enabled=true Delimiters for detection=null Empty value= Escape unquoted values=false Header extraction enabled=null Headers=null Ignore leading whitespaces=false Ignore leading whitespaces in quotes=false Ignore trailing whitespaces=false Ignore trailing whitespaces in quotes=false Input buffer size=1048576 Input reading on separate thread=false Keep escape sequences=false Keep quotes=false Length of content displayed on error=1000 Line separator detection enabled=true Maximum number of characters per column=-1 Maximum number of columns=20480 Normalize escaped line separators=true Null value= Number of records to read=all Processor=none Restricting data in exceptions=false RowProcessor error handler=null Selected fields=none Skip bits as whitespace=true Skip empty lines=true Unescaped quote handling=STOP_AT_DELIMITERFormat configuration: CsvFormat: Comment character=# Field delimiter=, Line separator (normalized)=\n Line separator sequence=\n Quote character=" Quote escape character=\ Quote escape escape character=null Internal state when error was thrown: line=38938, column=20481, record=38938, charIndex=361731462, headers=[#Code_commune_INSEE, Nom_de_la_commune, Code_postal, Libellé_d_acheminement, Ligne_5, _contours_commune.geometry] at com.univocity.parsers.common.AbstractParser.handleException(AbstractParser.java:402) at com.univocity.parsers.common.AbstractParser.parseNext(AbstractParser.java:623) at org.apache.spark.sql.catalyst.csv.UnivocityParser$$anon$1.next(UnivocityParser.scala:433) at scala.collection.Iterator$$anon$11.next(Iterator.scala:496) at scala.collection.Iterator.foreach(Iterator.scala:943) at scala.collection.Iterator.foreach$(Iterator.scala:943) at scala.collection.AbstractIterator.foreach(Iterator.scala:1431) at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199) at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192) at scala.collection.AbstractIterator.foldLeft(Iterator.scala:1431) at scala.collection.TraversableOnce.aggregate(TraversableOnce.scala:260) at scala.collection.TraversableOnce.aggregate$(TraversableOnce.scala:260) at scala.collection.AbstractIterator.aggregate(Iterator.scala:1431) at org.apache.spark.rdd.RDD.$anonfun$aggregate$2(RDD.scala:1226) at org.apache.spark.SparkContext.$anonfun$runJob$6(SparkContext.scala:2487) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166) at org.apache.spark.scheduler.Task.run(Task.scala:141) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:621) at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64) at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:624) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.lang.ArrayIndexOutOfBoundsException Driver stacktrace: |