hadoop - Getting java.lang.NoSuchFieldError: INT_8 error while running spark job through oozie -

i getting java.lang.nosuchfielderror: int_8 error when trying execute spark job using oozie on cloudera 5.5.1 version. on appreciated.

please find error stackstrace below.

16/01/28 11:21:17 warn tasksetmanager: lost task 0.2 in stage 20.0 (tid 40, zlab-physrv1): java.lang.nosuchfielderror: int_8     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:327)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:312)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter$$anonfun$convertfield$1.apply(catalystschemaconverter.scala:517)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter$$anonfun$convertfield$1.apply(catalystschemaconverter.scala:516)     @ scala.collection.indexedseqoptimized$class.foldl(indexedseqoptimized.scala:51)     @ scala.collection.indexedseqoptimized$class.foldleft(indexedseqoptimized.scala:60)     @ scala.collection.mutable.arrayops$ofref.foldleft(arrayops.scala:108)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:516)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:312)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:521)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convertfield(catalystschemaconverter.scala:312)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter$$anonfun$convert$1.apply(catalystschemaconverter.scala:305)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter$$anonfun$convert$1.apply(catalystschemaconverter.scala:305)     @ scala.collection.traversablelike$$anonfun$map$1.apply(traversablelike.scala:244)     @ scala.collection.traversablelike$$anonfun$map$1.apply(traversablelike.scala:244)     @ scala.collection.iterator$class.foreach(iterator.scala:727)     @ scala.collection.abstractiterator.foreach(iterator.scala:1157)     @ scala.collection.iterablelike$class.foreach(iterablelike.scala:72)     @ org.apache.spark.sql.types.structtype.foreach(structtype.scala:92)     @ scala.collection.traversablelike$class.map(traversablelike.scala:244)     @ org.apache.spark.sql.types.structtype.map(structtype.scala:92)     @ org.apache.spark.sql.execution.datasources.parquet.catalystschemaconverter.convert(catalystschemaconverter.scala:305)     @ org.apache.spark.sql.execution.datasources.parquet.parquettypesconverter$.convertfromattributes(parquettypesconverter.scala:58)     @ org.apache.spark.sql.execution.datasources.parquet.rowwritesupport.init(parquettablesupport.scala:55)     @ parquet.hadoop.parquetoutputformat.getrecordwriter(parquetoutputformat.java:277)     @ parquet.hadoop.parquetoutputformat.getrecordwriter(parquetoutputformat.java:251)     @ org.apache.spark.sql.execution.datasources.parquet.parquetoutputwriter.<init>(parquetrelation.scala:94)     @ org.apache.spark.sql.execution.datasources.parquet.parquetrelation$$anon$3.newinstance(parquetrelation.scala:272)     @ org.apache.spark.sql.execution.datasources.defaultwritercontainer.writerows(writercontainer.scala:233)     @ org.apache.spark.sql.execution.datasources.insertintohadoopfsrelation$$anonfun$run$1$$anonfun$apply$mcv$sp$3.apply(insertintohadoopfsrelation.scala:150)     @ org.apache.spark.sql.execution.datasources.insertintohadoopfsrelation$$anonfun$run$1$$anonfun$apply$mcv$sp$3.apply(insertintohadoopfsrelation.scala:150)     @ org.apache.spark.scheduler.resulttask.runtask(resulttask.scala:66)     @ org.apache.spark.scheduler.task.run(task.scala:88)     @ org.apache.spark.executor.executor$taskrunner.run(executor.scala:214)     @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145)     @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615)     @ java.lang.thread.run(thread.java:745) 

as per idea used error when ever there difference on jars have used generate code , jars have used currently.

note: when trying submit same 1 using spark-submit command it's running fine.

regards nisith

finally able debug , fix issue. issue installation 1 of data nodes having older version of parquet jars(5.2 cdh distribution). after replacing jars current version jars working fine.