solr5 - Nutch cause error while using solrindex command -
i using nutch 1.11 (published in 07 december 2015) , using bin/crawl command me work , ok until reaches solrindex command put data solr search engine, cause error:
solrindexwriter solr.server.type : type of solrserver communicate (default 'http' options include 'cloud', 'lb' , 'concurrent') solr.server.url : url of solr instance (mandatory) solr.zookeeper.url : url of zookeeper url (mandatory if 'cloud' value solr.server.type) solr.loadbalance.urls : comma-separated string of solr server strings used (madatory if 'lb' value solr.server.type) solr.mapping.file : name of mapping file fields (default solrindex-mapping.xml) solr.commit.size : buffer size when sending solr (default 1000) solr.auth : use authentication (default false) solr.auth.username : username authentication solr.auth.password : password authentication 2016-01-28 02:49:41,422 info indexer.indexermapreduce - indexermapreduce: crawldb: nutchweb/crawldb 2016-01-28 02:49:41,425 info indexer.indexermapreduce - indexermapreduce: linkdb: nutchweb/linkdb 2016-01-28 02:49:41,425 info indexer.indexermapreduce - indexermapreduces: adding segment: nutchweb/segments/20160127234706 2016-01-28 02:49:41,652 warn util.nativecodeloader - unable load native-hadoop library platform... using builtin-java classes applicable 2016-01-28 02:49:42,586 warn conf.configuration - file:/tmp/hadoop-micky/mapred/staging/micky810285982/.staging/job_local810285982_0001/job.xml:an attempt override final parameter: mapreduce.job.end-notification.max.retry.interval; ignoring. 2016-01-28 02:49:42,587 warn conf.configuration - file:/tmp/hadoop-micky/mapred/staging/micky810285982/.staging/job_local810285982_0001/job.xml:an attempt override final parameter: mapreduce.job.end-notification.max.attempts; ignoring. 2016-01-28 02:49:42,751 warn conf.configuration - file:/tmp/hadoop-micky/mapred/local/localrunner/micky/job_local810285982_0001/job_local810285982_0001.xml:an attempt override final parameter: mapreduce.job.end-notification.max.retry.interval; ignoring. 2016-01-28 02:49:42,752 warn conf.configuration - file:/tmp/hadoop-micky/mapred/local/localrunner/micky/job_local810285982_0001/job_local810285982_0001.xml:an attempt override final parameter: mapreduce.job.end-notification.max.attempts; ignoring. 2016-01-28 02:49:43,342 info anchor.anchorindexingfilter - anchor deduplication is: off 2016-01-28 02:49:49,230 info indexer.indexwriters - adding org.apache.nutch.indexwriter.solr.solrindexwriter 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: content dest: content 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: title dest: title 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: host dest: host 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: segment dest: segment 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: boost dest: boost 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: digest dest: digest 2016-01-28 02:49:50,627 info solr.solrmappingreader - source: tstamp dest: tstamp 2016-01-28 02:49:50,959 info solr.solrindexwriter - indexing 250 documents 2016-01-28 02:49:50,960 info solr.solrindexwriter - deleting 0 documents 2016-01-28 02:49:54,346 info solr.solrindexwriter - indexing 250 documents 2016-01-28 02:50:06,471 warn mapred.localjobrunner - job_local810285982_0001 java.lang.exception: org.apache.solr.client.solrj.impl.httpsolrserver$remotesolrexception: exception writing document id http://nutch.apache.org/apidocs/apidocs-1.1/overview-tree.html index; possible analysis error. @ org.apache.hadoop.mapred.localjobrunner$job.runtasks(localjobrunner.java:462) @ org.apache.hadoop.mapred.localjobrunner$job.run(localjobrunner.java:529) caused by: org.apache.solr.client.solrj.impl.httpsolrserver$remotesolrexception: exception writing document id http://nutch.apache.org/apidocs/apidocs-1.1/overview-tree.html index; possible analysis error. @ org.apache.solr.client.solrj.impl.httpsolrserver.executemethod(httpsolrserver.java:552) @ org.apache.solr.client.solrj.impl.httpsolrserver.request(httpsolrserver.java:210) @ org.apache.solr.client.solrj.impl.httpsolrserver.request(httpsolrserver.java:206) @ org.apache.solr.client.solrj.request.abstractupdaterequest.process(abstractupdaterequest.java:124) @ org.apache.nutch.indexwriter.solr.solrindexwriter.write(solrindexwriter.java:134) @ org.apache.nutch.indexer.indexwriters.write(indexwriters.java:85) @ org.apache.nutch.indexer.indexeroutputformat$1.write(indexeroutputformat.java:50) @ org.apache.nutch.indexer.indexeroutputformat$1.write(indexeroutputformat.java:41) @ org.apache.hadoop.mapred.reducetask$oldtrackingrecordwriter.write(reducetask.java:493) @ org.apache.hadoop.mapred.reducetask$3.collect(reducetask.java:422) @ org.apache.nutch.indexer.indexermapreduce.reduce(indexermapreduce.java:356) @ org.apache.nutch.indexer.indexermapreduce.reduce(indexermapreduce.java:56) @ org.apache.hadoop.mapred.reducetask.runoldreducer(reducetask.java:444) @ org.apache.hadoop.mapred.reducetask.run(reducetask.java:392) @ org.apache.hadoop.mapred.localjobrunner$job$reducetaskrunnable.run(localjobrunner.java:319) @ java.util.concurrent.executors$runnableadapter.call(executors.java:471) @ java.util.concurrent.futuretask.run(futuretask.java:262) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615) @ java.lang.thread.run(thread.java:745) 2016-01-28 02:50:07,330 error indexer.indexingjob - indexer: java.io.ioexception: job failed! @ org.apache.hadoop.mapred.jobclient.runjob(jobclient.java:836) @ org.apache.nutch.indexer.indexingjob.index(indexingjob.java:145) @ org.apache.nutch.indexer.indexingjob.run(indexingjob.java:222) @ org.apache.hadoop.util.toolrunner.run(toolrunner.java:70) @ org.apache.nutch.indexer.indexingjob.main(indexingjob.java:231)
i found out error
exception writing document id http://nutch.apache.org/apidocs/apidocs-1.1/overview-tree.html index; possible analysis error.
and seems no 1 encounter error before, please help.
Comments
Post a Comment