hadoop - How to tell distcp to ignore "file not found ..." and fall through to the next files? -
we have full hdfs backup using distcp
takes long time run, of data on hdfs "moving", is created , deleted. results in mappers failing java.io.filenotfoundexception: no such file or directory
. such files unimportant, want backup best can.
now seems -i
"ignore failures" not quite want because ignore @ map level rather file level, if map task fails files associated map task ignored. want file ignored.
Comments
Post a Comment