How to debug distcp jobs
Category : Bigdata
Some time when you run distcp jobs on cluster and you see some failure or performance then you want to debug it then you can go by using following command.
To turn on debug mode on the job level, issue the following command before executing the distcp job:
[root@m1.hdp22] export HADOOP_ROOT_LOGGER=hadoop.root.logger=Debug,console
To turn on debugmode on the mapper level, run distcp with mapper debug option as following:
[root@m1.hdp22] hadoop distcp -Dmapreduce.map.java.opts-="-Xmxyyyy -Dhadoop.root.logger=DEBUG,console"