script to kill yarn application if it is running more than x mins
Sometime we get a situation where we have to get lists of all long running and based on threshold we need to kill them.Also sometime we need to do it for a specific yarn queue. In such situation following script will help you to do your job.
[root@m1.hdp22~]$ vi kill_application_after_some_time.sh
#!/bin/bash
if [ “$#” -lt 1 ]; then
echo “Usage: $0 <max_life_in_mins>“
exit 1
fi
yarn application -list 2>/dev/null | grep “<queue_name>“ | grep “RUNNING“ | awk ‘{print $1}‘ > job_list.txt
for jobId in `cat job_list.txt`
do
finish_time=`yarn application -status $jobId 2>/dev/null | grep “Finish-Time“ | awk ‘{print $NF}‘`
if [ $finish_time -ne 0 ]; then
echo “App $jobId is not running“
exit 1
fi
time_diff=`date +%s`–`yarn application -status $jobId 2>/dev/null | grep “Start-Time“ | awk ‘{print $NF}‘ | sed ‘s!$!/1000!‘`
time_diff_in_mins=`echo “(“$time_diff“)/60“ | bc`
echo “App $jobId is running for $time_diff_in_mins min(s)“
if [ $time_diff_in_mins -gt $1 ]; then
echo “Killing app $jobId“
yarn application -kill $jobId
else
echo “App $jobId should continue to run“
fi
done
[yarn@m1.hdp22 ~]$ ./kill_application_after_some_time.sh 30 (pass x tim in mins)
App application_1487677946023_5995 is running for 0 min(s)
App application_1487677946023_5995 should continue to run
I hope it would help you but please feel free to give your valuable feedback or suggestion.