Ssh action with oozie
Category : Oozie
When you want to run your shell script via oozie then following article will help you to do your job in easy way.
Following steps you need to setup Oozie workflow using ssh-action:
1. Configure job.properties
Example:
[s0998dnz@m1.hdp22 oozie_ssh_action]$ cat job.properties
#*************************************************
# job.properties
#oozie-action for ssh
#*************************************************
nameNode=hdfs://m1.hdp22:8020
jobTracker=m2.hdp22:8050
queueName=default
oozie.libpath=${nameNode}/user/oozie/share/lib
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
oozieProjectRoot=${nameNode}/user/${user.name}/ooziesshaction
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
focusNodeLogin=s0998dnz@m1.hdp22
shellScriptPath=~/oozie_ssh_action/sampletest.sh
2. Configure workflow.xml
Example:
<!--******************************************-->
<!--workflow.xml -->
<!--******************************************-->
<workflow-app name="WorkFlowForSshAction" xmlns="uri:oozie:workflow:0.1">
<start to="sshAction"/>
<action name="sshAction">
<ssh xmlns="uri:oozie:ssh-action:0.1">
<host>${focusNodeLogin}</host>
<command>${shellScriptPath}</command>
<capture-output/>
</ssh>
<ok to="end"/>
<error to="killAction"/>
</action>
<!-- <action name="sendEmail">
<email xmlns="uri:oozie:email-action:0.1">
<to>${emailToAddress}</to>
<subject>Output of workflow ${wf:id()}</subject>
<body>Status of the file move: ${wf:actionData('sshAction')['STATUS']}</body>
</email>
<ok to="end"/>
<error to="end"/>
</action>
--> <kill name="killAction">
<message>"Killed job due to error"</message>
</kill>
<end name="end"/>
</workflow-app>
3. Write sample sampletest.sh script
Example:
[s0998dnz@m1.hdp22 oozie_ssh_action]$ cat sampletest.sh #!/bin/bash hadoop fs -ls / > /home/s0998dnz/oozie_ssh_action/output.txt
4. Upload workflow.xml to ${appPath} defined in job.properties
[s0998dnz@m1.hdp22 oozie_ssh_action]$ hadoop fs -put workflow.xml /user/s0998dnz/ooziesshaction/
5. Login to Oozie host by “oozie” user.
[oozie@m2.hdp22 ~]$
6. Generate a key pair,if it doesn’t exist already, using ‘ssh-keygen’ command:
[oozie@m2.hdp22 ~]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/oozie/.ssh/id_rsa): Created directory '/home/oozie/.ssh' Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oozie/.ssh/id_rsa. Your public key has been saved in /home/oozie/.ssh/id_rsa.pub. The key fingerprint is: SHA256:EW8WSDG3QnVjGf65znS8bP0AeOrgoQuteYl3hIunO8c oozie@m1.hdp22 The key's randomart image is: +---[RSA 2048]----+ .*++ =o <span class="Apple-converted-space"> ..= *.. <span class="Apple-converted-space"> o = . = . . . .S . o o .. . o . o .+.+o . + + ++Eo.+ +.+o +----[SHA256]-----+
7. On Oozie Server node copy ~/.ssh/id_rsa.pub and paste it to remote-node’s ~/.ssh/authorized_keys file (focus node)
8. Test password-less ssh from oozie@oozie-host to <username>@<remote-host>
9. Follow below command to run Oozie workflow
oozie job -oozie http://<oozie-server-hostname>:11000/oozie -config /$PATH/job.properties -run
I hope it helped you to do you job in quick time,please feel free to give your valuable feedback or suggestion.