Ssh action with oozie

  • 2

Ssh action with oozie

When you want to run your shell script via oozie then following article will help you to do your job in easy way.

Following steps you need to setup Oozie workflow using ssh-action:

1. Configure job.properties
Example:

[s0998dnz@m1.hdp22 oozie_ssh_action]$ cat job.properties
#*************************************************
#  job.properties
#oozie-action for ssh
#*************************************************
nameNode=hdfs://m1.hdp22:8020
jobTracker=m2.hdp22:8050
queueName=default
oozie.libpath=${nameNode}/user/oozie/share/lib
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
oozieProjectRoot=${nameNode}/user/${user.name}/ooziesshaction
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
focusNodeLogin=s0998dnz@m1.hdp22
shellScriptPath=~/oozie_ssh_action/sampletest.sh

2. Configure workflow.xml

Example:


<!--******************************************-->
<!--workflow.xml -->
<!--******************************************-->
<workflow-app name="WorkFlowForSshAction" xmlns="uri:oozie:workflow:0.1">
 <start to="sshAction"/>
 <action name="sshAction">
 <ssh xmlns="uri:oozie:ssh-action:0.1">
 <host>${focusNodeLogin}</host>
 <command>${shellScriptPath}</command>
 <capture-output/>
 </ssh>
 <ok to="end"/>
 <error to="killAction"/>
 </action>
<!-- <action name="sendEmail">
 <email xmlns="uri:oozie:email-action:0.1">
 <to>${emailToAddress}</to>
 <subject>Output of workflow ${wf:id()}</subject>
 <body>Status of the file move: ${wf:actionData('sshAction')['STATUS']}</body>
 </email>
 <ok to="end"/>
 <error to="end"/>
 </action>
 --> <kill name="killAction">
 <message>"Killed job due to error"</message>
 </kill>
 <end name="end"/>
</workflow-app>

3. Write sample sampletest.sh script

Example:

[s0998dnz@m1.hdp22 oozie_ssh_action]$ cat sampletest.sh 
#!/bin/bash
hadoop fs -ls / > /home/s0998dnz/oozie_ssh_action/output.txt

4. Upload workflow.xml to ${appPath} defined in job.properties

[s0998dnz@m1.hdp22 oozie_ssh_action]$ hadoop fs -put workflow.xml /user/s0998dnz/ooziesshaction/

5. Login to Oozie host by “oozie” user.

[oozie@m2.hdp22 ~]$

6. Generate a key pair,if it doesn’t exist already, using ‘ssh-keygen’ command:

[oozie@m2.hdp22 ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oozie/.ssh/id_rsa): 
Created directory '/home/oozie/.ssh'
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/oozie/.ssh/id_rsa.
Your public key has been saved in /home/oozie/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:EW8WSDG3QnVjGf65znS8bP0AeOrgoQuteYl3hIunO8c oozie@m1.hdp22
The key's randomart image is:
+---[RSA 2048]----+
.*++ =o <span class="Apple-converted-space">
..= *.. <span class="Apple-converted-space">  
o = .
= . . . 
.S . o o
.. .
o . o 
.+.+o .
+ +
++Eo.+ 
+.+o
+----[SHA256]-----+

7. On Oozie Server node copy ~/.ssh/id_rsa.pub and paste it to remote-node’s ~/.ssh/authorized_keys file (focus node)

8. Test password-less ssh from oozie@oozie-host to <username>@<remote-host>

9. Follow below command to run Oozie workflow

oozie job -oozie http://<oozie-server-hostname>:11000/oozie -config /$PATH/job.properties -run

I hope it helped you to do you job in quick time,please feel free to give your valuable feedback or suggestion.


2 Comments

kumar

November 10, 2018 at 5:28 am

Hbase Ave Load is high and red colour in Ambari UI(Hbase), if any RCA or Performance tuning documents kindly share.

Shweta Grewal

November 20, 2018 at 6:49 am

Hi,

I am running oozie ssh example on MapR distribution and the passwordless ssh is enabled, but I still get the issue:
Message [AUTH_FAILED: Not able to perform operation [ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 oozie@localhost mkdir -p oozie-mapr/0000024-181113103854803-oozie-mapr-W/ssh–ssh/ ] | ErrorStream: *******************************************************************************
* *
* This is a private computer system containing confidential information. *
* Any unauthorized attempt to access or use this computer system or any *
* information on it by employees or other persons may result in termination *
* of employment, civil fines, and criminal penalties. This system must be *
* used for authorized business purposes only. *
* *
*******************************************************************************
Permission denied (publickey,password).
]

When I run the ssh command manually it runs okay. Please help.

Regards,
Shweta

Leave a Reply