java.lang.IllegalArgumentException: stream exceeds limit [2,048]

  • 0

java.lang.IllegalArgumentException: stream exceeds limit [2,048]

When we run oozie job with SSH action and we use capture output then it may fail with following error.

java.lang.IllegalArgumentException: stream exceeds limit [2,048]
at org.apache.oozie.util.IOUtils.getReaderAsString(IOUtils.java:84)
at org.apache.oozie.servlet.CallbackServlet.doPost(CallbackServlet.java:117)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:304)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)

Root Cause: It is because of insufficient value for oozie.servlet.CallbackServlet.max.data.len property in oozie-site.xml. In this case it was set to 2048, which wasn’t sufficient for the output.
Resolution:

Option 1: If you are using oozie version before 4.2 then add the following property in oozie-site.xml or add in custom oozie-site via Ambari. 

<property>
 <name>oozie.servlet.CallbackServlet.max.data.len</name>
 <value>16000</value>
</property>

Option 2: If you are using oozie version after 4.2 then add the following property in oozie-site.xml or add in custom oozie-site via Ambari.

<property>
<name>oozie.action.max.output.data</name>
<value>16000</value>
</property>

 

I hope it helped you,feel free to give your valuable feedback suggestions.