First Build, DevOps Journey Starts here

 My first successful build in Oracle Cloud Services. My DevOps Journey Starts here.

Useful Script to Analyze last 30 days of OutofMemory Error on Oracle E-Business Suite OPP

 

Useful Script to Analyze last 30 days of OutofMemory Error on Oracle E-Business Suite OPP ( Output Post Processor ) Logs

First Spool the logfile that is required for analysis. Here I am spooling from fnd tables. you can use find command to get the last 30 days of OPP logs.

cd /tmp;mkdir OPPAnalysis;cd OPPAnalysis
sqlplus apps
spool opp_last30DaysLog.lst
set linesize 1000 pagesize 0
select logfile_name from fnd_concurrent_processes where trunc(creation_date)>trunc(sysdate-30) and logfile_name like '%OPP%';
spool off

Script below gets into each of the logs generated above and finds the OutOfMemoryError Tag then pulls the Request ID. Then for each of the Request ID it fetches if the failue is on OutOfMemory and then pulls the relevant templates. These templates are the cause of Out of Memory.

cat OPPOutOfMemoryDiag.sh

for log in $(cat opp_last30DaysLog.lst|grep OPP|grep txt)

do

 opp_request=$(grep -i UNEXPECTED $log|grep -i OutOfMemoryError|grep ":RT"|cut -d'[' -f4|cut -d']' -f1|cut -d':' -f2|cut -c 3-)

 if [ "$1" = "-debug" ]

  then

   echo

   echo $log

   echo $opp_request

  fi

 if [ ! -z $opp_request ]

 then

  echo $(echo $log|rev|cut -d'/' -f1|rev)" - "$(stat $log|grep Modify|cut -d":" -f2-)" - "$(grep -b1 $opp_request $log|grep -i Template|cut -d'-' -f2)

 fi

done

[oracle@testserver OPPAnalysis]$ sh OPPOutOfMemoryDiag.sh

FNDOPP548310.txt -  2091-11-04 07:05:02.190691000 -0600 - Template code: EXCELLOAD

FNDOPP548311.txt -  2091-11-04 07:06:32.195461000 -0600 - Template code: EXCELLOAD

FNDOPP548345.txt -  2091-11-04 09:50:20.855625000 -0600 - Template code: EXCELLOAD

FNDOPP547385.txt -  2091-11-23 11:50:16.076419000 -0600 - Template code: PDFOUTUT

FNDOPP548385.txt -  2091-11-06 07:20:16.937702000 -0600 - Template code: EXCELLOAD

FNDOPP548395.txt -  2091-11-06 07:20:19.119176000 -0600 - Template code: EXCELLOAD

FNDOPP548320.txt -  2091-11-07 07:20:12.012239000 -0600 - Template code: EXCELLOAD

FNDOPP548347.txt -  2091-11-03 08:27:41.649406000 -0600 - Template code: EXCELLOAD

FNDOPP548359.txt -  2091-11-08 09:02:02.276215000 -0600 - Template code: EXCELLOAD

FNDOPP548368.txt -  2091-11-08 10:22:16.784284000 -0600 - Template code: EXCELLOAD

FNDOPP548377.txt -  2091-11-08 10:35:22.024312000 -0600 - Template code: PDFOUTUT

FNDOPP548393.txt -  2091-11-03 09:18:51.866628000 -0600 - Template code: PDFOUTUT

FNDOPP548392.txt -  2091-11-08 12:47:33.709433000 -0600 - Template code: PDFOUTUT

FNDOPP548311.txt -  2091-11-09 07:08:49.778074000 -0600 - Template code: PDFOUTUT

FNDOPP548312.txt -  2091-11-09 07:17:04.803232000 -0600 - Template code: PDFOUTUT

FNDOPP548320.txt -  2091-11-10 07:11:10.239378000 -0600 - Template code: PDFOUTUT

FNDOPP548325.txt -  2091-11-10 07:15:11.158641000 -0600 - Template code: PDFOUTUT

FNDOPP549346.txt -  2091-11-12 07:19:48.004048000 -0600 - Template code: PDFOUTUT

FNDOPP549349.txt -  2091-11-12 07:09:20.415082000 -0600 - Template code: EXCELLOAD

FNDOPP549368.txt -  2091-11-13 07:20:13.214100000 -0600 - Template code: EXCELLOAD

FNDOPP549375.txt -  2091-11-13 07:15:32.330791000 -0600 - Template code: PDFOUTUT

FNDOPP549391.txt -  2091-11-14 07:16:11.077493000 -0600 - Template code: EXCELLOAD

FNDOPP549394.txt -  2091-11-14 07:09:31.057564000 -0600 - Template code: EXCELLOAD

[oracle@testserver OPPAnalysis]$ 

sh OPPOutOfMemoryDiag.sh -debug 

the above -debug options helps get more details

Next Steps

Now that you have found the template that is causing Heap Out Of Memory Error , the next step is to fix it.

There are two options

Option 1: more Heap upto 2gb , if you need more than 2gb Heap move to 64 bit Java

Option 2: move from ram to disk based procesing

Both these options are covered in detail

E-Business Suite Output BI Publisher Post Processor (OPP) Log Contains Error "java.lang.OutOfMemoryError: Java heap space" (Doc ID 1268217.1)

The best recommended for EXCEL and BULK programs is to move to Disk Based Processing.

Configure the XML Publisher Administrator Configuration settings.

1. As XML Publisher Administrator navigate to Administration->Configuration.
2. Under Temporary Directory pick a temporary file location on your concurrent processing node. This should be at least 5GB or 20x larger than largest XML data file you generate
3. Under FO Processing, set:
• Use XML Publisher's XSLT processor set to True
• Confirm the profile 'scalable feature of XSLT processor' is set to False at the global level
• Enable profile 'scalable feature of XSLT processor' be setting to True at the individual template level for the failing template
• Enable XSLT runtime optimization set to True