Useful Script to Analyze last 30 days of OutofMemory Error on Oracle E-Business Suite OPP ( Output Post Processor ) Logs
First Spool the logfile that is required for analysis. Here I am spooling from fnd tables. you can use find command to get the last 30 days of OPP logs.
cd /tmp;mkdir OPPAnalysis;cd OPPAnalysis
sqlplus apps
spool opp_last30DaysLog.lst
set linesize 1000 pagesize 0
select logfile_name from fnd_concurrent_processes where trunc(creation_date)>trunc(sysdate-30) and logfile_name like '%OPP%';
spool off
sqlplus apps
spool opp_last30DaysLog.lst
set linesize 1000 pagesize 0
select logfile_name from fnd_concurrent_processes where trunc(creation_date)>trunc(sysdate-30) and logfile_name like '%OPP%';
spool off
Script below gets into each of the logs generated above and finds the OutOfMemoryError Tag then pulls the Request ID. Then for each of the Request ID it fetches if the failue is on OutOfMemory and then pulls the relevant templates. These templates are the cause of Out of Memory.
cat OPPOutOfMemoryDiag.sh
for log in $(cat opp_last30DaysLog.lst|grep OPP|grep txt)
do
opp_request=$(grep -i UNEXPECTED $log|grep -i OutOfMemoryError|grep ":RT"|cut -d'[' -f4|cut -d']' -f1|cut -d':' -f2|cut -c 3-)
if [ "$1" = "-debug" ]
then
echo
echo $log
echo $opp_request
fi
if [ ! -z $opp_request ]
then
echo $(echo $log|rev|cut -d'/' -f1|rev)" - "$(stat $log|grep Modify|cut -d":" -f2-)" - "$(grep -b1 $opp_request $log|grep -i Template|cut -d'-' -f2)
fi
done
[oracle@testserver OPPAnalysis]$ sh OPPOutOfMemoryDiag.sh
FNDOPP548310.txt - 2091-11-04 07:05:02.190691000 -0600 - Template code: EXCELLOAD
FNDOPP548311.txt - 2091-11-04 07:06:32.195461000 -0600 - Template code: EXCELLOAD
FNDOPP548345.txt - 2091-11-04 09:50:20.855625000 -0600 - Template code: EXCELLOAD
FNDOPP547385.txt - 2091-11-23 11:50:16.076419000 -0600 - Template code: PDFOUTUT
FNDOPP548385.txt - 2091-11-06 07:20:16.937702000 -0600 - Template code: EXCELLOAD
FNDOPP548395.txt - 2091-11-06 07:20:19.119176000 -0600 - Template code: EXCELLOAD
FNDOPP548320.txt - 2091-11-07 07:20:12.012239000 -0600 - Template code: EXCELLOAD
FNDOPP548347.txt - 2091-11-03 08:27:41.649406000 -0600 - Template code: EXCELLOAD
FNDOPP548359.txt - 2091-11-08 09:02:02.276215000 -0600 - Template code: EXCELLOAD
FNDOPP548368.txt - 2091-11-08 10:22:16.784284000 -0600 - Template code: EXCELLOAD
FNDOPP548377.txt - 2091-11-08 10:35:22.024312000 -0600 - Template code: PDFOUTUT
FNDOPP548393.txt - 2091-11-03 09:18:51.866628000 -0600 - Template code: PDFOUTUT
FNDOPP548392.txt - 2091-11-08 12:47:33.709433000 -0600 - Template code: PDFOUTUT
FNDOPP548311.txt - 2091-11-09 07:08:49.778074000 -0600 - Template code: PDFOUTUT
FNDOPP548312.txt - 2091-11-09 07:17:04.803232000 -0600 - Template code: PDFOUTUT
FNDOPP548320.txt - 2091-11-10 07:11:10.239378000 -0600 - Template code: PDFOUTUT
FNDOPP548325.txt - 2091-11-10 07:15:11.158641000 -0600 - Template code: PDFOUTUT
FNDOPP549346.txt - 2091-11-12 07:19:48.004048000 -0600 - Template code: PDFOUTUT
FNDOPP549349.txt - 2091-11-12 07:09:20.415082000 -0600 - Template code: EXCELLOAD
FNDOPP549368.txt - 2091-11-13 07:20:13.214100000 -0600 - Template code: EXCELLOAD
FNDOPP549375.txt - 2091-11-13 07:15:32.330791000 -0600 - Template code: PDFOUTUT
FNDOPP549391.txt - 2091-11-14 07:16:11.077493000 -0600 - Template code: EXCELLOAD
FNDOPP549394.txt - 2091-11-14 07:09:31.057564000 -0600 - Template code: EXCELLOAD
[oracle@testserver OPPAnalysis]$
sh OPPOutOfMemoryDiag.sh -debug
the above -debug options helps get more details
Next Steps
Now that you have found the template that is causing Heap Out Of Memory Error , the next step is to fix it.
There are two options
Option 1: more Heap upto 2gb , if you need more than 2gb Heap move to 64 bit Java
Option 2: move from ram to disk based procesing
Both these options are covered in detail
E-Business Suite Output BI Publisher Post Processor (OPP) Log Contains Error "java.lang.OutOfMemoryError: Java heap space" (Doc ID 1268217.1)
The best recommended for EXCEL and BULK programs is to move to Disk Based Processing.
Configure the XML Publisher Administrator Configuration settings.
- As XML Publisher Administrator navigate to Administration->Configuration.
- Under Temporary Directory pick a temporary file location on your concurrent processing node. This should be at least 5GB or 20x larger than largest XML data file you generate
- Under FO Processing, set:
- Use XML Publisher's XSLT processor set to True
- Confirm the profile 'scalable feature of XSLT processor' is set to False at the global level
- Enable profile 'scalable feature of XSLT processor' be setting to True at the individual template level for the failing template
- Enable XSLT runtime optimization set to True
