On HP UX platform I recently did an upgrade project of a 1TB warehouse system from 10.2.0.2 to 10.2.0.4. After upgrade I saw a strange behaviour that Oracle Database alert.log started reporting ORA-07445:[$PXNMOVE] and ORA-07445:[$COLD_KKERFP()+1984]. These errors are diagnosed as faulty CPU , so had to replace all Cell Boards on the Faulty HP Superdome Hardware.
Later the performance went down and we could not load the data on time. Things went bad.
All I saw is the first three CPUs were used by the system and not the rest of the CPUs.
With a lot of investigation between HP & Oracle , it was finally discovered to be the NUMA setup within Oracle Code to use the first cell board and ignore the rest.
alert.log looked like the below for NUMA configuration
and NUMA PG = 3, CPUs = 3
Statement from Oracle was "NUMA is turned on by default 10.2.0.4 and as per the observation only 3 CPU is used."
Later the Init.ora parameter were setup to disable the NUMA.
_enable_numa_optimization=FALSE
_db_block_numa=1
Database was restarted and we were able to see the Oracle processes working on all the 10 CPUS.
Now the alert.log will showed like
and NUMA PG = 1, CPUs = 10
I hope this information will help HP Customers running Oracle Database.
The voice in my head may not be real , but they have some good ideas !!!
Fail Fast, Fail Forward, Fail Often, Fail Better , Standup Every Time
Every problem has at least one solution. Only some solutions are harder to find.
Saturday, September 27, 2008
Subscribe to:
Posts (Atom)
Popular Posts
-
[oracle@testebsop3app01 ~]$ perl /u01/install/APPS/fs1/EBSapps/comn/clone/bin/adcfgclone.pl appltop /u01/install/APPS/fs1/inst/apps/SATURN_...
-
FS_CLONE Phase It is a stand-alone command used for file system cloning. Standard cloning (using adcfgclone.pl) cannot be used to synch...
-
Suddenly all production reports in BI Publisher failed with message " ORA-01017: invalid username/password; logon denied" The da...
-
Abort Phase If for some reason either the prepare or apply phase failed or gave problems, you can abort the patching cycle. After runnin...
-
As a green field expert in data science, you would be responsible for designing and implementing data science projects from scratch. This i...
-
If you are an EBS administrator the first thing some asks you to do a health check of an environment , you return to him with a Request ID o...
-
PRVG-2031 : Owner of file "/u01/app/oracle/diag/crs/rac01/crs/lck" did not match the expected value on node "HOST1". [Ex...
-
ADOPValidations_detailed.log reported a warning message for one of the critical error "Cross Reference" ValidateFSConfigFile...
-
Ready the instance for cutover Perform the final operations like Compile Invalids , Generate Derived Objects and Pre-compute DDL to be run...
-
Scenario You have noticed that the on the patch edition there are lot of patches applied. You did the cutover and now customer wants t...