On HP UX platform I recently did an upgrade project of a 1TB warehouse system from 10.2.0.2 to 10.2.0.4. After upgrade I saw a strange behaviour that Oracle Database alert.log started reporting ORA-07445:[$PXNMOVE] and ORA-07445:[$COLD_KKERFP()+1984]. These errors are diagnosed as faulty CPU , so had to replace all Cell Boards on the Faulty HP Superdome Hardware.
Later the performance went down and we could not load the data on time. Things went bad.
All I saw is the first three CPUs were used by the system and not the rest of the CPUs.
With a lot of investigation between HP & Oracle , it was finally discovered to be the NUMA setup within Oracle Code to use the first cell board and ignore the rest.
alert.log looked like the below for NUMA configuration
and NUMA PG = 3, CPUs = 3
Statement from Oracle was "NUMA is turned on by default 10.2.0.4 and as per the observation only 3 CPU is used."
Later the Init.ora parameter were setup to disable the NUMA.
_enable_numa_optimization=FALSE
_db_block_numa=1
Database was restarted and we were able to see the Oracle processes working on all the 10 CPUS.
Now the alert.log will showed like
and NUMA PG = 1, CPUs = 10
I hope this information will help HP Customers running Oracle Database.
The voice in my head may not be real , but they have some good ideas !!!
Fail Fast, Fail Forward, Fail Often, Fail Better , Standup Every Time
Every problem has at least one solution. Only some solutions are harder to find.
Saturday, September 27, 2008
Popular Posts
-
Grafana for OCI Monitoring in Docker on Windows Step 1: Install Docker Desktop on Windows https://docs.docker.com/docker-for-windows/inst...
-
[oracle@testebsop3app01 ~]$ perl /u01/install/APPS/fs1/EBSapps/comn/clone/bin/adcfgclone.pl appltop /u01/install/APPS/fs1/inst/apps/SATURN_...
-
FS_CLONE Phase It is a stand-alone command used for file system cloning. Standard cloning (using adcfgclone.pl) cannot be used to synch...
-
Abort Phase If for some reason either the prepare or apply phase failed or gave problems, you can abort the patching cycle. After runnin...
-
Suddenly all production reports in BI Publisher failed with message " ORA-01017: invalid username/password; logon denied" The da...
-
If you are an EBS administrator the first thing some asks you to do a health check of an environment , you return to him with a Request ID o...
-
As a green field expert in data science, you would be responsible for designing and implementing data science projects from scratch. This i...
-
PRVG-2031 : Owner of file "/u01/app/oracle/diag/crs/rac01/crs/lck" did not match the expected value on node "HOST1". [Ex...
-
ADOPValidations_detailed.log reported a warning message for one of the critical error "Cross Reference" ValidateFSConfigFile...
-
ADOP patching on a downtime mode with [ERROR] Patch service is not exist or running Please note , if you get ADOP issues on PROD ple...