Scenario
You
have been asked to apply a patch and you completed the prepare phase and apply
phase. Now customer has given a downtime for the cutover. You have run the
finalize phase and the instance is ready for cutover.
- How do we do it?
- Pre-Health Check
- Blackout
- Cutover
- Unset Blackout
- Post-Health Check
$ adop -status
Enter the APPS
username: apps
Enter the APPS
password:
Current
Patching Session ID: 9
Node Name Node Type Phase
Status Started Finished Elapsed
---------------
--------------- ----------- --------------- ------------------------------
------------------------------ ------------
testserver01 master APPLY
ACTIVE 17-NOV-13 02:49:24 -06:00
17-NOV-13 02:52:46 -06:00 0:03:22
PREPARE COMPLETED 17-NOV-13 12:46:19 -06:00
17-NOV-13 01:00:28 -06:00 0:14:09
CUTOVER NOT STARTED
CLEANUP NOT STARTED
File System
Synchronization Used in this Patching Cycle: None
For more
information, run ADOP Status Report by using -detail option
Generating
ADOP Status Report at location: /inst122i/applmgr/fs_ne/EBSapps/log/status_20131117_031159/adzdshowstatus.out
Please wait...
Done...!
adop exiting
with status = 0 (Success)
$
adop phase=cutover
Enter the APPS
password:
Enter the
SYSTEM password:
Enter the
WLSADMIN password:
Please wait. Validating credentials...
RUN file system context file: /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
PATCH file system context file: /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
*************
Start of session *************
version: 12.2.0
started at: Sun Nov 17 2013 03:12:38
APPL_TOP is
set to /inst122i/applmgr/fs2/EBSapps/appl
Using ADOP Session ID from currently
incomplete patching cycle
[START
2013/11/17 03:13:12] adzdoptl.pl run
ADOP Session ID: 9
Phase: cutover
Log file: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/adop_20131117_031215.log
[EVENT]
Executing pre-cutover call-out script.
[START 2013/11/17 03:13:33] cutover phase
[EVENT]
[START 2013/11/17 03:13:43] Performing Cutover Phase steps
[EVENT] [START 2013/11/17 03:13:59] ICM Shutdown
You are
running adcmctl.sh version 120.19.12020000.3
Shutting down
concurrent managers for INST122I ...
ORACLE
Password:
Submitted
request 476567 for CONCURRENT FND SHUTDOWN
adcmctl.sh:
exiting with status 0
adcmctl.sh:
check the logfile /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/logs/appl/admin/log/adcmctl.txt
for more information ...
[EVENT] [END
2013/11/17 03:14:04] ICM Shutdown
[EVENT] [START 2013/11/17 03:14:10] Waiting for
ICM to go down
[EVENT] [END
2013/11/17 03:17:45] Waiting for ICM to go down
[EVENT] [START 2013/11/17 03:17:50] Bringing down
Middle Tier Services
[EVENT] Calling: /inst122i/applmgr/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl
Program : started @ Sun Nov 17 03:17:53 2013
*** Log File =
/inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/TXK_CTRL_forceshutdown_Sun_Nov_17_03_17_53_2013/txkADOPCutOverPhaseCtrlScript_Sun_Nov_17_03_17_53_2013.log
Program : completed @ Sun Nov 17 03:19:28 2013
[EVENT] [END
2013/11/17 03:19:28] Bringing down Middle Tier Services
Calling: adpatch options=hotpatch,nocompiledb
interactive=no console=no workers=4 restart=no abandon=yes defaultsfile=/inst122i/applmgr/fs1/EBSapps/appl/admin/INST122I_patch/adalldefaults.txt
patchtop=/inst122i/applmgr/fs1/EBSapps/appl/ad/12.0.0/patch/115/driver
logfile=cutover.log driver=ucutover.drv
ADPATCH Log directory: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/log
[EVENT] [START 2013/11/17 03:21:16] Atomic File System
Cutover
Program : started @ Sun Nov 17 03:21:17 2013
*** Log File =
/inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/TXK_CTRL_ctxupdate_Sun_Nov_17_03_21_17_2013/txkADOPCutOverPhaseCtrlScript_Sun_Nov_17_03_21_17_2013.log
Program : completed @ Sun Nov 17 03:22:53 2013
[EVENT] [END
2013/11/17 03:22:53] Atomic File System Cutover
[EVENT] [START 2013/11/17 03:23:00] Starting Middle
Tier Services
Program : started @ Sun Nov 17 03:23:00 2013
*** Log File =
/inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/TXK_CTRL_forcestartup_Sun_Nov_17_03_23_00_2013/txkADOPCutOverPhaseCtrlScript_Sun_Nov_17_03_23_00_2013.log
Program : completed @ Sun Nov 17 03:32:21 2013
[EVENT] [END
2013/11/17 03:32:21] Starting Middle Tier Services
[EVENT]
[END 2013/11/17 03:32:30]
Performing Cutover Phase steps
[END
2013/11/17 03:32:37] cutover phase
[START 2013/11/17 03:32:38] Generating Post
Cutover Reports
[END
2013/11/17 03:32:39] Generating Post Cutover Reports
[START 2013/11/17 03:32:43] Generating
AD_ZD_LOGS Report
[EVENT] Report: /inst122i/applmgr/fs1/EBSapps/appl/ad/12.0.0/sql/ADZDSHOWLOG.sql
[EVENT] Output: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/adzdshowlog.out
[END
2013/11/17 03:32:48] Generating AD_ZD_LOGS Report
[EVENT]
Executing post-cutover call-out script.
[END 2013/11/17 03:32:50] adzdoptl.pl run
adop phase=cutover - Completed
Successfully
Log file: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/adop_20131117_031215.log
adop exiting
with status = 0 (Success)
you have mail
in /var/spool/mail/apinst122i
$
Internal Logs for ICM Shutdown
[EVENT] [START 2013/11/17 03:13:59] ICM Shutdown
[STATEMENT] Log file: UNDEF
[EVENT] [END 2013/11/17 03:14:04] ICM Shutdown
[PROCEDURE] [START 2013/11/17 03:14:05] Cancelling ADZDPATCH concurrent
program
[STATEMENT] Concurrent Request Number: 476555
[PROCEDURE] [END 2013/11/17
03:14:10] Cancelling ADZDPATCH concurrent program
[EVENT] [START 2013/11/17
03:14:10] Waiting for ICM to go down
[STATEMENT] [START 2013/11/17 03:14:11] Check if ICM is running
[PROCEDURE] [START 2013/11/17 03:14:14] ICM Status
[PROCEDURE] [END 2013/11/17
03:14:16] ICM Status
[STATEMENT] ICM is active
[STATEMENT] [END 2013/11/17
03:14:17] Check if ICM is running
[PROCEDURE] ICM is still not down . Will wait for another minute.
[STATEMENT] [START 2013/11/17 03:15:21] Check if ICM is running
[PROCEDURE] [START 2013/11/17 03:15:24] ICM Status
[PROCEDURE] [END 2013/11/17
03:15:25] ICM Status
[STATEMENT] ICM is active
[STATEMENT] [END 2013/11/17
03:15:27] Check if ICM is running
[PROCEDURE] ICM is still not down . Will wait for another minute.
[STATEMENT] [START 2013/11/17 03:16:31] Check if ICM is running
[PROCEDURE] [START 2013/11/17
03:16:34] ICM Status
[PROCEDURE] [END 2013/11/17
03:16:36] ICM Status
[STATEMENT] ICM is active
[STATEMENT] [END 2013/11/17
03:16:37] Check if ICM is running
[PROCEDURE] ICM is still not down . Will wait for another minute.
[STATEMENT] [START 2013/11/17 03:17:41] Check if ICM is running
[STATEMENT] ICM is not active
[STATEMENT] [END 2013/11/17
03:17:44] Check if ICM is running
[EVENT] [END 2013/11/17 03:17:45] Waiting for ICM to go
down
Internal Logs for Shutdown Middle Tier
Services
[EVENT]
[START 2013/11/17 03:17:50] Bringing down Middle Tier Services
[EVENT] Calling: /inst122i/applmgr/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPCutOverPhaseCtrlScript.pl
[STATEMENT] Output directory: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01
[EVENT] [END 2013/11/17 03:19:28] Bringing down Middle
Tier Services
-----------
Values used
-----------
Context File
: /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
PATCH Context File
: /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
OUT Directory
: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01
Action
: forceshutdown
Console Logging
: off
Multi-node instance : NO
Executing SYSTEM command: /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/admin/scripts/adstpall.sh
-nopromptmsg -abortCM -console=off
EXIT STATUS: 0
Copying the file
----------------
SOURCE : /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/logs/appl/admin/log/adstpall.log
TARGET : /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/TXK_CTRL_forceshutdown_Sun_Nov_17_03_17_53_2013/adstpall.log
/inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/admin/scripts/adstpall.sh
went through successfully.
Internal Logs for Database Cutover
[STATEMENT] Sourcing the Patch Edition environment: /inst122i/applmgr/fs1/EBSapps/appl/APPSINST122I_testserver01.env
[PROCEDURE] [START 2013/11/17 03:19:35] Performing Automatic DB cutover
[PROCEDURE] [Start 2013/11/17 03:19:36]
ADOP::DefaultsFileUtil::substituteDefaultsFileTokensForPatchApplTop
[PROCEDURE] Reading defaults file /inst122i/applmgr/fs1/EBSapps/appl/admin/INST122I_patch/adalldefaults.txt
[PROCEDURE] Substituting tokens ...
[PROCEDURE] Regenerating defaults file ...
[PROCEDURE] [End 2013/11/17
03:19:37] ADOP::DefaultsFileUtil::substituteDefaultsFileTokensForPatchApplTop
[STATEMENT] Log file: adop_20131117_031215.log
Calling: adpatch
options=hotpatch,nocompiledb interactive=no console=no workers=4 restart=no
abandon=yes defaultsfile=/inst122i/applmgr/fs1/EBSapps/appl/admin/INST122I_patch/adalldefaults.txt
patchtop=/inst122i/applmgr/fs1/EBSapps/appl/ad/12.0.0/patch/115/driver
logfile=cutover.log driver=ucutover.drv
ADPATCH Log directory: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/log
[PROCEDURE] Running: adpatch options=hotpatch,nocompiledb interactive=no
console=no workers=4 restart=no abandon=yes defaultsfile=/inst122i/applmgr/fs1/EBSapps/appl/admin/INST122I_patch/adalldefaults.txt
patchtop=/inst122i/applmgr/fs1/EBSapps/appl/ad/12.0.0/patch/115/driver
logfile=cutover.log driver=ucutover.drv stdin=yes
[PROCEDURE] [END 2013/11/17
03:20:52] Performing Automatic DB cutover
Additional Tasks
[PROCEDURE] [START 2013/11/17 03:20:53] Loading Jar
files to DB
[STATEMENT] No jars found to load
[PROCEDURE] [END
2013/11/17 03:20:54] Loading Jar
files to DB
Internal Logs for Setting Patch
Edition as Default Edition
[PROCEDURE] [START 2013/11/17 03:20:55] Setting
Patch Edition as Default Edition
[STATEMENT] sql stmt : <
declare
result varchar2(10);
begin
ad_zd_adop.cutover;
exception when others
then
update
ad_adop_sessions set status='F'
where
adop_session_id=9;
commit;
raise_application_error(-20001,'Error while calling ad_zd_adop.cutover.'
|| sqlerrm);
end;>
[STATEMENT] Sourcing the Run Edition environment: /inst122i/applmgr/fs2/EBSapps/appl/APPSINST122I_testserver01.env
[PROCEDURE] [END 2013/11/17
03:21:00] Setting Patch Edition as Default Edition
Internal Logs for Flipping Snapshots
[PROCEDURE] [START 2013/11/17 03:21:07] Flipping
Snapshots
[STATEMENT] sql stmt : <
declare
result varchar2(10);
begin
ad_zd_adop.flip_snapshots;
exception when others then
update ad_adop_sessions
set status='F'
where
adop_session_id=9;
commit;
raise_application_error(-20001,'Error while calling ad_zd_adop.flip_snapshots.'
|| sqlerrm);
end;>
[PROCEDURE] [END 2013/11/17
03:21:10] Flipping Snapshots
Internal Logs for Atomic File System
Cutover
Script Name : txkADOPCutOverPhaseCtrlScript.pl
-----------
Values used
-----------
Context File
: /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
PATCH Context File
: /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
OUT Directory
: /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01
Action : ctxupdate
Console Logging
: off
Multi-node instance : NO
What it does ?
·
Backup the run and patch edition
context files to the FS_NE filesystem.
·
Backup the run and patch edition
environment variables + EBSapps.env to the FS_NE filesystem.
·
Backup the run and patch edition
fsclone_config.txt files to the FS_NE filesystem.
·
Reads the RUN edition context file
version from the database and compares it with the filesystem.
Context File Version = 120.401.12020000.21
FS: serial_no_fs = 0
DB: serial_no_db = 0
FS: ctx_version_fs = 120.401.12020000.21
DB: ctx_version_db = 120.401.12020000.21
Context File: /inst122i/applmgr/fs2/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
is in SYNC with DB, no need to download.
·
Reads the PATCH edition context file
version from the database and compares it with the filesystem.
Context File Version = 120.401.12020000.21
FS: serial_no_fs = 0
DB: serial_no_db = 0
FS: ctx_version_fs = 120.401.12020000.21
DB: ctx_version_db = 120.401.12020000.21
Context File: /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/appl/admin/INST122I_testserver01.xml
is in SYNC with DB, no need to download.
·
Updating Context Variable
NAME : s_file_edition_type
VALUE : patch
Update successful
Updating Context Variable
-------------------------
NAME : s_file_edition_type
VALUE : run
Update successful
·
Executes Autoconfig and creates the
following files using its corresponding template files
o INST122I_testserver01.env – environment variable for patch and run
edition.
o fsclone_config.txt – used by fsclone tool internally
o EBSapps.env
·
Uploads the updated context file back
to the database.
·
Copies the new RUN and PATCH configuration
files to the cutover folder
o INST122I_testserver01.xml
·
Copies the new RUN and PATCH deployment
plans to the cutover folder
o FMW_Home/Oracle_EBS-app1/deployment_plans
·
Copies the new RUN and PATCH OHS
webtire configuration files to the cutover folder
o FMW_Home/webtier/instances/EBS_web_INST122I_OHS1/config/OHS
·
Finally does cleanup activity
o cleanupCaboStylesCache() - FMW_Home/Oracle_EBS-app1/applications/oacore/html/cabo/styles/cache
Internal Logs for Startup Services
[EVENT] [START 2013/11/17 03:23:00] Starting
Middle Tier Services
[EVENT] [END 2013/11/17 03:32:21] Starting Middle Tier
Services
================================
Inside forceStartupServices()...
================================
Executing SYSTEM command: /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/admin/scripts/adstrtal.sh
-nopromptmsg -console=off
EXIT STATUS: 0
Copying the file
----------------
SOURCE : /inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/logs/appl/admin/log/adstrtal.log
TARGET : /inst122i/applmgr/fs_ne/EBSapps/log/adop/9/cutover_20131117_031215/INST122I_testserver01/TXK_CTRL_forcestartup_Sun_Nov_17_03_23_00_2013/adstrtal.log
/inst122i/applmgr/fs1/inst/apps/INST122I_testserver01/admin/scripts/adstrtal.sh
went through successfully.
