Thursday, May 23, 2013

Rman Duplicate (RAC 2-Node)… ASM (same storage/disk group)….

I would like to share the some of issues and things which we should concentrate while performing duplicate of production/development databases on same server and same shared storage. Most of the Administrators performed the task but still most of the upcoming dba’s didn’t get good hold of rman stuff. Just and lesson learning's.  I tried to play around on server as I was known with consequences (don’t play as I do – do take your management approval).

I have created initially static listener and tns entry for duplicate database which I intended to do build on same cluster.

### dup ####       
dup,dup.test.com =
  (DESCRIPTION =
   (LOAD_BALANCE=ON)
    (ADDRESS_LIST =
     (ADDRESS=(PROTOCOL=TCP)(HOST=node4-vip)(PORT=1536))
     (ADDRESS=(PROTOCOL=TCP)(HOST=node5-vip)(PORT=1536))
    )
   (CONNECT_DATA =
     (SERVICE_NAME = dup)
   )
)
### dup ####       

##########################################################
# listener_dup
##########################################################

listener_dup =
(ADDRESS_LIST=
  (ADDRESS=(PROTOCOL=tcp)
           (HOST=node4-vip.us.oracle.com)
           (QUEUESIZE=256)
           (PORT=1536)
  )
)
sid_list_listener_dup =
(SID_LIST =
  (SID_DESC= (GLOBAL_DBNAME = dup.test.com)
             (SID_NAME=dup1)
             (SDU=32768)
             (ORACLE_HOME =/u01/app/oracle/product/dup )
  )
)

########################################################

At first glance, I tot the entries are good and didn’t have any issue and then I continued with task of rman duplicate. Next step we need to create initORACLE_SID.ora file to start the instance in nomount mode.

DB_NAME=DUP
DB_UNIQUE_NAME=DUP
DB_FILE_NAME_CONVERT='+DATA/PROD','+DATA/DUP','+RECO/PROD','+RECO/DUP'
LOG_FILE_NAME_CONVERT='+DATA/PROD','+DATA/DUP','+RECO/PROD','+RECO/DUP'
sga_target=18G
pga_aggregate_target=4G
diagnostic_dest=/u02/diag/dup

I have started the instance and continued with rman duplicate (Note- I’m using the 11203 binaries)

rman target sys/sys1@prod auxiliary /

You will face any issue as below

Starting Duplicate Db at 22-MAY-13
released channel: prim1
released channel: dup1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 05/22/2013 02:21:15
RMAN-05501: aborting duplication of target database
RMAN-06217: not connected to auxiliary database with a net service name

Never try to connect to auxiliary instance as Host Authentication you will face an issue. Next continued with below

rman targetsys/sys1@prod auxiliary sys/sys1@dup (Note –Copy the password file of production,else it won’t work)

Now this time it failed with no listener issue. Now, I have reviewed my tnsnames.ora and listener.ora (bounced the listener) many times, still issue persist. Couldn’t able figure out the issue/cause. Then if you carefully observe,

service_name and global_dbname in listener.ora file and tnsnames.ora both mis-matches with domain name, which caused me to pause my work for a while. (To be honest, I made my entries to upper case and made sure that both matches then I was able to proceed further.

Now, next I have resolved my connections issue and continued with task, and next issue continues

Errors in file /u02/oraout/dup/diag/rdbms/dup/dup1/trace/dup1_ora_18406.trc:
ORA-19505: failed to identify file "+DATA/dup/controlfile/control_01.ctl"
ORA-17502: ksfdcre:3 Failed to create file +DATA/dup/controlfile/control_01.ctl
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15204: database version 11.2.0.0.0 is incompatible with diskgroup DATA

As my compatible parameter maps to lower version (perhaps I didn’t placed any parameter in my auxiliary init.ora file), in order to resolve this I modified my initialization parameter with “compatible=11.2.0.3.0” and continued with task.

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 05/22/2013 03:01:48
RMAN-05501: aborting duplication of target database
RMAN-05001: auxiliary file name +DATA/admin/prod/db/test_02.dbf conflicts with a file used by the target database
RMAN-05001: auxiliary file name +DATA/admin/prod/db/users_06.dbf conflicts with a file used by the target database

Now, the cause of this issue was, I missed one of the entry to place (which production datafiles get maps to) an entry in db_file_name_convert, due to which the task was aborted.

Note – In case if any one missed out the entries to place the in parameters (db_file_name_convert or log_file_name_convert), rman takes care of aborting the process, instead of overriding /corrupting production data files.

Now, I have updated my db_file_name_convert parameter as below

DB_FILE_NAME_CONVERT='+DATA/PROD','+DATA/DUP','+RECO/PROD','+RECO_UBA/DUP','+DATA/admin/prod','+DATA/admin/dup'

 

RMAN> run {
2> allocate channel prim1 device type disk;
3> allocate auxiliary channel dup1 device type disk;
4> duplicate database to dup from active database;
5> }

Let ‘s  me provide the actual process of duplication,

Task 1 = (Spfile and control file)
    Rebounce of instance (opens in mount mode)
    set’s db_name and db_unique_name to spfile
    Rebounce of instance (opens in mount mode)
    Take’s backup of controlfile from production
    Now mount the database with controlfile.

 

Task 2 = Data file copy

    set newname for datafile n (setting new location in current rman session)
    backup as copy reuse datafile  1 auxiliary format
    Note - AS COPY --performing image copy of datafile
    So, it copies from prod === dup over network(via.. sqlnet)
    sql 'alter system archive log current';
    Note - Last Step making sure that, we are making/recovering our db till the last Archived Online Redo log file.


Task 3 =

    Archive Files Copy and re-using
    backup as copy reuse
       archivelog like  "+RECO/prod/archivelog/2013_05_22/thread_2_seq_248.126795.816051805"
       auxiliary format "/u01/app/oracle/product/dup/dbs/arch2_248_809881982.dbf" 
       .....
       .....
       ...
       catalog clone archivelog "/u01/app/oracle/product/dup/dbs/arch2_248_809881982.dbf" (updating information to controlfile)
    switch clone datafile all;

Task 4 = Last task is recovery

    set until scn  13239404328241;
       recover
       clone database
        delete archivelog ;

    archived log for thread 1 with sequence 303 is already on disk as file /u01/app/oracle/product/dup/dbs/arch1_303_809881982.dbf
    archived log for thread 2 with sequence 249 is already on disk as file /u01/app/oracle/product/dup/dbs/arch2_249_809881982.dbf
    archived log file name=/u01/app/oracle/product/dup/dbs/arch1_303_809881982.dbf thread=1 sequence=303
    archived log file name=/u01/app/oracle/product/dup/dbs/arch2_249_809881982.dbf thread=2 sequence=249
    media recovery complete, elapsed time: 00:00:02
    Finished recover at 22-MAY-13
    Oracle instance started

Note - we can controlfile the archive log generation by using parameter "db_recovery_file_dest" or "log_archive_dest_%' else it maps to %ORACLE_HOME/dbs" folder, that's what we have seen here.

Further, I would like to show the importance of alert log monitoring, if you see the below messages

Reconfiguration complete
ORACLE_BASE from environment = /u01/app/oracle
Wed May 22 03:20:07 2013
alter database mount
Set as converted control file due to db_unique_name mismatch
Changing di2dbun from prod to DUP
Successful mount of redo thread 1, with mount id 1039698535
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: alter database mount
Wed May 22 03:20:17 2013
Starting background process ASMB

If you try to understand, due the process/execution of  Task –1, the Oracle is setting the db_name and db_unique_name @rman level, same mapped/written to alert log, perhaps the difference of communication to alert log is informative. So, do monitor alert log and traces files during task/execution.

I hope it the post helps to visualize things in different way…!!

Let me know your comments and things which you would like to see in upcoming posts.

1 comments:

Anonymous said...

Is this an OMF database you are trying to duplicate? As per documentation

"If the source database files are in the Oracle Managed Files (OMF) format, then you cannot use the DB_FILE_NAME_CONVERT and LOG_FILE_NAME_CONVERT initialization parameters or the fileNameConversionSpec clause to generate new OMF filenames for the duplicate database. OMF filenames are unique and generated by Oracle Database"

 
© 2013 Oracle Internals from basics to advanced....