Wednesday, 20 October 2021

RMAN Duplicate failed , errors RMAN-05501: aborting duplication of target database,RMAN-03015,RMAN-04006,ORA-12514

rman duplicate command failed with error while allocating  multiple channel through run block


$ cat Proddb_standby_build.sh

connect target sys/Password@PRODDB_PRIM

connect auxiliary  sys/Password@PRODDB_OCI_STDBY

run {

allocate channel ch1 type disk;

allocate channel ch2 type disk;

allocate channel ch3 type disk;

allocate channel ch4 type disk;

allocate channel ch5 type disk;

allocate channel ch6 type disk;

allocate auxiliary channel ch7 type disk;

allocate auxiliary channel ch8 type disk;

DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE DORECOVER;

}



error details:

$ tail -90f Proddb_standby_build.log


Spooling started in log file: /home/oracle/scripts/Migration/proddb/Proddb_standby_build.log


Recovery Manager19.10.0.0.0


RMAN>

PL/SQL package SYS.DBMS_BACKUP_RESTORE version 19.08.00.00 in TARGET database is not current

PL/SQL package SYS.DBMS_RCVMAN version 19.08.00.00 in TARGET database is not current

connected to target database: PRODDB (DBID=2143832746)


RMAN>

connected to auxiliary database: PRODDB (not mounted)


RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11>

using target database control file instead of recovery catalog

allocated channel: ch1

channel ch1: SID=1565 instance=proddb1 device type=DISK


allocated channel: ch2

channel ch2: SID=1711 instance=proddb1 device type=DISK


allocated channel: ch3

channel ch3: SID=1778 instance=proddb1 device type=DISK


allocated channel: ch4

channel ch4: SID=1850 instance=proddb1 device type=DISK


allocated channel: ch5

channel ch5: SID=2134 instance=proddb1 device type=DISK


allocated channel: ch6

channel ch6: SID=2205 instance=proddb1 device type=DISK


allocated channel: ch7

channel ch7: SID=2411 device type=DISK


allocated channel: ch8

channel ch8: SID=2477 device type=DISK


Starting Duplicate Db at 19-OCT-21

current log archived


contents of Memory Script:

{

   backup as copy reuse

   passwordfile auxiliary format  '/u02/app/oracle/product/19.0.0.0/dbhome_2/dbs/orapwproddb11'   ;

}

executing Memory Script


Starting backup at 19-OCT-21

Finished backup at 19-OCT-21

duplicating Online logs to Oracle Managed File (OMF) location

duplicating Datafiles to Oracle Managed File (OMF) location


contents of Memory Script:

{

   sql clone "alter system set  control_files =

  ''+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267'' comment=

 ''Set by RMAN'' scope=spfile";

   backup as copy current controlfile for standby auxiliary format  '+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267';

   sql clone "alter system set  control_files =

  ''+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267'' comment=

 ''Set by RMAN'' scope=spfile";

   shutdown clone immediate;

   startup clone nomount;

}

executing Memory Script


sql statement: alter system set  control_files =   ''+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267'' comment= ''Set by RMAN'' scope=spfile


Starting backup at 19-OCT-21

channel ch1: starting datafile copy

copying standby control file

output file name=+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267 tag=TAG20211019T015841

channel ch1: datafile copy complete, elapsed time: 00:00:03

Finished backup at 19-OCT-21


sql statement: alter system set  control_files =   ''+DATAC1/PRODDB_OCI/CONTROLFILE/current.328.1079736267'' comment= ''Set by RMAN'' scope=spfile


Oracle instance shut down


released channel: ch1

released channel: ch2

released channel: ch3

released channel: ch4

released channel: ch5

released channel: ch6

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of Duplicate Db command at 10/19/2021 01:59:21

RMAN-05501: aborting duplication of target database

RMAN-03015: error occurred in stored script Memory Script

RMAN-04006: error from auxiliary database: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor



Solution: Configure parallelism in rman and run duplicate 


Connect to rman target:

rman 

connect target sys/Password@PRODDB_PRIM  

connect auxiliary  sys/Password@PRODDB_OCI_STDBY


Configure parallelism:

CONFIGURE DEVICE TYPE DISK PARALLELISM 8 BACKUP TYPE TO BACKUPSET;


Duplicate will run without any errors now

DUPLICATE TARGET DATABASE FOR STANDBY FROM ACTIVE DATABASE DORECOVER;



Saturday, 2 October 2021

Failed to start Oracle OHASD service error while running 18c grid patch

18c grid opatchauto apply failed with errors, lets find root cause of failure and issue fix 


Apply grid patch with opatchauto:

root@racnode01:# opatchauto apply /u01/app/19c_Software/27494830  -oh /u01/app/18.3.0.0/grid


OPatchauto session is initiated at Tue Sep 28 04:08:55 2021


System initialization log file is /u01/app/18.3.0.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-09-28_04-08-58AM.log.


Session log file is /u01/app/18.3.0.0/grid/cfgtoollogs/opatchauto/opatchauto2021-09-28_04-11-20AM.log

The id for this session is 5INE


Executing OPatch prereq operations to verify patch applicability on home /u01/app/18.3.0.0/grid

Patch applicability verified successfully on home /u01/app/18.3.0.0/grid


Bringing down CRS service on home /u01/app/18.3.0.0/grid

CRS service brought down successfully on home /u01/app/18.3.0.0/grid


Start applying binary patch on home /u01/app/18.3.0.0/grid

Binary patch applied successfully on home /u01/app/18.3.0.0/grid


Starting CRS service on home /u01/app/18.3.0.0/grid

Failed to start CRS service on home /u01/app/18.3.0.0/grid


Execution of [GIStartupAction] patch action failed, check log for more details. Failures:

Patch Target : racnode01->/u01/app/18.3.0.0/grid Type[crs]

Details: [

---------------------------Patching Failed---------------------------------

Command execution failed during patching in home: /u01/app/18.3.0.0/grid, host: racnode01.

Command failed:  /u01/app/18.3.0.0/grid/perl/bin/perl -I/u01/app/18.3.0.0/grid/perl/lib -I/u01/app/18.3.0.0/grid/OPatch/auto/dbtmp/bootstrap_racnode01/patchwork/crs/install /u01/app/18.3.0.0/grid/OPatch/auto/dbtmp/bootstrap_racnode01/patchwork/crs/install/rootcrs.pl -postpatch

Command failure output: 

Using configuration parameter file: /u01/app/18.3.0.0/grid/OPatch/auto/dbtmp/bootstrap_racnode01/patchwork/crs/install/crsconfig_params

The log of current session can be found at:

  /app/grid/crsdata/racnode01/crsconfig/crs_postpatch_racnode01_2021-09-28_04-13-13AM.log

2021/09/28 04:13:37 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'

2021/09/28 04:14:13 CLSRSC-318: Failed to start Oracle OHASD service 


After fixing the cause of failure Run opatchauto resume]

OPATCHAUTO-68061: The orchestration engine failed.

OPATCHAUTO-68061: The orchestration engine failed with return code 1

OPATCHAUTO-68061: Check the log for more details.

OPatchAuto failed.


OPatchauto session completed at Tue Sep 28 04:15:27 2021

Time taken to complete the session 6 minutes, 32 seconds


opatchauto failed with error code 42

 

From patch log file:

2021-09-28 05:47:03: The unit oracle-ohasd.service may not be installed

2021-09-28 05:47:03: isRunning: 0; isEnabled: 0

2021-09-28 05:47:03: remove service file: /etc/systemd/system/oracle-ohasd.service

2021-09-28 05:47:03: Removing file /etc/systemd/system/oracle-ohasd.service

2021-09-28 05:47:03: Successfully removed file: /etc/systemd/system/oracle-ohasd.service

2021-09-28 05:47:03: SYSTEMD: Copying /u01/app/18.3.0.0/grid/crs/install/oracle-ohasd.service to /etc/systemd/system/oracle-ohasd.service

2021-09-28 05:47:03: Executing cmd: /usr/bin/systemctl daemon-reload

2021-09-28 05:47:34: Command output:

>  Failed to execute operation: Connection timed out 

2021-09-28 05:47:34: failed to reload systemd for scanning for changed units

2021-09-28 05:47:34: Executing cmd: /u01/app/18.3.0.0/grid/bin/clsecho -p has -f clsrsc -m 213 'oracle-ohasd.service' '25'

2021-09-28 05:48:47: Executing cmd: /u01/app/18.3.0.0/grid/bin/clsecho -p has -f clsrsc -m 213 'oracle-ohasd.service' '25'

2021-09-28 05:48:47: Command output:

>  CLSRSC-213: Failure in reading file 'oracle-ohasd.service' (error: 25) 

>End Command output

2021-09-28 05:48:47: CLSRSC-213: Failure in reading file 'oracle-ohasd.service' (error: 25)

2021-09-28 05:48:47: Executing cmd: /u01/app/18.3.0.0/grid/bin/clsecho -p has -f clsrsc -m 318

2021-09-28 05:48:47: Executing cmd: /u01/app/18.3.0.0/grid/bin/clsecho -p has -f clsrsc -m 318

2021-09-28 05:48:47: Command output:

>  CLSRSC-318: Failed to start Oracle OHASD service 



From patch logs we see that connection timed out errors while running systemctl daemon-reload command,  it might be a reason for patch failure 

lets try daemon-reload 

root@racnode01:# /usr/bin/systemctl daemon-reload

Failed to execute operation: Connection timed out -- same error reported


Check os process related to systemd

root@racnode01:# ps -ef | grep systemd

root   1     0  0 Sep20 ?      00:01:07 /usr/lib/systemd/systemd --switched-root --system --deserialize 22


Issue:

process 1 (systemd) was started with --switched-root and --deserialize.looks it's trying to reload with previously saved state.


Solution: 

1. System reboot will clear all child process and help in applying patch successfully -- we rebooted node and resumed patching 

2. kill -9 process 1 also will help 


Resume failed patch, it will complete successfully now 

root@racnode01:# opatchauto resume

OPatchauto session is initiated at Tue Sep 28 23:17:38 2021

Session log file is /u01/app/18.3.0.0/grid/cfgtoollogs/opatchauto/opatchauto2021-09-28_11-17-40PM.log

Resuming existing session with id 5INE

Checking shared status of home.....


Starting CRS service on home /u01/app/18.3.0.0/grid

CRS service started successfully on home /u01/app/18.3.0.0/grid


OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:


Host:iadstgracdb44

CRS Home:/u01/app/18.3.0.0/grid

Version:18.0.0.0.0

Summary:


==Following patches were SUCCESSFULLY applied:


Patch: /u01/app/19c_Software/27494830/27494830

Log: /u01/app/18.3.0.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-09-28_04-12-27AM_1.log


OPatchauto session completed at Tue Sep 28 23:29:36 2021

Time taken to complete the session 11 minutes, 59 seconds

root@racnode01:# 



Sunday, 22 August 2021

RMAN Stored backup scripts

RMAN stored scripts used to store backup scripts in catalog database with local and global options, 

Local script are specific to target database which is connected when script created,
Global scripts can be run in any db registered with catalog database.

Create stored scripts:
need to connect catalog db to create stored scripts,

/home/oracle> rman target /
Recovery Manager: Release 19.0.0.0.0 - Production on Sun Aug 22 22:57:24 2021
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.
connected to target database: LABDB19C (DBID=4148792351)

RMAN> connect catalog rcat/rcat@labdb19c_test1
connected to recovery catalog database

before creating scripts, lets check if any existing stored scripts available in db or not
RMAN> LIST GLOBAL SCRIPT NAMES;
List of Stored Scripts in Recovery Catalog
        No scripts in recovery catalog

RMAN> LIST SCRIPT NAMES;
List of Stored Scripts in Recovery Catalog
        No scripts in recovery catalog

catalog db don't have stored scripts, 
Create global script: 
RMAN> CREATE GLOBAL SCRIPT DB_BACKUP_FULL
{     
  BACKUP DATABASE PLUS ARCHIVELOG format '/backup/exports/lab19c_bkps/rman_bkp/%d_db_full_%u_%t';
}
created global script DB_BACKUP_FULL

Global script created ,validate with list command 
RMAN>  LIST GLOBAL SCRIPT NAMES;
List of Stored Scripts in Recovery Catalog
    Global Scripts

       Script Name
       Description
       ---------------------------------------------
       DB_BACKUP_FULL   
       
Create local script:
RMAN> CREATE SCRIPT DB_BACKUP_FULL
{     
  BACKUP DATABASE PLUS ARCHIVELOG format '/backup/exports/lab19c_bkps/rman_bkp/%d_db_full_%u_%t';
}
created script DB_BACKUP_FULL

Local script created ,validate with list command 
RMAN>  LIST SCRIPT NAMES;
List of Stored Scripts in Recovery Catalog

    Scripts of Target Database LABDB19C
       Script Name
       Description
       --------------------------------------
       DB_BACKUP_FULL

    Global Scripts
       Script Name
       Description
       ---------------------------------------
       DB_BACKUP_FULL

Create script from file:
/home/oracle/jay> cat /home/oracle/jay/Delete_archivelog.txt
{
delete noprompt archivelog until time 'sysdate-6/24';
}

RMAN> CREATE GLOBAL SCRIPT Delete_archives FROM FILE '/home/oracle/jay/Delete_archivelog.txt';

script commands will be loaded from file /home/oracle/jay/Delete_archivelog.txt
created global script Delete_archives

RMAN>  LIST GLOBAL SCRIPT NAMES;

List of Stored Scripts in Recovery Catalog

    Global Scripts
       Script Name
       Description
       -------------------------------------
       DB_BACKUP_FULL
       Delete_archives

Stored script backup to file:
RMAN> PRINT GLOBAL SCRIPT Delete_archives TO FILE '/home/oracle/jay/Delete_archivelog.txt_bkp';

global script Delete_archives written to file /home/oracle/jay/Delete_archivelog.txt_bkp

validate backup file:
/home/oracle/jay > cat /home/oracle/jay/Delete_archivelog.txt_bkp
{
delete noprompt archivelog until time 'sysdate-6/24';
}

Validate backup scripts:
RMAN> PRINT SCRIPT DB_BACKUP_FULL;
printing stored script: DB_BACKUP_FULL
{     
  BACKUP DATABASE PLUS ARCHIVELOG format '/backup/exports/lab19c_bkps/rman_bkp/%d_db_full_%u_%t';
}

RMAN> PRINT GLOBAL SCRIPT Delete_archives;
printing stored global script: Delete_archives
{
delete noprompt archivelog until time 'sysdate-6/24';
}

List commands:
LIST SCRIPT NAMES;
LIST GLOBAL SCRIPT NAMES;
LIST ALL SCRIPT NAMES;

Delete stored scripts:
DELETE SCRIPT global_full_backup;

example:
RMAN> DELETE SCRIPT Delete_archives;
deleted global script: Delete_archives


Run backups using stored scripts:
Commands:
RUN { EXECUTE GLOBAL SCRIPT DB_BACKUP_FULL ;}
RUN { EXECUTE SCRIPT DB_BACKUP_FULL; }

example:
RMAN> RUN { EXECUTE GLOBAL SCRIPT Delete_archives ;} 

executing global script: Delete_archives

released channel: ORA_DISK_1
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=285 device type=DISK
List of Archived Log Copies for database with db_unique_name LABDB19C
======================================================
Key     Thrd Seq     S Low Time       
------- ---- ------- - ---------------
1230    1    1119    A 20210802.082001
        Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_02/thread_1_seq_1119.290.1079511603
1231    1    1120    A 20210802.082002
        Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_02/thread_1_seq_1120.309.1079511603

deleted archived log
archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_02/thread_1_seq_1119.290.1079511603 RECID=17 STAMP=1079511602
deleted archived log
archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_02/thread_1_seq_1120.309.1079511603 RECID=18 STAMP=1079511603
Deleted 2 objects

Saturday, 21 August 2021

Restart Oracle Enterprise Manager 13c Cloud Control(OMS)

# Stop oms, agent and oms db 

$OMS_HOME/bin/emctl stop oms -all

$AGENT_HOME/bin/emctl stop agent

$ORACLE_HOME/bin/dbshut $ORACLE_HOME

# Start oms, agent and oms db 

$ORACLE_HOME/bin/dbstart $ORACLE_HOME

$OMS_HOME/bin/emctl start oms

$AGENT_HOME/bin/emctl start agent 

Check OMS status:

$OMS_HOME/bin/emctl status oms

$OMS_HOME/bin/emctl status oms  -details


Check agent status:
$AGENT_HOME/bin/emctl status agent

ERROR OGG-01260 Oracle GoldenGate Capture for Oracle, No sufficient space on directories

Golden Gate extract struck from last 26 hours, tried to restart extract process but it’s not allowing to stop  

GGSCI (racnode01)> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT     RUNNING     DINCS3      00:00:00      00:00:03      

EXTRACT     RUNNING     EINCS3      00:00:03      26:38:07    

GGSCI (racnode01)>stop EINCS3

OGG-15163  There was a problem sending a message to EXTRACT EINCS3 (Timeout waiting for message).

Review  ggserr.log: 

2021-08-21T10:09:41.158-0700  ERROR   OGG-01260  Oracle GoldenGate Capture for Oracle, eincs3.prm:  No sufficient space on directories : de: /ACFS_OGG/gg/dirtmp, bytes: 0x00000000400000, co_uid: 0x000000029415bc, mb_uid: 0x000000002006, co: 0x00007f26cff020, mb: 0x000072f41c70, mf: 0x00007f2640140.

2021-08-21T10:09:41.158-0700  ERROR   OGG-01668  Oracle GoldenGate Capture for Oracle, eincs3.prm:  PROCESS ABENDING.


Process abended with error  No sufficient space on directories : de: /ACFS_OGG/gg/dirtmp,

But in ggsci it’s still show as running , we can kill process forcefully 

GGSCI (racnode01)> kill extract EINCS3


CACHEMGR parameter is to control the amount of virtual memory and temporary disk space that is available for caching uncommitted transaction data,

Add CACHEMGR, CACHEDIRECTORY size 100G to extract param file:

GGSCI (racnode01)>edit params EINCS3 

CACHEMGR CACHESIZE 2G,CACHEDIRECTORY /ACFS_OGG/gg/dirtmp 100G

Start extract:

GGSCI (racnode01)>start EINCS3

Monitor extract lag timing:

GGSCI (racnode01)> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT     RUNNING     DINCS3      00:00:00      00:00:03      

EXTRACT     RUNNING     EINCS3      20:50:03      00:00:07    


GGSCI (racnode01)> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING

EXTRACT     RUNNING     DINCS3      00:00:00      00:00:03      

EXTRACT     RUNNING     EINCS3      00:50:03      00:00:07    


Thursday, 12 August 2021

RMAN Point-in-Time Recovery (DBPITR) in 19c to before resetlogs

RMAN point-in-time recovery  will restore database to the exact time in the past, it helps database recovery in db crash, data corrupt and erroneous delete situations

lets try database restore using time stamps, 

Start db in nomount:

/home/oracle> sqlplus "/as sysdba"

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Aug 12 00:00:47 2021

Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

SQL> startup nomount;

ORACLE instance started.

Total System Global Area 1.0737E+10 bytes

Fixed Size                 12446800 bytes

Variable Size            1711276032 bytes

Database Buffers         8992587776 bytes

Redo Buffers               21106688 bytes

SQL> exit

Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0


Restore controlfile and mount db;

[oracle@racnode01] -->labdb19c<--

/home/oracle> rman target /

Recovery Manager: Release 19.0.0.0.0 - Production on Thu Aug 12 00:04:27 2021

Version 19.8.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: LABDB19C (not mounted)

RMAN> restore controlfile from '+POC19C_DATA/LABDB19C/AUTOBACKUP/2021_08_09/s_108104810.310.108104811';

Starting restore at 20210812.000509

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=277 device type=DISK

channel ORA_DISK_1: restoring control file

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

output file name=+POC19C_DATA/LABDB19C/CONTROLFILE/current.344.1077323871

output file name=+POC19C_DATA/LABDB19C/CONTROLFILE/current.345.1077323871

Finished restore at 20210812.000510

released channel: ORA_DISK_1

RMAN> sql 'alter database mount';

sql statement: alter database mount


Restore database until time:

RMAN> run{

allocate channel ch1 type disk;

allocate channel ch2 type disk;

set until time="to_date('2021/08/11 02:55:48', 'yyyy/mm/dd hh24:mi:ss')";

restore database;

recover database;

}

allocated channel: ch1

channel ch1: SID=352 device type=DISK

allocated channel: ch2

channel ch2: SID=122 device type=DISK

executing command: SET until clause

Starting restore at 20210812.000856

Starting implicit crosscheck backup at 20210812.000856

Crosschecked 9 objects

Crosschecked 11 objects

Finished implicit crosscheck backup at 20210812.000858

Starting implicit crosscheck copy at 20210812.000858

Finished implicit crosscheck copy at 20210812.000858

searching for all files in the recovery area

cataloging files...

cataloging done

List of Cataloged Files

=======================

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_1158.314.108272057

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_1.340.108342897

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_2.281.108342905

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_3.282.108342907

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_4.727.108342909

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_5.726.108342913

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_6.725.108342915

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_7.724.108343051

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_8.723.108343593

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_9.722.108343607

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1156.311.108176411

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1157.312.108252007

File Name: +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_09/thread_1_seq_1155.283.108122643

File Name: +POC19C_DATA/LABDB19C/AUTOBACKUP/2021_08_11/s_108272060.316.108272061

File Name: +POC19C_DATA/LABDB19C/AUTOBACKUP/2021_08_09/s_108104810.310.108104811

File Name: +POC19C_DATA/LABDB19C/AUTOBACKUP/2021_08_09/s_108105347.285.108105347

File Name: +POC19C_DATA/LABDB19C/DATAFILE/tb01.352.1078956129

released channel: ch1

released channel: ch2

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of restore command at 08/12/2021 00:08:58

RMAN-20207: UNTIL TIME or RECOVERY WINDOW is before RESETLOGS time


Note: Db restore failed with error RMAN-20207: UNTIL TIME or RECOVERY WINDOW is before RESETLOGS time

Check db incarnation details:

RMAN> list incarnation;

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       LABDB19C 4148792351       PARENT  1          20190417.005559

2       2       LABDB19C 4148792351       PARENT  1920977    20210708.003753

3       3       LABDB19C 4148792351       CURRENT 6299511    20210811.033416


Current incarnation reset time is 2021-08-11 3.34AM , we are trying to restore db before current incarnation time , 

current incarnation time should be earlier than restore time to resolve rman error 

reset incarnation:

RMAN> reset database to incarnation 2;

database reset to incarnation 2

RMAN> list incarnation;

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       LABDB19C 4148792351       PARENT  1          20190417.005559

2       2       LABDB19C 4148792351       CURRENT 1920977    20210708.003753

3       3       LABDB19C 4148792351       ORPHAN  6299511    20210811.033416


Now current incarnation reset time ‘2021-07-08’  is earlier than restore time '2021-08-11’

Try  restore db until time now:

RMAN> run{

allocate channel ch1 type disk;

allocate channel ch2 type disk;

set until time="to_date('2021/08/11 02:55:48', 'yyyy/mm/dd hh24:mi:ss')";

restore database;

recover database;

}

allocated channel: ch1

channel ch1: SID=352 device type=DISK

allocated channel: ch2

channel ch2: SID=122 device type=DISK

executing command: SET until clause

Starting restore at 20210812.001339

channel ch1: starting datafile backup set restore

channel ch1: specifying datafile(s) to restore from backup set

channel ch1: restoring datafile 00001 to +POC19C_DATA/LABDB19C/DATAFILE/system.349.1077323759

channel ch1: restoring datafile 00002 to +POC19C_DATA/LABDB19C/DATAFILE/mts01.288.1077348575

channel ch1: restoring datafile 00003 to +POC19C_DATA/LABDB19C/DATAFILE/sysaux.348.1077323793

channel ch1: restoring datafile 00004 to +POC19C_DATA/LABDB19C/DATAFILE/undotbs1.347.1077323809

channel ch1: restoring datafile 00005 to +POC19C_DATA/LABDB19C/DATAFILE/tejaswi.313.1077349553

channel ch1: restoring datafile 00007 to +POC19C_DATA/LABDB19C/DATAFILE/users.346.1077323809

channel ch1: restoring datafile 00008 to +POC19C_DATA/LABDB19C/DATAFILE/mts01.287.1077350069

channel ch1: restoring datafile 00009 to +POC19C_DATA/LABDB19C/DATAFILE/jts0101.dbf

channel ch1: restoring datafile 00010 to +POC19C_DATA/LABDB19C/DATAFILE/jts01.279.1077352257

channel ch1: restoring datafile 00011 to +POC19C_DATA/LABDB19C/DATAFILE/mts01.dbf

channel ch1: restoring datafile 00012 to +POC19C_DATA/LABDB19C/DATAFILE/jts01.343.1077353335

channel ch1: restoring datafile 00013 to +POC19C_DATA/LABDB19C/DATAFILE/teja.350.1078365639

channel ch1: restoring datafile 00014 to +POC19C_DATA/LABDB19C/DATAFILE/tb01

channel ch1: restoring datafile 00015 to +POC19C_DATA/LABDB19C/DATAFILE/tb02.351.1078956401

channel ch1: restoring datafile 00016 to +POC19C_DATA/LABDB19C/DATAFILE/users.339.1078977183

channel ch1: restoring datafile 00017 to +POC19C_DATA/LABDB19C/DATAFILE/users.338.1078979345

channel ch1: restoring datafile 00018 to +POC19C_DATA/LABDB19C/DATAFILE/users.337.1078979349

channel ch1: restoring datafile 00019 to +POC19C_DATA/LABDB19C/DATAFILE/incent_data.335.1078983489

channel ch1: restoring datafile 00020 to +POC19C_DATA/LABDB19C/DATAFILE/users.334.1079063905

channel ch1: restoring datafile 00021 to +POC19C_DATA/LABDB19C/DATAFILE/users.333.1079063949

channel ch1: restoring datafile 00022 to +POC19C_DATA/LABDB19C/DATAFILE/users.329.1079067891

channel ch1: restoring datafile 00023 to +POC19C_DATA/LABDB19C/DATAFILE/users.328.1079067923

channel ch1: restoring datafile 00024 to +POC19C_DATA/LABDB19C/DATAFILE/users.327.1079069033

channel ch1: restoring datafile 00025 to +POC19C_DATA/LABDB19C/DATAFILE/incent_data.326.1079069273

channel ch1: reading from backup piece /flashblade_backup/exports/lab19c_bkps/LABDB19C_full_108103964

channel ch1: piece handle=/flashblade_backup/exports/lab19c_bkps/LABDB19C_full_108103964 tag=TAG2021089T045244

channel ch1: restored backup piece 1

channel ch1: restore complete, elapsed time: 00:13:45

Finished restore at 20210812.002725

Starting recover at 20210812.002725

starting media recovery

archived log for thread 1 with sequence 1155 is already on disk as file +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_09/thread_1_seq_1155.283.108122643

archived log for thread 1 with sequence 1156 is already on disk as file +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1156.311.108176411

archived log for thread 1 with sequence 1157 is already on disk as file +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1157.312.108252007

archived log for thread 1 with sequence 1158 is already on disk as file +POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_1158.314.108272057

archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_09/thread_1_seq_1155.283.108122643 thread=1 sequence=1155

archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1156.311.108176411 thread=1 sequence=1156

archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_10/thread_1_seq_1157.312.108252007 thread=1 sequence=1157

archived log file name=+POC19C_DATA/LABDB19C/ARCHIVELOG/2021_08_11/thread_1_seq_1158.314.108272057 thread=1 sequence=1158

media recovery complete, elapsed time: 00:00:05

Finished recover at 20210812.002731

released channel: ch1

released channel: ch2


Restore and recover database completed successfully,

rman restore database from full and incremental backups , once restore completed it will recover database up to specified time using archive backups and available archives in db server.

Point-in-Time Recovery(PITR) is sometimes called incomplete recovery because it does not use all of the available redo or completely recover all changes to database. 

Incomplete recovery allows db to open in resetlogs options only,

Open database:

/home/oracle> sqlplus "/as sysdba"

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Aug 12 00:30:00 2021

Version 19.8.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

Connected to:

Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Version 19.8.0.0.0

SQL> alter database open resetlogs;

Database altered.

SQL> select name,open_mode from v$database;

NAME      OPEN_MODE

--------- --------------------

LABDB19C  READ WRITE


Once db open with resetlogs option it will create new incarnation 

/home/oracle> rman target /

Recovery Manager: Release 19.0.0.0.0 - Production on Thu Aug 12 00:31:03 2021

Version 19.8.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.

connected to target database: LABDB19C (DBID=4148792351)

RMAN> list incarnation;

using target database control file instead of recovery catalog

List of Database Incarnations

DB Key  Inc Key DB Name  DB ID            STATUS  Reset SCN  Reset Time

------- ------- -------- ---------------- --- ---------- ----------

1       1       LABDB19C 4148792351       PARENT  1          20190417.005559

2       2       LABDB19C 4148792351       PARENT  1920977    20210708.003753

3       3       LABDB19C 4148792351       ORPHAN  6299511    20210811.033416

4       4       LABDB19C 4148792351       CURRENT 6300585    20210812.003015