How to Backup and Restore a RAC Database

This example illustrates how to backup and recover a 2-node, 10gR2 RAC cluster on

Linux. Backup and recovery operations for RAC databases are similar to that of single

instance databases, however there are some key differences – the goal of this post is to

highlight the steps required for recoverying RAC databases.

Backup the Database

Use the method you prefer. I used a NOCATALOG, disk-based backup for this example,

using the scripts found here.

Simulate the Failure

Shutdown the database and simulate a failure by dropping and/or corrupting a datafile.

# export ORACLE_SID=em1

# . oraenv

# srvctl stop database -d em

# srvctl status database -d em

Instance em1 is not running on node rac1

Instance em2 is not running on node rac2

# cd /u02/oracle/ora1/em

# rm sysaux01.dbf

Verify the Failure

# srvctl start instance -d em -i em1

PRKP-1001 : Error starting instance em1 on node rac1

CRS-0215: Could not start resource ‘ora.em.em1.inst’.

You will find something similar to the following in the instance’s alert log:

Sat May 24 16:53:47 2008

Errors in file /u02/oracle/logs/em/bdump/em1_dbw0_16947.trc:

ORA-01157: cannot identify/lock data file 3 – see DBWR trace file

ORA-01110: data file 3: ‘/u02/oracle/ora1/em/sysaux01.dbf’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-1157 signalled during: ALTER DATABASE OPEN…

Sat May 24 16:53:49 2008

Shutting down instance (abort)

Restore the Database

First, take the database out of cluster mode via:

# sqlplus “/ as sysdba”

SQL*Plus: Release – Production on Sat May 24 17:02:17 2008

Copyright (c) 1982, 2006, Oracle.  All Rights Reserved.

Connected to an idle instance.

SQL> startup nomount;

ORACLE instance started.

Total System Global Area  268435456 bytes

Fixed Size                  1261300 bytes

Variable Size             251658508 bytes

Database Buffers            8388608 bytes

Redo Buffers                7127040 bytes

SQL> alter system set cluster_database=false scope=spfile sid=’*’;

System altered.

SQL> shutdown immediate;

ORA-01507: database not mounted

ORACLE instance shut down.

Then restore the database via RMAN:

# rman target=/

Recovery Manager: Release – Production on Sat May 24

17:04:10 2008

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

connected to target database (not started)

RMAN> startup mount;

Oracle instance started

database mounted

Total System Global Area     268435456 bytes

Fixed Size                     1261300 bytes

Variable Size                251658508 bytes

Database Buffers               8388608 bytes

Redo Buffers                   7127040 bytes

RMAN> restore database;

Starting restore at 24-MAY-08

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: sid=154 devtype=DISK

channel ORA_DISK_1: starting datafile backupset restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

restoring datafile 00001 to /u02/oracle/ora1/em/system01.dbf

restoring datafile 00002 to /u02/oracle/ora1/em/undotbs01.dbf

restoring datafile 00003 to /u02/oracle/ora1/em/sysaux01.dbf

restoring datafile 00004 to /u02/oracle/ora1/em/users01.dbf

restoring datafile 00005 to /u02/oracle/ora1/em/example01.dbf

restoring datafile 00006 to /u02/oracle/ora1/em/undotbs02.dbf

restoring datafile 00007 to /u02/oracle/ora2/em/mgmt.dbf

restoring datafile 00008 to /u02/oracle/ora1/em/mgmt_ecm_depot1.dbf

channel ORA_DISK_1: reading from backup piece


channel ORA_DISK_1: restored backup piece 1

piece handle=/u02/oracle/ora3/em/backups/0ijh6j4t_1_1


channel ORA_DISK_1: restore complete, elapsed time: 00:06:36

Finished restore at 24-MAY-08

Since my database is so small, I will simply restore the entire thing, however, since you

know which datafile is missing, you could simply restore it and then recover the database

as necessary.

Recover the Database

RMAN> recover database;

Starting recover at 24-MAY-08

using channel ORA_DISK_1

starting media recovery

media recovery complete, elapsed time: 00:00:03

Finished recover at 24-MAY-08

RMAN> alter database open;

database opened

RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT


RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and


ORACLE error from target database:

ORA-06502: PL/SQL: numeric or value error: character to number

conversion error

Afterwards, place the database back into cluster mode and startup both instances:

# sqlplus “/ as sysdba”

SQL*Plus: Release – Production on Sat May 24 17:16:36 2008

Copyright (c) 1982, 2006, Oracle.  All Rights Reserved.

Connected to:

Oracle Database 10g Enterprise Edition Release – Production

With the Partitioning, Real Application Clusters, OLAP and Data Mining


SQL> alter system set cluster_database=true scope=spfile sid=’*’;

System altered.

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

# srvctl start database -d em

[oracle@rac1 bdump]$ srvctl status database -d em

Instance em1 is running on node rac1

Instance em2 is running on node rac2

Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: