<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>惜分飞 &#187; ORA-600 3600</title>
	<atom:link href="https://www.xifenfei.com/tag/ora-600-3600/feed" rel="self" type="application/rss+xml" />
	<link>https://www.xifenfei.com</link>
	<description>提供7*24专业数据库(Oracle,SQL Server,MySQL,PostgreSQL等)恢复和技术支持@Tel:17813235971</description>
	<lastBuildDate>Tue, 17 Nov 2026 13:36:46 +0000</lastBuildDate>
	<language>zh-CN</language>
		<sy:updatePeriod>hourly</sy:updatePeriod>
		<sy:updateFrequency>1</sy:updateFrequency>
	
	<item>
		<title>先offline数据文件，再resetlogs导致恢复复杂的故障处理</title>
		<link>https://www.xifenfei.com/2026/04/offline-resetlogs-recovery.html</link>
		<comments>https://www.xifenfei.com/2026/04/offline-resetlogs-recovery.html#comments</comments>
		<pubDate>Thu, 23 Apr 2026 14:23:01 +0000</pubDate>
		<dc:creator><![CDATA[惜分飞]]></dc:creator>
				<category><![CDATA[Oracle备份恢复]]></category>
		<category><![CDATA[MISSING]]></category>
		<category><![CDATA[ORA-600 2662]]></category>
		<category><![CDATA[ORA-600 3600]]></category>
		<category><![CDATA[resetlogs scn 不一致]]></category>

		<guid isPermaLink="false">https://www.xifenfei.com/?p=12109</guid>
		<description><![CDATA[联系：手机/微信(+86 17813235971) QQ(107644445)  &#8230; <a href="https://www.xifenfei.com/2026/04/offline-resetlogs-recovery.html">继续阅读 <span class="meta-nav">&#8594;</span></a>]]></description>
				<content:encoded><![CDATA[<div id="xifenfeicopyright">
<p>联系：<a href="https://www.xifenfei.com/about_xifenfei" target="_blank">手机/微信(+86 17813235971)   QQ(107644445)</a><a target="_blank" href="http://wpa.qq.com/msgrd?v=3&#038;uin=107644445&#038;site=qq&#038;menu=yes"><img src="https://www.xifenfei.com/wp-content/themes/img/site_qq.jpg" alt="QQ咨询惜分飞" title="QQ咨询惜分飞"/></a></p>
<p>标题：<a href="https://www.xifenfei.com/2026/04/offline-resetlogs-recovery.html" target="_blank">先offline数据文件，再resetlogs导致恢复复杂的故障处理</a></p>
<p>作者：<a href="https://www.xifenfei.com" target="_blank">惜分飞</a>©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]</p>
</div>
<p>本来是一个简单的数据文件被误删除,然后通过底层恢复出来数据文件,再启动库就可以的事情,结果由于对oracle的不了解和自以为是,直接把丢失的文件不存在的情况下,offline文件,然后尝试resetlogs打开库,并且进行了各种尝试,结果使得问题比较麻烦.<br />
<strong>故障之后现象</strong><br />
通过分析alert日志大概的主要错误,大概梳理故障情况<br />
<br />
1. 启动数据库报control03.ctl丢失</p>
<pre class="brush: bash; title: ; notranslate">
Fri Apr 17 21:53:03 2026
MMNL started with pid=16, OS id=3613 
ORACLE_BASE from environment = /data/oracle
Fri Apr 17 21:53:08 2026
alter database mount
ORA-00210: cannot open the specified control file
ORA-00202: control file: '/data/oracle/oradata/orcl/control03.ctl'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00210: cannot open the specified control file
ORA-00202: control file: '/data/oracle/oradata/orcl/control02.ctl'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-205 signalled during: alter database mount...
</pre>
<p>如果只是这个文件丢失（这里还没有看到其他数据文件丢失的报错）,本身是一个非常简单的故障,直接修改control_files参数即可<br />
<br />
2. 结果当时操作的人直接rectl</p>
<pre class="brush: bash; title: ; notranslate">
Fri Apr 17 21:57:01 2026
Successful mount of redo thread 1, with mount id 1758675116
Completed: CREATE CONTROLFILE REUSE DATABASE &quot;ORCL&quot; NORESETLOGS NOARCHIVELOG
    MAXLOGFILES 16
    MAXLOGMEMBERS 3
    MAXDATAFILES 100
    MAXINSTANCES 8
    MAXLOGHISTORY 292
LOGFILE
  GROUP 1 '/data/oracle/oradata/orcl/redo01.log'  SIZE 50M BLOCKSIZE 512,
  GROUP 2 '/data/oracle/oradata/orcl/redo02.log'  SIZE 50M BLOCKSIZE 512,
  GROUP 3 '/data/oracle/oradata/orcl/redo03.log'  SIZE 50M BLOCKSIZE 512
DATAFILE
  '/data/oracle/oradata/orcl/system01.dbf',
  '/data/oracle/oradata/orcl/sysaux01.dbf',
  '/data/oracle/oradata/orcl/undotbs01.dbf',
  '/data/oracle/oradata/orcl/users01.dbf'
CHARACTER SET ZHS16GBK
</pre>
<p>3.然后启动数据库报错</p>
<pre class="brush: bash; title: ; notranslate">
Fri Apr 17 22:02:43 2026
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 3 processes
Started redo scan
Completed redo scan
 read 39020 KB redo, 0 data blocks need recovery
Started redo application at
 Thread 1: logseq 11590, block 2, scn 137806010
Recovery of Online Redo Log: Thread 1 Group 1 Seq 11590 Reading mem 0
  Mem# 0: /data/oracle/oradata/orcl/redo01.log
Completed redo application of 0.00MB
Completed crash recovery at
 Thread 1: logseq 11590, block 78042, scn 137831847
 0 data blocks read, 0 data blocks written, 39020 redo k-bytes read
Fri Apr 17 22:02:44 2026
Thread 1 advanced to log sequence 11591 (thread open)
Thread 1 opened at log sequence 11591
  Current log# 2 seq# 11591 mem# 0: /data/oracle/oradata/orcl/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Fri Apr 17 22:02:44 2026
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Dictionary check beginning
Tablespace 'TEMP' #3  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ERP_XXXX' #6  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ERP_AAAA' #7  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ABCD' #8  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ERP_BBBB' #9  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ERP_XXD' #10  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'ERP_12SF' #11  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'XXX14' #12  found in data dictionary,
but not in the controlfile. Adding to controlfile.
Tablespace 'P_ZY' #13 found in data dictionary,
but not in the controlfile. Adding to controlfile.
File #5 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00005' in the controlfile.
File #6 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00006' in the controlfile.
File #7 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00007' in the controlfile.
File #8 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00008' in the controlfile.
File #9 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00009' in the controlfile.
File #10  found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00010' in the controlfile.
File #11  found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00011' in the controlfile.
File #12 found in data dictionary but not in controlfile.
Creating OFFLINE file 'MISSING00012' in the controlfile.
</pre>
<p>4.然后尝试resetlogs操作</p>
<pre class="brush: bash; title: ; notranslate">
Sat Apr 18 05:55:10 2026
ALTER DATABASE   MOUNT
Successful mount of redo thread 1, with mount id 1758652862
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Sat Apr 18 05:55:14 2026
ALTER DATABASE OPEN RESETLOGS
ORA-1139 signalled during: ALTER DATABASE OPEN RESETLOGS...
Sat Apr 18 05:56:29 2026
Starting ORACLE instance (normal)
ALTER DATABASE RECOVER  DATABASE UNTIL CANCEL  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 4 slaves
Sat Apr 18 05:56:29 2026
Warning: Datafile 5 (/data/oracle/orcl/xxxx.dbf) is offline during full database recovery and will not be recovered
Warning: Datafile 6 (/data/oracle/orcl/xxxx.dbf) is offline during full database recovery and will not be recovered
Warning: Datafile 7 (/data/oracle/orcl/xxxx.dbf) is offline during full database recovery and will not be recovered
Warning: Datafile 8 (/data/oracle/orcl/xxxx.dbf) is offline during full database recovery and will not be recovered
Media Recovery Not Required
Completed: ALTER DATABASE RECOVER  DATABASE UNTIL CANCEL  
Sat Apr 18 05:57:45 2026
ALTER DATABASE OPEN RESETLOGS
RESETLOGS after complete recovery through change 137865786
Resetting resetlogs activation ID 1645665187 (0x6216dba3)
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2549.trc:
ORA-00367: checksum error in log file header
ORA-00322: log 1 of thread 1 is not current copy
ORA-00312: online log 1 thread 1: '/data/oracle/oradata/orcl/redo01.log'
Sat Apr 18 05:57:45 2026
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_2554.trc:
ORA-00316: log 1 of thread 1, type 0 in header is not log file
ORA-00312: online log 1 thread 1: '/data/oracle/oradata/orcl/redo01.log'
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2549.trc:
ORA-00367: checksum error in log file header
ORA-00322: log 2 of thread 1 is not current copy
ORA-00312: online log 2 thread 1: '/data/oracle/oradata/orcl/redo02.log'
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_2554.trc:
ORA-00316: log 2 of thread 1, type 0 in header is not log file
ORA-00312: online log 2 thread 1: '/data/oracle/oradata/orcl/redo02.log'
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_m000_2554.trc:
ORA-00322: log 3 of thread 1 is not current copy
ORA-00312: online log 3 thread 1: '/data/oracle/oradata/orcl/redo03.log'
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_2549.trc:
ORA-00367: checksum error in log file header
ORA-00322: log 3 of thread 1 is not current copy
ORA-00312: online log 3 thread 1: '/data/oracle/oradata/orcl/redo03.log'
Sat Apr 18 05:57:46 2026
Setting recovery target incarnation to 2
Sat Apr 18 05:57:46 2026
Assigning activation ID 1758652862 (0x68d2e9be)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /data/oracle/oradata/orcl/redo01.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Apr 18 05:57:46 2026
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Dictionary check beginning
File #5 is offline, but is part of an online tablespace.
data file 5: '/data/oracle/oradata/orcl/xxxx.dbf'
File #6 is offline, but is part of an online tablespace.
data file 6: '/data/oracle/oradata/orcl/xxxx.dbf'
File #7 is offline, but is part of an online tablespace.
data file 7: '/data/oracle/oradata/orcl/xxxx.dbf'
File #8 is offline, but is part of an online tablespace.
data file 8: '/data/oracle/oradata/orcl/xxxx.dbf'
</pre>
<p>到这一步悲剧基本上已经发生，犯了一个在oracle恢复里面比较忌讳的事情，有数据文件offline的情况下，执行resetlogs操作，导致部分数据文件的resetlogs信息没有被及时更新，导致一套库里面，被offline的这个部分数据文件resetlogs信息小于其他online的数据文件的。<br />
<br />
5. 后续其他操作各种报错</p>
<pre class="brush: bash; title: ; notranslate">
Completed: ALTER DATABASE   MOUNT
Sun Apr 19 08:13:02 2026
ALTER DATABASE DATAFILE 5 OFFLINE DROP
Sun Apr 19 08:13:02 2026
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_dbw0_9212.trc  (incident=67094):
ORA-00600: internal error code, arguments: [3600], [5], [14], [], [], [], [], [], [], [], [], []
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_67094/orcl_dbw0_9212_i67094.trc
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_dbw0_9212.trc:
ORA-00600: internal error code, arguments: [3600], [5], [14], [], [], [], [], [], [], [], [], []
DBW0 (ospid: 9212): terminating the instance due to error 471
</pre>
<pre class="brush: bash; title: ; notranslate">
Tue Apr 21 22:31:23 2026
Assigning activation ID 1758985759 (0x68d7fe1f)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: /data/oracle/oradata/orcl/redo01.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Tue Apr 21 22:31:23 2026
SMON: enabling cache recovery
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4951.trc  (incident=87950):
ORA-00600: internal error code, arguments: [2662], [0], [137890858], [0], [137891091], [12583056], []
Incident details in: /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_87950/orcl_ora_4951_i87950.trc
Errors in file /data/oracle/diag/rdbms/orcl/orcl/incident/incdir_87950/orcl_ora_4951_i87950.trc:
ORA-00339: archived log does not contain any redo
ORA-00334: archived log: '/data/oracle/oradata/orcl/redo03.log'
ORA-00339: archived log does not contain any redo
ORA-00334: archived log: '/data/oracle/oradata/orcl/redo02.log'
ORA-00339: archived log does not contain any redo
ORA-00334: archived log: '/data/oracle/oradata/orcl/redo02.log'
ORA-00339: archived log does not contain any redo
ORA-00334: archived log: '/data/oracle/oradata/orcl/redo03.log'
ORA-00600: internal error code, arguments: [2662], [0], [137890858], [0], [137891091], [12583056], []
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4951.trc:
ORA-00600: internal error code, arguments: [2662], [0], [137890858], [0], [137891091], [12583056], []
Errors in file /data/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_4951.trc:
ORA-00600: internal error code, arguments: [2662], [0], [137890858], [0], [137891091], [12583056], []
Error 600 happened during db open, shutting down database
USER (ospid: 4951): terminating the instance due to error 600
</pre>
<p><strong>接手故障之后分析</strong><br />
使用obet工具直接快速的检查坏块情况和文件头信息，关于obet的介绍参考:<br />
<a href="https://www.xifenfei.com/2026/01/obet-dbv.html" title="obet实现对数据文件坏块检测功能" rel="noopener" target="_blank">obet实现对数据文件坏块检测功能</a><br />
<a href="https://www.xifenfei.com/2025/11/obet.html" title="Oracle数据块编辑工具( Oracle Block Editor Tool)-obet" rel="noopener" target="_blank">Oracle数据块编辑工具( Oracle Block Editor Tool)-obet</a><br />
<a href="https://www.xifenfei.com/wp-content/uploads/2026/04/dbv.jpg"><img src="https://www.xifenfei.com/wp-content/uploads/2026/04/dbv.jpg" alt="dbv" width="987" height="849" class="alignnone size-full wp-image-12110" /></a><br />
<hr />
dbv检测没任何坏块，比较好好的消息<br />
<a href="https://www.xifenfei.com/wp-content/uploads/2026/04/reset.png"><img src="https://www.xifenfei.com/wp-content/uploads/2026/04/reset.png" alt="reset" width="1073" height="238" class="alignnone size-full wp-image-12111" /></a><br />
<hr />
但是检测数据文件头信息，发现有三种类型的resetlogs的信息，证明进行了多次部分文件的情况下进行了resetlogs操作<br />
<br />
<strong>恢复处理</strong><br />
1. 使用Oracle Recovery Tools工具修改 resetlogs 信息<br />
由于大量reseltogs 信息不一致，先使用Oracle Recovery Tools修改scn等相关信息<a href="https://www.xifenfei.com/2025/05/oracle-recovery-tools-202505.html" title="Oracle Recovery Tools恢复案例总结—202505" rel="noopener" target="_blank">Oracle Recovery Tools恢复案例总结—202505</a>（注意选择resetlogs scn最大的文件为参照文件）<br />
<a href="https://www.xifenfei.com/wp-content/uploads/2026/04/orarec.jpg"><img src="https://www.xifenfei.com/wp-content/uploads/2026/04/orarec.jpg" alt="orarec" width="787" height="495" class="alignnone size-full wp-image-12112" /></a><br />
<hr />
2. 重建ctl，打开库<br />
<a href="https://www.xifenfei.com/wp-content/uploads/2026/04/open-db.jpg"><img src="https://www.xifenfei.com/wp-content/uploads/2026/04/open-db.jpg" alt="open-db" width="710" height="471" class="alignnone size-full wp-image-12113" /></a><br />
<hr />
比较幸运直接打开成功（本来也就应该成功，因为客户本身之前丢失主要业务文件的时候多次打开过库）</p>
<hr />
3. 然后增加temp文件，并expdp导出数据，完成本次恢复工作</p>
<div id="crp_related"><ul><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2026/04/obet-offline-datafile.html" rel="bookmark" class="crp_title">obet一键恢复offline数据文件</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2024/09/analyze-alert-logs-parse-recovery-process.html" rel="bookmark" class="crp_title">通过alert日志分析客户自行对一个数据库恢复的来龙去脉和点评</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/01/orarecovery-ora-19909.html" rel="bookmark" class="crp_title">Oracle Recovery Tools快速恢复ORA-19909</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2026/07/ora-00314-ora-00312-recovery.html" rel="bookmark" class="crp_title">ORA-00314: log 3 of thread 1, expected sequence# N doesn&#8217;t match 0</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2022/08/oracle-recovery-tools-ora-600-3020.html" rel="bookmark" class="crp_title">Oracle Recovery Tools 解决ORA-600 3020故障</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/08/ora-600-ksuloget2.html" rel="bookmark" class="crp_title">ORA-600 ksuloget2 恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2026/06/last-mile-db-rescue.html" rel="bookmark" class="crp_title">接手一个只差临门一脚的数据库恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/12/ora-600-3020-ora-600-2662.html" rel="bookmark" class="crp_title">ORA-600 3020/ORA-600 2662故障</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2026/02/ora-600-kcratr_nab_less_than_odr-ora-600-4193.html" rel="bookmark" class="crp_title">ORA-600 kcratr_nab_less_than_odr和ORA-600 4193故障处理</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2022/02/ora-00742-ora-00312-recovery.html" rel="bookmark" class="crp_title">ORA-00742 ORA-00312故障恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2026/02/review-alert-logs-and-open-database.html" rel="bookmark" class="crp_title">通过alert日志回顾其他dba oracle异常恢复故障处理以及后续open数据库操作</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2022/01/ora-600-3600.html" rel="bookmark" class="crp_title">ORA-600 3600恢复&#8212;-resetlogs scn异常</a></li></ul></div>]]></content:encoded>
			<wfw:commentRss>https://www.xifenfei.com/2026/04/offline-resetlogs-recovery.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>ORA-600 3600恢复&#8212;-resetlogs scn异常</title>
		<link>https://www.xifenfei.com/2022/01/ora-600-3600.html</link>
		<comments>https://www.xifenfei.com/2022/01/ora-600-3600.html#comments</comments>
		<pubDate>Thu, 27 Jan 2022 16:10:33 +0000</pubDate>
		<dc:creator><![CDATA[惜分飞]]></dc:creator>
				<category><![CDATA[Oracle]]></category>
		<category><![CDATA[ORA-19909]]></category>
		<category><![CDATA[ORA-600 3600]]></category>
		<category><![CDATA[Oracle Recovery Tools]]></category>

		<guid isPermaLink="false">https://www.xifenfei.com/?p=9579</guid>
		<description><![CDATA[联系：手机/微信(+86 17813235971) QQ(107644445)  &#8230; <a href="https://www.xifenfei.com/2022/01/ora-600-3600.html">继续阅读 <span class="meta-nav">&#8594;</span></a>]]></description>
				<content:encoded><![CDATA[<div id="xifenfeicopyright">
<p>联系：<a href="https://www.xifenfei.com/about_xifenfei" target="_blank">手机/微信(+86 17813235971)   QQ(107644445)</a><a target="_blank" href="http://wpa.qq.com/msgrd?v=3&#038;uin=107644445&#038;site=qq&#038;menu=yes"><img src="https://www.xifenfei.com/wp-content/themes/img/site_qq.jpg" alt="QQ咨询惜分飞" title="QQ咨询惜分飞"/></a></p>
<p>标题：<a href="https://www.xifenfei.com/2022/01/ora-600-3600.html" target="_blank">ORA-600 3600恢复&#8212;-resetlogs scn异常</a></p>
<p>作者：<a href="https://www.xifenfei.com" target="_blank">惜分飞</a>©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]</p>
</div>
<p>由于客户误操作,在有文件offline的情况下进行resetlogs操作,导致有文件resetlogs scn不对<br />
<a href="https://www.xifenfei.com/wp-content/uploads/2022/01/20220128000040.jpg"><img src="https://www.xifenfei.com/wp-content/uploads/2022/01/20220128000040.jpg" alt="20220128000040" width="1699" height="444" class="alignnone size-full wp-image-9580" /></a><br />
<hr />
尝试offline异常文件,均报ORA-600 3600</p>
<pre class="brush: bash; title: ; notranslate">
---直接offline
Wed Jan 26 11:08:15 2022
ALTER DATABASE RECOVER  database until cancel  
Media Recovery Start
 started logmerger process
Wed Jan 26 11:08:17 2022
Datafile 8 (ckpscn 731239901) is orphaned on incarnation#=2
Media Recovery failed with error 19909
Slave exiting with ORA-283 exception
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_pr00_133504.trc:
ORA-00283: recovery session canceled due to errors
ORA-19909: datafile 8 belongs to an orphan incarnation
ORA-01110: data file 8: 'D:\APP\ADMINISTRATOR\PRODUCT\11.2.0\DBHOME_1\DATABASE\XIFENFEI.DBF'
Recovery Slave PR00 previously exited with exception 283
ORA-283 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Wed Jan 26 11:08:31 2022
alter database datafile 8 offline
Wed Jan 26 11:08:31 2022
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_133948.trc  (incident=134637):
ORA-00600: internal error code, arguments: [3600], [8], [14], [], [], [], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_134637\orcl_dbw0_133948_i134637.trc
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_133948.trc:
ORA-00600: internal error code, arguments: [3600], [8], [14], [], [], [], [], [], [], [], [], []
DBW0 (ospid: 133948): terminating the instance due to error 471
Instance terminated by DBW0, pid = 133948

---offline drop
Wed Jan 26 11:09:20 2022
alter database datafile 8 offline drop
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_133932.trc  (incident=135837):
ORA-00600: internal error code, arguments: [3600], [8], [14], [], [], [], [], [], [], [], [], []
Incident details in: d:\app\administrator\diag\rdbms\orcl\orcl\incident\incdir_135837\orcl_dbw0_133932_i135837.trc
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_133932.trc:
ORA-00600: internal error code, arguments: [3600], [8], [14], [], [], [], [], [], [], [], [], []
DBW0 (ospid: 133932): terminating the instance due to error 471
Wed Jan 26 11:09:22 2022
Instance terminated by DBW0, pid = 133932
</pre>
<p>因为resetlogs scn不对,也无法正常重建控制文件,对于这样的case,可以<a href="https://www.xifenfei.com/2021/01/oracle-recovery-tools-ora-01190-ora-01248.html" title="Oracle Recovery Tools 解决ORA-01190 ORA-01248等故障" rel="noopener" target="_blank">Oracle Recovery Tools</a>进行修复resetlogs scn,然后直接open库</p>
<pre class="brush: bash; title: ; notranslate">
Wed Jan 26 11:15:12 2022
SMON: enabling cache recovery
Dictionary check beginning
Archived Log entry 3 added for thread 1 sequence 381 ID 0x60b930a1 dest 1:
Dictionary check complete
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Starting background process QMNC
Wed Jan 26 11:15:15 2022
QMNC started with pid=25, OS id=131784 
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Completed: alter database open
</pre>
<p>软件下载:<a href="https://www.xifenfei.com/wp-content/uploads/recover_check/OraRecovery.zip">OraRecovery下载</a><br />
使用说明:<a href="https://www.xifenfei.com/wp-content/uploads/2020/08/OracleRecoveryTools.pdf">使用说明</a></p>
<div id="crp_related"><ul><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2022/02/kcratr_scan_lastbwr.html" rel="bookmark" class="crp_title">ora-600 kcratr_scan_lastbwr</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/%e5%88%86%e4%ba%abora-01113-ora-01110%e6%95%85%e9%9a%9c%e5%a4%84%e7%90%86%e8%bf%87%e7%a8%8b.html" rel="bookmark" class="crp_title">分享一次ORA-01113 ORA-01110故障处理过程</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2021/07/win-ora-600-kokasgi1.html" rel="bookmark" class="crp_title">win环境报ora-600 kokasgi1处理</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/08/ora-600-ksuloget2.html" rel="bookmark" class="crp_title">ORA-600 ksuloget2 恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2021/03/ora-600-kcratr_scan_lastbwr.html" rel="bookmark" class="crp_title">ORA-600 kcratr_scan_lastbwr 恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2024/12/oracle-recovery-tools-ora-00376-ora-01110.html" rel="bookmark" class="crp_title">Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障（文件offline）</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2024/03/ora-00600-16703-1403-4.html" rel="bookmark" class="crp_title">ORA-00600: internal error code, arguments: [16703], [1403], [4] 原因</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2019/03/noarchivelog-recovery.html" rel="bookmark" class="crp_title">非归档数据库异常恢复一例</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2016/08/ora-600-999.html" rel="bookmark" class="crp_title">ORA-600 999 异常恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2022/08/oracle-recovery-tools-ora-600-3020.html" rel="bookmark" class="crp_title">Oracle Recovery Tools 解决ORA-600 3020故障</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/01/19c-power-failure-recovery.html" rel="bookmark" class="crp_title">Oracle 19c 断电异常恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2023/01/orarecovery-ora-19909.html" rel="bookmark" class="crp_title">Oracle Recovery Tools快速恢复ORA-19909</a></li></ul></div>]]></content:encoded>
			<wfw:commentRss>https://www.xifenfei.com/2022/01/ora-600-3600.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>SYSTEM表空间坏块恢复&#8212;C_TS#对象坏块恢复(file 1 block 60)</title>
		<link>https://www.xifenfei.com/2014/11/system%e8%a1%a8%e7%a9%ba%e9%97%b4%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8d-c_ts%e5%af%b9%e8%b1%a1%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8dfile-1-block-60.html</link>
		<comments>https://www.xifenfei.com/2014/11/system%e8%a1%a8%e7%a9%ba%e9%97%b4%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8d-c_ts%e5%af%b9%e8%b1%a1%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8dfile-1-block-60.html#comments</comments>
		<pubDate>Sat, 22 Nov 2014 15:24:47 +0000</pubDate>
		<dc:creator><![CDATA[惜分飞]]></dc:creator>
				<category><![CDATA[非常规恢复]]></category>
		<category><![CDATA[ORA-600 2662]]></category>
		<category><![CDATA[ORA-600 2663]]></category>
		<category><![CDATA[ORA-600 3600]]></category>
		<category><![CDATA[ORA-600 4137]]></category>
		<category><![CDATA[ORA-600 4198]]></category>
		<category><![CDATA[ORA-600 4511]]></category>
		<category><![CDATA[ORA-600 6807]]></category>
		<category><![CDATA[ORA-600 krhpfh_03-1209]]></category>
		<category><![CDATA[ORA-600 ktsitbs_info1]]></category>

		<guid isPermaLink="false">https://www.xifenfei.com/?p=5649</guid>
		<description><![CDATA[联系：手机/微信(+86 17813235971) QQ(107644445)  &#8230; <a href="https://www.xifenfei.com/2014/11/system%e8%a1%a8%e7%a9%ba%e9%97%b4%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8d-c_ts%e5%af%b9%e8%b1%a1%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8dfile-1-block-60.html">继续阅读 <span class="meta-nav">&#8594;</span></a>]]></description>
				<content:encoded><![CDATA[<div id="xifenfeicopyright">
<p>联系：<a href="https://www.xifenfei.com/about_xifenfei" target="_blank">手机/微信(+86 17813235971)   QQ(107644445)</a><a target="_blank" href="http://wpa.qq.com/msgrd?v=3&#038;uin=107644445&#038;site=qq&#038;menu=yes"><img src="https://www.xifenfei.com/wp-content/themes/img/site_qq.jpg" alt="QQ咨询惜分飞" title="QQ咨询惜分飞"/></a></p>
<p>标题：<a href="https://www.xifenfei.com/2014/11/system%e8%a1%a8%e7%a9%ba%e9%97%b4%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8d-c_ts%e5%af%b9%e8%b1%a1%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8dfile-1-block-60.html" target="_blank">SYSTEM表空间坏块恢复&#8212;C_TS#对象坏块恢复(file 1 block 60)</a></p>
<p>作者：<a href="https://www.xifenfei.com" target="_blank">惜分飞</a>©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]</p>
</div>
<p>一朋友给我电话,说他们客户公司数据库故障,被另外一家公司恢复了一天不能正常恢复,请求我协助解决.接手一看数据库已经被破坏的不像样子了,根据alert日志信息大概分析了故障原因和上家公司处理情况。后面接手后通过bbed修复block数据库恢复过程,在本次恢复中出现大量ORA-600错误,主要包括ORA-00600 400,ORA-00600 2662,ORA-00600 2663,ORA-00600 krhpfh_03-1209,ORA-00600 3600,ORA-00600 ktsitbs_info1,ORA-00600 4137,ORA-00600 4511,ORA-00600 4198,ORA-00600 6807等<br />
<strong>故障原因redo文件丢失</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 11:28:39 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_lgwr_1404.trc:
ORA-00313: open failed for members of log group 7 of thread 1
ORA-00312: online log 9 thread 1: '/data2/oradata/redo0902.log'
ORA-27037: unable to obtain file status
SVR4 Error: 2: No such file or directory
Additional information: 3
Thu Nov 20 11:28:39 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_lgwr_1404.trc:
ORA-00313: open failed for members of log group 7 of thread 1
ORA-00312: online log 9 thread 1: '/data2/oradata/redo0902.log'
ORA-27037: unable to obtain file status
SVR4 Error: 2: No such file or directory
Additional information: 3
Thu Nov 20 11:28:39 2014
LGWR: terminating instance due to error 313
Thu Nov 20 11:28:39 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_pmon_1394.trc:
ORA-00313: open failed for members of log group  of thread 
Thu Nov 20 11:28:39 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_ckpt_1406.trc:
ORA-00313: open failed for members of log group  of thread 
Instance terminated by LGWR, pid = 1404
</pre>
<p><strong>尝试clear redo文件方式恢复</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 13:04:16 2014
alter database clear logfile group 9
Thu Nov 20 13:04:16 2014
ORA-1624 signalled during: alter database clear logfile group 9...
Thu Nov 20 13:04:45 2014
alter database clear logfile group 9
Thu Nov 20 13:04:46 2014
ORA-1624 signalled during: alter database clear logfile group 9...
Thu Nov 20 13:04:59 2014
alter database clear unarchived logfile group 9
Thu Nov 20 13:04:59 2014
ORA-1624 signalled during: alter database clear unarchived logfile group 9...
Thu Nov 20 13:05:00 2014
alter database clear unarchived logfile group 9
Thu Nov 20 13:05:00 2014
ORA-1624 signalled during: alter database clear unarchived logfile group 9...
</pre>
<p><strong>不完全恢复resetlogs尝试打开数据库</strong></p>
<pre class="brush: bash; title: ; notranslate">
ORA-279 signalled during: ALTER DATABASE RECOVER  database using backup controlfile  ...
Thu Nov 20 13:49:01 2014
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Thu Nov 20 13:49:02 2014
Media Recovery Log /opt/oracle/flash_recovery_area/xifenfei/archivelog/2014_11_20/o1_mf_1_285999_%u_.arc
Errors with log /opt/oracle/flash_recovery_area/xifenfei/archivelog/2014_11_20/o1_mf_1_285999_%u_.arc
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Thu Nov 20 13:49:02 2014
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Thu Nov 20 13:49:02 2014
Media Recovery Log /opt/oracle/flash_recovery_area/xifenfei/archivelog/2014_11_20/o1_mf_1_285999_%u_.arc
Errors with log /opt/oracle/flash_recovery_area/xifenfei/archivelog/2014_11_20/o1_mf_1_285999_%u_.arc
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Thu Nov 20 13:49:02 2014
ALTER DATABASE RECOVER CANCEL 
Thu Nov 20 13:49:03 2014
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER CANCEL 
Thu Nov 20 13:49:33 2014
alter database open resetlogs
Thu Nov 20 13:49:34 2014
ORA-1113 signalled during: alter database open resetlogs...
</pre>
<p><strong>使用隐含参数</strong></p>
<pre class="brush: bash; title: ; notranslate">
_allow_resetlogs_corruption= TRUE
</pre>
<p><strong>进行不完全恢复,尝试open数据库报ORA-600 4000错误</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 14:35:02 2014
ALTER DATABASE   MOUNT
Thu Nov 20 14:35:07 2014
Setting recovery target incarnation to 2
Thu Nov 20 14:35:07 2014
Successful mount of redo thread 1, with mount id 4039504598
Thu Nov 20 14:35:07 2014
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Thu Nov 20 14:40:33 2014
ALTER DATABASE RECOVER  database until cancel  
Thu Nov 20 14:40:33 2014
Media Recovery Start
Thu Nov 20 14:40:33 2014
Media Recovery failed with error 1610
ORA-283 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Thu Nov 20 14:41:23 2014
ALTER DATABASE RECOVER  database using backup controlfile until cancel  

Thu Nov 20 14:43:08 2014
alter database open resetlogs
Thu Nov 20 14:43:08 2014
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 31293973571
Resetting resetlogs activation ID 3855216310 (0xe5c9eeb6)
Online log /data2/oradata/redo0802.log: Thread 1 Group 8 was previously cleared
Online log /data2/oradata/redo0902.log: Thread 1 Group 9 was previously cleared
Thu Nov 20 14:43:14 2014
Setting recovery target incarnation to 3
Thu Nov 20 14:43:14 2014
Assigning activation ID 4039504598 (0xf0c5f2d6)
Thread 1 opened at log sequence 1
  Current log# 9 seq# 1 mem# 0: /data2/oradata/redo0902.log
Successful open of redo thread 1
Thu Nov 20 14:43:14 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Nov 20 14:43:14 2014
SMON: enabling cache recovery
Thu Nov 20 14:43:14 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_1844.trc:
ORA-00600: internal error code, arguments: [4000], [17], [], [], [], [], [], []
Thu Nov 20 14:43:16 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_1844.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [4000], [17], [], [], [], [], [], []
Thu Nov 20 14:43:16 2014
Error 704 happened during db open, shutting down database
USER: terminating instance due to error 704
Instance terminated by USER, pid = 1844
ORA-1092 signalled during: alter database open resetlogs...
</pre>
<p><strong>尝试隐含屏蔽回滚段</strong></p>
<pre class="brush: bash; title: ; notranslate">
_corrupted_rollback_segments= _SYSSMU1$, _SYSSMU2$,…………
</pre>
<p><strong>错误依旧ORA-600 4000</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 15:09:21 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Nov 20 15:09:21 2014
SMON: enabling cache recovery
Thu Nov 20 15:09:21 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_624.trc:
ORA-00600: internal error code, arguments: [4000], [17], [], [], [], [], [], []
Thu Nov 20 15:09:23 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_624.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00600: internal error code, arguments: [4000], [17], [], [], [], [], [], []
Thu Nov 20 15:09:23 2014
Error 704 happened during db open, shutting down database
USER: terminating instance due to error 704
Instance terminated by USER, pid = 624
ORA-1092 signalled during: alter database open
</pre>
<p><strong>多次重启,resetlogs后,数据库出现ORA-600 2662错误</strong></p>
<pre class="brush: bash; title: ; notranslate">
Successful open of redo thread 1
Thu Nov 20 17:13:24 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Nov 20 17:13:24 2014
SMON: enabling cache recovery
Thu Nov 20 17:13:24 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_7967.trc:
ORA-00600: internal error code, arguments: [2662], [7], [1229382552], [7], [1229560642], [8388633], [], []
Thu Nov 20 17:13:25 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_7967.trc:
ORA-00600: internal error code, arguments: [2662], [7], [1229382552], [7], [1229560642], [8388633], [], []
Thu Nov 20 17:13:25 2014
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 7967
ORA-1092 signalled during: ALTER DATABASE OPEN...
Thu Nov 20 17:18:23 2014
USER: terminating instance due to error 1092
Instance terminated by USER, pid = 7967
</pre>
<p><strong>offline undo相关文件,尝试打开数据库</strong></p>
<pre class="brush: bash; title: ; notranslate">
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Thu Nov 20 17:52:31 2014
ALTER DATABASE RECOVER  database until cancel  
Thu Nov 20 17:52:31 2014
Media Recovery Start
 parallel recovery started with 15 processes
ORA-279 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Thu Nov 20 17:53:42 2014
ALTER DATABASE RECOVER CANCEL 
Thu Nov 20 17:53:44 2014
ORA-1547 signalled during: ALTER DATABASE RECOVER CANCEL ...
Thu Nov 20 17:56:34 2014
alter database datafile '/opt/oracle/oradata/xifenfei/undotbs01.dbf' offline
Thu Nov 20 17:56:35 2014
Completed: alter database datafile '/opt/oracle/oradata/xifenfei/undotbs01.dbf' offline
Thu Nov 20 17:57:01 2014
alter database datafile '/data2/oradata/undotbs02.dbf' offline
Thu Nov 20 17:57:02 2014
Completed: alter database datafile '/data2/oradata/undotbs02.dbf' offline
Thu Nov 20 17:57:26 2014
alter database datafile '/data2/oradata/undotbs03.dbf' offline
Thu Nov 20 17:57:27 2014
Completed: alter database datafile '/data2/oradata/undotbs03.dbf' offline
Thu Nov 20 17:57:43 2014
alter database open resetlogs
Thu Nov 20 17:57:43 2014
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
ORA-1245 signalled during: alter database open resetlogs...
Thu Nov 20 17:58:58 2014
alter database datafile '/opt/oracle/oradata/xifenfei/undotbs01.dbf' offline drop
Thu Nov 20 17:58:58 2014
Completed: alter database datafile '/opt/oracle/oradata/xifenfei/undotbs01.dbf' offline drop
Thu Nov 20 17:59:15 2014
alter database open resetlogs
Thu Nov 20 17:59:15 2014
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
ORA-1245 signalled during: alter database open resetlogs...
Thu Nov 20 17:59:35 2014
alter database datafile '/data2/oradata/undotbs02.dbf' offline drop
Thu Nov 20 17:59:35 2014
Completed: alter database datafile '/data2/oradata/undotbs02.dbf' offline drop
Thu Nov 20 17:59:50 2014
alter database datafile '/data2/oradata/undotbs03.dbf' offline drop
Thu Nov 20 17:59:50 2014
Completed: alter database datafile '/data2/oradata/undotbs03.dbf' offline drop
Thu Nov 20 18:00:07 2014
alter database open resetlogs
Thu Nov 20 18:00:07 2014
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 31294173628
Resetting resetlogs activation ID 4039492628 (0xf0c5c414)
Online log /data2/oradata/redo0802.log: Thread 1 Group 8 was previously cleared
Thu Nov 20 18:00:14 2014
Setting recovery target incarnation to 8
Thu Nov 20 18:00:14 2014
Assigning activation ID 4039504142 (0xf0c5f10e)
Thread 1 opened at log sequence 1
  Current log# 9 seq# 1 mem# 0: /data2/oradata/redo0902.log
Successful open of redo thread 1
Thu Nov 20 18:00:15 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Thu Nov 20 18:00:15 2014
SMON: enabling cache recovery
Thu Nov 20 18:00:15 2014
Successfully onlined Undo Tablespace 1.
Dictionary check beginning
File #2 is offline, but is part of an online tablespace.
data file 2: '/opt/oracle/oradata/xifenfei/undotbs01.dbf'
File #100 is offline, but is part of an online tablespace.
data file 100: '/data2/oradata/undotbs02.dbf'
Thu Nov 20 18:00:28 2014
File #185 is offline, but is part of an online tablespace.
data file 185: '/data2/oradata/undotbs03.dbf'
Dictionary check complete
Thu Nov 20 18:00:35 2014
SMON: enabling tx recovery
Thu Nov 20 18:00:36 2014
Database Characterset is ZHS16CGB231280
Thu Nov 20 18:00:37 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_28472.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 185 cannot be read at this time
ORA-01110: data file 185: '/data2/oradata/undotbs03.dbf'
Error 604 happened during db open, shutting down database
USER: terminating instance due to error 604
Thu Nov 20 18:00:37 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_lgwr_28450.trc:
ORA-00604: error occurred at recursive SQL level 
Thu Nov 20 18:00:37 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_dbw0_28446.trc:
ORA-00604: error occurred at recursive SQL level 
Instance terminated by USER, pid = 28472
ORA-1092 signalled during: alter database open resetlogs...
</pre>
<p><strong>不知道做了什么操作出现file 1 block 60坏块,很可能bbed修改错误导致</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 19:18:15 2014
SMON: enabling cache recovery
Thu Nov 20 19:18:16 2014
Hex dump of (file 1, block 60) in trace file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_13232.trc
Corrupt block relative dba: 0x0040003c (file 1, block 60)
Bad header found during buffer read
Data in bad block:
 type: 128 format: 0 rdba: 0x0040003c
 last change scn: 0x0005.ebe04bc9 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x4bc90602
 check value in block header: 0x6faa
 computed block checksum: 0x0
Reread of rdba: 0x0040003c (file 1, block 60) found same corrupted data
Successfully onlined Undo Tablespace 1.
Thu Nov 20 19:18:16 2014
SMON: enabling tx recovery
Thu Nov 20 19:18:17 2014
Database Characterset is ZHS16CGB231280
Thu Nov 20 19:18:17 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_13232.trc:
ORA-00604: error occurred at recursive SQL level 1
ORA-00376: file 185 cannot be read at this time
ORA-01110: data file 185: '/data2/oradata/undotbs03.dbf'
Error 604 happened during db open, shutting down database
USER: terminating instance due to error 604
Instance terminated by USER, pid = 13232
ORA-1092 signalled during: alter database open...
</pre>
<p><strong>尝试不完全恢复,并resetlogs操作</strong></p>
<pre class="brush: bash; title: ; notranslate">
ALTER DATABASE RECOVER  database until cancel  
Thu Nov 20 19:33:41 2014
Media Recovery Start
Datafile 2 is on orphaned branch
          File status = 4
        Abs fuzzy SCN = 0
 Hot backup fuzzy SCN = 0
Thu Nov 20 19:33:41 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_20878.trc:
ORA-00600: internal error code, arguments: [krhpfh_03-1209], [2], [864151207], [864153315], [1229402557], [7], [0], [0]
ORA-01110: data file 2: '/opt/oracle/oradata/xifenfei/undotbs01.dbf'
Thu Nov 20 19:33:42 2014
Media Recovery failed with error 600
ORA-283 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Thu Nov 20 19:34:06 2014
alter database open resetlogs
Thu Nov 20 19:34:06 2014
ORA-1139 signalled during: alter database open resetlogs...
Thu Nov 20 19:34:17 2014
alter database open
Thu Nov 20 19:34:17 2014
ORA-1190 signalled during: alter database open...
Thu Nov 20 19:35:57 2014
ALTER DATABASE RECOVER  database until cancel  
Thu Nov 20 19:35:57 2014
Media Recovery Start
Datafile 2 is on orphaned branch
          File status = 4
        Abs fuzzy SCN = 0
 Hot backup fuzzy SCN = 0
Thu Nov 20 19:35:58 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_20878.trc:
ORA-00600: internal error code, arguments: [krhpfh_03-1209], [2], [864151207], [864153315], [1229402557], [7], [0], [0]
ORA-01110: data file 2: '/opt/oracle/oradata/xifenfei/undotbs01.dbf'
Thu Nov 20 19:35:59 2014
Media Recovery failed with error 600
ORA-283 signalled during: ALTER DATABASE RECOVER  database until cancel  ...
Thu Nov 20 19:37:19 2014
alter database open resetlogs
Thu Nov 20 19:37:19 2014
ORA-1139 signalled during: alter database open resetlogs...
</pre>
<p><strong>继续打开报 ORA-600 3600错误</strong></p>
<pre class="brush: bash; title: ; notranslate">
Thu Nov 20 19:43:14 2014
alter database datafile '/opt/oracle/oradata/xifenfei/undotbs01.dbf' offline drop
Thu Nov 20 19:43:14 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_dbw0_20856.trc:
ORA-00600: internal error code, arguments: [3600], [2], [14], [], [], [], [], []
Thu Nov 20 19:43:15 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_dbw0_20856.trc:
ORA-00600: internal error code, arguments: [3600], [2], [14], [], [], [], [], []
Thu Nov 20 19:43:15 2014
DBW0: terminating instance due to error 471
Instance terminated by DBW0, pid = 20856
1
&lt;strong&gt;中间多次重启和resetlogs，还出现ORA-600 2663错误&lt;/strong&gt;
1
Fri Nov 21 12:35:12 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Fri Nov 21 12:35:12 2014
SMON: enabling cache recovery
Fri Nov 21 12:35:13 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_15596.trc:
ORA-00600: internal error code, arguments: [2663], [7], [1229543007], [7], [1229560642], [], [], []
Fri Nov 21 12:35:14 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_15596.trc:
ORA-00600: internal error code, arguments: [2663], [7], [1229543007], [7], [1229560642], [], [], []
Fri Nov 21 12:35:14 2014
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Fri Nov 21 12:35:14 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_mman_15572.trc:
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], []
Fri Nov 21 12:35:14 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_dbw1_15576.trc:
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], []
Instance terminated by USER, pid = 15596
ORA-1092 signalled during: ALTER DATABASE OPEN..
</pre>
<p><strong>继续尝试打开数据库出现ORA-600 ktsitbs_info1错误</strong></p>
<pre class="brush: bash; title: ; notranslate">
SMON: enabling cache recovery
Fri Nov 21 13:54:25 2014
Hex dump of (file 1, block 60) in trace file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_21111.trc
Corrupt block relative dba: 0x0040003c (file 1, block 60)
Bad header found during buffer read
Data in bad block:
 type: 128 format: 0 rdba: 0x0040003c
 last change scn: 0x0005.ebe04bc9 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x4bc90602
 check value in block header: 0x6faa
 computed block checksum: 0x0
Reread of rdba: 0x0040003c (file 1, block 60) found same corrupted data
Fri Nov 21 13:54:25 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_21111.trc:
ORA-00600: internal error code, arguments: [ktsitbs_info1], [2], [], [], [], [], [], []
Fri Nov 21 13:54:27 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_21111.trc:
ORA-00600: internal error code, arguments: [ktsitbs_info1], [2], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER: terminating instance due to error 600
Instance terminated by USER, pid = 21111
ORA-1092 signalled during: alter database open...
</pre>
<p>以上是客户数据库故障原因和问题大概的处理过程,下面是我接手后的处理过程</p>
<hr />
<strong>dbv 检查system01.dbf文件,得到结果</strong></p>
<pre class="brush: bash; title: ; notranslate">
HNDX-DB% dbv file=/opt/oracle/oradata/xifenfei/system01.dbf

DBVERIFY: Release 10.2.0.1.0 - Production on Fri Nov 21 16:22:37 2014

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

DBVERIFY - Verification starting : FILE = /opt/oracle/oradata/xifenfei/system01.dbf
Page 60 is marked corrupt
Corrupt block relative dba: 0x0040003c (file 1, block 60)
Bad header found during dbv: 
Data in bad block:
 type: 128 format: 0 rdba: 0x0040003c
 last change scn: 0x0005.ebe04bc9 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x4bc90602
 check value in block header: 0x6faa
 computed block checksum: 0x0

Corrupt block relative dba: 0x004001f2 (file 1, block 498)
Bad check value found during buffer read
Data in bad block:
 type: 6 format: 2 rdba: 0x004001f2
 last change scn: 0x0007.49499ca1 seq: 0x1 flg: 0x06
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x9ca10601
 check value in block header: 0xe458
 computed block checksum: 0x9720


DBVERIFY - Verification complete

Total Pages Examined         : 786432
Total Pages Processed (Data) : 201131
Total Pages Failing   (Data) : 2
Total Pages Processed (Index): 221394
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 60265
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 303641
Total Pages Marked Corrupt   : 2
Total Pages Influx           : 0
Highest block SCN            : 1229823477 (7.1229823477)
</pre>
<p>这里知道数据库有两个坏块,而且根据对于bootstrap$的经验,可以大概确定60坏块很可能是C_TS#,第一反应type异常,498可能是seq$<br />
<br />
<strong>对数据库启动过程做10046,得到trace文件</strong></p>
<pre class="brush: bash; title: ; notranslate">
PARSING IN CURSOR #1 len=275 dep=2 uid=0 oct=3 lid=0 tim=27978051403575 hv=3408408745 ad='7df93cd0'
select name,online$,contents$,undofile#,undoblock#,blocksize,dflmaxext,dflinit,dflincr,dflextpct,dflminext, 
dflminlen, owner#,scnwrp,scnbas, NVL(pitrscnwrp, 0), NVL(pitrscnbas, 0), dflogging, bitmapped, inc#, flags, 
plugged, NVL(spare1,0), NVL(spare2,0) from ts$ where ts#=:1
END OF STMT
PARSE #1:c=0,e=92,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=27978051403569
BINDS #1:
kkscoacd
 Bind#0
  oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
  oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0
  kxsbbbfp=ffffffff7dbac9a8  bln=22  avl=02  flg=05
  value=2
EXEC #1:c=0,e=310,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=27978051404296
WAIT #1: nam='db file sequential read' ela= 42 file#=1 block#=60 blocks=1 obj#=-1 tim=27978051404449
Hex dump of (file 1, block 60)
Corrupt block relative dba: 0x0040003c (file 1, block 60)
Bad header found during buffer read
Data in bad block:
 type: 128 format: 0 rdba: 0x0040003c
 last change scn: 0x0005.ebe04bc9 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x4bc90602
 check value in block header: 0x6faa
 computed block checksum: 0x0
Reread of rdba: 0x0040003c (file 1, block 60) found same corrupted data
FETCH #1:c=10000,e=4072,p=1,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=27978051408438
STAT #1 id=1 cnt=0 pid=0 pos=1 obj=16 op='TABLE ACCESS CLUSTER TS$ (cr=2 pr=1 pw=0 time=4075 us)'
STAT #1 id=2 cnt=1 pid=1 pos=1 obj=7 op='INDEX UNIQUE SCAN I_TS# (cr=1 pr=0 pw=0 time=13 us)'
*** 2014-11-22 14:44:43.235
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [ktsitbs_info1], [2], [], [], [], [], [], []
Current SQL statement for this session:
select max(maxconcurrency) from sys.wrh$_undostat  where instance_number = :1 and dbid = :2 
and snap_id in   (select snap_id from dba_hist_snapshot where end_interval_time &gt;     
(select max(end_interval_time)-7 from dba_hist_snapshot))
</pre>
<p>这里显示了数据库启动报ORA-00600[ktsitbs_info1],[2]，明显的表示了b中的2是表示表空间号,由于ts$坏块,无法读取ts$中表空间信息,从而出现数据字典不一致,从而出现该错误。所以恢复该库的关键是修复file 1 block 60.<br />
<br />
<strong>bbed尝试修复file 1 block 60</strong></p>
<pre class="brush: bash; title: ; notranslate">
HNDX-DB% bbed password=blockedit mode=edit

BBED: Release 2.0.0.0.0 - Limited Production on Sat Nov 22 15:16:26 2014

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

************* !!! For Oracle Internal Use only !!! ***************

BBED&gt; set filename '/opt/oracle/oradata/xifenfei/system01.dbf'
        FILENAME        /opt/oracle/oradata/xifenfei/system01.dbf

BBED&gt; set block 8192
        BLOCK#          8192

BBED&gt; set block 60
        BLOCK#          60

BBED&gt; set count 64
        COUNT           64

BBED&gt; map
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 60                                    Dba:0x00000000
------------------------------------------------------------
BBED-00400: invalid blocktype (128)


BBED&gt; set block 61
        BLOCK#          61

BBED&gt; map
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 61                                    Dba:0x00000000
------------------------------------------------------------
 KTB Data Block (Table/Cluster)

 struct kcbh, 20 bytes                      @0       

 struct ktbbh, 72 bytes                     @20      

 struct kdbh, 14 bytes                      @92      

 struct kdbt[3], 12 bytes                   @106     

 sb2 kdbr[2]                                @118     

 ub1 freespace[7959]                        @122     

 ub1 rowdata[107]                           @8081    

 ub4 tailchk                                @8188    


BBED&gt; p kcbh
struct kcbh, 20 bytes                       @0       
   ub1 type_kcbh                            @0        0x06
   ub1 frmt_kcbh                            @1        0xa2
   ub1 spare1_kcbh                          @2        0x00
   ub1 spare2_kcbh                          @3        0x00
   ub4 rdba_kcbh                            @4        0x0040003d
   ub4 bas_kcbh                             @8        0x0000235b
   ub2 wrp_kcbh                             @12       0x0000
   ub1 seq_kcbh                             @14       0x01
   ub1 flg_kcbh                             @15       0x04 (KCBHFCKV)
   ub2 chkval_kcbh                          @16       0x7a85
   ub2 spare3_kcbh                          @18       0x0000

BBED&gt; set block 60
        BLOCK#          60

BBED&gt; d
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 60               Offsets:    0 to   63           Dba:0x00000000
------------------------------------------------------------------------
 80000000 0040003c ebe04bc9 00050204 6faa0000 01000000 00000006 29b3a204 
 00040ca0 00020200 00000000 000a0000 00000002 0080009b 00000100 80000000 

 &lt;32 bytes per line&gt;

BBED&gt; d block 61
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 61               Offsets:    0 to   63           Dba:0x00000000
------------------------------------------------------------------------
 06a20000 0040003d 0000235b 00000104 7a850000 01000000 00000006 00001837 
 00001738 00020200 00000000 0007002e 00000002 00800075 00012300 80000000 

 &lt;32 bytes per line&gt;

BBED&gt; set block 60
        BLOCK#          60

BBED&gt; m /x 06a2
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 60               Offsets:    0 to   63           Dba:0x00000000
------------------------------------------------------------------------
 06a20000 0040003c ebe04bc9 00050204 6faa0000 01000000 00000006 29b3a204 
 00040ca0 00020200 00000000 000a0000 00000002 0080009b 00000100 80000000 

 &lt;32 bytes per line&gt;

BBED&gt; map
 File: /opt/oracle/oradata/xifenfei/system01.dbf (0)
 Block: 60                                    Dba:0x00000000
------------------------------------------------------------
 KTB Data Block (Table/Cluster)

 struct kcbh, 20 bytes                      @0       

 struct ktbbh, 72 bytes                     @20      

 struct kdbh, 14 bytes                      @92      

 struct kdbt[3], 12 bytes                   @106     

 sb2 kdbr[2]                                @118     

 ub1 freespace[7598]                        @122     

 ub1 rowdata[468]                           @7720    

 ub4 tailchk                                @8188    


BBED&gt; sum apply
Check value for File 0, Block 60:
current = 0xe908, required = 0xe908

BBED&gt; verify
DBVERIFY - Verification starting
FILE = /opt/oracle/oradata/xifenfei/system01.dbf
BLOCK = 60


DBVERIFY - Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 1
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0

BBED&gt; 
</pre>
<p><strong>尝试启动数据库</strong></p>
<pre class="brush: bash; title: ; notranslate">
Sat Nov 22 15:51:33 2014
alter database open
Sat Nov 22 15:51:34 2014
Thread 1 opened at log sequence 7
  Current log# 8 seq# 7 mem# 0: /data2/oradata/redo0802.log
Successful open of redo thread 1
Sat Nov 22 15:51:34 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Nov 22 15:51:34 2014
SMON: enabling cache recovery
SMON: enabling tx recovery
Sat Nov 22 15:51:34 2014
Database Characterset is ZHS16CGB231280
Hex dump of (file 1, block 498) in trace file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_2818.trc
Corrupt block relative dba: 0x004001f2 (file 1, block 498)
Bad check value found during buffer read
Data in bad block:
 type: 6 format: 2 rdba: 0x004001f2
 last change scn: 0x0007.49499ca1 seq: 0x1 flg: 0x06
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x9ca10601
 check value in block header: 0xe458
 computed block checksum: 0x9720
Reread of rdba: 0x004001f2 (file 1, block 498) found same corrupted data
Sat Nov 22 15:51:35 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_smon_2803.trc:
ORA-00600: internal error code, arguments: [4000], [12], [], [], [], [], [], []
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=18, OS id=3000
Sat Nov 22 15:51:36 2014
Completed: alter database open
Sat Nov 22 15:51:36 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_3010.trc:
ORA-00600: internal error code, arguments: [6807], [AUDSES$], [144], [], [], [], [], []
Sat Nov 22 15:51:37 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_mmon_2809.trc:
ORA-00600: internal error code, arguments: [6807], [WRI$_ALERT_SEQUENCE], [8783], [], [], [], [], []
Sat Nov 22 15:51:37 2014
Non-fatal internal error happenned while SMON was doing non-existent object cleanup.
SMON encountered 1 out of maximum 100 non-fatal internal errors.
Sat Nov 22 15:51:38 2014
ORA-600 encountered when generating server alert SMG-3000
Sat Nov 22 15:51:38 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_mmon_2809.trc:
ORA-00600: internal error code, arguments: [ktcpoptx_0], [0x772705E60], [], [], [], [], [], []
</pre>
<p>只要出现ORA-600 4000和ORA-600 6807错误,其中ORA-600 6807错误比较明显是由于seq$坏块,导致AUDSES$ seq异常导致。ORA-600 4000应该是回滚段异常,继续分析回滚段</p>
<pre class="brush: sql; title: ; notranslate">
SQL&gt; select name,ts#,status$ from undo$;

NAME                                  TS#    STATUS$
------------------------------ ---------- ----------
SYSTEM                                  0          2
_SYSSMU1$                               1          2
_SYSSMU2$                               1          2
_SYSSMU3$                               1          2
…………
_SYSSMU168$                             1          2
_SYSSMU169$                             1          2
</pre>
<p>这里很异常,system回滚段在数据库open之后,按照常理不可能处于STATUS$=2(OFFLINE)状态。而且其他回滚段全部为OFFLINE状态也属于异常情况.而且尝试drop undo报ORA-01561,另外在dba_rollback_segs中无SYSTEM(查询结果忘记保存)</p>
<pre class="brush: bash; title: ; notranslate">
SQL&gt; drop tablespace undotbs1 including contents; 
drop tablespace undotbs1 including contents
*
ERROR at line 1:
ORA-01561: failed to remove all objects in the tablespace specified
</pre>
<p>通过这一系列很怀疑是由于bbed 修改了undo$等相关基表信息导致现在system中的undo信息混乱.信息反馈给客户后,客户想起来昨天给他们恢复的公司在bbed操作前备份了system01.dbf.突然感觉救星来了.实在怕不懂bbed的人折腾bbed<br />
<br />
<strong>dbv检测备份文件</strong></p>
<pre class="brush: bash; title: ; notranslate">
DBVERIFY - Verification starting : FILE = /data3/backup/system01.dbf_bak
Page 60 is marked corrupt
Corrupt block relative dba: 0x0040003c (file 1, block 60)
Bad header found during dbv: 
Data in bad block:
 type: 128 format: 0 rdba: 0x0040003c
 last change scn: 0x0005.ebe04bc9 seq: 0x2 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x4bc90602
 check value in block header: 0x6faa
 computed block checksum: 0x0

Block Checking: DBA = 4194802, Block Type = KTB-managed data block
data header at 0x1002ef05c
kdbchk: row locked by non-existent transaction
        table=0   slot=4
        lockid=1   ktbbhitc=2
Page 498 failed with check code 6101


DBVERIFY - Verification complete

Total Pages Examined         : 786432
Total Pages Processed (Data) : 201131
Total Pages Failing   (Data) : 1
Total Pages Processed (Index): 221394
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 60265
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 303641
Total Pages Marked Corrupt   : 1
Total Pages Influx           : 0
Highest block SCN            : 1229823477 (7.1229823477)
</pre>
<p>好家伙只有一个物理坏块和一个逻辑坏块,而对于物理坏块block 60已经知道如何修复,逻辑坏块可以尝试设置隐含参数跳过去,bbed修改相关block(同上步骤)<br />
<br />
<strong>再次启动数据库</strong></p>
<pre class="brush: sql; title: ; notranslate">
dd if=/opt/oracle/oradata/xifenfei/system01.dbf bs=8192 count=2 of=/tmp/system01.2
dd if=/tmp/system01.2 of=/data3/backup/system01.dbf_bak bs=8192 count=2 conv=notrunc 
</pre>
<pre class="brush: bash; title: ; notranslate">
Sat Nov 22 17:52:50 2014
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Sat Nov 22 17:53:38 2014
alter database rename file '/opt/oracle/oradata/xifenfei/system01.dbf' to '/data3/backup/system01.dbf_bak'
Sat Nov 22 17:53:39 2014
Completed: alter database rename file '/opt/oracle/oradata/xifenfei/system01.dbf' to '/data3/backup/system01.dbf_bak'
Sat Nov 22 17:55:43 2014
alter database open
Sat Nov 22 17:55:48 2014
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=18, OS id=15858
Sat Nov 22 17:56:10 2014
ARC0: Archival started
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=17, OS id=15879
Sat Nov 22 17:56:19 2014
Thread 1 opened at log sequence 7
  Current log# 8 seq# 7 mem# 0: /data2/oradata/redo0802.log
Successful open of redo thread 1
Sat Nov 22 17:56:19 2014
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Sat Nov 22 17:56:19 2014
SMON: enabling cache recovery
SMON: enabling tx recovery
Sat Nov 22 17:56:20 2014
ARC1: STARTING ARCH PROCESSES
Sat Nov 22 17:56:20 2014
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Sat Nov 22 17:56:22 2014
Database Characterset is ZHS16CGB231280
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Sat Nov 22 17:56:33 2014
ARC2: Archival started
ARC1: STARTING ARCH PROCESSES COMPLETE
ARC1: Becoming the heartbeat ARCH
ARC2 started with pid=23, OS id=15928
QMNC started with pid=25, OS id=15996
Sat Nov 22 17:57:11 2014
Completed: alter database open
Sat Nov 22 17:57:18 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_16010.trc:
ORA-00600: internal error code, arguments: [4511], [], [], [], [], [], [], []
Sat Nov 22 17:57:26 2014
Errors in file /opt/oracle/admin/xifenfei/udump/xifenfei_ora_16012.trc:
ORA-00600: internal error code, arguments: [4511], [], [], [], [], [], [], []
Sat Nov 22 17:58:17 2014
Starting background process EMN0
Sat Nov 22 18:00:03 2014
Shutting down instance: further logons disabled
EMN0 started with pid=71, OS id=16421
Sat Nov 22 18:00:12 2014
SMON: Restarting fast_start parallel rollback
Sat Nov 22 18:00:23 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_p000_15951.trc:
ORA-00600: internal error code, arguments: [4198], [9], [], [], [], [], [], []
Sat Nov 22 18:00:24 2014
Stopping background process CJQ0
Sat Nov 22 18:00:24 2014
Stopping background process QMNC
Sat Nov 22 18:00:27 2014
Doing block recovery for file 2 block 41
Block recovery from logseq 7, block 180883 to scn 214748389244
Sat Nov 22 18:00:27 2014
Recovery of Online Redo Log: Thread 1 Group 8 Seq 7 Reading mem 0
  Mem# 0 errs 0: /data2/oradata/redo0802.log
Block recovery stopped at EOT rba 7.180988.16
Block recovery completed at rba 7.180988.16, scn 50.24441
Sat Nov 22 18:00:32 2014
Stopping background process MMNL
Sat Nov 22 18:00:38 2014
Stopping background process MMON
Sat Nov 22 18:00:41 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_smon_15395.trc:
ORA-00600: internal error code, arguments: [4137], [], [], [], [], [], [], []
Sat Nov 22 18:00:42 2014
ORACLE Instance xifenfei (pid = 9) - Error 600 encountered while recovering transaction (3, 4).
Sat Nov 22 18:00:42 2014
Errors in file /opt/oracle/admin/xifenfei/bdump/xifenfei_smon_15395.trc:
ORA-00600: internal error code, arguments: [4137], [], [], [], [], [], [], []
</pre>
<p>这里都是很常规的错误,查询undo$也已经正常,重建新undo表空间删除老undo,然后alert日志中无其他报错,数据库恢复至此完成,建议客户导出导入重建数据库</p>
<div id="crp_related"><ul><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/06/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1ora-00600kdxlinpsno-out-of-rangeora-006003020ora-006004000ora-006004193%e7%9a%84%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">记录一次ORA-00600[kdxlin:psno out of range]/ORA-00600[3020]/ORA-00600[4000]/ORA-00600[4193]的数据库恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1ora-600-3004-%e6%81%a2%e5%a4%8d%e8%bf%87%e7%a8%8b%e5%92%8c%e5%a4%84%e7%90%86%e6%80%9d%e8%b7%af.html" rel="bookmark" class="crp_title">记录一次ORA-600 3004 恢复过程和处理思路</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/08/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1system%e5%9d%8f%e5%9d%97%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">记录一次system表空间坏块(ORA-01578)数据库恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/08/%e5%8f%88%e4%b8%80%e8%b5%b7%e5%ad%98%e5%82%a8%e6%95%85%e9%9a%9c%e5%af%bc%e8%87%b4ora-00333-ora-00312%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">又一起存储故障导致ORA-00333 ORA-00312恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/08/oracle-8-1-7-%e6%95%b0%e6%8d%ae%e5%ba%93ora-600-4000%e6%95%85%e9%9a%9c%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORACLE 8.1.7 数据库ORA-600 4000故障恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/10/%e9%80%9a%e8%bf%87%e5%a4%9a%e6%ac%a1%e8%a7%84%e9%81%bf%e7%b1%bb%e4%bc%bcora-01248-file-n-was-created-in-the-future-of-incomplete-recovery%e9%94%99%e8%af%af.html" rel="bookmark" class="crp_title">通过多次resetlogs规避类似ORA-01248: file N was created in the future of incomplete recovery错误</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/05/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e9%81%ad%e9%81%87ora-006003705.html" rel="bookmark" class="crp_title">数据库恢复遭遇ORA-00600[3705]</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/%e6%9f%90%e9%9b%86%e5%9b%a2ebs%e6%95%b0%e6%8d%ae%e5%ba%93redo-undo%e4%b8%a2%e5%a4%b1%e5%af%bc%e8%87%b4%e6%82%b2%e5%89%a7.html" rel="bookmark" class="crp_title">某集团ebs数据库redo undo丢失导致悲剧</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/817%e6%95%b0%e6%8d%ae%e5%ba%93ora-600-4194%e6%95%85%e9%9a%9c%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORACLE 8.1.7 数据库ORA-600 4194故障恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2024/12/controlfile-corrupt-block.html" rel="bookmark" class="crp_title">ORA-00227: corrupt block detected in control file</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/02/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1ora-00316-ora-00312-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">记录一次ORA-00316 ORA-00312 redo异常恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/10/ora-27069-skgfdisp-%e5%b0%9d%e8%af%95%e5%9c%a8%e6%96%87%e4%bb%b6%e8%8c%83%e5%9b%b4%e5%a4%96%e6%89%a7%e8%a1%8c-io.html" rel="bookmark" class="crp_title">ORA-27069: skgfdisp: 尝试在文件范围外执行 I/O</a></li></ul></div>]]></content:encoded>
			<wfw:commentRss>https://www.xifenfei.com/2014/11/system%e8%a1%a8%e7%a9%ba%e9%97%b4%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8d-c_ts%e5%af%b9%e8%b1%a1%e5%9d%8f%e5%9d%97%e6%81%a2%e5%a4%8dfile-1-block-60.html/feed</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>数据库恢复历史再次刷新到Oracle 7.3.2版本&#8212;redo异常恢复</title>
		<link>https://www.xifenfei.com/2014/06/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e5%8e%86%e5%8f%b2%e5%86%8d%e6%ac%a1%e5%88%b7%e6%96%b0%e5%88%b0oracle-7-3-2%e7%89%88%e6%9c%ac-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html</link>
		<comments>https://www.xifenfei.com/2014/06/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e5%8e%86%e5%8f%b2%e5%86%8d%e6%ac%a1%e5%88%b7%e6%96%b0%e5%88%b0oracle-7-3-2%e7%89%88%e6%9c%ac-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html#comments</comments>
		<pubDate>Tue, 17 Jun 2014 02:24:15 +0000</pubDate>
		<dc:creator><![CDATA[惜分飞]]></dc:creator>
				<category><![CDATA[非常规恢复]]></category>
		<category><![CDATA[ORA-600 3600]]></category>
		<category><![CDATA[ORA-600 4306]]></category>
		<category><![CDATA[Oracle 7.3]]></category>
		<category><![CDATA[ORACLE恢复]]></category>
		<category><![CDATA[数据库恢复]]></category>

		<guid isPermaLink="false">https://www.xifenfei.com/?p=5378</guid>
		<description><![CDATA[联系：手机/微信(+86 17813235971) QQ(107644445)  &#8230; <a href="https://www.xifenfei.com/2014/06/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e5%8e%86%e5%8f%b2%e5%86%8d%e6%ac%a1%e5%88%b7%e6%96%b0%e5%88%b0oracle-7-3-2%e7%89%88%e6%9c%ac-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html">继续阅读 <span class="meta-nav">&#8594;</span></a>]]></description>
				<content:encoded><![CDATA[<div id="xifenfeicopyright">
<p>联系：<a href="https://www.xifenfei.com/about_xifenfei" target="_blank">手机/微信(+86 17813235971)   QQ(107644445)</a><a target="_blank" href="http://wpa.qq.com/msgrd?v=3&#038;uin=107644445&#038;site=qq&#038;menu=yes"><img src="https://www.xifenfei.com/wp-content/themes/img/site_qq.jpg" alt="QQ咨询惜分飞" title="QQ咨询惜分飞"/></a></p>
<p>标题：<a href="https://www.xifenfei.com/2014/06/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e5%8e%86%e5%8f%b2%e5%86%8d%e6%ac%a1%e5%88%b7%e6%96%b0%e5%88%b0oracle-7-3-2%e7%89%88%e6%9c%ac-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html" target="_blank">数据库恢复历史再次刷新到Oracle 7.3.2版本&#8212;redo异常恢复</a></p>
<p>作者：<a href="https://www.xifenfei.com" target="_blank">惜分飞</a>©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]</p>
</div>
<p>有网友在QQ上找我,说Oracle 7.3的数据库,因为redo异常咨询我是否可以恢复<br />
<img src="//www.xifenfei.com/wp-content/uploads/2014/06/qq1.jpg" alt="qq咨询" width="390" height="74" class="alignleft size-full wp-image-5379" /></p>
<hr />
检查数据库得到以下信息</p>
<pre class="brush: sql; title: ; notranslate">
SVRMGR&gt; select * from v$version;
BANNER
----------------------------------------------------------------
Oracle7 Workgroup Server Release 7.3.2.2.1 - Production Release
PL/SQL Release 2.3.2.2.0 - Production
CORE Version 3.5.2.0.0 - Production
TNS for 32-bit Windows: Version 2.3.2.1.0 - Production
NLSRTL Version 3.2.2.0.0 - Production
已选择 5 行
</pre>
<p>数据文件信息<br />
<img src="//www.xifenfei.com/wp-content/uploads/2014/06/qq.png" alt="qq" width="322" height="485" class="alignleft size-full wp-image-5381" /><br />
<img src="//www.xifenfei.com/wp-content/uploads/2014/06/qq2.png" alt="qq2" width="533" height="340" class="alignleft size-full wp-image-5382" /></p>
<hr />
redo信息<br />
<img src="//www.xifenfei.com/wp-content/uploads/2014/06/qq3.png" alt="qq3" width="478" height="88" class="alignleft size-full wp-image-5380" /></p>
<hr />
跳过redo进行恢复,在resetlogs过程中报rbs表空间坏块,然后通过dul工具获得回滚段名称,然后使用隐含参数屏蔽掉</p>
<pre class="brush: bash; title: ; notranslate">
License high water mark = 2
Starting up ORACLE RDBMS Version: 7.3.2.2.1.
System parameters with non-default values:
  processes                = 800
  shared_pool_size         = 540000000
  control_files            = D:\ORANT\DATABASE\ctl1orcl.ora, D:\ORANT\DATABASE\ctl2orcl.ora
  compatible               = 7.3.0.0.0
  log_buffer               = 327680
  log_checkpoint_interval  = 1000000
  db_files                 = 40
  db_file_simultaneous_writes= 1280
  max_rollback_segments    = 12800
  _offline_rollback_segments= RB13, RB14, RB15, RB16, RB20
  _corrupted_rollback_segments= RB13, RB14, RB15, RB16, RB20
  sequence_cache_entries   = 100
  sequence_cache_hash_buckets= 100
  remote_login_passwordfile= SHARED
  mts_servers              = 0
  mts_max_servers          = 0
  mts_max_dispatchers      = 0
  audit_trail              = NONE
  sort_area_retained_size  = 65536
  sort_direct_writes       = AUTO
  db_name                  = oracle
  open_cursors             = 800
  text_enable              = TRUE
  snapshot_refresh_processes= 1
  background_dump_dest     = %RDBMS73%\trace
  user_dump_dest           = %RDBMS73%\trace

Mon Jun 16 16:46:57 2014

PMON started
Mon Jun 16 16:46:57 2014

DBWR started
Mon Jun 16 16:46:57 2014

LGWR started
Mon Jun 16 16:46:57 2014

RECO started
Mon Jun 16 16:46:57 2014

SNP0 started
Mon Jun 16 16:46:57 2014
alter database  mount exclusive
Mon Jun 16 16:46:58 2014
Successful mount of redo thread 1.
Mon Jun 16 16:46:58 2014
Completed: alter database  mount exclusive
Mon Jun 16 16:48:15 2014
alter database open
Mon Jun 16 16:48:16 2014
Beginning crash recovery of 1 threads
Crash recovery completed successfully
Mon Jun 16 16:48:17 2014
Thread 1 advanced to log sequence 9
  Current log# 1 seq# 9 mem# 0: D:\ORANT\DATABASE\LOG2ORCL.ORA
Thread 1 opened at log sequence 9
  Current log# 1 seq# 9 mem# 0: D:\ORANT\DATABASE\LOG2ORCL.ORA
Successful open of redo thread 1.
Mon Jun 16 16:48:18 2014
SMON: enabling cache recovery
Mon Jun 16 16:48:19 2014
Completed: alter database open
Mon Jun 16 16:48:20 2014
SMON: enabling tx recovery
SMON: about to recover undo segment 14
SMON: mark undo segment 14 as needs recovery
SMON: about to recover undo segment 15
SMON: mark undo segment 15 as needs recovery
SMON: about to recover undo segment 16
SMON: mark undo segment 16 as needs recovery
SMON: about to recover undo segment 17
SMON: mark undo segment 17 as needs recovery
SMON: about to recover undo segment 18
SMON: mark undo segment 18 as needs recovery
Mon Jun 16 16:48:20 2014
Errors in file D:\ORANT\RDBMS73\trace\orclSMON.TRC:
ORA-00600: internal error code, arguments: [4306], [21], [2], [], [], [], [], []
</pre>
<p>数据库在启动过程中出现ORA-00600[4306]，导致smon异常。该错误是因为在数据库open过程中smon会清理临时段从而出现该错误,通过设置event跳过，数据库算整体打开，不过在恢复过程中还遇到了</p>
<pre class="brush: bash; title: ; notranslate">
Mon Jun 16 17:53:10 2014
Errors in file D:\ORANT\RDBMS73\trace\orclDBWR.TRC:
ORA-00600: internal error code, arguments: [3600], [3], [14], [], [], [], [], []


Mon Jun 16 18:05:12 2014
Errors in file D:\ORANT\RDBMS73\trace\ORA06880.TRC:
ORA-01578: ORACLE数据块有错(文件号12, 块号46644)
ORA-01110: 文件'12'没有联机
ORA-00600: 内部错误码, 变元: [4194], [18], [5], [], [], []
ORA-00600: 内部错误码, 变元: [4194], [18], [5], [], [], []
</pre>
<p>ORA-00600[3600]是因为在offline 回滚段所在表空间锁出现的问题<br />
ORA-00600[4194]是因为回滚段所在的表空间数据文件出现坏块所导致</p>
<div id="crp_related"><ul><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/08/oracle-8-1-7-%e6%95%b0%e6%8d%ae%e5%ba%93ora-600-4000%e6%95%85%e9%9a%9c%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORACLE 8.1.7 数据库ORA-600 4000故障恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2011/08/undo%e5%bc%82%e5%b8%b8%e5%a4%84%e7%90%86%e6%ad%a5%e9%aa%a4%ef%bc%889i%ef%bc%89.html" rel="bookmark" class="crp_title">undo异常处理步骤（9i）</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/08/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1system%e5%9d%8f%e5%9d%97%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">记录一次system表空间坏块(ORA-01578)数据库恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/817%e6%95%b0%e6%8d%ae%e5%ba%93ora-600-4194%e6%95%85%e9%9a%9c%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORACLE 8.1.7 数据库ORA-600 4194故障恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/10/%e9%80%9a%e8%bf%87%e5%a4%9a%e6%ac%a1%e8%a7%84%e9%81%bf%e7%b1%bb%e4%bc%bcora-01248-file-n-was-created-in-the-future-of-incomplete-recovery%e9%94%99%e8%af%af.html" rel="bookmark" class="crp_title">通过多次resetlogs规避类似ORA-01248: file N was created in the future of incomplete recovery错误</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/10/ora-27069-skgfdisp-%e5%b0%9d%e8%af%95%e5%9c%a8%e6%96%87%e4%bb%b6%e8%8c%83%e5%9b%b4%e5%a4%96%e6%89%a7%e8%a1%8c-io.html" rel="bookmark" class="crp_title">ORA-27069: skgfdisp: 尝试在文件范围外执行 I/O</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/03/ora-00600kcrf_resilver_log_1%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORA-00600[kcrf_resilver_log_1]异常恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2012/05/ora-00600kcratr1_lostwrtora-006003020%e9%94%99%e8%af%af%e6%81%a2%e5%a4%8d.html" rel="bookmark" class="crp_title">ORA-00600[kcratr1_lostwrt]/ORA-00600[3020]错误恢复</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/%e6%9f%90%e9%9b%86%e5%9b%a2ebs%e6%95%b0%e6%8d%ae%e5%ba%93redo-undo%e4%b8%a2%e5%a4%b1%e5%af%bc%e8%87%b4%e6%82%b2%e5%89%a7.html" rel="bookmark" class="crp_title">某集团ebs数据库redo undo丢失导致悲剧</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/06/oracle%e5%ae%89%e5%85%a8%e8%ad%a6%e7%a4%ba%e5%bd%95%e5%8a%a0%e9%94%99%e8%a3%b8%e8%ae%be%e5%a4%87%e5%af%bc%e8%87%b4redo%e5%bc%82%e5%b8%b8.html" rel="bookmark" class="crp_title">Oracle安全警示录:加错裸设备导致redo异常</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2014/07/%e8%ae%b0%e5%bd%95%e4%b8%80%e6%ac%a1ora-600-3004-%e6%81%a2%e5%a4%8d%e8%bf%87%e7%a8%8b%e5%92%8c%e5%a4%84%e7%90%86%e6%80%9d%e8%b7%af.html" rel="bookmark" class="crp_title">记录一次ORA-600 3004 恢复过程和处理思路</a></li><li style="list-style-type:decimal;font-size:11px;" ><a href="https://www.xifenfei.com/2024/03/ora-600-ktsiseginfo1.html" rel="bookmark" class="crp_title">ORA-600 ktsiseginfo1故障</a></li></ul></div>]]></content:encoded>
			<wfw:commentRss>https://www.xifenfei.com/2014/06/%e6%95%b0%e6%8d%ae%e5%ba%93%e6%81%a2%e5%a4%8d%e5%8e%86%e5%8f%b2%e5%86%8d%e6%ac%a1%e5%88%b7%e6%96%b0%e5%88%b0oracle-7-3-2%e7%89%88%e6%9c%ac-redo%e5%bc%82%e5%b8%b8%e6%81%a2%e5%a4%8d.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>
