惜分飞 » obet bbed

kcratr_nab_less_than_odr和system坏块故障处理

惜分飞 — Mon, 24 Aug 2026 16:01:34 +0000

标题：kcratr_nab_less_than_odr和system坏块故障处理

学校客户由于机房断电,导致数据库异常.现场工程师进行了现场备份,然后尝试强制拉库,结果没有成功,大概报错和操作过程如下:
断电之后启动数据库,数据库报ORA-600 kcratr_nab_less_than_odr故障解决错误,这个是一种非常常见的操作,一般是由于写丢失导致,以前有过很多类似恢复经历:
ORA-600 kcratr_nab_less_than_odr故障解决
 差点被误操作的ORA-600 kcratr_nab_less_than_odr故障
 ORA-600 kcratr_nab_less_than_odr和ORA-600 2662故障处理
 ORA-600 kcratr_nab_less_than_odr和ORA-600 4193故障处理

Tue Aug 18 23:59:19 2026
ALTER DATABASE OPEN
This instance was first to open
Beginning crash recovery of 2 threads
 parallel recovery started with 19 processes
Started redo scan
Completed redo scan
 read 21634 KB redo, 800 data blocks need recovery
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_275950.trc  (incident=703037):
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6916], [3], [4], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_703037/orcl2_ora_275950_i703037.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Abort recovery for domain 0
Aborting crash recovery due to error 600
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_275950.trc:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6916], [3], [4], [], [], [], []
Abort recovery for domain 0
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_275950.trc:
ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6916], [3], [4], [], [], [], []
ORA-600 signalled during: ALTER DATABASE OPEN...

现场恢复人员上来之后,直接尝试做强制resetlogs操作,数据库报ORA-600 ORA-600 krsi_al_hdr_update.15错误,主要是由于redo写丢失导致无法resetlogs成功,具体参考：Alter Database Open Resetlogs returns error ORA-00600: [krsi_al_hdr_update.15], (Doc ID 2026541.1) Oracle断电故障处理

Wed Aug 19 00:28:47 2026
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 29721342127
Archived Log entry 38935 added for thread 1 sequence 6916 ID 0xceea62af dest 1:
ARCH: All Archive destinations made inactive due to error 742
ARCH: Closing local archive destination LOG_ARCHIVE_DEST_1: '+ARCHDG/2_19951_1224781365.arc' (error 742)(orcl2)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_287355.trc  (incident=727038):
ORA-00600: internal error code, arguments: [krsi_al_hdr_update.15], [4294967295], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_727038/orcl2_ora_287355_i727038.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Master archival failure: 600
Archive all online redo logfiles failed:600
ORA-600 signalled during: alter database open resetlogs...

通过 ALTER DATABASE RECOVER database using backup controlfile until cancel之后,继续尝试强制打开库，报ORA-600 2662错误.

Wed Aug 19 00:40:25 2026
Checker run found 32 new persistent data failures
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 29721342127
Archived Log entry 38936 added for thread 1 sequence 6915 ID 0xceea62af dest 1:
Archived Log entry 38937 added for thread 1 sequence 6916 ID 0xceea62af dest 1:
Archived Log entry 38938 added for thread 2 sequence 19951 ID 0xceea62af dest 1:
Archived Log entry 38939 added for thread 2 sequence 19950 ID 0xceea62af dest 1:
Clearing online redo logfile 1 +DATADG/orcl/onlinelog/group_1.319.1224781365
Clearing online log 1 of thread 1 sequence number 6915
Wed Aug 19 00:40:34 2026
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 +DATADG/orcl/onlinelog/group_2.320.1224781367
Clearing online log 2 of thread 1 sequence number 6916
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 +DATADG/orcl/onlinelog/group_3.323.1224781463
Clearing online log 3 of thread 2 sequence number 19951
Clearing online redo logfile 3 complete
Clearing online redo logfile 4 +DATADG/orcl/onlinelog/group_4.324.1224781465
Clearing online log 4 of thread 2 sequence number 19950
Clearing online redo logfile 4 complete
Resetting resetlogs activation ID 3471467183 (0xceea62af)
Online log +DATADG/orcl/onlinelog/group_1.319.1224781365: Thread 1 Group 1 was previously cleared
Online log +ARCHDG/orcl/onlinelog/group_1.4574.1224781367: Thread 1 Group 1 was previously cleared
Online log +DATADG/orcl/onlinelog/group_2.320.1224781367: Thread 1 Group 2 was previously cleared
Online log +ARCHDG/orcl/onlinelog/group_2.10729.1224781369: Thread 1 Group 2 was previously cleared
Online log +DATADG/orcl/onlinelog/group_3.323.1224781463: Thread 2 Group 3 was previously cleared
Online log +ARCHDG/orcl/onlinelog/group_3.12866.1224781463: Thread 2 Group 3 was previously cleared
Online log +DATADG/orcl/onlinelog/group_4.324.1224781465: Thread 2 Group 4 was previously cleared
Online log +ARCHDG/orcl/onlinelog/group_4.7527.1224781465: Thread 2 Group 4 was previously cleared
Wed Aug 19 00:40:43 2026
Setting recovery target incarnation to 3
Wed Aug 19 00:40:43 2026
Assigning activation ID 3488382552 (0xcfec7e58)
Thread 2 opened at log sequence 1
  Current log# 3 seq# 1 mem# 0: +DATADG/orcl/onlinelog/group_3.323.1224781463
  Current log# 3 seq# 1 mem# 1: +ARCHDG/orcl/onlinelog/group_3.12866.1224781463
Successful open of redo thread 2
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Aug 19 00:40:43 2026
SMON: enabling cache recovery
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_291405.trc  (incident=739007):
ORA-00600: internal error code, arguments: [2662], [6], [3951555809], [6], [3951556555], [12583040], [], []
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_739007/orcl2_ora_291405_i739007.trc
Wed Aug 19 00:40:45 2026
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_291405.trc:
ORA-00600: internal error code, arguments: [2662], [6], [3951555809], [6], [3951556555], [12583040], [], []
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_291405.trc:
ORA-00600: internal error code, arguments: [2662], [6], [3951555809], [6], [3951556555], [12583040], [], []
Error 600 happened during db open, shutting down database
USER (ospid: 291405): terminating the instance due to error 600
Instance terminated by USER, pid = 291405
ORA-1092 signalled during: alter database open resetlogs...

到这一步,现场停止了继续尝试,我接手故障处理.先dbv检测坏块,发现system有两个坏块

[oracle@db3 ~]$ dbv userid=sys/oracle file=/datapool/cold_backup_20260818/SYSTEM.314.1224781265

DBVERIFY: Release 11.2.0.4.0 - Production on Sat Aug 22 10:39:26 2026

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /datapool/cold_backup_20260818/SYSTEM.314.1224781265
Page 94587 is marked corrupt
Corrupt block relative dba: 0x0041717b (file 1, block 94587)
Bad header found during dbv: 
Data in bad block:
 type: 11 format: 2 rdba: 0x00400001
 last change scn: 0x0000.00000000 seq: 0x1 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000b01
 check value in block header: 0xd49
 computed block checksum: 0x0

Page 95021 is marked corrupt
Corrupt block relative dba: 0x0041732d (file 1, block 95021)
Bad header found during dbv: 
Data in bad block:
 type: 11 format: 2 rdba: 0x00400001
 last change scn: 0x0000.00000000 seq: 0x1 flg: 0x04
 spare1: 0x0 spare2: 0x0 spare3: 0x0
 consistency value in tail: 0x00000b01
 check value in block header: 0xd49
 computed block checksum: 0x0



DBVERIFY - Verification complete

Total Pages Examined         : 157440
Total Pages Processed (Data) : 72079
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 21854
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 49484
Total Pages Processed (Seg)  : 1
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 14021
Total Pages Marked Corrupt   : 2
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 0 (0.0)

OBET> dbv file 1

===============================================
DBV (Data Block Verification)
Block Size: 8192 bytes
Endian:     little-endian (x86)
Target File: #1(only)
===============================================

Verifying file #1: /datapool/cold_backup_20260818/SYSTEM.314.1224781265 (157441 blocks) - Started: 2026-08-22 11:14:48
File #32: rfile=1 (0x00000001)  header_block_num=157440 (0x00026700)  filesize_status:OK
  Progress: 100000 / 157441 blocks checked...
  File #1completed: 0 all zero, 0 soft corrupted, 2 tailchk error, 0 checksum error, 0 rdba error

DBV completed at: 2026-08-22 11:14:59

===============================================
DBV Summary:
Total blocks checked: 157439
Total all zero blocks found: 0
Total all rdba error blocks found: 0
Total all tailchk error blocks found: 2
Total all soft corrupted blocks found: 0
Total all checksum error blocks found: 0
Total bad blocks found: 2
Execution time: 11.00 seconds
Throughput: 111.82 MB/s
===============================================

Detailed report saved to: dbv_file_32_20260822111448.log
Files processed: 1

OBET> list corrupt
file 1(/datapool/cold_backup_20260818/SYSTEM.314.1224781265) total bad blocks: 2
block#   bad block type
94587   tailchk
95021   tailchk

使用obet修复坏块
Oracle Block Editor Tool使用手册

OBET> set file 32
filename set to: /datapool/cold_backup_20260818/SYSTEM.314.1224781265 (file#1)

OBET> set block 94587
block set to: 94587

OBET> set mode edit
mode set to: edit

OBET> repair block
Warning: Missing value for 'block', using global setting: 94587

Repairing block 94587 in file /datapool/cold_backup_20260818/SYSTEM.314.1224781265...

Repair analysis for block 94587:
1. seq_kcbh check: 0x01 -> OK
2. Tailchk check: 0x0106C0E4 -> needs repair (0xE4C00601)
3. Checksum check: 0x5EC5 -> needs repair (0x7DE6)

Confirm repair operations:
File: /datapool/cold_backup_20260818/SYSTEM.314.1224781265
Block: 94587
Operations needed: fix tailchk, fix checksum
Confirm? (Y/YES to proceed): y

Verification after repair:
1. seq_kcbh: 0x01 OK
2. Tailchk: 0x0106C0E4 OK
3. Checksum: 0x7DE6 OK

Block 94587 repair completed successfully.

OBET> set block 95021
block set to: 95021

OBET> repair block 
Warning: Missing value for 'block', using global setting: 95021

Repairing block 95021 in file /datapool/cold_backup_20260818/SYSTEM.314.1224781265...

Repair analysis for block 95021:
1. seq_kcbh check: 0x01 -> OK
2. Tailchk check: 0xF8FA0601 -> needs repair (0xE06C0601)
3. Checksum check: 0x81BA -> OK

Confirm repair operations:
File: /datapool/cold_backup_20260818/SYSTEM.314.1224781265
Block: 95021
Operations needed: fix tailchk
Confirm? (Y/YES to proceed): y

Verification after repair:
1. seq_kcbh: 0x01 OK
2. Tailchk: 0x01066CE0 OK
3. Checksum: 0x81BA OK

Block 95021 repair completed successfully.

dbv检查确认坏块修复成功

[oracle@db3 ~]$ dbv file=/datapool/cold_backup_20260818/SYSTEM.314.1224781265

DBVERIFY: Release 11.2.0.4.0 - Production on Sat Aug 22 11:17:30 2026

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /datapool/cold_backup_20260818/SYSTEM.314.1224781265


DBVERIFY - Verification complete

Total Pages Examined         : 157440
Total Pages Processed (Data) : 72080
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 21855
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 49484
Total Pages Processed (Seg)  : 1
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 14021
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 3951556890 (6.3951556890)

后面的恢复比较简单,使用客户恢复之前的备份,直接重建ctl,然后打开库成功,并且做rman校验没有异常,直接把恢复之后的库备份还原到asm里面,完成本次恢复任务

分享一例运行在aix上的sap系统数据库恢复过程

惜分飞 — Fri, 14 Aug 2026 15:49:53 +0000

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：分享一例运行在aix上的sap系统数据库恢复过程

情况描述
客户sap系统运行在aix系统,ibm v7000存储上,数据存放在三个1T的lun组成的vg的多个lv中.异常断电之后,但是给数据库使用的三个lun丢失了2个,从而使得所有vg/lv异常,通过硬件恢复出来异常的2个lun和剩余的1个lun一起,把所有的数据文件恢复出来.但是由于某种原因,出现部分block被覆盖(其中还包括两个文件头损坏).

坏块检测
对于恢复的所有文件,为了快速做一遍坏块检查,直接在恢复的win机器上使用过obet做了一次dbv检查obet实现对数据文件坏块检测功能(使用obet的dbv检查有几个好处:1>可以在win上面检测aix的数据文件;2>可以检测数据文件头损坏的数据文件其他block;3>检测速度比原生dbv快[每个文件内部加了并行检测]),检测结果如下

--其中两个文件头损坏
File #62: E:\sr3_54\sr3.data54 (1280000 blocks) - Started: 2026-07-28 23:57:53
File #62: rfile=0 (0x00000000)  header_block_num=0 (0x00000000)  filesize_status:NO
file 62, block 0: block all zero
file 62, block 1: block all zero

File #84: E:\sr3_76\sr3.data76 (4185601 blocks) - Started: 2026-07-29 00:55:32
File #84: rfile=638937491 (0x26156993)  header_block_num=77152556 (0x0499412C)  filesize_status:NO
file 84, block 1: rdba error (expected 1, got 69760), bad block
file 84, block 2: rdba error (expected 2, got 589836), bad block

--坏块汇总
DBV completed at: 2026-07-29 00:58:10
===============================================
DBV Summary:
Total blocks checked: 196287237
Total all zero blocks found: 337417
Total all rdba error blocks found: 1062468
Total all tailchk error blocks found: 1
Total all soft corrupted blocks found: 0
Total all checksum error blocks found: 0
Total bad blocks found: 1399886
Execution time: 10507.00 seconds
===============================================

这个统计下来好的block在99.3%左右,证明硬件层面的会效果已经非常好.

碎片工具进一步恢复
恢复公司文件系统层面恢复有数据块遗漏的可能,通过碎片工具(OraScan(Oracle 碎片扫描工具) 使用说明)进一步扫描

通过确认62号文件还有少量block可以进一步恢复（也就是说碎片扫描到的62号文件的block多于硬件公司恢复出来的62号文件里面好的block数量）,通过obet的merge功能进行填补

基于上述操作,对于lun里面的数据文件实现了最大效果恢复.

数据库恢复操作
1. 上次恢复文件到aix,offline掉异常文件头数据文件打开数据库

sapprd2:oraprd 8> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Mon Aug 3 18:41:28 2026

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select status from v$instance;

STATUS
------------------------------------
STARTED

SQL> alter database mount;
alter database mount
*
ERROR at line 1:
ORA-00214: control file '/oracle/PRD/origlogA/cntrl/cntrlPRD.dbf' version
9742943 inconsistent with file '/oracle/PRD/sapdata1/cntrl/cntrlPRD.dbf'
version 9742931

解决ctl不一致问题之后继续mount库恢复

sapprd2:oraprd 11> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Mon Aug 3 18:42:53 2026

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> alter database mount;

Database altered.

SQL> alter database datafile 62,84 offline;

Database altered.

SQL> recover database ;
Media recovery complete.
SQL> alter database open;

Database altered.

2.对于两个损坏的数据文件头进行修复
由于现在aix环境的客户比较少,obet没有在aix环境下进行编译,因此直接使用bbed工具进行文件头修复（列举了主要操作过程）

BBED> copy file 83 block 1 to file 84 block 1
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:    0 to   31           Dba:0x15000001
------------------------------------------------------------------------
 0ba20000 14c00001 00000000 00000104 22250000 00000000 0b200000 72bc991d

 <32 bytes per line>

BBED> set offset 368
        OFFSET          368

BBED> d
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:  368 to  399           Dba:0x15000001
------------------------------------------------------------------------
 00000053 00000000 00000000 495dc097 00000000 00000000 00000000 00000000

 <32 bytes per line>

BBED> m /x 00000054
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:  368 to  399           Dba:0x15000001
------------------------------------------------------------------------
 00000054 00000000 00000000 495dc097 00000000 00000000 00000000 00000000

 <32 bytes per line>

BBED> set offset 52
        OFFSET          52

BBED> d
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:   52 to   83           Dba:0x15000001
------------------------------------------------------------------------
 00530003 00000000 00000000 00000000 00000000 00000000 00000000 00000000

 <32 bytes per line>

BBED> m /x 0054
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:   52 to   83           Dba:0x15000001
------------------------------------------------------------------------
 00540003 00000000 00000000 00000000 00000000 00000000 00000000 00000000

 <32 bytes per line>

BBED> set offset 4
        OFFSET          4

BBED> d
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:    4 to   35           Dba:0x15000001
------------------------------------------------------------------------
 14c00001 00000000 00000104 22250000 00000000 0b200000 72bc991d 50524400

 <32 bytes per line>

BBED> m /x 15000001
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:    4 to   35           Dba:0x15000001
------------------------------------------------------------------------
 15000001 00000000 00000104 22250000 00000000 0b200000 72bc991d 50524400

 <32 bytes per line>


BBED> m /x E1B0
Warning: contents of previous BIFILE will be lost. Proceed? (Y/N) y
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:  100 to  611           Dba:0x15000001
------------------------------------------------------------------------
 e1b0853f 00040000 4083fcfe 33606b63 02502452 0000ca28 49680fb3 93b9f6b9

 <32 bytes per line>

BBED> set offset +2
        OFFSET          102

BBED> m /x 8542
 File: /oracle/PRD/sapdata3/sr3_76/sr3.data76 (84)
 Block: 1                Offsets:  102 to  613           Dba:0x15000001
------------------------------------------------------------------------
 85420004 00004083 fcfe3360 6b630250 24520000 ca284968 0fb393b9 f6b90005

 <32 bytes per line>


BBED> sum
Check value for File 84, Block 1:
current = 0x2225, required = 0x23e5

BBED> sum apply
Check value for File 84, Block 1:
current = 0x23e5, required = 0x23e5

BBED> verify
DBVERIFY - Verification starting
FILE = /oracle/PRD/sapdata3/sr3_76/sr3.data76
BLOCK = 1


DBVERIFY - Verification complete

Total Blocks Examined         : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing   (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing   (Index): 0
Total Blocks Empty            : 0
Total Blocks Marked Corrupt   : 0
Total Blocks Influx           : 0
Message 531 not found;  product=RDBMS; facility=BBED

修改完成之后,还出现过几个错误

SQL> alter database open ;
alter database open
*
ERROR at line 1:
ORA-01122: database file 62 failed verification check
ORA-01110: data file 62: '/oracle/PRD/sapdata2/sr3_54/sr3.data54'
ORA-01200: actual file size of 1279999 is smaller than correct size of 1280000

ORA-01200是由于数据文件比文件头记录信息小一个block,通过补上这个block解决

Read of datafile '/oracle/PRD/sapdata2/sr3_54/sr3.data54' (fno 62) header failed with ORA-01202
Rereading datafile 62 header failed with ORA-01202
Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_11272278.trc:
ORA-01122: database file 62 failed verification check
ORA-01110: data file 62: '/oracle/PRD/sapdata2/sr3_54/sr3.data54'
ORA-01202: wrong incarnation of this file - wrong creation time

ORA-01202是由于create time没有修改正确导致,重新修改解决

Rereading datafile 84 header failed with ORA-01203
Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_17105252.trc:
ORA-01122: database file 84 failed verification check
ORA-01110: data file 84: '/oracle/PRD/sapdata3/sr3_76/sr3.data76'
ORA-01203: wrong incarnation of this file - wrong creation SCN

ORA-01203是由于create scn没有修改正确导致,重新修改解决

Errors in file /oracle/PRD/saptrace/diag/rdbms/prd/PRD/trace/PRD_ora_17105278.trc:
ORA-01122: database file 62 failed verification check
ORA-01110: data file 62: '/oracle/PRD/sapdata2/sr3_54/sr3.data54'
ORA-01207: file is more recent than control file - old control file
ORA-1122 signalled during: alter database open .

ORA-01207是由于数据文件的ckp信息比控制文件的新,重建ctl解决,解决这些问题之后,顺利打开数据库

SQL> startup mount;
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
ORACLE instance started.

Total System Global Area 3.7548E+10 bytes
Fixed Size                  2254136 bytes
Variable Size            1.9193E+10 bytes
Database Buffers         1.8254E+10 bytes
Redo Buffers               98996224 bytes
Database mounted.
SQL> alter database open;

Database altered.

SQL> select status,count(1) from v$datafile_header group by status;

STATUS    COUNT(1)
------- ----------
ONLINE          84

然后跳过坏块,导出数据,对于无法导出的异常表进行特殊处理,完成本次恢复工作,最终恢复结果总结

obet一键恢复offline数据文件

惜分飞 — Tue, 31 Mar 2026 16:09:17 +0000

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：obet一键恢复offline数据文件

客户有一个数据库由于异常断电之后无法正常启动,自行尝试恢复之后但是没有open成功,让可以通过Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)脚本收集信息进行评估,发现两个问题:
1. 根据查询信息确认users01.dbf（file# 4）文件处于offline状态,而且checkpoint scn明显小于其他文件

2. 通过分析alert日志确认客户在尝试offline file 4 之后open数据库报ORA-600 4194错误,数据库没有open成功

Mon Mar 30 06:26:30 2026
Starting ORACLE instance (normal)
Mon Mar 30 06:27:00 2026
ALTER DATABASE DATAFILE 4 OFFLINE DROP
Completed: ALTER DATABASE DATAFILE 4 OFFLINE DROP
Mon Mar 30 06:27:26 2026
ALTER DATABASE OPEN
Beginning crash recovery of 1 threads
 parallel recovery started with 7 processes
Started redo scan
Completed redo scan
 read 88 KB redo, 103 data blocks need recovery
Started redo application at
 Thread 1: logseq 3, block 3
Recovery of Online Redo Log: Thread 1 Group 3 Seq 3 Reading mem 0
  Mem# 0: /home/oracle/app/oracle/oradata/orcl/redo03.log
Completed redo application of 0.07MB
Completed crash recovery at
 Thread 1: logseq 3, block 180, scn 415466134
 103 data blocks read, 103 data blocks written, 88 redo k-bytes read
Mon Mar 30 06:27:28 2026
Thread 1 advanced to log sequence 4 (thread open)
Thread 1 opened at log sequence 4
  Current log# 1 seq# 4 mem# 0: /home/oracle/app/oracle/oradata/orcl/redo01.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Mar 30 06:27:28 2026
SMON: enabling cache recovery
Successfully onlined Undo Tablespace 2.
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is ZHS16GBK
No Resource Manager plan active
Errors in file /home/oracle/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_17628.trc(incident=186839):
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /home/oracle/app/oracle/diag/rdbms/orcl/orcl/orcl_smon_17628_i186839.trc
Exception [type: SIGBUS, Non-existent physical address] [ADDR:0x6C0C4B62] [PC:0x2297750, kgegpa()+40]
Exception [type: SIGBUS, Non-existent physical address] [ADDR:0x6C0C4B62] [PC:0x229597B, kgebse()+279]
Mon Mar 30 06:27:28 2026
PMON (ospid: 17604): terminating the instance due to error 397
Errors in file /home/oracle/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_smon_17628.trc:
ORA-00328: archived log ends at change 415466135, need later change 415466136
ORA-00334: archived log: '/home/oracle/app/oracle/oradata/orcl/redo03.log'
ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], [], [], [], [], []
Instance terminated by PMON, pid = 17604

接手这个故障之后,由于数据库是非归档模式,而且已经被屏蔽一致性强制打开过(open过程没成功),redo已经被clear过,因此基于这样的情况,直接上obet工具(Oracle Block Editor Tool修改file# 4的文件头状态
obet修复之前文件状态

STATUS	CHECKPOINT_TIME 			 FUZ CHECKPOINT_CHANGE# 	 ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
OFFLINE 2026-03-30 06:24:41			 YES	      415446014 	       1
ONLINE	2026-03-30 06:28:26			 YES	      415486184 	       7

obet修改文件头操作

OBET> open listfile.txt
Loaded 8 files from  datafile list 'listfile.txt'.

OBET> info

Loaded files (2 total):
----------------------------------------
Number  Path
----------------------------------------
     1  /home/oracle/app/oracle/oradata/orcl/system01.dbf
     4  /opt/oradata/orcl/users01.dbf
----------------------------------------

OBET> set mode edit
mode set to: edit


OBET> set file 4
filename set to: /opt/oradata/orcl/users01.dbf (file#4)

OBET> backup block 1
Created backup directory: backup_blk
Successfully backed up block 1 from current file to /tmp/backup_blk/users01.dbf_1.20260331092333


OBET> copy chkscn file 1 to file 4

Confirm Modify chkscn:
Source: file#1 (/home/oracle/app/oracle/oradata/orcl/system01.dbf)
Target: file#4 (/opt/oradata/orcl/users01.dbf)
Proceed? (Y/YES to confirm): y
Successfully copied checkpoint SCN information from file#1 to file#4.

OBET> exit
Exiting OBET.

再次查询文件头scn信息

STATUS	CHECKPOINT_TIME 			 FUZ CHECKPOINT_CHANGE# 	 ROW_NUM
------- ---------------------------------------- --- ------------------ ----------------
OFFLINE 2026-03-30 06:28:26			 NO	      415486184 	       1
ONLINE	2026-03-30 06:28:26			 YES	      415486184 	       7

尝试online文件,并open库成功

SQL> recover datafile 4;
Media recovery complete.
SQL> alter database datafile 4 online;

Database altered.

SQL> alter database open;

Database altered.

然后expdp导出数据成功,基本完成本次数据恢复工作

obet快速修改scn/resetlogs恢复数据库(缺少归档,ORA-00308）

惜分飞 — Thu, 08 Jan 2026 13:15:40 +0000

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：obet快速修改scn/resetlogs恢复数据库(缺少归档,ORA-00308）

最初是由于业务系统报ORA-01578导致

然后相关人员上来进行了一系列操作,包括但不限于offline文件,create datafile 44,recover datafile,还尝试做resetlogs,使用备份控制文件还原等等.我接手该故障的时候状态

44号文件状态是12月1日的,而且resetlogs信息也不对.通过和客户沟通,确认是他们在没有备份44号文件的前提下直接执行了类似alter database create datafile 44的命令,但是在应用了写归档之后,发现提示有归档不存在

ALTER DATABASE RECOVER    CONTINUE DEFAULT
Media Recovery Log /data/app/oracle/archive/orcl/1_181285_1103629544.dbf
ORA-279 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
ALTER DATABASE RECOVER    CONTINUE DEFAULT
Media Recovery Log /data/app/oracle/archive/orcl/1_181286_1103629544.dbf
Tue Jan 06 12:48:12 2026
Errors with log /data/app/oracle/archive/orcl/1_181286_1103629544.dbf
Errors in file /data/app/oracle/diag/rdbms/orcl8/orcl/trace/orcl_pr00_8338.trc:
ORA-00308: cannot open archived log '/data/app/oracle/archive/orcl/1_181286_1103629544.dbf'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

分析现存归档,确实发现12月1日之后丢失了部分归档日志(归档不连续)

基于这种情况(故障数据文件直接被覆盖,归档又出现了多天的连续断档,而且没有有效备份),只能先打开数据库,然后根据情况导出数据,然后导入到新库中.这里我直接使用obet来修改resetlogs和checkpoint 信息,实现数据库快速打开

OBET> help
OBET (Oracle Block Editor Tool) commands:
  open     - Load file list from config file (format:  )
  info                  - Show loaded file list (from open command)
  set filename    - Set target file path (required)
  set file         - Set filename using loaded file number (from open list)
  set blocksize   - Set block size (2048,4096,[8192],16384,32768)
  set block        - Set block number (starts from 0, default: 1)
  set offset    - Set offset within block (< blocksize, default: 0)
  set count      - Set number of bytes to read (default: 32)
  set mode edit/browse  - Enable edit/browse mode
  d/dump [options]      - Display data (options: block X, offset Y, count N)
  m/modify  [opts] - Modify data with hex (opts: block X, offset Y)
  undo                  - Undo last modification
  sum [block X]         - Calculate checksum for block (default: current block)
  sum apply [block X]   - Apply checksum: write calculated value to block
  tailchk [block X]     - Calculate tailchk for block (default: current block)
  tailchk apply [block X] - Apply tailchk: write calculated value to block
  repair [block X]      - Repair bad block (fix seq_kcbh, tailchk, checksum)
  copy data  to   - Copy data between files
     format: file,block,offset,count (e.g., 1,1,10,64)
     format: file[,block][,offset] (e.g., 3 or 3,1 or 3,1,128)
  copy block file#,block# to file#,block#     - Copy entire data block
  copy chkscn file n to file m         - Copy datafile header checkpoint SCN info
  copy resetlogscn file n to file m    - Copy datafile header resetlogs info
  backup [file n] [block m] - Backup data block (optional file and block parameters)
  corrupt [block X]     - Mark block as corrupted (default: current block)
  show                  - Display current settings (filename, blocksize, block, offset, count, mode)
  license               - Show/manage software license (registration code required)
  version               - Show software version and developer information
  p/print        - Print Oracle structure,Use the 'p/print' command to see details
  undo                  - Undo the last copy chkscn or copy resetlogscn operation
  spool           - Start logging to file (spool off to stop)
  help                  - Show this help message
  quit/exit             - Exit OBET

OBET> open 1.txt                 
Loaded 2 files from config file '1.txt'.

OBET> info

Loaded files (2 total):
----------------------------------------
Number  Path
----------------------------------------
     1  /data/app/oracle/oradata/orcl/system01.dbf
    44  /data/app/oracle/oradata/orcl/xff26.dbf
----------------------------------------

OBET> set file 44
filename set to: /data/app/oracle/oradata/orcl/xff26.dbf (file#44)

OBET> d

File: /data/app/oracle/oradata/orcl/xff26.dbf
Block: 1                Offsets:     0 to    31
--------------------------------------------------------------------------------
0xifenfei000 0BA20000 0100000B 00000000 00000104 B05F0000 00000000 0004200B 65583D61 

<32 bytes read>


OBET> backup file 44 block 1
Created backup directory: backup_blk
Successfully backed up block 1 from file#44 to /tmp/backup_blk/xff26.dbf_1.20260106182405

OBET> copy resetlogscn file 1 to file 44
Error: Edit mode not enabled. Use 'set mode edit' first.

OBET> set mode edit     
mode set to: edit

OBET> copy resetlogscn file 1 to file 44

Confirm Modify resetlogscn:
Source: file#1 (/data/app/oracle/oradata/orcl/system01.dbf)
Target: file#44 (/data/app/oracle/oradata/orcl/xff26.dbf)
Proceed? (Y/YES to confirm): y
Successfully copied resetlog SCN information from file#1 to file#44.

OBET> copy chkscn file 1 to file 44

Confirm Modify chkscn:
Source: file#1 (/data/app/oracle/oradata/orcl/system01.dbf)
Target: file#44 (/data/app/oracle/oradata/orcl/xff26.dbf)
Proceed? (Y/YES to confirm): y
Successfully copied checkpoint SCN information from file#1 to file#44.

OBET> tailchk
Check tailchk for File /data/app/oracle/oradata/orcl/xff26.dbf, Block 1:
current = 0x010B0000, required = 0x010B0000

OBET> sum
Check value for File /data/app/oracle/oradata/orcl/xff26.dbf, Block 1:
current = 0xB05F, required = 0x6DA3

OBET> sum apply

Confirm applying checksum:
File: /data/app/oracle/oradata/orcl/xff26.dbf
Block: 1
Offset in block: 16 (file offset: 0x0xifenfei010)
Original value: 0xB05F
New value:      0x6DA3
Confirm? (Y/YES to proceed): y
Verification successful: Stored checksum matches calculated value (0x6DA3).
Checksum applied successfully.

然后重建ctl正常打开数据库

[oracle@xifenfei check_db]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Tue Jan 6 18:29:22 2026

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup mount pfile='/tmp/pfile';
ORACLE instance started.

Total System Global Area 6.8413E+10 bytes
Fixed Size		    2270360 bytes
Variable Size		 3.3554E+10 bytes
Database Buffers	 3.4762E+10 bytes
Redo Buffers		   93585408 bytes
Database mounted.
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01190: control file or data file 1 is from before the last RESETLOGS
ORA-01110: data file 1: '/data/app/oracle/oradata/orcl/system01.dbf'


SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.
SQL> 
SQL> 
SQL> startup nomount pfile='/tmp/pfile
';
ORACLE instance started.

Total System Global Area 6.8413E+10 bytes
Fixed Size		    2270360 bytes
Variable Size		 3.3554E+10 bytes
Database Buffers	 3.4762E+10 bytes
Redo Buffers		   93585408 bytes
SQL> @ctl.sql

Control file created.

SQL> SQL> recover database;
ORA-00283: recovery session canceled due to errors
ORA-00264: no recovery required


SQL> alter database open;

Database altered.

后续根据情况选择导出表,对于导出报错的表再进行二次处理.
obet工具下载地址:obet下载
obet工具使用说明:obet使用说明