标签云
asm恢复 bbed bootstrap$ dul kcbzib_kcrsds_1 kccpb_sanity_check_2 kcratr_nab_less_than_odr kgegpa MySQL恢复 ORA-00312 ORA-00704 ORA-00742 ORA-01110 ORA-01200 ORA-01555 ORA-01578 ORA-01595 ORA-600 2662 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-600 kdsgrp1 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (110)
- 数据库 (1,821)
- DB2 (22)
- MySQL (80)
- Oracle (1,651)
- Data Guard (53)
- EXADATA (8)
- GoldenGate (24)
- ORA-xxxxx (168)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (15)
- ORACLE 21C (3)
- Oracle 23ai (8)
- Oracle ASM (69)
- Oracle Bug (8)
- Oracle RAC (54)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (29)
- Oracle备份恢复 (620)
- Oracle安装升级 (102)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (86)
- PostgreSQL (36)
- pdu工具 (7)
- PostgreSQL恢复 (13)
- SQL Server (34)
- SQL Server恢复 (14)
- TimesTen (7)
- 达梦数据库 (3)
- 达梦恢复 (1)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (45)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (28)
-
最近发表
- Patch_SCN快速解决ORA-600 2663故障
- 在生产环境错误执行dd命令破坏asm磁盘故障恢复
- obet实现对数据文件坏块检测功能
- oracle linux 8.10注意pmlogger导致空间被大量占用
- obet快速修改scn/resetlogs恢复数据库(缺少归档,ORA-00308)
- 使用DBMS_PDB.RECOVER抢救单个pdb
- aix环境写入大文件设置combehin提高效率
- 记录一次国产数据库被rm -rf /*删除的救援过程
- 数据库启动报 maximum number of processes () exceeded分析
- ORA-600 [ksunfy : too few sessions]
- 由于数据块scn大于数据库scn导致ORA-600 kcbzib_kcrsds_1错误
- ORA-600 ktbair2: illegal inheritance恢复
- 一键恢复ORA-00704 ORA-00702故障—202512
- PostgreSQL查询一个表相关的所有oid
- PostgreSQL oid文件替换实现数据访问
- 模拟sql server故障备份完成恢复实现数据0丢失
- sql server 事务日志备份异常恢复案例
- win平台挂起Oracle数据库启动进程
- linux异常磁盘lvm恢复操作演示
- open数据库报ora-600 kdsgrp1故障处理
分类目录归档:Oracle
Patch_SCN快速解决ORA-600 2663故障
一个维保客户和我说他们测试库删除了日志文件导致库无法启动,让我帮忙看看
客户现场现况
1. 磁盘空间使用100%
[oracle@We1-db_Test ~]$ df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 3.9G 880K 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/vda1 40G 38G 92M 100% / tmpfs 783M 0 783M 0% /run/user/0
2. 数据库redo被删除了部分,而且是active状态的被删除
[oracle@We1-db_Test ~]$ ls -l /opt/app/oracle/oradata/orcl/redo0*
-rw-r----- 1 oracle oinstall 52429312 Jan 15 15:29 /opt/app/oracle/oradata/orcl/redo04.log
-rw-r----- 1 oracle oinstall 52429312 Jan 15 16:26 /opt/app/oracle/oradata/orcl/redo05.log
SQL> select group#,SEQUENCE#,STATUS FROM V$lOG;
GROUP# SEQUENCE# STATUS
---------- ---------- ----------------
1 8989 CURRENT
2 0 UNUSED
5 0 UNUSED
4 0 UNUSED
3 8988 ACTIVE
SQL> select member from v$logfile;
MEMBER
-----------------------------------------------------
/opt/app/oracle/oradata/orcl/redo03.log
/opt/app/oracle/oradata/orcl/redo02.log
/opt/app/oracle/oradata/orcl/redo01.log
/opt/app/oracle/oradata/orcl/redo04.log
/opt/app/oracle/oradata/orcl/redo05.log
基于当前情况,直接open库无望,但是空间不足问题需要先解决,不然恢复过程中创建redo空间不足依旧会报错卡死,所以先清理了监听和alert等日志,系统空闲了3G多空间,可以进行恢复操作
[oracle@We1-db_Test trace]$ df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 3.9G 880K 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/vda1 40G 34G 3.9G 90% / tmpfs 783M 0 783M 0% /run/user/0
恢复数据库
1. 由于active redo丢失,毫无疑问,直接强制拉库,使用_allow_resetlogs_corruption参数开干
SQL> startup mount pfile='/tmp/pfile';
ORACLE instance started.
Total System Global Area 2455228416 bytes
Fixed Size 2255712 bytes
Variable Size 905970848 bytes
Database Buffers 1526726656 bytes
Redo Buffers 20275200 bytes
Database mounted.
SQL> recover database until cancel;
ORA-00283: recovery session canceled due to errors
ORA-01610: recovery using the BACKUP CONTROLFILE option must be done
SQL> recover database using backup controlfile;
ORA-00279: change 311982775 generated at 12/31/2025 17:35:11 needed for thread
1
ORA-00289: suggestion :
/opt/app/oracle/fast_recovery_area/ORCL/archivelog/2026_01_16/o1_mf_1_8988_%u_.a
rc
ORA-00280: change 311982775 for thread 1 is in sequence #8988
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
cancel
Media recovery cancelled.
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [2663], [0], [311982792], [0],
[311982833], [], [], [], [], [], [], []
Process ID: 11917
Session ID: 576 Serial number: 3
alert日志报错
Fri Jan 16 21:25:31 2026 Assigning activation ID 1750515127 (0x6856bdb7) Thread 1 opened at log sequence 1 Current log# 1 seq# 1 mem# 0: /opt/app/oracle/oradata/orcl/redo01.log Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Fri Jan 16 21:25:31 2026 SMON: enabling cache recovery Errors in file /opt/app/oracle/diag/rdbms/orcl/we1db/trace/we1db_ora_11917.trc (incident=81753): ORA-00600: internal error code, arguments: [2663], [0], [311982792], [0], [311982833], [], [] Incident details in: /opt/app/oracle/diag/rdbms/orcl/we1db/incident/incdir_81753/we1db_ora_11917_i81753.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /opt/app/oracle/diag/rdbms/orcl/we1db/trace/we1db_ora_11917.trc: ORA-00600: internal error code, arguments: [2663], [0], [311982792], [0], [311982833], [], [] Errors in file /opt/app/oracle/diag/rdbms/orcl/we1db/trace/we1db_ora_11917.trc: ORA-00600: internal error code, arguments: [2663], [0], [311982792], [0], [311982833], [], [] Error 600 happened during db open, shutting down database USER (ospid: 11917): terminating the instance due to error 600 Instance terminated by USER, pid = 11917 ORA-1092 signalled during: alter database open resetlogs... opiodr aborting process unknown ospid (11917) as a result of ORA-1092 Fri Jan 16 21:25:33 2026 ORA-1092 : opitsk aborting process
不幸数据库遇到ORA-600 2663错误,这个故障在以前的文章中描述过,基本上和ORA-600 2662的处理思路类似,这里直接使用:Patch_SCN for Linux进行恢复
2. 使用Patch_SCN处理数据库SCN
SQL> startup nomount pfile='/tmp/pfile'; ORACLE instance started. Total System Global Area 2455228416 bytes Fixed Size 2255712 bytes Variable Size 905970848 bytes Database Buffers 1526726656 bytes Redo Buffers 20275200 bytes SQL>@rectl Control file created. SQL> recover database; Media recovery complete.
SQL> alter database open;
Database altered.
SQL> SELECT CURRENT_SCN FROM V$DATABASE;
CURRENT_SCN
----------------
322002903
到这里完成数据库open操作,后续逻辑导出完成恢复任务
在生产环境错误执行dd命令破坏asm磁盘故障恢复
由于ssh登录错误,客户对生产环境进行了误操作把系统的一块磁盘dd到另外两个磁盘上,由于及时发现立马进行了终止操作,但是还是分别破坏了一点数据(一块盘破坏了2G多,另外一块盘破坏了1G多)

通过分析udev的绑定关系确认被破坏的asm disk名称

再通过asm alert日志确认破坏磁盘在asm disk中情况

通过上述信息基本上可以确认,asmdisk13被分别dd到了asmdisk11和asmdisk26中了部分


基于这种情况,由于asm disk被破坏了1-2G多,直接修复然后正常mount磁盘组基本上没有希望,经过分析以及与客户沟通,确认他们改系统是4节点组成的集群,1/2节点上面跑2套库,3/4节点上跑2套库,数据整体放在data_dg磁盘组中,需要恢复的库是第二个顺序创建的1套库(4套库中只需恢复一套即可),由于破坏的数据本身不多,而且需要恢复的数据不是最初写入asm磁盘组,基于这样的情况,需要恢复的数据库机会比较大.
由于现在三个磁盘头信息一致(一个磁盘被dd到另外两个磁盘上),因此第一步先把损坏的两个磁盘头进行简单修复,为了便于amdu(找回ASM中数据文件)等之类数据可以识别到正确的磁盘头信息,然后进行后续的数据文件提取恢复.使用工具对数据文件进行了批量提取,提取数据完成之后,尝试recover和open库

虽然数据库正常打开了,不过很不幸,后台还是有一些坏块报错,通过dbv检查发现有文件有一部分坏块,类似dbv报错信息

通过分析该文件在磁盘组中各个磁盘的分布情况

确认该文件确实有部分block分布在被dd磁盘的破坏的范围内这个部分的数据丢失无法挽回,只能是定位到具体对象然后由业务想办法处理.相对以往的各种dd破坏的案例恢复而言,这个应该是效果比较好的一个,而且也是恢复比较容易的一个,没有使用到asm disk 基于asm au/oracle block 扫描的级别,而且system表空间没有任何损坏,数据库甚至直接open成功了,以往的一些dd案例列举:
asm磁盘dd破坏恢复
dd破坏asm磁盘头恢复
asm disk 磁盘部分被清空恢复
obet实现对数据文件坏块检测功能
通过一段时间的测试和使用,obet修复了不少bug,关于obet的以往功能和特性的文章:
Oracle数据块编辑工具( Oracle Block Editor Tool)-obet
obet(Oracle Block Editor Tool)第二版发布
并且也在客户的生产环境上进行了实战:obet快速修改scn/resetlogs恢复数据库(缺少归档,ORA-00308).利用周末的时间又对obet的工具进行了功能增强,增加了dbv(数据块校验)功能.
Oracle dbv的不足
1. oracle dbv需要在安装oracle服务端的环境下才能执行
2. oracle dbv对于文件大小不正确(文件头记录block数和实际文件大小不匹配),文件头损坏等情况都可能导致dbv无法执行,类似下面的报错
C:\Users\XFF>dbv file=H:\BaiduNetdisk\kingdee\system01.dbf DBVERIFY: Release 11.2.0.4.0 - Production on 星期日 1月 11 17:30:29 2026 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. DBV-00107: 未知标头格式 (0) (2054913149) C:\Users\XFF>
C:\Users\XFF>dbv file=H:\BaiduNetdisk\kingdee\users01.dbf DBVERIFY: Release 11.2.0.4.0 - Production on 星期日 1月 11 20:10:05 2026 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. DBV-00102: FILE (H:\BAIDUNETDISK\KINGDEE\USERS01.DBF) 在 end read 操作 (-1) 期间出现文件 I/O 错误 C:\Users\XFF>
3. oracle dbv一条命令执行检测一个数据文件,如果数据文件多,检查起来很繁琐
4. oracle dbv么有检查进度,对于io性能较慢,数据文件较大的情况,无法跟踪检查进度.
obet的dbv功能使用
1. 配置listfile.txt文件,格式为file# name
1 H:\BaiduNetdisk\kingdee\system01.dbf 5 H:\BaiduNetdisk\kingdee\EAS_D_EAS_STANDARD.ORA
2. 启动obet,执行open listfile.txt(如果不在obet目录提供完整路径)
OBET> open listfile.txt
Loaded 2 files from config file 'listfile.txt'.
OBET> info
Loaded files (2 total):
----------------------------------------
Number Path
----------------------------------------
1 H:\BaiduNetdisk\kingdee\system01.dbf
5 H:\BaiduNetdisk\kingdee\EAS_D_EAS_STANDARD.ORA
----------------------------------------
3. 执行dbv命令(logfile
OBET> dbv =============================================== DBV (Data Block Verification) Block Size: 8192 bytes =============================================== Verifying file #1: H:\BaiduNetdisk\kingdee\system01.dbf (131841 blocks) - Started: 2026-01-11 18:03:58 file 1, block 0: checksum error (expected 0xC4DA, got 0xC478), bad block file 1, block 1: checksum error (expected 0xF835, got 0xB835), bad block Progress: 10000 / 131841 blocks checked... Progress: 20000 / 131841 blocks checked... ………………. Progress: 120000 / 131841 blocks checked... Progress: 130000 / 131841 blocks checked... File #1 completed: 0 all zero, 0 soft corrupted, 0 tailchk error, 2 checksum error, 0 rdba error Verifying file #5: H:\BaiduNetdisk\kingdee\EAS_D_EAS_STANDARD.ORA (4194303 blocks) - Started: 2026-01-11 18:04:08 Progress: 10000 / 4194303 blocks checked... Progress: 20000 / 4194303 blocks checked... …………………… Progress: 260000 / 4194303 blocks checked... Progress: 270000 / 4194303 blocks checked... file 5, block 277678: tailchk error (expected 0x0228BB21, got 0x0228AD21), bad block file 5, block 277679: rdba error (expected 277679, got 277615), bad block file 5, block 277680: rdba error (expected 277680, got 277616), bad block file 5, block 277681: rdba error (expected 277681, got 277617), bad block ……………… file 5, block 277692: rdba error (expected 277692, got 277628), bad block file 5, block 277693: rdba error (expected 277693, got 277629), bad block file 5, block 277694: rdba error (expected 277694, got 277630), bad block file 5, block 279406: tailchk error (expected 0x02281E22, got 0x00000700), bad block file 5, block 279407: rdba error (expected 279407, got 448), bad block file 5, block 279408: rdba error (expected 279408, got 0), bad block ……………… Progress: 280000 / 4194303 blocks checked... Progress: 290000 / 4194303 blocks checked... Progress: 300000 / 4194303 blocks checked... Progress: 310000 / 4194303 blocks checked... file 5, block 312932: tailchk error (expected 0x0106C0B8, got 0x010629B8), bad block file 5, block 312933: rdba error (expected 312933, got 312869), bad block file 5, block 312934: rdba error (expected 312934, got 312870), bad block file 5, block 312935: rdba error (expected 312935, got 312871), bad block file 5, block 312936: rdba error (expected 312936, got 312872), bad block file 5, block 312937: rdba error (expected 312937, got 312873), bad block file 5, block 312938: rdba error (expected 312938, got 312874), bad block file 5, block 312939: rdba error (expected 312939, got 312875), bad block ……………… Progress: 4180000 / 4194303 blocks checked... Progress: 4190000 / 4194303 blocks checked... File #5 completed: 1 all zero, 0 soft corrupted, 13 tailchk error, 1 checksum error, 255 rdba error DBV completed at: 2026-01-11 18:09:30 =============================================== DBV Summary: Total blocks checked: 4325872 Total all zero blocks found: 1 Total all rdba error blocks found: 255 Total all tailchk error blocks found: 13 Total all soft corrupted blocks found: 0 Total all checksum error blocks found: 3 Total bad blocks found: 272 =============================================== Detailed report saved to: dbv_20260111180358.log OBET>



加我微信(17813235971)
加我QQ(107644445)

