标签云
asm恢复 bbed bootstrap$ dul kcbzib_kcrsds_1 kccpb_sanity_check_2 kcratr_nab_less_than_odr MySQL恢复 obet ORA-00312 ORA-00704 ORA-00742 ORA-01110 ORA-01200 ORA-01555 ORA-01578 ORA-01595 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-600 kcratr_nab_less_than_odr ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 ORACLE恢复 Oracle 恢复 ORACLE数据库恢复 oracle碎片 OSD-04016 YOUR FILES ARE ENCRYPTED 比特币加密文章分类
- MongoDB (1)
- MongoDB恢复 (1)
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (112)
- 数据库 (1,880)
- DB2 (22)
- MySQL (82)
- Oracle (1,705)
- Data Guard (53)
- EXADATA (8)
- GoldenGate (24)
- ORA-xxxxx (168)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (15)
- ORACLE 21C (3)
- Oracle 23ai (8)
- Oracle ASM (72)
- Oracle Bug (8)
- Oracle RAC (56)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (29)
- Oracle备份恢复 (659)
- Oracle安装升级 (108)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (91)
- PostgreSQL (37)
- pdu工具 (7)
- PostgreSQL恢复 (13)
- SQL Server (34)
- SQL Server恢复 (14)
- TimesTen (7)
- 达梦数据库 (5)
- 达梦恢复 (3)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (51)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (34)
- MongoDB (1)
-
最近发表
- Firebird 损坏数据库数据抢救工具—firebirdext
- Oracle 19c 202610补丁(RUs+OJVM)-19.33
- 由于con$.con#值达到极限,触发数据库增加约束报ORA-600 12807错误
- MongoDB 离线提取工具—mongoext
- .bres勒索病毒加密数据库dmp文件可恢复
- ORA-600 krsm_mrp_rebuild_lf_list.lno_mismatch
- 记录一次0丢失的ORA-00354: 损坏重做日志块标头故障恢复
- 不当数据库恢复操作导致一个月数据丢失
- obet快速修复oracle 位图损坏块
- 不太常见的10.2.0.1的oracle redo损坏恢复
- obet forcecopy功能抢救硬件故障中的数据文件
- kcratr_nab_less_than_odr和system坏块故障处理
- 通过obet 恢复system坏块,打开数据库
- obet dbv功能完整说明
- 分享一例运行在aix上的sap系统数据库恢复过程
- OBET-Oracle Block Editor Tool使用说明
- 几乎动用了所有手段的Oracle故障恢复
- Oracle Block Edit Tool (obet) 功能增强–2026.07
- Oracle 19c 202607补丁(RUs+OJVM)-19.32
- Patch_SCN快速修复ORA-01555数据库open故障
分类目录归档:Oracle备份恢复
win asm disk header 异常恢复
有朋友反馈win环境下rac异常,asm无法正常mount,检查日志发现
Fri Jul 03 03:55:46 2020 Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7004.trc: ORA-15025: could not open disk "\\.\ORCLDISKDATA1" ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7004.trc: ORA-15025: could not open disk "\\.\ORCLDISKDATA1" ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 WARNING: failed to read mirror side 1 of virtual extent 0 logical extent 0 of file 267 in group [2.2254399778] from disk DATA_0000 allocation unit 3502 reason error; if possible, will try another mirror side Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7004.trc: ORA-15081: failed to submit an I/O operation to a disk Fri Jul 03 03:59:46 2020 Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7328.trc: ORA-15025: could not open disk "\\.\ORCLDISKDATA1" ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7328.trc: ORA-15025: could not open disk "\\.\ORCLDISKDATA1" ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 WARNING: failed to read mirror side 1 of virtual extent 0 logical extent 0 of file 267 in group [2.2254399778] from disk DATA_0000 allocation unit 3502 reason error; if possible, will try another mirror side Errors in file C:\APP\ADMINISTRATOR\diag\asm\+asm\+asm2\trace\+asm2_ora_7328.trc: ORA-15081: failed to submit an I/O operation to a disk
报错信息比较明显是由于无法找到\\.\ORCLDISKDATA1磁盘,因此异常,通过asmtool查看磁盘信息
C:\app\11.2.0\grid>asmtool -list
NTFS \Device\Harddisk0\Partition3 81920M
NTFS \Device\Harddisk0\Partition4 200000M
NTFS \Device\Harddisk0\Partition5 4293849M
\Device\Harddisk1\Partition2 4062M
\Device\Harddisk2\Partition2 2097022M
ORCLDISKFRA0 \Device\Harddisk3\Partition2 511870M
明显的发现ORCLDISKDATA1磁盘丢失,通过对磁盘dd到本地然后进行分析发现,asm disk header损坏
C:\Users\Administrator>kfed read F:\temp\disk3\1\disk2.dd kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 006B38C00 00000000 00000000 00000000 00000000 [................] Repeat 255 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0] C:\Users\Administrator>kfed read F:\temp\disk3\1\disk2.dd blkn=2 kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL kfbh.datfmt: 2 ; 0x003: 0x02 kfbh.block.blk: 2 ; 0x004: blk=2 kfbh.block.obj: 2147483648 ; 0x008: disk=0 kfbh.check: 2349305287 ; 0x00c: 0x8c078dc7 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdatb.aunum: 0 ; 0x000: 0x00000000 kfdatb.shrink: 448 ; 0x004: 0x01c0 kfdatb.ub2pad: 0 ; 0x006: 0x0000 kfdatb.auinfo[0].link.next: 8 ; 0x008: 0x0008 kfdatb.auinfo[0].link.prev: 8 ; 0x00a: 0x0008 kfdatb.auinfo[1].link.next: 12 ; 0x00c: 0x000c kfdatb.auinfo[1].link.prev: 12 ; 0x00e: 0x000c kfdatb.auinfo[2].link.next: 456 ; 0x010: 0x01c8 kfdatb.auinfo[2].link.prev: 456 ; 0x012: 0x01c8 kfdatb.auinfo[3].link.next: 488 ; 0x014: 0x01e8 kfdatb.auinfo[3].link.prev: 488 ; 0x016: 0x01e8 kfdatb.auinfo[4].link.next: 24 ; 0x018: 0x0018 kfdatb.auinfo[4].link.prev: 24 ; 0x01a: 0x0018 kfdatb.auinfo[5].link.next: 28 ; 0x01c: 0x001c kfdatb.auinfo[5].link.prev: 28 ; 0x01e: 0x001c kfdatb.auinfo[6].link.next: 552 ; 0x020: 0x0228 kfdatb.auinfo[6].link.prev: 3112 ; 0x022: 0x0c28 kfdatb.spare: 0 ; 0x024: 0x00000000 kfdate[0].discriminator: 1 ; 0x028: 0x00000001 kfdate[0].allo.lo: 0 ; 0x028: XNUM=0x0 kfdate[0].allo.hi: 8388608 ; 0x02c: V=1 I=0 H=0 FNUM=0x0 kfdate[1].discriminator: 1 ; 0x030: 0x00000001 kfdate[1].allo.lo: 0 ; 0x030: XNUM=0x0 kfdate[1].allo.hi: 8388608 ; 0x034: V=1 I=0 H=0 FNUM=0x0 kfdate[2].discriminator: 1 ; 0x038: 0x00000001 kfdate[2].allo.lo: 0 ; 0x038: XNUM=0x0 kfdate[2].allo.hi: 8388609 ; 0x03c: V=1 I=0 H=0 FNUM=0x1
fra磁盘虽然磁盘asm label信息存在,但是其他信息依旧损坏,但是也只是磁盘头信息损坏

通过现场分析,基本上可以确定是由于某种原因导致win asm 的磁盘的所有磁盘头都损坏(两个磁盘头被置空,另外一个磁盘头基本上损坏),基于原因未知
基于客户现场的情况,以及他们有前一天的rman备份,而且客户有保障现场(进一步故障原因分析)的需求,未在现场环境进行恢复,而是在不对现场环境做任何修改的情况下,直接恢复fra里面的redo和归档日志,进而结合备份异地实现数据库恢复,实现数据0丢失,又不破坏现场的效果

以前遇到过类似我其他操作系统平台中asm disk header异常的case:
asm磁盘分区丢失恢复
pvid=yes导致asm无法mount
asm磁盘头全部损坏数据0丢失恢复
分区无法识别导致asm diskgroup无法mount
asm disk误设置pvid导致asm diskgroup无法mount恢复
系统故障oracle数据库恢复
由于系统故障,导致操作系统进入,客户通过其他方式进入系统拷贝出来数据文件,redo,ctl等文件,安装版本相同的数据库,修改相关路径,启动数据库,但是启动报错,让我们给予技术支持.数据库open报ORA-600 2662错误
SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-00603: ORACLE server session terminated by fatal error ORA-00600: internal error code, arguments: [2662], [2], [2313731576], [2], [2313735660], [12583040], [], [], [], [], [], [] ORA-00600: internal error code, arguments: [2662], [2], [2313731575], [2], [2313735660], [12583040], [], [], [], [], [], [] ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [2662], [2], [2313731573], [2], [2313735660], [12583040], [], [], [], [], [], [] Process ID: 22446 Session ID: 577 Serial number: 1
alert日志报错
Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /home/oracle/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_22446.trc: ORA-00600: internal error code, arguments:[2662],[2],[2313731573],[2],[2313735660],[12583040],[],[],[],[],[],[] Errors in file /home/oracle/app/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_22446.trc: ORA-00600: internal error code, arguments:[2662],[2],[2313731573],[2],[2313735660],[12583040],[],[],[],[],[],[] Error 600 happened during db open, shutting down database USER (ospid: 22446): terminating the instance due to error 600
这个错误比较常见,特别是使用了_allow_resetlogs_corruption屏蔽一致性强制拉库的时候.解决该问题比较简单,修改数据库scn,然后open数据库成功,参考部分案例_allow_resetlogs_corruption
[oracle@localhost ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.3.0 Production on Sat Jun 20 08:31:55 2020 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount pfile='/tmp/pfile'; ORACLE instance started. Total System Global Area 4275781632 bytes Fixed Size 2235208 bytes Variable Size 2902459576 bytes Database Buffers 1325400064 bytes Redo Buffers 45686784 bytes Database mounted. SQL> alter database open; Database altered.
尝试导出数据
[oracle@localhost ~]$ tail -f nohup.out
. exporting foreign function library names for user XIFENFEI
. exporting PUBLIC type synonyms
. exporting private type synonyms
. exporting object type definitions for user XIFENFEI
About to export LIOVBJ2017's objects ...
. exporting database links
. exporting sequence numbers
. exporting cluster definitions
. about to export LIOVBJ2017's tables via Conventional Path ...
. . exporting table ABCD
1 rows exported
. . exporting table TB_D_RECORD
EXP-00008: ORACLE error 1578 encountered
ORA-01578: ORACLE data block corrupted (file # 1, block # 290344)
ORA-01110: data file 1: '/home/oracle/app/oracle/oradata/orcl/system01.dbf'
. . exporting table TB_DRIVER
EXP-00008: ORACLE error 1578 encountered
ORA-01578: ORACLE data block corrupted (file # 1, block # 290344)
ORA-01110: data file 1: '/home/oracle/app/oracle/oradata/orcl/system01.dbf'
. . exporting table TB_XFF
EXP-00008: ORACLE error 1578 encountered
ORA-01578: ORACLE data block corrupted (file # 1, block # 290344)
ORA-01110: data file 1: '/home/oracle/app/oracle/oradata/orcl/system01.dbf'
. . exporting table TB_XFF_TM_REL
EXP-00008: ORACLE error 1578 encountered
ORA-01578: ORACLE data block corrupted (file # 1, block # 290344)
ORA-01110: data file 1: '/home/oracle/app/oracle/oradata/orcl/system01.dbf'
. . exporting table TB_LOCATION
由于file # 1, block # 290344坏块导致数据无法导出,通过dbv检查数据文件
[oracle@localhost trace]$ dbv file=/home/oracle/app/oracle/oradata/orcl/system01.dbf DBVERIFY: Release 11.2.0.3.0 - Production on Sat Jun 20 08:43:49 2020 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. DBVERIFY - Verification starting : FILE = /home/oracle/app/oracle/oradata/orcl/system01.dbf Page 290344 is influx - most likely media corrupt Corrupt block relative dba: 0x00446e28 (file 1, block 290344) Fractured block found during dbv: Data in bad block: type: 6 format: 2 rdba: 0x00446e28 last change scn: 0x0002.89e2b718 seq: 0x1 flg: 0x06 spare1: 0x0 spare2: 0x0 spare3: 0x0 consistency value in tail: 0x61980601 check value in block header: 0xe118 computed block checksum: 0xd680 DBVERIFY - Verification complete Total Pages Examined : 298240 Total Pages Processed (Data) : 257035 Total Pages Failing (Data) : 0 Total Pages Processed (Index): 13457 Total Pages Failing (Index): 0 Total Pages Processed (Other): 3598 Total Pages Processed (Seg) : 1 Total Pages Failing (Seg) : 0 Total Pages Empty : 24149 Total Pages Marked Corrupt : 1 Total Pages Influx : 1 Total Pages Encrypted : 0 Highest block SCN : 3221247831 (2.3221247831)
确认只有一个坏块,尝试通过bbed进行坏块修复
BBED> set blocksize 8192
BLOCKSIZE 8192
BBED> set block 290344
BLOCK# 290344
BBED> map
File: /home/oracle/app/oracle/oradata/orcl/system01.dbf (0)
Block: 290344 Dba:0x00000000
------------------------------------------------------------
KTB Data Block (Index Leaf)
struct kcbh, 20 bytes @0
struct ktbbh, 72 bytes @20
struct kdxle, 32 bytes @92
sb2 kd_off[231] @124
ub1 freespace[3026] @586
ub1 rowdata[4508] @3612
ub4 tailchk @8188
BBED> verify
DBVERIFY - Verification starting
FILE = /home/oracle/app/oracle/oradata/orcl/system01.dbf
BLOCK = 290344
Block 290344 is corrupt
Corrupt block relative dba: 0x00446e28 (file 0, block 290344)
Fractured block found during verification
Data in bad block:
type: 6 format: 2 rdba: 0x00446e28
last change scn: 0x0002.89e2b718 seq: 0x1 flg: 0x06
spare1: 0x0 spare2: 0x0 spare3: 0x0
consistency value in tail: 0x61980601
check value in block header: 0xe118
computed block checksum: 0xd680
DBVERIFY - Verification complete
Total Blocks Examined : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing (Index): 0
Total Blocks Empty : 0
Total Blocks Marked Corrupt : 1
Total Blocks Influx : 2
Message 531 not found; product=RDBMS; facility=BBED
BBED> set mode edit
MODE Edit
BBED> p kcbh
struct kcbh, 20 bytes @0
ub1 type_kcbh @0 0x06
ub1 frmt_kcbh @1 0xa2
ub1 spare1_kcbh @2 0x00
ub1 spare2_kcbh @3 0x00
ub4 rdba_kcbh @4 0x00446e28
ub4 bas_kcbh @8 0x89e2b718
ub2 wrp_kcbh @12 0x0002
ub1 seq_kcbh @14 0x01
ub1 flg_kcbh @15 0x06 (KCBHFDLC, KCBHFCKV)
ub2 chkval_kcbh @16 0xe118
ub2 spare3_kcbh @18 0x0000
BBED> p tailchk
ub4 tailchk @8188 0x61980601
BBED> d /v
File: /home/oracle/app/oracle/oradata/orcl/system01.dbf (0)
Block: 290344 Offsets: 8188 to 8191 Dba:0x00000000
-------------------------------------------------------
01069861 l ...a
<16 bytes per line>
BBED> m /x 010618b7
File: /home/oracle/app/oracle/oradata/orcl/system01.dbf (0)
Block: 290344 Offsets: 8188 to 8191 Dba:0x00000000
------------------------------------------------------------------------
010618b7
<32 bytes per line>
BBED> sum apply
Check value for File 0, Block 290344:
current = 0xe118, required = 0xe118
BBED> verify
DBVERIFY - Verification starting
FILE = /home/oracle/app/oracle/oradata/orcl/system01.dbf
BLOCK = 290344
DBVERIFY - Verification complete
Total Blocks Examined : 1
Total Blocks Processed (Data) : 0
Total Blocks Failing (Data) : 0
Total Blocks Processed (Index): 1
Total Blocks Failing (Index): 0
Total Blocks Empty : 0
Total Blocks Marked Corrupt : 0
Total Blocks Influx : 0
Message 531 not found; product=RDBMS; facility=BBED
继续尝试导出数据,遭遇ORA-08103,参考相关文章:
模拟普通ORA-08103并解决
模拟极端ORA-08103并解决
数据库启动ORA-08103故障恢复
EXP-00056: ORACLE error 8103 encountered ORA-08103: object no longer exists
通过对其进行处理,恢复该记录之外的所有记录,客户创建新库导入数据,数据库恢复基本完成
asm磁盘dd破坏恢复
有客户和我们反馈,由于运维人员操作错误,对一个磁盘组的asm disk进行了dd操作,导致部分数据丢失(客户数据文件存放在两个磁盘组中,其中一个被dd掉[误以为只是存放归档,其实由于第一个磁盘组空间不足,把部分数据文件放进该磁盘组])

通过对asm 日志进行分析发现被dd的磁盘是一个磁盘组,以前恢复过类似的asm 磁盘被dd的案例(asm磁盘头全部损坏数据0丢失恢复,上次因为dd破坏较少,所以可以通过修复磁盘组直接恢复出来里面数据,但是这次被dd了50M,直接修复磁盘头恢复数据基本上不太可能.通过工具对其进行磁盘扫描,参考:asm disk header 彻底损坏恢复,对扫描结果进行分析,发现不少数据块是重复,无法较好的实现重组效果.

类似出现这样的情况一般是由于该asm磁盘组中有同一个文件号的数据多份(比如一个磁盘组中有两个库,同一个数据文件存储多份),通过方面分析,该库没有文件多份存储而且该磁盘组中只有一个数据库.进一步分析仅有的asm alert日志(大部分日志被清除),发现类似信息
Sun Mar 14 05:25:40 CST 2020 NOTE: F1X0 found on disk 0 fcn 0.60289025 NOTE: cache opening disk 0 of grp 2: HIS_FLASH00 label:HIS_FLASH00 NOTE: cache opening disk 1 of grp 2: HIS_FLASH01 label:HIS_FLASH01 NOTE: cache opening disk 2 of grp 2: HIS_FLASH02 label:HIS_FLASH02 NOTE: cache opening disk 3 of grp 2: HIS_DATA03 label:HIS_DATA03 NOTE: cache mounting (first) group 2/0xCCD84BCB (HIS_FLASH) * allocate domain 2, invalid = TRUE kjbdomatt send to node 0 Sun Mar 14 05:25:40 CST 2020 NOTE: attached to recovery domain 2 Sun Mar 14 05:25:40 CST 2020 NOTE: starting recovery of thread=1 ckpt=405.816 group=2 NOTE: advancing ckpt for thread=1 ckpt=405.819 NOTE: cache recovered group 2 to fcn 0.65493064 Sun Mar 14 05:25:40 CST 2020 NOTE: LGWR attempting to mount thread 1 for disk group 2 NOTE: LGWR mounted thread 1 for disk group 2 NOTE: opening chunk 1 at fcn 0.65493064 ABA NOTE: seq=406 blk=820 Sun Mar 14 05:25:40 CST 2020 NOTE: cache mounting group 2/0xCCD84BCB (HIS_FLASH) succeeded SUCCESS: diskgroup HIS_FLASH was mounted Sun Mar 14 05:25:42 CST 2020 NOTE: recovering COD for group 2/0xccd84bcb (HIS_FLASH) SUCCESS: completed COD recovery for group 2/0xccd84bcb (HIS_FLASH) Sun Mar 14 05:25:47 CST 2020 Starting background process ASMB ASMB started with pid=17, OS id=14599
初步可以定位,很可能是由于在3月份对该磁盘组进行了扩容,从而发生了数据文件的rebalance操作,从而出现了某些block有重复现象,对于这类情况,通过结合asm字典信息进行分析可以完全规避该问题,对数据文件进行恢复,dbv进行检查,一切正常

对所有文件类似处理,结合正常磁盘组中数据文件,对数据库进行直接open,实现完美恢复.
如果您遇到此类情况,无法解决请联系我们,提供专业ORACLE数据库恢复技术支持
Phone:17813235971 Q Q:107644445
E-Mail:dba@xifenfei.com这次数据能够完美恢复属于侥幸,因为asm disk被dd了50M(正常情况下4个磁盘的磁盘组每个磁盘dd 50M之后,很可能有部分数据文件被覆盖,该客户该磁盘组最初是存储归档日志,因此数据文件写入位置相对比较靠后,从而没有被dd破坏)

加我微信(17813235971)
加我QQ(107644445)

