标签云
asm mount asm恢复 asm 恢复 bbed bootstrap$ dul eking In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2662 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 sql加密恢复 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币 oracle 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (91)
- 数据库 (1,431)
- DB2 (22)
- MySQL (58)
- Oracle (1,314)
- Data Guard (41)
- EXADATA (7)
- GoldenGate (20)
- ORA-xxxxx (150)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (12)
- ORACLE 21C (3)
- Oracle ASM (60)
- Oracle Bug (7)
- Oracle RAC (42)
- Oracle 安全 (6)
- Oracle 开发 (26)
- Oracle 监听 (26)
- Oracle备份恢复 (445)
- Oracle安装升级 (71)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (63)
- PostgreSQL (13)
- PostgreSQL恢复 (3)
- SQL Server (26)
- SQL Server恢复 (7)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (26)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (9)
-
最近发表
- ORA-600 2032故障处理
- Oracle Recovery Tools实战批量坏块修复
- ORA-15063: ASM discovered an insufficient number of disks for diskgroup 恢复
- 修改oracle scn小工具(patch scn)
- ORA-01110 ORA-17070 OSD-04006 故障恢复
- fdisk分区导致asm disk破坏数据库恢复
- ORA-600 kcvent_internal_02故障处理
- PostgreSQL恢复系列:pg_filedump基本使用
- Oracle断电故障处理
- PostgreSQL恢复系列:pg_control异常恢复
- PostgreSQL恢复系列:wal日志丢失恢复
- pvcreate asm disk导致asm磁盘组异常恢复
- O/S-Error: (OS 23) 数据错误(循环冗余检查) 数据库恢复
- 数据库open报ORA-00959: tablespace ‘UNDOTBS1′ does not exist分析
- ASM删除表空间恢复
- ORA-12518 ORA-27302: 错误发生在: ssthrddcr
- plsql 插入blob
- 分享运气超级好的一次drop tablespace 数据恢复
- segment header异常对象删除处理
- Oracle 数据库(RDBMS)的版本支持状态摘要 (Doc ID 1674382.1)
友情链接
标签归档:KFED-00322
ORA-15196: invalid ASM block header [kfc.c:26368]故障恢复
有客户对asm的data磁盘组增加磁盘进行扩容,在做reblance的过程中重启了主机,结果导致data磁盘组mount之后自动dismount
Fri Oct 09 20:48:06 2020 NOTE: PST enabling heartbeating (grp 1) Fri Oct 09 20:48:06 2020 NOTE: ASM did background COD recovery for group 1/0x739536c (DATA) NOTE: starting rebalance of group 1/0x739536c (DATA) at power 10 Starting background process ARB0 Fri Oct 09 20:48:07 2020 ARB0 started with pid=28, OS id=39278 NOTE: assigning ARB0 to group 1/0x739536c (DATA) with 10 parallel I/Os cellip.ora not found. WARNING:cache read a corrupt block:group=1(DATA) dsk=8 blk=7 disk=8(DATA_0008)incarn=3916014506 au=0 blk=7 count=1 Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_39278.trc: ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] NOTE: a corrupted block from group DATA was dumped to /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_39278.trc WARNING:cache read(retry)a corrupt block:group=1(DATA) dsk=8 blk=7 disk=8(DATA_0008)incarn=3916014506 au=0 blk=7 count=1 Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_39278.trc: ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] ERROR: cache failed to read group=1(DATA) dsk=8 blk=7 from disk(s): 8(DATA_0008) Fri Oct 09 20:48:13 2020 NOTE: GroupBlock outside rolling migration privileged region ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] NOTE: requesting all-instance membership refresh for group=1 NOTE: cache initiating offline of disk 8 group DATA NOTE: process _arb0_+asm1 (39278) initiating offline of disk 8.3916014506 (DATA_0008) with mask 0x7e in group 1 NOTE: initiating PST update: grp = 1, dsk = 8/0xe969a3aa, mask = 0x6a, op = clear GMON updating disk modes for group 1 at 7 for pid 28, osid 39278 ERROR: Disk 8 cannot be offlined, since diskgroup has external redundancy. ERROR: too many offline disks in PST (grp 1) Fri Oct 09 20:48:13 2020 NOTE: cache dismounting (not clean) group 1/0x0739536C (DATA) NOTE: messaging CKPT to quiesce pins Unix process pid: 39346, image: oracle@rac1 (B000) Fri Oct 09 20:48:13 2020 NOTE: halting all I/Os to diskgroup 1 (DATA) Fri Oct 09 20:48:13 2020 NOTE: LGWR doing non-clean dismount of group 1 (DATA) NOTE: LGWR sync ABA=32.4749 last written ABA 32.4749 WARNING: Offline for disk DATA_0008 in mode 0x7f failed. Fri Oct 09 20:48:13 2020 kjbdomdet send to inst 2 detach from dom 1, sending detach message to inst 2 Fri Oct 09 20:48:13 2020 List of instances: 1 2 Dirty detach reconfiguration started (new ddet inc 2, cluster inc 4) Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_39278.trc (incident=337185): ORA-15335: ASM metadata corruption detected in disk group 'DATA' ORA-15130: diskgroup "DATA" is being dismounted ORA-15066: offlining disk "DATA_0008" in group "DATA" may result in a data loss ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483656] [7] [2182009786 != 2190395015] Incident details in: /u01/grid/diag/asm/+asm/+ASM1/incident/incdir_337185/+ASM1_arb0_39278_i337185.trc Global Resource Directory partially frozen for dirty detach * dirty detach - domain 1 invalid = TRUE 2341 GCS resources traversed, 0 cancelled Dirty Detach Reconfiguration complete freeing rdom 1 Fri Oct 09 20:48:13 2020 WARNING: dirty detached from domain 1 NOTE: cache dismounted group 1/0x0739536C (DATA)
错误信息比较明显dsk=8 blk=7 au=0 blk=7 的check值不对,本来应该是2190395015现在变为了2182009786,通过kfed分析确实如此
C:\Users\Administrator>kfed read f:/temp/xff/2.dd blkn=7|more kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL kfbh.datfmt: 2 ; 0x003: 0x02 kfbh.block.blk: 7 ; 0x004: blk=7 kfbh.block.obj: 2147483656 ; 0x008: disk=8 kfbh.check: 2182009786 ; 0x00c: 0x820ed3ba kfbh.fcn.base: 2711248 ; 0x010: 0x00295ed0 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdatb.aunum: 2240 ; 0x000: 0x000008c0 kfdatb.shrink: 448 ; 0x004: 0x01c0 kfdatb.ub2pad: 0 ; 0x006: 0x0000 kfdatb.auinfo[0].link.next: 8 ; 0x008: 0x0008 kfdatb.auinfo[0].link.prev: 8 ; 0x00a: 0x0008 kfdatb.auinfo[1].link.next: 12 ; 0x00c: 0x000c kfdatb.auinfo[1].link.prev: 12 ; 0x00e: 0x000c kfdatb.auinfo[2].link.next: 16 ; 0x010: 0x0010 kfdatb.auinfo[2].link.prev: 16 ; 0x012: 0x0010 kfdatb.auinfo[3].link.next: 20 ; 0x014: 0x0014 kfdatb.auinfo[3].link.prev: 20 ; 0x016: 0x0014 kfdatb.auinfo[4].link.next: 24 ; 0x018: 0x0018 kfdatb.auinfo[4].link.prev: 24 ; 0x01a: 0x0018 kfdatb.auinfo[5].link.next: 28 ; 0x01c: 0x001c kfdatb.auinfo[5].link.prev: 28 ; 0x01e: 0x001c kfdatb.auinfo[6].link.next: 32 ; 0x020: 0x0020 kfdatb.auinfo[6].link.prev: 32 ; 0x022: 0x0020 kfdatb.spare: 0 ; 0x024: 0x00000000
修改该值之后,再次mount data磁盘组,报错如下
Sat Oct 10 13:49:22 2020 ARB0 started with pid=28, OS id=10329 NOTE: assigning ARB0 to group 1/0x3759521c (DATA) with 10 parallel I/Os cellip.ora not found. Sat Oct 10 13:49:26 2020 NOTE: GroupBlock outside rolling migration privileged region NOTE: requesting all-instance membership refresh for group=1 WARNING: cache read a corrupt block: group=1(DATA) dsk=8 blk=8 disk=8 (DATA_0008) incarn=3916014011 au=0 blk=8 count=1 Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_10329.trc: ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] NOTE: a corrupted block from group DATA was dumped to /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_10329.trc WARNING:cache read(retry)a corrupt block: group=1(DATA)dsk=8 blk=8 disk=8(DATA_0008)incarn=3916014011 au=0 blk=8 count=1 Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_10329.trc: ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] ERROR: cache failed to read group=1(DATA) dsk=8 blk=8 from disk(s): 8(DATA_0008) ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] NOTE: cache initiating offline of disk 8 group DATA NOTE: process _arb0_+asm1 (10329) initiating offline of disk 8.3916014011 (DATA_0008) with mask 0x7e in group 1 NOTE: initiating PST update: grp = 1, dsk = 8/0xe969a1bb, mask = 0x6a, op = clear GMON updating disk modes for group 1 at 64 for pid 28, osid 10329 ERROR: Disk 8 cannot be offlined, since diskgroup has external redundancy. ERROR: too many offline disks in PST (grp 1) Sat Oct 10 13:49:28 2020 NOTE: cache dismounting (not clean) group 1/0x3759521C (DATA) WARNING: Offline for disk DATA_0008 in mode 0x7f failed. Sat Oct 10 13:49:28 2020 NOTE: halting all I/Os to diskgroup 1 (DATA) NOTE: messaging CKPT to quiesce pins Unix process pid: 10346, image: oracle@rac1 (B000) Errors in file /u01/grid/diag/asm/+asm/+ASM1/trace/+ASM1_arb0_10329.trc (incident=363107): ORA-15335: ASM metadata corruption detected in disk group 'DATA' ORA-15130: diskgroup "DATA" is being dismounted ORA-15066: offlining disk "DATA_0008" in group "DATA" may result in a data loss ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483656] [8] [0 != 1] Incident details in: /u01/grid/diag/asm/+asm/+ASM1/incident/incdir_363107/+ASM1_arb0_10329_i363107.trc
该报错为:dsk=8 blk=7 au=0 blk=8异常,通过kfed查看发现
C:\Users\Administrator>kfed read f:/temp/xff/2.dd blkn=8 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 006BE8C00 00000000 00000000 00000000 00000000 [................] Repeat 31 times 006BE8E00 012C0000 04AFFC07 003BFFCD 03F15BD0 [..,.......;..[..] 006BE8E10 012BFC30 00000000 00000002 00000002 [0.+.............] 006BE8E20 00008000 00008000 00002000 564F22AF [......... ..."OV] 006BE8E30 5F805293 FFFF0002 0001EF53 00000001 [.R._....S.......] 006BE8E40 545AE384 00000000 00000000 00000001 [..ZT............] 006BE8E50 00000000 0000000B 00000100 0000003C [............<...] 006BE8E60 00000242 0000007B 52438BA0 C44FFA90 [B...{.....CR..O.] 006BE8E70 33B6F381 919E2DBA 00000000 00000000 [...3.-..........] 006BE8E80 00000000 00000000 6361622F 0070756B [......../backup.] 006BE8E90 00000000 00000000 00000000 00000000 [................] Repeat 2 times 006BE8EC0 00000000 00000000 00000000 03ED0000 [................] 006BE8ED0 00000000 00000000 00000000 00000000 [................] 006BE8EE0 00000008 00000000 00000000 AC3C87D6 [..............<.] 006BE8EF0 F1401174 F4F036BD 274FB92F 00000101 [t.@..6../.O'....] 006BE8F00 0000000C 00000000 545AE384 0002F30A [..........ZT....] 006BE8F10 00000004 00000000 00000000 00007FFF [................] 006BE8F20 02508000 00007FFF 00000001 0250FFFF [..P...........P.] 006BE8F30 00000000 00000000 00000000 00000000 [................] 006BE8F40 00000000 00000000 00000000 08000000 [................] 006BE8F50 00000000 00000000 00000000 001C001C [................] 006BE8F60 00000001 00000000 00000000 00000000 [................] 006BE8F70 00000000 00000004 A9AF72B9 0000003B [.........r..;...] 006BE8F80 00000000 00000000 00000000 00000000 [................] Repeat 167 times 006BE9A00 00001CC4 00800101 00001CC9 00800101 [................] 006BE9A10 00001CCD 00800101 00001CD2 00800101 [................] 006BE9A20 00001CD7 00800101 00001CDE 00800101 [................] 006BE9A30 00001CE3 00800101 00001CE8 00800101 [................] 006BE9A40 00001CEC 00800101 00000000 00000000 [................] 006BE9A50 00000000 00000000 00000000 00000000 [................] Repeat 26 times KFED-00322:Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
该block完全损坏,基本上无直接修复的可能,通过对data 磁盘组进行patch操作,让其mount之后不再dismount
NOTE: GMON heartbeating for grp 1 GMON querying group 1 at 76 for pid 27, osid 14466 NOTE: cache opening disk 0 of grp 1: DATA_0000 path:/dev/emcpowere NOTE: F1X0 found on disk 0 au 2 fcn 0.2708382 NOTE: cache opening disk 1 of grp 1: DATA_0001 path:/dev/emcpowerf NOTE: cache opening disk 2 of grp 1: DATA_0002 path:/dev/emcpowerg NOTE: cache opening disk 3 of grp 1: DATA_0003 path:/dev/emcpowerh NOTE: cache opening disk 4 of grp 1: DATA_0004 path:/dev/emcpoweri NOTE: cache opening disk 5 of grp 1: DATA_0005 path:/dev/emcpowerj NOTE: cache opening disk 6 of grp 1: DATA_0006 path:/dev/emcpowerk NOTE: cache opening disk 7 of grp 1: DATA_0007 path:/dev/emcpowerl NOTE: cache opening disk 8 of grp 1: DATA_0008 path:/dev/emcpowerc NOTE: cache mounting (first) external redundancy group 1/0x47495222 (DATA) Sat Oct 10 13:59:38 2020 * allocate domain 1, invalid = TRUE Sat Oct 10 13:59:38 2020 NOTE: attached to recovery domain 1 NOTE: starting recovery of thread=1 ckpt=53.6778 group=1 (DATA) NOTE: advancing ckpt for group 1 (DATA) thread=1 ckpt=53.6778 NOTE: cache recovered group 1 to fcn 0.2961429 NOTE: redo buffer size is 256 blocks (1053184 bytes) Sat Oct 10 13:59:38 2020 NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (DATA) NOTE: LGWR found thread 1 closed at ABA 53.6777 NOTE: LGWR mounted thread 1 for diskgroup 1 (DATA) NOTE: LGWR opening thread 1 at fcn 0.2961429 ABA 54.6778 NOTE: cache mounting group 1/0x47495222 (DATA) succeeded NOTE: cache ending mount (success) of group DATA number=1 incarn=0x47495222 NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1 SUCCESS: diskgroup DATA was mounted SUCCESS: alter diskgroup data mount
然后通过rman备份数据库,删除老磁盘组,创建新磁盘组,恢复数据,实现数据库完美恢复,数据0丢失.
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type]
在oracle asm的使用过程中由于操作系统层面的错误操作导致asm disk 被破坏,这里列举了几种破坏之后的kfed报错现象(KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type])
asm mount 磁盘组报错(ORA-15040 ORA-15042)
SQL> alter diskgroup DATA mount; alter diskgroup DATA mount * ERROR at line 1: ORA-15032: not all alterations performed ORA-15040: diskgroup is incomplete ORA-15042: ASM disk "2" is missing from group number "2"
asm alert日志报错(ORA-15335 ORA-15066 ORA-15196等)
ORA-15335: ASM metadata corruption detected in disk group 'DATA' ORA-15130: diskgroup "DATA" is being dismounted ORA-15066: offlining disk "DATA_0002" in group "DATA" may result in a data loss ORA-15196: invalid ASM block header [kfc.c:26368] [endian_kfbh] [2147483651] [48] [0 != 1]
kfed查看磁盘头报错
文件文件头(不光是disk header的4k,可能是连续的几个au,甚至更多)可能彻底损坏,一般kfed 读取都会看到KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type]之类错误
[oracle@fcomtaep2 disks]$ kfed read ASMRECO03 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 7FC18D899400 00000000 00000000 00000000 00000000 [................] Repeat 27 times 7FC18D8995C0 FEEE0001 0001FFFF FFFF0000 00000FFF [................] 7FC18D8995D0 00000000 00000000 00000000 00000000 [................] Repeat 1 times 7FC18D8995F0 00000000 00000000 00000000 AA550000 [..............U.] 7FC18D899600 20494645 54524150 00010000 0000005C [EFI PART....\...] <==== **** Here ****** 7FC18D899610 BD82BBB3 00000000 00000001 00000000 [................] 7FC18D899620 0FFFFFFF 00000000 00000022 00000000 [........".......] 7FC18D899630 0FFFFFDE 00000000 FD8857E5 42D7B49B [.........W.....B] 7FC18D899640 0901FA87 6B3DB5AA 00000002 00000000 [......=k........] 7FC18D899650 00000080 00000080 FE48EB77 00000000 [........w.H.....] 7FC18D899660 00000000 00000000 00000000 00000000 [................] Repeat 25 times 7FC18D899800 EBD0A0A2 4433B9E5 B668C087 C79926B7 [......3D..h..&..] 7FC18D899810 5381F6DF 4626F988 0E4F468D D78D3B28 [...S..&F.FO.(;..] 7FC18D899820 000007A1 00000000 0FFFF85F 00000000 [........_.......] 7FC18D899830 00000000 00000000 00720070 006D0069 [........p.r.i.m.] 7FC18D899840 00720061 00000079 00000000 00000000 [a.r.y...........] 7FC18D899850 00000000 00000000 00000000 00000000 [................] Repeat 186 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
“EFI PART”是分区的元数据,一般是被分区导致asm disk损坏.
[ebernal@dbaasm new2]$ kfed read emcpowerl | head -25 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 2ABD671E9400 00000000 00000000 00000000 00000000 [................] Repeat 31 times 2ABD671E9600 4542414C 454E4F4C 00000001 00000000 [LABELONE........] 2ABD671E9610 E4E1DDB1 00000020 324D564C 31303020 [.... ...LVM2 001] <==== **** Here ****** 2ABD671E9620 50365A77 71327874 34303156 4B4E6136 [wZ6Ptx2qV1046aNK] 2ABD671E9630 35395159 5147634C 487A5A38 63575A37 [YQ95LcGQ8ZzH7ZWc] 2ABD671E9640 00000000 00000019 00030000 00000000 [................] 2ABD671E9650 00000000 00000000 00000000 00000000 [................] 2ABD671E9660 00000000 00000000 00001000 00000000 [................] 2ABD671E9670 0002F000 00000000 00000000 00000000 [................] 2ABD671E9680 00000000 00000000 00000000 00000000 [................] Repeat 215 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
“LVM2 001” 是逻辑卷的名字,该asm disk很可能被做为lvm管理而被破坏
[ebernal@dbaasm tars]$ kfed read rhdisk16 kfbh.endian: 65 ; 0x000: 0x41 kfbh.hard: 73 ; 0x001: 0x49 kfbh.type: 88 ; 0x002: *** Unknown Enum *** kfbh.datfmt: 32 ; 0x003: 0x20 kfbh.block.blk: 1111709260 ; 0x004: blk=1111709260 kfbh.block.obj: 1634861056 ; 0x008: file=131072 kfbh.check: 119 ; 0x00c: 0x00000077 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 2B6FE2AC1400 20584941 4243564C 61720000 00000077 [AIX LVCB..raw...] <==== **** Here ****** 2B6FE2AC1410 00000000 00000000 00000000 00000000 [................] 2B6FE2AC1420 00000000 00000000 30300000 38306430 [..........000d08] 2B6FE2AC1430 30306131 34643030 30303030 31303030 [1a0000d400000001] 2B6FE2AC1440 61006533 766C6D73 7461645F 00003161 [3e.asmlv_data1..] 2B6FE2AC1450 00000000 00000000 00000000 00000000 [................] Repeat 2 times 2B6FE2AC1480 54000000 4D206575 20207961 31312037 [...Tue May 7 11] 2B6FE2AC1490 3A33343A 32203633 0A333130 00000000 [:43:36 2013.....] 2B6FE2AC14A0 65755400 79614D20 20372020 343A3131 [.Tue May 7 11:4] 2B6FE2AC14B0 34323A38 31303220 00000A33 44000000 [8:24 2013......D] 2B6FE2AC14C0 41313830 30303444 6D6D7900 02007900 [081AD400.ymm.y..] 2B6FE2AC14D0 0100E40C 656E6F4E 00000000 00000000 [....None........] 2B6FE2AC14E0 00000000 00000000 00000000 00000000 [................] Repeat 14 times 2B6FE2AC15D0 00000000 00000000 65310000 61653934 [..........1e49ea] 2B6FE2AC15E0 342E3862 00000000 00000000 00000000 [b8.4............] 2B6FE2AC15F0 00000000 00000000 00000000 00000000 [................] Repeat 224 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][88]
这里的“AIX LVCB..raw” 是AIX OS volume 的元数据库,也就是说,asm disk 被作为了aix os层面破坏
[oracle@dbep2 disks]$ kfed read asm-disk3 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 0 ; 0x001: 0x00 kfbh.type: 0 ; 0x002: KFBTYP_INVALID kfbh.datfmt: 0 ; 0x003: 0x00 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 06000000 00000000 00000000 00000000 00000000 [................] Repeat 25 times 0602100 51e2b7f6 00ed4e00 00000000 00000001 [...Q.N..........] 0602120 00000000 0000000b 00000100 0000003c [............<...] 0602140 00000242 0000007b 5d8468e7 6147782a [B...{....h.]*xGa] 0602160 d17851a2 327552e2 00000000 00000000 [.Qx..Ru2........] 0602200 00000000 00000000 3130752f 91a4f000 [......../u01....] <==== **** Here ****** 0602220 ff8808e4 d5104cff 000000ac 00000100 [.....L..........] 0602240 00000000 00000000 00000000 09d18000 [................] Repeat 254 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][88]
这里的/u01很可能表明该asm disk被文件系统覆盖
对于asm disk的各种破坏情况,如果是normal/high冗余,那么asm dg没有问题,可以考虑通过删除异常盘,然后重新加入;如果是外部冗余遭遇到asm disk 被破坏,一般asm disk 会dismount,而且无法正常mount,如果有备份的磁盘头,可以尝试还原磁盘头,mount 磁盘组,然后只读方式迁移数据;如果没有备份磁盘头或者还原之后也无法mount,可能需要通过一些额外的方式处理比如通过工具在asm dismount状态下恢复数据文件,甚至通过对asm block/oracle block碎片重组的方式恢复数据.参考相关文章:
oracle asm系列文章汇总
pvid=yes导致asm无法mount
asm disk header 彻底损坏恢复
分区无法识别导致asm diskgroup无法mount
oracle asm disk格式化恢复—格式化为ext4文件系统
oracle asm disk格式化恢复—格式化为ntfs文件系统
asm disk误设置pvid导致asm diskgroup无法mount恢复
分享oracleasm createdisk重新创建asm disk后数据0丢失恢复案例
ORA-15042: ASM disk “N” is missing from group number “M” 故障恢复
如果您遇到此类情况,无法解决请联系我们,提供专业ORACLE数据库恢复技术支持
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
发表在 Oracle ASM, Oracle备份恢复
标签为 endian_kfbh, Invalid OSM block type, kfbtTraverseBlock, KFED-00322, ORA-15042, ORA-15196
评论关闭
asm disk误设置pvid导致asm diskgroup无法mount恢复
有朋友找到我说他们把以前存储到AIX直连的存储切换为含光纤交换机的存储网络后,RAC无法启动,让我给予支持.通过分析是由于换链路之后开始磁盘顺序不对,维护人员对其asm disk 设置了pvid,导致asm 磁盘组无法正常mount,从而使得含votedisk的dg的asm disk无法正常访问,从而RAC的cssd进程无法启动,同样数据文件的磁盘组也无法mount,通过kfed修复成功,实现数据0丢失.
平台版本信息(2节点RAC)
$ sqlplus -v SQL*Plus: Release 11.2.0.4.0 Production $ uname -a AIX db2 1 7 00F9733E4C00
GI日志报错信息
2014-12-20 16:44:08.769: [ohasd(6946818)]CRS-2769:Unable to failover resource 'ora.diskmon'. 2014-12-20 16:44:11.775: [cssd(9502756)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log 2014-12-20 16:44:26.791: [cssd(9502756)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; 、Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log 2014-12-20 16:44:41.812: [cssd(9502756)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log
从这里可以看出来是由于RAC启动过程中无法获得votedisk使得其无法正常启动,通过分析日志找出来votedisk相关磁盘
2014-12-15 17:36:15.424: [cssd(10027070)]CRS-1605:CSSD voting file is online: /dev/rhdisk4; details in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log 2014-12-15 17:36:15.433: [cssd(10027070)]CRS-1605:CSSD voting file is online: /dev/rhdisk5; details in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log 2014-12-15 17:36:15.445: [cssd(10027070)]CRS-1605:CSSD voting file is online: /dev/rhdisk6; details in /u01/app/11.2.0/grid/log/db1/cssd/ocssd.log
从这里可以知道rhdisk4,5,6为votedisk对应磁盘,使用kfed查看磁盘头信息
$kfed read /dev/rhdisk4 kfbh.endian: 201 ; 0x000: 0xc9 kfbh.hard: 194 ; 0x001: 0xc2 kfbh.type: 212 ; 0x002: *** Unknown Enum *** kfbh.datfmt: 193 ; 0x003: 0xc1 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 0 ; 0x008: file=0 kfbh.check: 0 ; 0x00c: 0x00000000 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 1102BEE00 C9C2D4C1 00000000 00000000 00000000 [................] 1102BEE10 00000000 00000000 00000000 00000000 [................] Repeat 6 times 1102BEE80 00F9733D 67553E0A 00000000 00000000 [..s=gU>.........] 1102BEE90 00000000 00000000 00000000 00000000 [................] Repeat 246 times KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][212] $kfed read /dev/rhdisk4 blkn=1 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 2 ; 0x002: KFBTYP_FREESPC kfbh.datfmt: 2 ; 0x003: 0x02 kfbh.block.blk: 1 ; 0x004: blk=1 kfbh.block.obj: 2147483648 ; 0x008: disk=0 kfbh.check: 3883664132 ; 0x00c: 0xe77c0304 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdfsb.aunum: 0 ; 0x000: 0x00000000 kfdfsb.max: 254 ; 0x004: 0x00fe kfdfsb.cnt: 23 ; 0x006: 0x0017 kfdfsb.bound: 0 ; 0x008: 0x0000 kfdfsb.flag: 1 ; 0x00a: B=1 kfdfsb.ub1spare: 0 ; 0x00b: 0x00 kfdfsb.spare[0]: 0 ; 0x00c: 0x00000000 kfdfsb.spare[1]: 0 ; 0x010: 0x00000000 kfdfsb.spare[2]: 0 ; 0x014: 0x00000000 kfdfse[0].fse: 119 ; 0x018: FREE=0x7 FRAG=0x7 kfdfse[1].fse: 16 ; 0x019: FREE=0x0 FRAG=0x1 ………… $kfed read /dev/rhdisk4 blkn=510 kfbh.endian: 0 ; 0x000: 0x00 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD kfbh.datfmt: 1 ; 0x003: 0x01 kfbh.block.blk: 254 ; 0x004: blk=254 kfbh.block.obj: 2147483648 ; 0x008: disk=0 kfbh.check: 3460116983 ; 0x00c: 0xce3d31f7 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8 kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000 kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000 kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000 kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000 kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000 kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000 kfdhdb.compat: 186646528 ; 0x020: 0x0b200000 kfdhdb.dsknum: 0 ; 0x024: 0x0000 kfdhdb.grptyp: 2 ; 0x026: KFDGTP_NORMAL kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER kfdhdb.dskname: CRS_0000 ; 0x028: length=8 kfdhdb.grpname: CRS ; 0x048: length=3 kfdhdb.fgname: CRS_0000 ; 0x068: length=8 …………
由上述分析可以基本上确定是asm disk header 被破坏,进一步分析破坏原因
[db2/dev#]lspv hdisk0 00f9733ef7cf27e9 rootvg active hdisk1 00f9733e21b953e6 rootvg active hdisk2 00f9733e21b97a83 appvg active hdisk3 00f9733e21b98434 appvg active hdisk4 00f9733d67553e0a None hdisk5 00f9733d67553f31 None hdisk6 00f9733d67554011 None hdisk7 00f9733d67554165 None hdisk8 00f9733d675541e5 None hdisk9 00f9733d675542e4 None hdisk10 none None [db2/dev#]ls -l rhdisk* crw------- 2 root system 24, 1 Oct 18 11:45 rhdisk0 crw------- 1 root system 24, 3 Oct 18 13:27 rhdisk1 crw------- 1 root system 24, 5 Dec 20 20:02 rhdisk10 crw------- 1 root system 24, 2 Oct 18 13:32 rhdisk2 crw------- 1 root system 24, 0 Oct 18 13:32 rhdisk3 crw-rw---- 1 grid asmadmin 24, 8 Dec 20 20:02 rhdisk4 crw-rw---- 1 grid asmadmin 24, 9 Dec 20 20:02 rhdisk5 crw-rw---- 1 grid asmadmin 24, 10 Dec 20 20:02 rhdisk6 crw-rw---- 1 grid asmadmin 24, 4 Dec 20 20:02 rhdisk7 crw-rw---- 1 grid asmadmin 24, 6 Dec 20 20:02 rhdisk8 crw-rw---- 1 grid asmadmin 24, 7 Dec 20 20:02 rhdisk9
从这里基本上可以看出来,是由于磁盘头被重写了pvid,导致asm disk header 被破坏.进一步分析asm log,确定哪些磁盘被用作asm disk
SQL> CREATE DISKGROUP CRS NORMAL REDUNDANCY DISK '/dev/rhdisk4', '/dev/rhdisk5', '/dev/rhdisk6' ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */ NOTE: Assigning number (1,0) to disk (/dev/rhdisk4) NOTE: Assigning number (1,1) to disk (/dev/rhdisk5) NOTE: Assigning number (1,2) to disk (/dev/rhdisk6) NOTE: initializing header on grp 1 disk CRS_0000 NOTE: initializing header on grp 1 disk CRS_0001 NOTE: initializing header on grp 1 disk CRS_0002 SQL> CREATE DISKGROUP DATA EXTERNAL REDUNDANCY DISK '/dev/rhdisk9' SIZE 614400M ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */ NOTE: Assigning number (2,0) to disk (/dev/rhdisk9) NOTE: initializing header on grp 2 disk DATA_0000 SQL> CREATE DISKGROUP FBA EXTERNAL REDUNDANCY DISK '/dev/rhdisk8' SIZE 204800M ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */ NOTE: Assigning number (3,0) to disk (/dev/rhdisk8) NOTE: initializing header on grp 3 disk FBA_0000 SQL> CREATE DISKGROUP ARCH EXTERNAL REDUNDANCY DISK '/dev/rhdisk7' SIZE 102400M ATTRIBUTE 'compatible.asm'='11.2.0.0.0','au_size'='1M' /* ASMCA */ NOTE: Assigning number (4,0) to disk (/dev/rhdisk7) NOTE: initializing header on grp 4 disk ARCH_0000
这里可以确定asm disk为rhdisk[4-9],通过kfed分析全部和rhdisk4一样的问题,也符合lspv查询出来的结果,使用kfed repair修复asm disk header后
SQL> alter diskgroup data mount; Diskgroup altered. SQL> alter diskgroup fba mount; Diskgroup altered. SQL> alter diskgroup arch mount; Diskgroup altered. SQL> alter diskgroup crs mount; Diskgroup altered. SQL> select group_number,disk_number,path from v$asm_disk; GROUP_NUMBER DISK_NUMBER PATH ------------ ----------- -------------------------------------------------- 2 0 /dev/rhdisk4 2 1 /dev/rhdisk5 2 2 /dev/rhdisk6 1 0 /dev/rhdisk7 4 0 /dev/rhdisk8 3 0 /dev/rhdisk9 6 rows selected. SQL> select group_number,name from v$asm_diskgroup; GROUP_NUMBER NAME ------------ ------------------------------------------------------------ 1 ARCH 2 CRS 3 DATA 4 FBA
这里证明通过kfed对磁盘头的修复,asm磁盘组已经全部mount成功,GI状态也恢复正常
[db2/#]crsctl status res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ARCH.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.CRS.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.FBA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.LISTENER.lsnr ONLINE ONLINE db1 ONLINE ONLINE db2 ora.asm ONLINE ONLINE db1 Started ONLINE ONLINE db2 Started ora.gsd OFFLINE OFFLINE db1 OFFLINE OFFLINE db2 ora.net1.network ONLINE ONLINE db1 ONLINE ONLINE db2 ora.ons ONLINE ONLINE db1 ONLINE ONLINE db2 ora.registry.acfs ONLINE ONLINE db1 ONLINE ONLINE db2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE db1 ora.cvu 1 ONLINE ONLINE db1 ora.db1.vip 1 ONLINE ONLINE db1 ora.db2.vip 1 ONLINE ONLINE db2 ora.nkora.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.oc4j 1 ONLINE ONLINE db1 ora.scan1.vip 1 ONLINE ONLINE db1
这里忽略了一个问题,在修复磁盘头之前没有清除pvid,导致磁盘头修复后,pvid依然存储在odm中
[db2/dev#]lspv hdisk0 00f9733ef7cf27e9 rootvg active hdisk1 00f9733e21b953e6 rootvg active hdisk2 00f9733e21b97a83 appvg active hdisk3 00f9733e21b98434 appvg active hdisk4 00f9733d67553e0a None hdisk5 00f9733d67553f31 None hdisk6 00f9733d67554011 None hdisk7 00f9733d67554165 None hdisk8 00f9733d675541e5 None hdisk9 00f9733d675542e4 None hdisk10 none None
通过分析发现fba磁盘组中无任何记录,使用该磁盘组进行直接清除pvid测试
$ sqlplus / as sysasm SQL*Plus: Release 11.2.0.4.0 Production on Sun Dec 21 03:13:31 2014 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> alter diskgroup fba dismount; Diskgroup altered. SQL> exit Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options $ exit You have mail in /usr/spool/mail/root [db2/#]chdev -l hdisk8 -a pv=clear hdisk8 changed [db2/#]lspv hdisk0 00f9733ef7cf27e9 rootvg active hdisk1 00f9733e21b953e6 rootvg active hdisk2 00f9733e21b97a83 appvg active hdisk3 00f9733e21b98434 appvg active hdisk4 00f9733d67553e0a None hdisk5 00f9733d67553f31 None hdisk6 00f9733d67554011 None hdisk7 00f9733d67554165 None hdisk8 none None hdisk9 00f9733d675542e4 None hdisk10 none None [db2/#]su - grid $ sqlplus / as sysasm SQL*Plus: Release 11.2.0.4.0 Production on Sun Dec 21 03:15:19 2014 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> alter diskgroup fba mount; Diskgroup altered. SQL> exit Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options
通过测试直接清除pvid asm 磁盘头依然工作正常,关闭GI,使用chdev清除hdisk[4-9]所有pvid,启动GI一切正常
[db1/#]crsctl status res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ARCH.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.CRS.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.FBA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.LISTENER.lsnr ONLINE ONLINE db1 ONLINE ONLINE db2 ora.asm ONLINE ONLINE db1 Started ONLINE ONLINE db2 Started ora.gsd OFFLINE OFFLINE db1 OFFLINE OFFLINE db2 ora.net1.network ONLINE ONLINE db1 ONLINE ONLINE db2 ora.ons ONLINE ONLINE db1 ONLINE ONLINE db2 ora.registry.acfs ONLINE ONLINE db1 ONLINE ONLINE db2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE db1 ora.cvu 1 ONLINE ONLINE db1 ora.db1.vip 1 ONLINE ONLINE db1 ora.db2.vip 1 ONLINE ONLINE db2 ora.nkora.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.oc4j 1 ONLINE ONLINE db1 ora.scan1.vip 1 ONLINE ONLINE db1 [db1/#]lspv hdisk0 00f9733df7c7a9db rootvg active hdisk1 00f9733d21dad8fe rootvg active hdisk2 00f9733d21dbd08b appvg active hdisk3 00f9733d21dbd2ab appvg active hdisk4 none None hdisk5 none None hdisk6 none None hdisk7 none None hdisk8 none None hdisk9 none None hdisk10 none None
至此设置pvid导致asm disk header损坏的asm 恢复正常,实现数据0丢失。
温馨提示:aix asm disk磁盘中不能设置pvid,否则将会导致asm disk header 损坏,无法正常mount
如果您遇到此类情况,无法解决请联系我们,提供专业ORACLE数据库恢复技术支持
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
发表在 Oracle ASM, 非常规恢复
标签为 asm, asm不能mount, asm恢复, kfbtTraverseBlock, KFED-00322, mount, ORA-15042, pvid
一条评论