标签云
asm mount asm恢复 asm 恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (97)
- 数据库 (1,545)
- DB2 (22)
- MySQL (65)
- Oracle (1,420)
- Data Guard (46)
- EXADATA (7)
- GoldenGate (21)
- ORA-xxxxx (157)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (12)
- ORACLE 21C (3)
- Oracle ASM (65)
- Oracle Bug (7)
- Oracle RAC (46)
- Oracle 安全 (6)
- Oracle 开发 (26)
- Oracle 监听 (26)
- Oracle备份恢复 (500)
- Oracle安装升级 (81)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (74)
- PostgreSQL (13)
- PostgreSQL恢复 (3)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (36)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (19)
-
最近发表
- ORA-07445: exception encountered: core dump [kdxlin()+4088]处理
- bbed解决ORA-01578
- asm disk被加入到另外一个磁盘组故障恢复
- ORA-600 ksuloget2 恢复
- ORA-10485: Real-Time Query cannot be enabled while applying migration redo.
- Patch SCN一键解决ORA-600 2662故障
- amdu参数详解
- ORA-600 kghstack_underflow_internal_2
- WRH$_LATCH, WRH$_SYSSTAT, WRH$_PARAMETER对象较大
- Oracle 19C 报ORA-704 ORA-01555故障处理
- Exadata磁盘损坏导致磁盘组无法mount恢复(oracle一体机磁盘组异常恢复)
- .faust加密勒索数据库恢复
- Assistant: Download Reference for Oracle Database/GI PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases-202307
- ORA-01122 ORA-01208 故障处理
- 硬件故障恢复出文件之后数据库故障处理
- dul支持arm版本Oracle数据库恢复
- ARM上的oracle数据库存储字节序分析—小字节序
- ARM Linux(麒麟操作系统)安装Oracle数据库
- win系统删除oracle数据文件恢复
- ORA-01122 ORA-01200故障处理
友情链接
分类目录归档:非常规恢复
asm disk被加入到另外一个磁盘组故障恢复
有朋友在aix环境对其中一个rac的asm磁盘组进行扩容
之后另外一套rac的磁盘组直接dismount
Wed Aug 23 12:44:02 2023 NOTE: SMON starting instance recovery for group DATA domain 2 (mounted) NOTE: F1X0 found on disk 0 au 2 fcn 0.128808679 NOTE: SMON skipping disk 7 - no header NOTE: cache initiating offline of disk 7 group DATA NOTE: process _smon_+asm1 (1770932) initiating offline of disk 7.3422955792 (DATA_0007) with mask 0x7e in group 2 NOTE: initiating PST update: grp = 2, dsk = 7/0xcc062910, mask = 0x6a, op = clear Wed Aug 23 12:44:02 2023 GMON updating disk modes for group 2 at 7 for pid 17, osid 1770932 ERROR: Disk 7 cannot be offlined, since diskgroup has external redundancy. ERROR: too many offline disks in PST (grp 2) Wed Aug 23 12:44:02 2023 NOTE: cache dismounting (not clean) group 2/0x7FE6D808 (DATA) WARNING: Offline for disk DATA_0007 in mode 0x7f failed. Wed Aug 23 12:44:02 2023 NOTE: halting all I/Os to diskgroup 2 (DATA) ERROR: No disks with F1X0 found on disk group DATA NOTE: aborting instance recovery of domain 2 due to diskgroup dismount NOTE: SMON skipping lock domain (2) validation because diskgroup being dismounted Abort recovery for domain 2 Wed Aug 23 12:44:02 2023 ERROR: ORA-15130 in COD recovery for diskgroup 2/0x7fe6d808 (DATA) ERROR: ORA-15130 thrown in RBAL for group number 2 Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_rbal_2360526.trc: ORA-15130: diskgroup "DATA" is being dismounted
再次尝试mount该磁盘组,报ORA-15042和ORA-15038错误
SQL> alter diskgroup data mount NOTE: cache registered group DATA number=2 incarn=0x79e6d861 NOTE: cache began mount (first) of group DATA number=2 incarn=0x79e6d861 NOTE: Assigning number (2,0) to disk (/dev/rhdisk31) NOTE: Assigning number (2,3) to disk (/dev/rhdisk33) NOTE: Assigning number (2,4) to disk (/dev/rhdisk34) NOTE: Assigning number (2,5) to disk (/dev/rhdisk35) NOTE: Assigning number (2,6) to disk (/dev/rhdisk36) NOTE: Assigning number (2,9) to disk (/dev/rhdisk39) NOTE: Assigning number (2,1) to disk (/dev/rhdisk8) NOTE: Assigning number (2,2) to disk (/dev/rhdisk9) Wed Aug 23 12:58:46 2023 NOTE: GMON heartbeating for grp 2 GMON querying group 2 at 11 for pid 27, osid 3736034 NOTE: Assigning number (2,7) to disk () NOTE: Assigning number (2,8) to disk () GMON querying group 2 at 12 for pid 27, osid 3736034 NOTE: cache dismounting (clean) group 2/0x79E6D861 (DATA) NOTE: messaging CKPT to quiesce pins Unix process pid: 3736034, image: oracle@hbbz01 (TNS V1-V3) NOTE: dbwr not being msg'd to dismount NOTE: lgwr not being msg'd to dismount NOTE: cache dismounted group 2/0x79E6D861 (DATA) NOTE: cache ending mount (fail) of group DATA number=2 incarn=0x79e6d861 NOTE: cache deleting context for group DATA 2/0x79e6d861 GMON dismounting group 2 at 13 for pid 27, osid 3736034 NOTE: Disk DATA_0000 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0001 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0002 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0003 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0004 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0005 in mode 0x7f marked for de-assignment NOTE: Disk DATA_0006 in mode 0x7f marked for de-assignment NOTE: Disk in mode 0x7f marked for de-assignment NOTE: Disk in mode 0x7f marked for de-assignment NOTE: Disk DATA_0009 in mode 0x7f marked for de-assignment ERROR: diskgroup DATA was not mounted ORA-15032: not all alterations performed ORA-15040: diskgroup is incomplete ORA-15042: ASM disk "8" is missing from group number "2" ORA-15042: ASM disk "7" is missing from group number "2" ORA-15038: disk '/dev/rhdisk37' mismatch on 'Time Stamp' with target disk group [2129689239] [2062898314] ERROR: alter diskgroup data mount
怀疑把报错这个磁盘组的rhdisk37加入到另外一套rac的asm中了(也就是说两套asm使用了同一块磁盘),aix操作系统层面分析确认
---对asm扩容的机器上 # lscfg -vpl hdisk15 hdisk15 U78C5.001.DQD076A-P2-C4-T1-W200C00A098BC9A83-L0 MPIO NetApp FCP Default PCM Disk Manufacturer................NETAPP Machine Type and Model......LUN C-Mode ROS Level and ID............9000 Serial Number...............80DYz]L/OpCA Device Specific.(Z0)........FAS8020 PLATFORM SPECIFIC Name: disk Node: disk Device Type: block ---磁盘组dismount的机器上 # lscfg -vpl hdisk37 hdisk37 U5802.001.9K87776-P1-C1-T1-W200500A098BC9A83-L0 MPIO NetApp FCP Default PCM Disk Manufacturer................NETAPP Machine Type and Model......LUN C-Mode ROS Level and ID............9000 Serial Number...............80DYz]L/OpCA Device Specific.(Z0)........FAS8020 PLATFORM SPECIFIC Name: disk Node: disk Device Type: block
通过lscfg 命令确认两套rac使用了同一块盘导致一个磁盘组异常,在新加的机器上查询确认新盘被破坏情况(新加入的磁盘由于reblance操作,已经被写入了380G左右数据[也就意味着这个磁盘在老磁盘组中最少会丢失380G数据]
对于这种情况,dismount磁盘组是外部冗余不可能直接mount起来,只能通过以前处理的类似方法:
asm disk header 彻底损坏恢复
asm磁盘加入vg恢复
asm磁盘dd破坏恢复
asm disk 磁盘部分被清空恢复
再一例asm disk被误加入vg并且扩容lv恢复
fdisk分区导致asm disk破坏数据库恢复
再一起asm disk被格式化成ext3文件系统故障恢复
oracle asm disk格式化恢复—格式化为ext4文件系统
oracle asm disk格式化恢复—格式化为ntfs文件系统
ORA-15063: ASM discovered an insufficient number of disks for diskgroup 恢复
通过底层处理恢复出来没有覆盖的数据块中数据

再使用dul恢复出来其中数据,完成这次故障的核心数据恢复
win系统删除oracle数据文件恢复
有客户联系我们,说win平台下的数据库,在由于空间紧张,在关闭数据库的情况下删除的两个数据文件,导致数据库无法正常访问很多业务表,需要对其进行恢复,查看alert日志发现大概操作,删除文件之后,启动数据库失败
Completed: alter database mount exclusive alter database open Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_4060.trc: ORA-01157: cannot identify/lock data file 6 - see DBWR trace file ORA-01110: data file 6: 'D:\DATASPACE\XXXXX.DBF' ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_4060.trc: ORA-01157: cannot identify/lock data file 38 - see DBWR trace file ORA-01110: data file 38: 'D:\DATASPACE\XXXXX24.DBF' ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Tue Jun 27 14:50:28 2023 Checker run found 2 new persistent data failures
人工创建被删除文件,启动库报ORA-27047,OSD-04006等错误
Tue Jun 27 16:45:10 2023 ALTER DATABASE OPEN Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_5456.trc: ORA-01157: cannot identify/lock data file 6 - see DBWR trace file ORA-01110: data file 6: 'D:\DATASPACE\XXXXX.DBF' ORA-27047: unable to read the header block of file OSD-04006: ReadFile() 失败, 无法读取文件 O/S-Error: (OS 38) 已到文件结尾。
offline相关数据文件,启动库成功,但是job开始报错
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_5456.trc: ORA-01157: cannot identify/lock data file 38 - see DBWR trace file ORA-01110: data file 38: 'D:\DATASPACE\XXXXX24.DBF' ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_648.trc: ORA-01157: 无法标识/锁定数据文件 6 - 请参阅 DBWR 跟踪文件 ORA-01110: 数据文件 6: 'D:\DATASPACE\XXXXX.DBF' ORA-1157 signalled during: ALTER DATABASE OPEN... Tue Jun 27 16:48:43 2023 alter database datafile 'D:\DATASPACE\XXXXX.DBF' offline drop Completed: alter database datafile 'D:\DATASPACE\XXXXX.DBF' offline drop Tue Jun 27 16:49:08 2023 alter database open Tue Jun 27 16:49:08 2023 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_dbw0_5456.trc: ORA-01157: cannot identify/lock data file 38 - see DBWR trace file ORA-01110: data file 38: 'D:\DATASPACE\XXXXX24.DBF' ORA-27041: unable to open file OSD-04002: 无法打开文件 O/S-Error: (OS 2) 系统找不到指定的文件。 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_3976.trc: ORA-01157: 无法标识/锁定数据文件 38 - 请参阅 DBWR 跟踪文件 ORA-01110: 数据文件 38: 'D:\DATASPACE\XXXXX24.DBF' ORA-1157 signalled during: alter database open... Tue Jun 27 16:49:08 2023 Checker run found 1 new persistent data failures Tue Jun 27 16:49:28 2023 alter database datafile 'D:\DATASPACE\XXXXX24.DBF' offline drop Completed: alter database datafile 'D:\DATASPACE\XXXXX24.DBF' offline drop alter database open Tue Jun 27 16:49:37 2023 Thread 1 opened at log sequence 145929 Current log# 3 seq# 145929 mem# 0: D:\APP\ADMINISTRATOR\ORADATA\ORCL\REDO03.LOG Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Tue Jun 27 16:49:37 2023 SMON: enabling cache recovery Successfully onlined Undo Tablespace 2. Verifying file header compatibility for 11g tablespace encryption.. Verifying 11g file header compatibility for tablespace encryption completed SMON: enabling tx recovery Database Characterset is ZHS16GBK No Resource Manager plan active Tue Jun 27 16:49:39 2023 replication_dependency_tracking turned off (no async multimaster replication found) Starting background process QMNC Tue Jun 27 16:49:40 2023 QMNC started with pid=21, OS id=6096 Completed: alter database open Tue Jun 27 16:49:43 2023 db_recovery_file_dest_size of 4096 MB is 0.00% used. This is a user-specified limit on the amount of space that will be used by this database for recovery-related files, and does not reflect the amount of space available in the underlying filesystem or ASM diskgroup. Tue Jun 27 16:49:44 2023 Starting background process CJQ0 Tue Jun 27 16:49:44 2023 CJQ0 started with pid=142, OS id=6036 Tue Jun 27 16:49:48 2023 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_j007_5184.trc: ORA-12012: 自动执行作业 64 出错 ORA-00376: 此时无法读取文件 6 ORA-01110: 数据文件 6: 'D:\DATASPACE\XXXXX.DBF' ORA-06512: 在 "XIFENFEI.XXXXXXXX", line 2897 ORA-06512: 在 line 1 Tue Jun 27 16:51:52 2023 Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_j000_2548.trc: ORA-12012: 自动执行作业 64 出错 ORA-00376: 此时无法读取文件 6 ORA-01110: 数据文件 6: 'D:\DATASPACE\XXXXX.DBF' ORA-06512: 在 "XIFENFEI.XXXXXXXX", line 2897 ORA-06512: 在 line 1 Tue Jun 27 16:54:44 2023 Starting background process SMCO Tue Jun 27 16:54:44 2023 SMCO started with pid=42, OS id=908 Tue Jun 27 16:55:52 2023
接手现场之后,关闭数据库,使用操作系统层面反删除工具进行扫描恢复,发现其中一个文件(另外一个文件os层面无法恢复)
通过工具检测恢复出来的数据文件,损坏的几个block是文件头部不涉及业务数据,运气不错

另外一个数据文件,从os层面无法恢复,对于这种情况,只能基于底层的block层面进行恢复(恢复没有覆盖的block)

参考类似恢复案例:
win文件系统损坏oracle恢复
Oracle 数据文件大小为0kb或者文件丢失恢复
分享运气超级好的一次drop tablespace 数据恢复
恢复出来的两个数据文件,结合该编辑的其他数据文件通过dul工具恢复其中数据,最大程度抢救客户数据,减少损失.
ORA-01122 ORA-01200故障处理
由于某种原因客户的数据库启动报ORA-01122 ORA-01200错误
让客户把system01.dbf文件发给我进行分析,发现system01.dbf文件大于32G(在8k的blocksize库中,默认情况system01.dbf文件不会超过32G),这个明显异常

检测坏块情况发现4096000之后的block全部为全0块

通过bbed分析文件头记录文件大小

通过bbed修改合适的值,并且把文件截取到适当大小,提供system文件给客户,直接启动库成功,实现数据库完美恢复

通过设置文件头大小和截断合适大小实现本次数据库恢复,以前有过类似恢复:
bbed处理ORA-01200故障