Oracle 19c故障恢复

有客户找到我们,他们的oracle 19c数据库由于异常断电,导致启动异常,经过一系列恢复之后,依旧无法解决问题,请求我们给予支持.通过我们的Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check),获取数据库当前信息如下:
数据库版本为19C并且安装了19.5.0.0.191015 (30125133)补丁
20200310220453
20200310220748


数据库使用pdb
20200310220610

数据库启动成功后,一会就crash掉

2020-03-10T01:44:41.018032+08:00
Pluggable database RACBAK opened read write
2020-03-10T01:44:41.018996+08:00
Pluggable database RAC opened read write
2020-03-10T01:44:51.244050+08:00
Completed: ALTER PLUGGABLE DATABASE ALL OPEN
Starting background process CJQ0
Completed: ALTER DATABASE OPEN
2020-03-10T01:44:51.317085+08:00
CJQ0 started with pid=224, OS id=32581 
2020-03-10T01:44:56.067043+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j001_32588.trc  (incident=1095281) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [27733], [27754], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095281/XFF_j001_32588_i1095281.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:44:56.073112+08:00
RAC(4):*****************************************************************
RAC(4):An internal routine has requested a dump of selected redo.
RAC(4):This usually happens following a specific internal error, when
RAC(4):analysis of the redo logs will help Oracle Support with the
RAC(4):diagnosis.
RAC(4):It is recommended that you retain all the redo logs generated (by
RAC(4):all the instances) during the past 12 hours, in case additional
RAC(4):redo dumps are required to help with the diagnosis.
RAC(4):*****************************************************************
2020-03-10T01:44:56.079228+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j002_32590.trc  (incident=1095289) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [2633], [2638], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095289/XFF_j002_32590_i1095289.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:44:56.085068+08:00
RAC(4):*****************************************************************
RAC(4):An internal routine has requested a dump of selected redo.
RAC(4):This usually happens following a specific internal error, when
RAC(4):analysis of the redo logs will help Oracle Support with the
RAC(4):diagnosis.
RAC(4):It is recommended that you retain all the redo logs generated (by
RAC(4):all the instances) during the past 12 hours, in case additional
RAC(4):redo dumps are required to help with the diagnosis.
RAC(4):*****************************************************************
2020-03-10T01:44:56.115765+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j004_32594.trc  (incident=1095305) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [63532], [63537], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095305/XFF_j004_32594_i1095305.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:46:48.202213+08:00
RAC(4):Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
RAC(4):  Mem# 0: /opt/oracle/oradata/XFF/redo02.log
RAC(4):Block recovery completed at rba 0.0.0, scn 0x0000000d3675e48e
RAC(4):DDE: Problem Key 'ORA 600 [4193]' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes
2020-03-10T01:46:48.384040+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_clmn_31741.trc:
ORA-00600: internal error code, arguments: [4193], [27733], [27754], [], [], [], [], [], [], [], [], []
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_clmn_31741.trc  (incident=1093505) (PDBNAME=CDB$ROOT):
ORA-501 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1093505/XFF_clmn_31741_i1093505.trc
2020-03-10T01:46:49.264624+08:00
USER (ospid: 31741): terminating the instance due to ORA error 501
2020-03-10T01:46:49.280664+08:00
System state dump requested by (instance=1, osid=31741 (CLMN)), summary=[abnormal instance termination].
System State dumped to trace file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_diag_31759.trc
2020-03-10T01:46:53.156926+08:00
ORA-00501: CLMN process terminated with error
2020-03-10T01:46:53.157103+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_diag_31759.trc:
ORA-00501: CLMN process terminated with error
2020-03-10T01:46:53.157211+08:00
Dumping diagnostic data in directory=[cdmp_20200310014649], requested by (instance=1, osid=31741 (CLMN)), 
summary=[abnormal instance termination].

通过报错信息判断,数据库open之后(特别是pdb 4 open之后),开始报ORA-600 4193错误.然后由于CLMN进程异常,最后数据库crash.对于这类故障,因为使用的pdb,而且是由于pdb的undo异常导致数据库启动之后crash,可以通过对于pdb进行特殊处理,从而实现数据库启动之后不再crash.

发表在 ORACLE 19C, Oracle备份恢复 | 标签为 , , , | 留下评论

oracle文件被删除且部分被覆盖恢复案例

有客户数据库异常,让我们对其进行分析,判断是否可以恢复,让客户通过Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check)收集数据库信息,发现有三个数据文件异常
20200306223254


通过和客户确认,大概情况是这样的:由于/opt目录满了,客户把/opt/oracle整体迁移到/home目录中,然后通过link目录的方式实现迁移,但是在迁移之前把/home/oracle中的所有数据文件给rm掉了,然后把/opt中的数据拷贝到/home/中,在启动数据库的时候提示/home/oracle/JXWR.dbf文件丢失,然后又人工创建了一个该文件,从而出现了上述的三个文件异常(其实删除的数据库文件有十几个,涉及该库的有三个,客户只要恢复JXWR表空间数据即可).对于这种情况,有可能有覆盖的风险,让客户提供空间对现有/home 分区进行镜像,通过工具分析镜像文件
20200305142707
20200305142745

发现需要恢复文件大小/时间均不对,查看内容发现是oracle的审计trace,证明该文件对应的位置已经有覆盖,对于这样的情况,无法从os层面反删除进行恢复,考虑通过oracle碎片层面进行处理,对其分析发现大量block依旧存在(情况有点复杂,因为该目录涉及多个库,通过分析确认相关段为该数据库文件)
20200306230009

检查重组出来的数据文件效果
20200306224628


检查效果比较乐观,因为根据这样的情况,丢失15%左右的block算是非常理想的效果,然后通过oracle dul恢复客户需要的数据1

完成数据库恢复任务.
这次的恢复效果不是太好主要就是由于客户删除文件之后,对被删除文件所在分区进行了大量写操作导致不少数据库block被覆盖,最终只能抱着试试看的态度最大限度恢复,属于比较侥幸的恢复成功,再次提醒各位:在数据被误删除之后,应该先保护现场(不要对其分区进行写操作),覆盖的越少,恢复的效果越好.

发表在 非常规恢复 | 评论关闭

数字后缀名加密数据库恢复

一个oracle dmp文件被加密破坏的case,加密提示如下
20200306192501


20200306191213

通过工具分析发现该文件前面1M被破坏
20200306191553

通过我们工具对头部损坏的1M数据进行特殊处理,数据直接使用imp命令导入
20200306191709
如果你有各种被类似病毒加密的数据库(oracle,sql server,mysql),我们可以提供专业的恢复支持,实现不给黑客交钱的情况下,数据几乎完美恢复
Tel/微信:13429648788    Q Q:107644445 QQ咨询惜分飞    E-Mail:dba@xifenfei.com提供专业的恢复服务.

发表在 勒索恢复, 非常规恢复 | 标签为 , , , | 评论关闭

.horseleader加密数据库恢复

被病毒加密的.horseleader后缀名的文件
20200304182923


通过分析,我们该病毒只是破坏了部分数据,我们可以恢复其中的绝大多数数据
20200304182504
20200304182522


通过底层进行处理,跳过损坏部分恢复出来没有破坏的数据
20200304182551


对于该类型加密,我们可以对sql server、mysql、oracle恢复出来绝大多数数据,通过不向黑客交赎金的方式,实现绝绝大部分业务数据恢复.

发表在 勒索恢复 | 标签为 , , | 评论关闭

sql server 数据库 mdf 0kb 恢复

以前恢复过oracle数据库dbf文件大小变为0kb的case(Oracle 数据文件大小为0kb或者文件丢失恢复),这次遇到一个客户由于主机重启导致sql server 数据库的mdf文件大小变为0kb,客户自己通过反删除软件无法正常恢复,我们通过磁盘底层block对其进行处理,实现大部分数据恢复(由于客户的一些操作导致部分数据覆盖)
该磁盘分区有多个mdf文件(多个sql server库)
20200303190055
通过底层block技术发现大量没有覆盖的该文件的block
20200303190141
20200303190332
通过block技术恢复出来mdf文件之后,然后恢复出来表数据情况
20200303190617


如果您遇到sql server 数据库由于某种原因导致mdf文件大小变为0kb,请第一时间保护现场,不要进行任何写操作,我们可以最大限度对其进行恢复,尽可能减少您的损失
Tel/微信:13429648788    Q Q:107644445 QQ咨询惜分飞    E-Mail:dba@xifenfei.com提供专业的恢复服务.

发表在 SQL Server恢复 | 标签为 , | 评论关闭