标签云
asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2131 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (102)
- 数据库 (1,670)
- DB2 (22)
- MySQL (73)
- Oracle (1,532)
- Data Guard (52)
- EXADATA (8)
- GoldenGate (21)
- ORA-xxxxx (159)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (14)
- ORACLE 21C (3)
- Oracle 23ai (7)
- Oracle ASM (65)
- Oracle Bug (8)
- Oracle RAC (52)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (28)
- Oracle备份恢复 (560)
- Oracle安装升级 (91)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (78)
- PostgreSQL (18)
- PostgreSQL恢复 (6)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (37)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (20)
-
最近发表
- ORA-600 krse_arc_complete.4
- Oracle 19c 202410补丁(RUs+OJVM)
- ntfs MFT损坏(ntfs文件系统故障)导致oracle异常恢复
- .mkp扩展名oracle数据文件加密恢复
- 清空redo,导致ORA-27048: skgfifi: file header information is invalid
- A_H_README_TO_RECOVER勒索恢复
- 通过alert日志分析客户自行对一个数据库恢复的来龙去脉和点评
- ORA-12514: TNS: 监听进程不能解析在连接描述符中给出的SERVICE_NAME
- ORA-01092 ORA-00604 ORA-01558故障处理
- ORA-65088: database open should be retried
- Oracle 19c异常恢复—ORA-01209/ORA-65088
- ORA-600 16703故障再现
- 数据库启动报ORA-27102 OSD-00026 O/S-Error: (OS 1455)
- .[metro777@cock.li].Elbie勒索病毒加密数据库恢复
- 应用连接错误,初始化mysql数据库恢复
- RAC默认服务配置优先节点
- Oracle 19c RAC 替换私网操作
- 监听报TNS-12541 TNS-12560 TNS-00511错误
- drop tablespace xxx including contents恢复
- Linux 8 修改网卡名称
标签归档:kgegpa
存储宕机导致Oracle异常故障处理
存储突然掉线,导致数据库crash,报大量ORA-00206 ORA-00202 ORA-15081以及Linux-x86_64 Error: 5: Input/output error之类的错误
Sun Jul 21 20:00:11 2024 Thread 1 advanced to log sequence 1594398 (LGWR switch) Current log# 5 seq# 1594398 mem# 0: +DATA/xff/onlinelog/group_5.412.906718739 Sun Jul 21 20:53:17 2024 WARNING: Write Failed. group:2 disk:0 AU:506916 offset:49152 size:16384 Sun Jul 21 20:53:17 2024 WARNING: Read Failed. group:2 disk:2 AU:506931 offset:49152 size:16384 WARNING: failed to read mirror side 1 of virtual extent 4 logical extent 0 of file 415 in group [2.34109396] from disk ORACLE_DATA_0002 allocation unit 506931 reason error; if possible, will try another mirror side Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ckpt_42142.trc: ORA-15080: 与磁盘的同步 I/O 操作失败 ORA-27061: 异步 I/O 等待失败 Linux-x86_64 Error: 5: Input/output error Additional information: -1 Additional information: 16384 WARNING: failed to write mirror side 1 of virtual extent 0 logical extent 0 of file 415 in group 2 on disk 0 allocation unit 506916 Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ckpt_42142.trc: ORA-00206: 写入控制文件时出错 (块 3, # 块 1) ORA-00202: 控制文件: ''+DATA/xff/controlfile/current.415.906718737'' ORA-15081: 无法将 I/O 操作提交到磁盘 ORA-15081: 无法将 I/O 操作提交到磁盘 Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ckpt_42142.trc: ORA-00221: 写入控制文件时出错 ORA-00206: 写入控制文件时出错 (块 3, # 块 1) ORA-00202: 控制文件: ''+DATA/xff/controlfile/current.415.906718737'' ORA-15081: 无法将 I/O 操作提交到磁盘 ORA-15081: 无法将 I/O 操作提交到磁盘 CKPT (ospid: 42142): terminating the instance due to error 221 Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_lmon_42087.trc: ORA-00202: 控制文件: ''+DATA/xff/controlfile/current.415.906718737'' ORA-15081: 无法将 I/O 操作提交到磁盘 ORA-27072: 文件 I/O 错误 Linux-x86_64 Error: 5: Input/output error Additional information: 4 Additional information: 1038194784 Additional information: -1 Sun Jul 21 20:53:19 2024 ORA-1092 : opitsk aborting process Sun Jul 21 20:53:24 2024 ORA-1092 : opitsk aborting process Sun Jul 21 20:53:24 2024 License high water mark = 59 Sun Jul 21 20:53:28 2024 Instance terminated by CKPT, pid = 42142 USER (ospid: 64660): terminating the instance Instance terminated by USER, pid = 64660
存储恢复之后启动数据库报ORA-600 2131错误
Mon Jul 22 09:10:04 2024 ALTER DATABASE MOUNT This instance was first to mount Mon Jul 22 09:10:04 2024 Sweep [inc][490008]: completed Sweep [inc2][490008]: completed NOTE: Loaded library: System SUCCESS: diskgroup ORACLE_DATA was mounted NOTE: dependency between database rac and diskgroup resource ora.ORACLE_DATA.dg is established Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ora_14301.trc (incident=492409): ORA-00600: ??????, ??: [2131], [33], [32], [], [], [], [], [], [], [], [], [] Incident details in: /users/oracle/app/db/diag/rdbms/xff/xff1/incident/incdir_492409/xff1_ora_14301_i492409.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ORA-600 signalled during: ALTER DATABASE MOUNT...
客户尝试重建ctl进行恢复,结果由于分析不正确,导致在重建ctl的时候,遗漏了3个数据文件,并且在屏蔽一致性的情况下,强制resetlogs操作,结果数据库没有被正常打开,而是报ORA-600 2662错误
alter database open resetlogs RESETLOGS is being done without consistancy checks. This may result in a corrupted database. The database should be recreated. RESETLOGS after incomplete recovery UNTIL CHANGE 9965567206652 Clearing online redo logfile 1 +DATA/xff/onlinelog/group_1.414.906718739 Clearing online log 1 of thread 1 sequence number 0 Clearing online redo logfile 1 complete Clearing online redo logfile 2 +DATA/xff/onlinelog/group_2.413.906718739 Clearing online log 2 of thread 1 sequence number 0 Clearing online redo logfile 2 complete Clearing online redo logfile 5 +DATA/xff/onlinelog/group_5.412.906718739 Clearing online log 5 of thread 1 sequence number 0 Clearing online redo logfile 5 complete Expanded controlfile section 2 from 1 to 63 records The number of logical blocks in section 2 remains the same Expanded controlfile section 1 from 4 to 66 records Requested to grow by 62 records; added 32 blocks of records Expanded controlfile section 30 from 1 to 63 records The number of logical blocks in section 30 remains the same Expanded controlfile section 29 from 1 to 63 records The number of logical blocks in section 29 remains the same Control file has been expanded to support 63 threads Mon Jul 22 23:04:07 2024 Redo thread 2 enabled by open resetlogs or standby activation Online log +DATA/xff/onlinelog/group_1.414.906718739: Thread 1 Group 1 was previously cleared Online log +DATA/xff/onlinelog/group_2.413.906718739: Thread 1 Group 2 was previously cleared Online log +DATA/xff/onlinelog/group_3.501.1175036643: Thread 2 Group 3 was previously cleared Online log +DATA/xff/onlinelog/group_4.502.1175036645: Thread 2 Group 4 was previously cleared Online log +DATA/xff/onlinelog/group_5.412.906718739: Thread 1 Group 5 was previously cleared Mon Jul 22 23:04:08 2024 Setting recovery target incarnation to 2 Initializing SCN for created control file Database SCN compatibility initialized to 3 Warning - High Database SCN: Current SCN value is 9965567206655, threshold SCN value is 0 If you have not previously reported this warning on this database, please notify Oracle Support so that additional diagnosis can be performed. Mon Jul 22 23:04:09 2024 Assigning activation ID 2763017873 (0xa4b04e91) Thread 1 opened at log sequence 1 Current log# 1 seq# 1 mem# 0: +DATA/xff/onlinelog/group_1.414.906718739 Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Mon Jul 22 23:04:10 2024 SMON: enabling cache recovery Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ora_64210.trc (incident=624374): ORA-00600: 内部错误代码, 参数: [2662], [2320], [1243079939], [2320], [1243211805], [12583040], [], [], [], [], [], [] Incident details in: /users/oracle/app/db/diag/rdbms/xff/xff1/incident/incdir_624374/xff1_ora_64210_i624374.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ora_64210.trc: ORA-00600: 内部错误代码, 参数: [2662], [2320], [1243079939], [2320], [1243211805], [12583040], [], [], [], [], [], [] Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_ora_64210.trc: ORA-00600: 内部错误代码, 参数: [2662], [2320], [1243079939], [2320], [1243211805], [12583040], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 64210): terminating the instance due to error 600 Instance terminated by USER, pid = 64210 ORA-1092 signalled during: alter database open resetlogs...
操作到这里,后续问题就比较麻烦了,因为在asm磁盘组中数据文件重建ctl的时候遗漏3个并且还被resetlogs操作过,导致这三个文件的resetlogs scn和其他数据文件不一致,对于这个问题,解决办法通过Oracle Recovery Tools工具或者bbed修改相关resetlogs scn,然后重建ctl
SQL> @rectl.sql Control file created. SQL> RECOVER DATABASE; Media recovery complete
然后解决之前数据库启动报ORA-600 2662问题,通过修改数据库scn进行解决,可以使用Patch_SCN工具进行快速解决,然后open数据库成功
SQL> ALTER DATABASE OPEN; Database altered.
但是查看alert日志数据库报大量ORA-600 4194、ORA-01595和Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0xC21D511] [PC:0x97F4EFA, kgegpa()+40]之类错误
Wed Jul 24 15:24:21 2024 alter database open Beginning crash recovery of 1 threads parallel recovery started with 32 processes Started redo scan Completed redo scan read 0 KB redo, 0 data blocks need recovery ………… Database Characterset is ZHS16GBK No Resource Manager plan active Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_smon_40279.trc (incident=777938): ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], [] Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. replication_dependency_tracking turned off (no async multimaster replication found) Starting background process QMNC Wed Jul 24 15:24:40 2024 QMNC started with pid=79, OS id=54632 Block recovery from logseq 2, block 74 to scn 9965587206835 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/redo02 LOGSTDBY: Validating controlfile with logical metadata Wed Jul 24 15:24:40 2024 Block recovery stopped at EOT rba 2.82.16 Block recovery completed at rba 2.82.16, scn 2320.1263080114 Block recovery from logseq 2, block 74 to scn 9965587206833 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/redo02 Block recovery completed at rba 2.82.16, scn 2320.1263080114 Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_smon_40279.trc: ORA-01595: 释放区 (4) 回退段 (20) 时出错 ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], [] LOGSTDBY: Validation complete Wed Jul 24 15:24:41 2024 Sweep [inc][777938]: completed Sweep [inc2][777938]: completed Wed Jul 24 15:24:41 2024 Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_q001_54657.trc (incident=778362): ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], [] Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Starting background process SMCO Wed Jul 24 15:24:42 2024 SMCO started with pid=83, OS id=54691 Block recovery from logseq 2, block 74 to scn 9965587206835 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/redo02 Block recovery completed at rba 2.82.16, scn 2320.1263080118 Block recovery from logseq 2, block 74 to scn 9965587206838 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/redo02 Block recovery completed at rba 2.83.16, scn 2320.1263080119 Error 600 in kwqmnpartition(), aborting txn Errors in file /users/oracle/app/db/diag/rdbms/xff/xff1/trace/xff1_q001_54657.trc (incident=778363): ORA-25319: 队列表重新分区已中止 Completed: alter database open Block recovery from logseq 2, block 74 to scn 9965587206835 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/rac/onlinelog/redo02 Block recovery completed at rba 2.82.16, scn 2320.1263080118 Block recovery from logseq 2, block 74 to scn 9965587207538 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/rac/onlinelog/redo02 Block recovery completed at rba 2.1097.16, scn 2320.1263080819 Errors in file /users/oracle/app/db/diag/rdbms/rac/rac1/trace/rac1_cjq0_55657.trc (incident=778427): ORA-00600: 内部错误代码, 参数: [600], [ORA-00600: 内部错误代码, 参数: [4194], [], [], [], [], [], [], [], [], [], [], []], [], [], [], [], [], [], [], [], [], [] Incident details in: /users/oracle/app/db/diag/rdbms/xff/xff1/incident/incdir_778427/xff1_cjq0_55657_i778427.trc Exception [type:SIGSEGV, Address not mapped to object][ADDR:0xC21D511][PC:0x97F4EFA, kgegpa()+40][flags: 0x0, count: 1] Exception [type:SIGSEGV, Address not mapped to object][ADDR:0xC21D511][PC:0x97F396E, kgebse()+776][flags: 0x2, count: 2] Exception [type:SIGSEGV, Address not mapped to object][ADDR:0xC21D511][PC:0x97F396E, kgebse()+776][flags: 0x2, count: 2]
从报错分析是由于undo异常导致,处理异常undo回滚段之后,数据库open正常,安排逻辑迁移数据,完成本次恢复
发表在 Oracle备份恢复
标签为 kgegpa, ORA-00206 ORA-00202 ORA-15081, ORA-600 2131, ORA-600 2662, ORA-600 4194, 存储宕机oracle处理
评论关闭
存储重启,oracle无法启动故障处理
有客户由于机房要停电,正常关闭两个节点数据库,通过数据库alert日志均可看到类似如下记录,证明数据库确实是正常shutdown immediate
然后关闭存储,启动存储之后发现数据库无法正常启动(数据scn不一致).相关信息如下:
最初报ORA-214错
that ORACLE_BASE be set in the environment Wed Jul 06 00:50:02 2022 ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=10.10.10.10)(PORT=1521))' SCOPE=MEMORY SID='xffdb2'; ALTER DATABASE MOUNT /* db agent *//* {1:42392:203} */ This instance was first to mount NOTE: Loaded library: System SUCCESS: diskgroup DATA1 was mounted SUCCESS: diskgroup DATA2 was mounted ORA-214 signalled during: ALTER DATABASE MOUNT /* db agent *//* {1:42392:203} */... NOTE: dependency between database xffdb and diskgroup resource ora.DATA1.dg is established NOTE: dependency between database xffdb and diskgroup resource ora.DATA2.dg is established
提示ctl不存在,通过处理之后报ORA-600 2131错误
Wed Jul 06 01:55:45 2022 ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=10.10.10.10)(PORT=1521))' SCOPE=MEMORY SID='xffdb2'; ALTER DATABASE MOUNT /* db agent *//* {1:42392:663} */ This instance was first to mount NOTE: Loaded library: System SUCCESS: diskgroup DATA1 was mounted SUCCESS: diskgroup DATA2 was mounted NOTE: dependency between database xffdb and diskgroup resource ora.DATA1.dg is established NOTE: dependency between database xffdb and diskgroup resource ora.DATA2.dg is established Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb2/trace/xffdb2_ora_47746.trc (incident=576488): ORA-00600: internal error code, arguments: [2131], [33], [32], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/xffdb/xffdb2/incident/incdir_576488/xffdb2_ora_47746_i576488.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ORA-600 signalled during: ALTER DATABASE MOUNT /* db agent *//* {1:42392:663} */...
重建控制文件后恢复报错
Parallel Media Recovery started with 127 slaves ORA-279 signalled during: ALTER DATABASE RECOVER database using BACKUP CONTROLFILE ... Wed Jul 06 02:41:04 2022 ALTER DATABASE RECOVER LOGFILE '+DATA3/xffdb/archivelog/2022_07_05/thread_2_seq_40889.18030.1109269215' Media Recovery Log +DATA3/xffdb/archivelog/2022_07_05/thread_2_seq_40889.18030.1109269215 Wed Jul 06 02:41:04 2022 Errors with log +DATA3/xffdb/archivelog/2022_07_05/thread_2_seq_40889.18030.1109269215 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_pr00_96503.trc: ORA-00325: archived log for thread 1, wrong thread # 2 in header ORA-00334: archived log: '+DATA3/xffdb/archivelog/2022_07_05/thread_2_seq_40889.18030.1109269215' ORA-325 signalled during: ALTER DATABASE RECOVER LOGFILE '+DATA3/thread_2_seq_40889.18030.1109269215' ... ALTER DATABASE RECOVER CANCEL Media Recovery Canceled Completed: ALTER DATABASE RECOVER CANCEL ………… Wed Jul 06 02:22:25 2022 ALTER DATABASE RECOVER DATABASE Media Recovery Start started logmerger process Only allocated 127 recovery slaves (requested 128) Parallel Media Recovery started with 127 slaves Wed Jul 06 02:22:28 2022 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_pr00_77044.trc: ORA-00313: open failed for members of log group 7 of thread 1 Media Recovery failed with error 313 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_pr00_77044.trc: ORA-00283: recovery session canceled due to errors ORA-00313: open failed for members of log group 7 of thread 1 Wed Jul 06 02:22:28 2022 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_m000_77318.trc: ORA-00322: log 4 of thread 2 is not current copy ORA-00312: online log 4 thread 2: '+DATA3/xffdb/onlinelog/group_4.16148.1107795635' Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_m000_77318.trc: ORA-00322: log 7 of thread 1 is not current copy ORA-00312: online log 7 thread 1: '+DATA3/xffdb/onlinelog/group_7.18959.1107796013' Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_m000_77318.trc: ORA-00314: log 9 of thread 1, expected sequence# 133495 doesn't match 133490 ORA-00312: online log 9 thread 1: '+DATA3/xffdb/onlinelog/group_9.3142.1107796071' Checker run found 208 new persistent data failures ORA-10877 signalled during: ALTER DATABASE RECOVER DATABASE ... ………… Only allocated 127 recovery slaves (requested 128) Parallel Media Recovery started with 127 slaves ORA-279 signalled during: ALTER DATABASE RECOVER database using backup controlfile ... Wed Jul 06 06:15:26 2022 ALTER DATABASE RECOVER LOGFILE '+DATA3/xffdb/onlinelog/group_4.16442.1107795653' Media Recovery Log +DATA3/xffdb/onlinelog/group_4.16442.1107795653 ORA-279 signalled during: ALTER DATABASE RECOVER LOGFILE '+DATA3/xffdb/onlinelog/group_4.16442.1107795653' ... Wed Jul 06 06:15:43 2022 ALTER DATABASE RECOVER LOGFILE '+DATA3/xffdb/onlinelog/group_7.18959.1107796013' Media Recovery Log +DATA3/xffdb/onlinelog/group_7.18959.1107796013 Wed Jul 06 06:15:50 2022 Errors with log +DATA3/xffdb/onlinelog/group_7.18959.1107796013 Wed Jul 06 06:15:50 2022 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_pr29_306479.trc (incident=961030): ORA-00600: internal error code, arguments: [6102], [13], [17], [], [], [], [], [], [], [], [], [] Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Wed Jul 06 06:15:54 2022 Sweep [inc][961030]: completed Sweep [inc2][961030]: completed Slave exiting with ORA-10562 exception Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_pr29_306479.trc: ORA-10562: Error occurred while applying redo to data block (file# 159, block# 3591756) ORA-10564: tablespace LIS ORA-01110: data file 159: '+DATA1/xffdb/datafile/lis.379.1080445903' ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 138875 ORA-00600: internal error code, arguments: [6102], [13], [17], [], [], [], [], [], [], [], [], [] Wed Jul 06 06:15:59 2022 Recovery Slave PR29 previously exited with exception 10562
基于上述情况,很可能是由于存储重启之后,cache或者某些数据没有写入到数据文件和redo中,数据库重启之后redo不是最新的[ORA-00322错误可以证明,],数据文件也需要进行恢复(不是数据库正常关闭之后该有的情况),而且redo和数据文件还不一致[ORA-00600 6102可以证明],对于类似这样的情况,只能尝试强制打开数据库,报ORA-600 2663
SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [2663], [1393], [4159455578], [1393], [4160374753], [], [], [], [], [], [], [] Process ID: 357910 Session ID: 1585 Serial number: 7
Wed Jul 06 06:57:25 2022 SMON: enabling cache recovery Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_ora_357910.trc (incident=1056360): ORA-00600: internal error code, arguments: [2663], [1393],[4159455578],[1393],[4160374753],[], [], [], [] Redo thread 2 internally disabled at seq 1 (CKPT) Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_arc3_360348.trc: ORA-00600: internal error code, arguments: [ORA_NPI_ERROR],[600], [ORA-00600: internal error code, arguments: [kffbAddBlk04] Unable to create archive log file '+DATA3' ARC3: Error 19504 Creating archive log file to '+DATA3' ARCH: Archival error occurred on a closed thread. Archiver continuing ORACLE Instance xffdb1 - Archival Error. Archiver continuing. ARCH: Archival error occurred on a closed thread. Archiver continuing ORACLE Instance xffdb1 - Archival Error. Archiver continuing. Wed Jul 06 06:57:34 2022 Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/incident/incdir_1056360/xffdb1_ora_357910_i1056360.trc: ORA-00339: archived log does not contain any redo ORA-00334: archived log: '+DATA1/xffdb/onlinelog/group_4.424.1109314453' ORA-00600: internal error code, arguments: [2663], [1393], [4159455578], [1393], [4160374753], [], [], Wed Jul 06 06:57:34 2022 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_ora_357910.trc: ORA-00600: internal error code, arguments: [2663], [1393], [4159455578], [1393], [4160374753], [], [], [] Errors in file /u01/app/oracle/diag/rdbms/xffdb/xffdb1/trace/xffdb1_ora_357910.trc: ORA-00600: internal error code, arguments: [2663], [1393], [4159455578], [1393], [4160374753], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 357910): terminating the instance due to error 600 Instance terminated by USER, pid = 357910 ORA-1092 signalled during: alter database open resetlogs... opiodr aborting process unknown ospid (357910) as a result of ORA-1092 Wed Jul 06 06:57:35 2022 ORA-1092 : opitsk aborting process
该错误比较常见,参考:ORA-600 2663,也可以利用我的Patch_SCN小工具快速解决,后续数据库报ORA-03113错
SQL> alter database open ; alter database open * ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 369324 Session ID: 1585 Serial number: 1
查看alert日志,确认具体报错为kgegpa
Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Wed Jul 06 07:17:08 2022 SMON: enabling cache recovery ARC1: Archiving disabled thread 2 sequence 1 Archived Log entry 1 added for thread 1 sequence 1 ID 0x36317f52 dest 1: Archived Log entry 2 added for thread 1 sequence 2 ID 0x36317f52 dest 1: Archived Log entry 3 added for thread 2 sequence 1 ID 0x0 dest 1: Exception [type:SIGSEGV, Address not mapped to object][ADDR:0x4D562123][PC:0x983CDD6,kgegpa()+40][flags: 0x0,count:1] Exception [type:SIGSEGV, Address not mapped to object][ADDR:0x4D562123][PC:0x983B84A, kgebse()+776][flags: 0x2,count:2] Exception [type:SIGSEGV, Address not mapped to object][ADDR:0x4D562123][PC:0x983B84A, kgebse()+776][flags: 0x2,count:2] Wed Jul 06 07:17:11 2022 PMON (ospid: 377647): terminating the instance due to error 397
该问题有过类似的案例通过处理数据库open成功:
在数据库恢复遭遇ORA-07445 kgegpa错误
Exception [type: SIGSEGV, Address not mapped to object] [] [ kgegpa()+36]
Exception [type: SIGSEGV, Address not mapped to object] [] [ kgegpa()+36]
有朋友找我,由于客户硬件故障,导致控制文件损坏,重建控制文件的时候,由于失误遗漏了100多个数据文件,并且进行了resetlogs操作,然后通过bbed修改文件头,再进行恢复,数据库在open过程中报ORA-03113错误
SQL> recover database; Media recovery complete. SQL> alter database Open ; ERROR: ORA-03113: end-of-file on communication channel Process ID: 11207284 Session ID: 4537 Serial number: 1
对应的alert日志信息
Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Sat Dec 04 23:17:02 2021 SMON: enabling cache recovery ARC3: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE Sat Dec 04 23:17:03 2021 Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x0,count:1] Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x2,count:2] Exception [type: SIGSEGV, Address not mapped to object][ADDR:0x1604971A0][PC:0x1000E1764,kgegpa()+36][flags:0x2,count:2] Sat Dec 04 23:17:07 2021 PMON (ospid: 10879866): terminating the instance due to error 397 Sat Dec 04 23:17:08 2021 System state dump requested by (instance=1, osid=10879866 (PMON)), summary=[abnormal instance termination]. System State dumped to trace file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_diag_11076320.trc Dumping diagnostic data in directory=[cdmp_20211204231708], requested by (instance=1, osid=10879866 (PMON)), summary=[abnormal instance termination]. Sat Dec 04 23:17:09 2021 ORA-1092 : opitsk aborting process
通过对其启动过程进行跟踪确认在update undo$ 语句中异常
PARSING IN CURSOR #11529215044940435280 len=148 dep=1 uid=0 oct=6 lid=0 tim=223080942765 hv=3540833987 ad='c000000d67a42778' sqlid='5ansr7r9htpq3' update undo$ set name=:2,file#=:3,block#=:4,status$=:5,user#=:6,undosqn=:7,xactsqn=:8, scnbas=:9,scnwrp=:10,inst#=:11,ts#=:12,spare1=:13 where us#=:1 END OF STMT PARSE #11529215044940435280:c=10000,e=8182,p=6,cr=55,cu=0,mis=1,r=0,dep=1,og=4,plh=0,tim=223080942764 BINDS #11529215044940435280: Bind#0 oacdty=01 mxl=32(20) mxlc=00 mal=00 scl=00 pre=00 oacflg=18 fl2=0001 frm=01 csi=873 siz=32 off=0 kxsbbbfp=c000000d5fd299aa bln=32 avl=20 flg=09 value="_SYSSMU29_3935275865$" Bind#1 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6e078 bln=24 avl=02 flg=05 value=3 Bind#2 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6e048 bln=24 avl=03 flg=05 value=128 Bind#3 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6e010 bln=24 avl=02 flg=05 value=5 Bind#4 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dfe0 bln=24 avl=02 flg=05 value=1 Bind#5 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dfb0 bln=24 avl=04 flg=05 value=28921 Bind#6 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6df80 bln=24 avl=05 flg=05 value=1245262 Bind#7 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6df48 bln=24 avl=06 flg=05 value=1217986655 Bind#8 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dc90 bln=24 avl=03 flg=05 value=3621 Bind#9 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dc60 bln=24 avl=01 flg=05 value=0 Bind#10 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dc30 bln=24 avl=02 flg=05 value=2 Bind#11 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6dc00 bln=24 avl=02 flg=05 value=2 Bind#12 oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00 oacflg=08 fl2=0001 frm=00 csi=00 siz=24 off=0 kxsbbbfp=9fffffffbcc6e0a8 bln=22 avl=02 flg=05 value=1 WAIT #4574659112: nam='db file sequential read' ela= 6835 file#=1 block#=842169 blocks=1 obj#=0 tim=90821816220340
通过分析,确认是由于undo事务异常,通过一系列处理之后,数据库启动报
SQL> alter database open; alter database open * ERROR at line 1: ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Process ID: 11206912 Session ID: 4537 Serial number: 1
对应alert日志报错
Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Sun Dec 05 02:42:02 2021 SMON: enabling cache recovery SUCCESS: diskgroup FRA was mounted Archived Log entry 1 added for thread 1 sequence 1 ID 0x14ba19ad dest 1: Sun Dec 05 02:42:03 2021 NOTE: dependency between database xff and diskgroup resource ora.FRA.dg is established Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc (incident=5681243): ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /oracle/app/oracle/diag/rdbms/xff/xff1/incident/incdir_5681243/xff1_ora_11206912_i5681243.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. ARC3: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE Sun Dec 05 02:42:05 2021 Dumping diagnostic data in directory=[cdmp_20211205024205], requested by (instance=1, osid=11206912), summary=[incident=5681243]. Sun Dec 05 02:42:05 2021 Block recovery from logseq 2, block 3 to scn 13867049493730 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/group_2.log Block recovery stopped at EOT rba 2.5.16 Block recovery completed at rba 2.5.16, scn 3228.2895062240 Block recovery from logseq 2, block 3 to scn 13867049493727 Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0 Mem# 0: +DATA/xff/onlinelog/group_2.log Block recovery completed at rba 2.5.16, scn 3228.2895062240 Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc: ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Errors in file /oracle/app/oracle/diag/rdbms/xff/xff1/trace/xff1_ora_11206912.trc: ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 11206912): terminating the instance due to error 600 Instance terminated by USER, pid = 11206912 ORA-1092 signalled during: alter database open... opiodr aborting process unknown ospid (11206912) as a result of ORA-1092 Sun Dec 05 02:42:07 2021 ORA-1092 : opitsk aborting process
trace中关键信息和bbed对看到结果对应,通过bbed进行修改,数据库启动正常
uba: 0x004cd9b9.02fb.0c ext: 0x19 spc: 0x1462 uba: 0x00000000.02fb.02 ext: 0x19 spc: 0x1f06 uba: 0x00000000.02fb.0d ext: 0x19 spc: 0x1970 uba: 0x00000000.02fa.03 ext: 0x18 spc: 0x1e34 uba: 0x00000000.02fa.03 ext: 0x18 spc: 0x1e34 struct ktuxcfbp[0], 12 bytes @4192 struct ktufbuba, 8 bytes @4192 ub4 kubadba @4192 0x004cd9b9 ub2 kubaseq @4196 0x02fb ub1 kubarec @4198 0x0c sb2 ktufbext @4200 25 sb2 ktufbspc @4202 5218
以前有过类似恢复硬件恢复之后,数据库无法open故障恢复
发表在 Oracle备份恢复
标签为 Address not mapped to object, kgegpa, ORA-600 4193, type: SIGSEGV, update undo$
评论关闭