分类目录归档：Oracle备份恢复

数据库启动报ORA-600 kcbgtcr_13处理

发表于 2023 年 4 月 20 日 由 惜分飞

数据库发生故障,经过第三方处理过,接手之后,尝试open库报ORA-01190错误

Thu Apr 20 16:51:25 2023
alter database open upgrade
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_4818.trc:
ORA-01190: control file or data file 3 is from before the last RESETLOGS
ORA-01110: data file 3: '/data/topprod/undotbs01.dbf'
ORA-1190 signalled during: alter database open upgrade...

这个问题是由于resetlogs的时候有文件遗漏导致resetlogs scn和其他数据文件/ctl不匹配导致,以前类似处理文章:
bbed解决ORA-01190
12C sysaux 异常恢复—ORA-01190错误恢复
Oracle Recovery Tools 解决ORA-01190 ORA-01248等故障

数据库启动报ORA-600 kcbgtcr_13错

</tmp> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Apr 20 17:05:24 2023

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> recover database ;
Media recovery complete.
SQL> alter database open ;
alter database open
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [],
[], [], [], [], [], []
Process ID: 5492
Session ID: 861 Serial number: 19

alert日志报错

Thu Apr 20 17:05:37 2023
SMON: enabling cache recovery
[5492] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:800184 end:800294 diff:110 (1 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is AL32UTF8
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_smon_4770.trc  (incident=2390097):
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u2/oracle/diag/rdbms/xifenfei/xifenfei/incident/incdir_2390097/xifenfei_smon_4770_i2390097.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_5492.trc  (incident=2390129):
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u2/oracle/diag/rdbms/xifenfei/xifenfei/incident/incdir_2390129/xifenfei_ora_5492_i2390129.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Block recovery from logseq 2, block 56 to scn 8615223701
Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
  Mem# 0: /u2/oracle/oradata/xifenfei/redo02.log
Block recovery completed at rba 2.60.16, scn 2.25289110
Block recovery from logseq 2, block 56 to scn 8615223600
Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
  Mem# 0: /u2/oracle/oradata/xifenfei/redo02.log
Block recovery completed at rba 2.59.16, scn 2.25289009
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_smon_4770.trc:
ORA-01595: error freeing extent (3) of rollback segment (5))
ORA-00600: internal error code, arguments: [4193], [], [], [], [], [], [], [], [], [], [], []
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_5492.trc:
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [], [], [], [], [], [], []
Errors in file /u2/oracle/diag/rdbms/xifenfei/xifenfei/trace/xifenfei_ora_5492.trc:
ORA-00600: internal error code, arguments: [kcbgtcr_13], [], [], [], [], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER (ospid: 5492): terminating the instance due to error 600
Thu Apr 20 17:05:40 2023
Instance terminated by USER, pid = 5492
ORA-1092 signalled during: alter database open upgrade...
opiodr aborting process unknown ospid (5492) as a result of ORA-1092

这个错误比较明显是由于undo异常导致,规避掉undo问题,数据库启动成功

SQL> startup mount pfile='/tmp/pfile'
ORACLE instance started.

Total System Global Area 4.2758E+10 bytes
Fixed Size                  2237776 bytes
Variable Size            3.7447E+10 bytes
Database Buffers         5234491392 bytes
Redo Buffers               74444800 bytes
Database mounted.
SQL> alter database open ;

Database altered.

然后逻辑方式导出数据,导入到新库即可,对于此类问题在2014年处理过类似的case:
记录一次ORA-600 kccpb_sanity_check_2和ORA-600 kcbgtcr_13 错误恢复

发表在 Oracle备份恢复 | 标签为 , , | 评论关闭

Oracle 启动后一会儿就挂掉故障处理—ORA-600 17182

发表于 2023 年 3 月 25 日 由 惜分飞

一例正常运行的数据库突然节点不停重启(因为是rac,启动一会儿就crash,然后又被crs给启动起来,然后有crash,依次循环),告警日志类似:

Fri Mar 24 13:36:07 2023
QMNC started with pid=124, OS id=188397 
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Completed: ALTER DATABASE OPEN
Fri Mar 24 13:36:08 2023
minact-scn: Inst 1 is now the master inc#:2 mmon proc-id:188028 status:0x7
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.00000000 gcalc-scn:0x0000.00000000
Fri Mar 24 13:36:08 2023
Starting background process CJQ0
Fri Mar 24 13:36:08 2023
CJQ0 started with pid=144, OS id=188451 
Fri Mar 24 13:36:09 2023
Redo thread 2 internally disabled at seq 44406 (CKPT)
Archived Log entry 135343 added for thread 2 sequence 44405 ID 0xcd7086e0 dest 1:
ARC0: Archiving disabled thread 2 sequence 44406
Archived Log entry 135344 added for thread 2 sequence 44406 ID 0xcd7086e0 dest 1:
Thread 1 advanced to log sequence 40030 (LGWR switch)
  Current log# 2 seq# 40030 mem# 0: +DATA/xff/onlinelog/group_2.310.1087136761
Archived Log entry 135345 added for thread 1 sequence 40029 ID 0xcd7086e0 dest 1:
Fri Mar 24 13:36:30 2023
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p200_188856.trc  (incident=1082418):
ORA-00600: internal error code, arguments: [17182], [0x7F4D2A13DBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1082418/xff1_p200_188856_i1082418.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Mar 24 13:36:30 2023
Dumping diagnostic data in directory=[cdmp_20230324133630], requested by (instance=1, osid=188856 (P200)), summary=[incident=1082418].
Fri Mar 24 13:36:54 2023
Decreasing number of real time LMS from 6 to 0
Fri Mar 24 13:36:54 2023
Block recovery from logseq 40030, block 259 to scn 17199959182
Recovery of Online Redo Log: Thread 1 Group 2 Seq 40030 Reading mem 0
  Mem# 0: +DATA/xff/onlinelog/group_2.310.1087136761
Block recovery stopped at EOT rba 40030.317.16
Block recovery completed at rba 40030.317.16, scn 4.20089998
Exception [type: SIGSEGV, SI_KERNEL(general_protection)] [ADDR:0x0] [PC:0x97E8579, kghrst()+1835] [flags: 0x0, count: 1]
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p200_188856.trc  (incident=1082419):
ORA-07445: exception encountered: core dump [kghrst()+1835] [SIGSEGV] [ADDR:0x0] [PC:0x97E8579] [SI_KERNEL(general_protection)] []
ORA-00600: internal error code, arguments: [17182], [0x7F4D2A13DBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1082419/xff1_p200_188856_i1082419.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p200_188856.trc  (incident=1082420):
ORA-00600: internal error code, arguments: [17147], [0x7F4D2A13DBD0], [], [], [], [], [], [], [], [], [], []
ORA-07445: exception encountered: core dump [kghrst()+1835] [SIGSEGV] [ADDR:0x0] [PC:0x97E8579] [SI_KERNEL(general_protection)] []
ORA-00600: internal error code, arguments: [17182], [0x7F4D2A13DBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1082420/xff1_p200_188856_i1082420.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p200_188856.trc  (incident=1082421):
ORA-00600: internal error code, arguments: [kghfrempty:ds], [0x7F4D2A13DBE8], [], [], [], [], [], [], [], [], [], []
ORA-07445: exception encountered: core dump [kghrst()+1835] [SIGSEGV] [ADDR:0x0] [PC:0x97E8579] [SI_KERNEL(general_protection)] []
ORA-00600: internal error code, arguments: [17182], [0x7F4D2A13DBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1082421/xff1_p200_188856_i1082421.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Mar 24 13:36:56 2023
Dumping diagnostic data in directory=[cdmp_20230324133656], requested by (instance=1, osid=188856 (P200)), summary=[incident=1082420].
SMON: Restarting fast_start parallel rollback
Fri Mar 24 13:37:12 2023
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p000_188229.trc  (incident=1080530):
ORA-00600: internal error code, arguments: [17182], [0x7F3AB22ADBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1080530/xff1_p000_188229_i1080530.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Mar 24 13:37:12 2023
Dumping diagnostic data in directory=[cdmp_20230324133712], requested by (instance=1, osid=188229 (P000)), summary=[incident=1080530].
Fri Mar 24 13:37:24 2023
Block recovery from logseq 40030, block 259 to scn 17199959182
Recovery of Online Redo Log: Thread 1 Group 2 Seq 40030 Reading mem 0
  Mem# 0: +DATA/xff/onlinelog/group_2.310.1087136761
Block recovery completed at rba 40030.317.16, scn 4.20089999
Fri Mar 24 13:37:37 2023
Exception [type: SIGSEGV, SI_KERNEL(general_protection)] [ADDR:0x0] [PC:0x97E8579, kghrst()+1835] [flags: 0x0, count: 1]
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_p000_188229.trc  (incident=1080531):
ORA-07445: exception encountered: core dump [kghrst()+1835] [SIGSEGV] [ADDR:0x0] [PC:0x97E8579] [SI_KERNEL(general_protection)] []
ORA-00600: internal error code, arguments: [17182], [0x7F3AB22ADBF8], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1080531/xff1_p000_188229_i1080531.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Mar 24 13:37:37 2023
Dumping diagnostic data in directory=[cdmp_20230324133737], requested by (instance=1, osid=188229 (P000)), summary=[incident=1080531].
Fri Mar 24 13:38:16 2023
SMON: slave died unexpectedly, downgrading to serial recovery
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_smon_188020.trc  (incident=1080418):
ORA-00600: internal error code, arguments: [17182], [0x7F9184B445C0], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1080418/xff1_smon_188020_i1080418.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Block recovery from logseq 40030, block 259 to scn 17199959182
Recovery of Online Redo Log: Thread 1 Group 2 Seq 40030 Reading mem 0
  Mem# 0: +DATA/xff/onlinelog/group_2.310.1087136761
Block recovery completed at rba 40030.317.16, scn 4.20089999
ORACLE Instance xff1 (pid = 56) - Error 600 encountered while recovering transaction (10, 26) on object 242112.
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_smon_188020.trc:
ORA-00600: internal error code, arguments: [17182], [0x7F9184B445C0], [], [], [], [], [], [], [], [], [], []
Fri Mar 24 13:38:17 2023
Dumping diagnostic data in directory=[cdmp_20230324133817], requested by (instance=1, osid=188020 (SMON)), summary=[incident=1080418].
Exception [type: SIGSEGV, SI_KERNEL(general_protection)] [ADDR:0x0] [PC:0x97E8579, kghrst()+1835] [flags: 0x0, count: 1]
Errors in file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_smon_188020.trc  (incident=1080419):
ORA-07445: exception encountered: core dump [kghrst()+1835] [SIGSEGV] [ADDR:0x0] [PC:0x97E8579] [SI_KERNEL(general_protection)] []
ORA-00600: internal error code, arguments: [17182], [0x7F9184B445C0], [], [], [], [], [], [], [], [], [], []
Incident details in: /oracle/database/diag/rdbms/xff/xff1/incident/incdir_1080419/xff1_smon_188020_i1080419.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Fri Mar 24 13:38:20 2023
PMON (ospid: 187888): terminating the instance due to error 474
System state dump requested by (instance=1, osid=187888 (PMON)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/database/diag/rdbms/xff/xff1/trace/xff1_diag_187902_20230324133820.trc
Fri Mar 24 13:38:21 2023
ORA-1092 : opitsk aborting process
Dumping diagnostic data in directory=[cdmp_20230324133820], requested by (instance=1, osid=187888 (PMON)), summary=[abnormal instance termination].
Instance terminated by PMON, pid = 187888

这类的故障在多年前处理过几次
ORA-600 17182导致oracle异常
ORA-00600[17182],ORA-00600[25027],ORA-00600[kghfrempty:ds]故障处理
这个故障的原因是由于block逻辑损坏,实例无法正常做回滚恢复,从而异常.处理异常回滚问题,就可以规避掉数据库启动后一会儿就crash问题.

发表在 Oracle备份恢复 | 标签为 , , , | 评论关闭

断电引起的oracle数据库异常恢复

发表于 2023 年 3 月 19 日 由 惜分飞

服务器断电，数据库mount失败

SQL> startup mount pfile='d:/pfile.txt'
ORACLE 例程已经启动。

Total System Global Area 1185853440 bytes
Fixed Size                  2175168 bytes
Variable Size             335548224 bytes
Database Buffers          838860800 bytes
Redo Buffers                9269248 bytes
ORA-00205: ?????????, ??????, ???????

alert日志报错信息

Sun Mar 19 20:18:29 2023
ALTER DATABASE   MOUNT
Errors in file d:\app\xifenfei\diag\rdbms\orcl\orcl\trace\orcl_ckpt_15064.trc  (incident=3697):
ORA-00227: ????????????: (? 1, # ? 1)
ORA-00202: ????: ''D:\BAIDUNETDISKDOWNLOAD\ORCL\CONTROL01.CTL''
Incident details in: d:\app\xifenfei\diag\rdbms\orcl\orcl\incident\incdir_3697\orcl_ckpt_15064_i3697.trc
Sun Mar 19 20:18:30 2023
Errors in file d:\app\xifenfei\diag\rdbms\orcl\orcl\trace\orcl_m000_18084.trc  (incident=3761):
ORA-00227: ????????????: (? 1, # ? 1)
ORA-00202: ????: ''D:\BAIDUNETDISKDOWNLOAD\ORCL\CONTROL01.CTL''
Incident details in: d:\app\xifenfei\diag\rdbms\orcl\orcl\incident\incdir_3761\orcl_m000_18084_i3761.trc
Sun Mar 19 20:18:29 2023
MMNL started with pid=16, OS id=9404 
ORA-00227: ????????????: (? 1, # ? 1)
ORA-00202: ????: ''D:\BAIDUNETDISKDOWNLOAD\ORCL\CONTROL01.CTL''
Checker run found 1 new persistent data failures
Trace dumping is performing id=[cdmp_20230319201831]
ORA-205 signalled during: ALTER DATABASE   MOUNT...

错误比较明显由于控制文件的block损坏导致数据库在mount的时候提示ORA-00205,重试重建ctl

SQL> CREATE CONTROLFILE REUSE DATABASE "orcl" NORESETLOGS  NOARCHIVELOG
  2      MAXLOGFILES 50
  3      MAXLOGMEMBERS 5
  4      MAXDATAFILES 100
  5      MAXINSTANCES 8
  6      MAXLOGHISTORY 226
  7  LOGFILE
  8    GROUP 1 'D:\BaiduNetdiskDownload\orcl/redo01.log'  SIZE 50M,
  9    GROUP 2 'D:\BaiduNetdiskDownload\orcl/redo02.log'  SIZE 50M,
 10    GROUP 3 'D:\BaiduNetdiskDownload\orcl/redo03.log'  SIZE 50M
 11  DATAFILE
 12  'D:\BaiduNetdiskDownload\orcl\EXAMPLE01.DBF',
 13  'D:\BaiduNetdiskDownload\orcl\GHZN.DBF',
 14  'D:\BaiduNetdiskDownload\orcl\GHZN2.DBF',
 15  'D:\BaiduNetdiskDownload\orcl\SYSAUX01.DBF',
 16  'D:\BaiduNetdiskDownload\orcl\SYSTEM01.DBF',
 17  'D:\BaiduNetdiskDownload\orcl\UNDOTBS01.DBF',
 18  'D:\BaiduNetdiskDownload\orcl\USERS01.DBF'
 19  CHARACTER SET ZHS16GBK
 20  ;
CREATE CONTROLFILE REUSE DATABASE "orcl" NORESETLOGS  NOARCHIVELOG
*
ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-01565: error in identifying file
'D:\BaiduNetdiskDownload\orcl\UNDOTBS01.DBF'
ORA-27041: unable to open file
OSD-04001: 逻辑块大小无效 (OS 2613931212)

由于undo文件异常(大小不是block size的整数倍）,因此报OSD-04001: 逻辑块大小无效错误.对undo文件及其其他文件进行检查发现数据库文件有不少坏块,而且undo文件的文件头损坏
20230319202417

通过抛弃undo文件并进行一些处理,重建ctl成功,并且recover 数据库成功,顺利open数据库

SQL> CREATE CONTROLFILE REUSE DATABASE "orcl" NORESETLOGS  NOARCHIVELOG
  2      MAXLOGFILES 50
  3      MAXLOGMEMBERS 5
  4      MAXDATAFILES 100
  5      MAXINSTANCES 8
  6      MAXLOGHISTORY 226
  7  LOGFILE
  8    GROUP 1 'D:\BaiduNetdiskDownload\orcl/redo01.log'  SIZE 50M,
  9    GROUP 2 'D:\BaiduNetdiskDownload\orcl/redo02.log'  SIZE 50M,
 10    GROUP 3 'D:\BaiduNetdiskDownload\orcl/redo03.log'  SIZE 50M
 11  DATAFILE
 12  'D:\BaiduNetdiskDownload\orcl\EXAMPLE01.DBF',
 13  'D:\BaiduNetdiskDownload\orcl\GHZN.DBF',
 14  'D:\BaiduNetdiskDownload\orcl\GHZN2.DBF',
 15  'D:\BaiduNetdiskDownload\orcl\SYSAUX01.DBF',
 16  'D:\BaiduNetdiskDownload\orcl\SYSTEM01.DBF',
 17  'D:\BaiduNetdiskDownload\orcl\USERS01.DBF'
 18  CHARACTER SET ZHS16GBK
 19  ;

Control file created.

SQL> recover database;
Media recovery complete.
SQL> shutdown immediate;
ORA-01109: database not open


Database dismounted.
ORACLE instance shut down.
SQL> startup mount pfile='d:/pfile.txt'
ORACLE instance started.

Total System Global Area 1185853440 bytes
Fixed Size                  2175168 bytes
Variable Size             335548224 bytes
Database Buffers          838860800 bytes
Redo Buffers                9269248 bytes
Database mounted.
SQL> alter database open;

Database altered.

然后使用逻辑方式导出数据,运气不错业务文件没有任何坏块,system坏块在aud$上,无任何业务数据丢失.

发表在 Oracle备份恢复 | 标签为 , , | 评论关闭