分类目录归档：Oracle备份恢复

重建ctl遭遇ORA-01163 ORA-01110和ORA-600 kccccfl_1错误

发表于 2023 年 11 月 9 日 由 惜分飞

有客户联系我,数据库强制关闭之后无法正常启动,通过他们一系列尝试,数据库报open 报ORA-01555错误
ORA-01555

通过分析确认是由于客户屏蔽一致性强制拉库失败导致

Wed Nov 01 19:49:03 2023
alter database open resetlogs
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 1199697777
Archived Log entry 68106 added for thread 1 sequence 3829 ID 0x761a769a dest 1:
Archived Log entry 68107 added for thread 1 sequence 3830 ID 0x761a769a dest 1:
Archived Log entry 68108 added for thread 1 sequence 3828 ID 0x761a769a dest 1:
Clearing online redo logfile 1 /u01/app/oracle/oradata/ORACLE/redo01.log
Clearing online log 1 of thread 1 sequence number 3829
Clearing online redo logfile 1 complete
Clearing online redo logfile 2 /u01/app/oracle/oradata/ORACLE/redo02.log
Clearing online log 2 of thread 1 sequence number 3830
Clearing online redo logfile 2 complete
Clearing online redo logfile 3 /u01/app/oracle/oradata/ORACLE/redo03.log
Clearing online log 3 of thread 1 sequence number 3828
Clearing online redo logfile 3 complete
Resetting resetlogs activation ID 1981445786 (0x761a769a)
Online log /u01/app/oracle/oradata/ORACLE/redo01.log: Thread 1 Group 1 was previously cleared
Online log /u01/app/oracle/oradata/ORACLE/redo02.log: Thread 1 Group 2 was previously cleared
Online log /u01/app/oracle/oradata/ORACLE/redo03.log: Thread 1 Group 3 was previously cleared
Wed Nov 01 19:49:09 2023
Setting recovery target incarnation to 3
Wed Nov 01 19:49:09 2023
Assigning activation ID 2024817472 (0x78b04340)
LGWR: STARTING ARCH PROCESSES
Wed Nov 01 19:49:09 2023
ARC0 started with pid=21, OS id=2078 
ARC0: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC0: STARTING ARCH PROCESSES
Wed Nov 01 19:49:10 2023
ARC1 started with pid=22, OS id=2080 
Thread 1 advanced to log sequence 2 (thread open)
Wed Nov 01 19:49:10 2023
ARC2 started with pid=23, OS id=2082 
Wed Nov 01 19:49:10 2023
ARC3 started with pid=24, OS id=2084 
ARC1: Archival started
ARC2: Archival started
ARC1: Becoming the 'no FAL' ARCH
ARC1: Becoming the 'no SRL' ARCH
ARC2: Becoming the heartbeat ARCH
Thread 1 opened at log sequence 2
  Current log# 2 seq# 2 mem# 0: /u01/app/oracle/oradata/ORACLE/redo02.log
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Nov 01 19:49:10 2023
SMON: enabling cache recovery
ARC2: LGWR is scheduled to archive destination LOG_ARCHIVE_DEST_2 after log switch
Wed Nov 01 19:49:10 2023
NSA2 started with pid=25, OS id=2086 
ORA-01555 caused by SQL statement below (SQL ID: 4krwuz0ctqxdt, SCN: 0x0000.4781ef78):
select ctime, mtime, stime from obj$ where obj# = :1
Errors in file /u01/app/oracle/diag/rdbms/oracle_pd/ORACLE/trace/ORACLE_ora_2076.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_467363169$" too small
Errors in file /u01/app/oracle/diag/rdbms/oracle_pd/ORACLE/trace/ORACLE_ora_2076.trc:
ORA-00704: bootstrap process failure
ORA-00704: bootstrap process failure
ORA-00604: error occurred at recursive SQL level 1
ORA-01555: snapshot too old: rollback segment number 11 with name "_SYSSMU11_467363169$" too small
Error 704 happened during db open, shutting down database
USER (ospid: 2076): terminating the instance due to error 704
Instance terminated by USER, pid = 2076
ORA-1092 signalled during: alter database open resetlogs...
opiodr aborting process unknown ospid (2076) as a result of ORA-1092

这种错误比较常见恢复简单,以前有过类似的文章:
在数据库open过程中常遇到ORA-01555汇总
数据库open过程遭遇ORA-1555对应sql语句补充
重建control遗漏数据文件,reseltogs报ORA-1555错误处理
Oracle Recovery Tools恢复—ORA-00704 ORA-01555故障
这个错误恢复本身不难,只是在数据库的恢复过程中需要重建文件,在这次的重建文件中由于ctl语句错误【语句是通过sql查询拼接而成】,导致重建文件报了两个错误,供参考:
ORA-01163 ORA-01110错误

SQL> CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS  NOARCHIVELOG
        MAXLOGFILES 50
       MAXLOGMEMBERS 5
       MAXDATAFILES 1000
       MAXINSTANCES 8
       MAXLOGHISTORY 2920
LOGFILE
       group 1   '/u01/app/oracle/oradata/ORACLE/redo01.log' size 200M,
  2         group 3   '/u01/app/oracle/oradata/ORACLE/redo03.log' size 200M,
       group 2   '/u01/app/oracle/oradata/ORACLE/redo02.log' size 200M,
DATAFILE
        '/u01/app/oracle/oradata/ORACLE/system01.dbf',
        …………
        '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf',
CHARACTER SET  ZHS16GBK ;
 23   24   25   26   27  CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS	NOARCHIVELOG
*
ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-01163: SIZE clause indicates 12800 (blocks), but should match header
1564672
ORA-01110: data file 15: '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf'

出现该错误是由于我写路径的时候在最后一个数据文件路径后面多了一个【,】导致

ORA-600 kccccfl_1错误

SQL> CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS  NOARCHIVELOG
        MAXLOGFILES 50
       MAXLOGMEMBERS 5
       MAXDATAFILES 1000
       MAXINSTANCES 8
       MAXLOGHISTORY 2920
LOGFILE
       group 1   '/u01/app/oracle/oradata/ORACLE/redo01.log'  size 200M,
       group 3   '/u01/app/oracle/oradata/ORACLE/redo03.log' size 200M,
       group 2   '/u01/app/oracle/oradata/ORACLE/redo02.log' size 200M,
DATAFILE
        '/u01/app/oracle/oradata/ORACLE/system01.dbf',
        ………… 
        '/u01/app/oracle/oradata/ORACLE/xifenfei06.dbf'
CHARACTER SET  ZHS16GBK ;
 23   24   25   26   27  CREATE CONTROLFILE REUSE DATABASE "ORACLE" NORESETLOGS	NOARCHIVELOG
*
ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-00600: internal error code, arguments: [kccccfl_1], [], [], [], [], [], [],
[], [], [], [], []

出现该错误是由于在最后redo文件路径后面多了【,】导致

发表在 Oracle备份恢复 | 标签为 , , | 评论关闭

再现ORA-600 4000故障处理

发表于 2023 年 10 月 18 日 由 惜分飞

有一个10g的库，由于redo损坏导致无法正常recover成功
20231018111911

正常途径无法open成功,尝试强制打开库

Wed Oct 18 11:23:25 2023
alter database open resetlogs
Wed Oct 18 11:23:25 2023
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 236715370
Resetting resetlogs activation ID 4009817354 (0xef00f50a)
Wed Oct 18 11:23:26 2023
Setting recovery target incarnation to 3
Wed Oct 18 11:23:26 2023
Assigning activation ID 4271120987 (0xfe94225b)
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: D:\ORADATA\EAIS\REDO01.LOG
Successful open of redo thread 1
Wed Oct 18 11:23:26 2023
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Wed Oct 18 11:23:26 2023
SMON: enabling cache recovery
Wed Oct 18 11:23:27 2023
Errors in file d:\hys_db\admin\eais\udump\eais_ora_10916.trc:
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []

Wed Oct 18 11:23:28 2023
Errors in file d:\hys_db\admin\eais\udump\eais_ora_10916.trc:
ORA-00704: 引导程序进程失败
ORA-00704: 引导程序进程失败
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []

Wed Oct 18 11:23:28 2023
Error 704 happened during db open, shutting down database
USER: terminating instance due to error 704
Wed Oct 18 11:23:28 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_pmon_1764.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_reco_9628.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_smon_9884.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_ckpt_10096.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_lgwr_8932.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_dbw0_9892.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_mman_9472.trc:
ORA-00704: bootstrap process failure

Wed Oct 18 11:23:29 2023
Errors in file d:\hys_db\admin\eais\bdump\eais_psp0_7692.trc:
ORA-00704: bootstrap process failure

Instance terminated by USER, pid = 10916
ORA-1092 signalled during: alter database open resetlogs...

resetlogs失败,报ora-600 4000错误,查看相关trace文件

*** 2023-10-18 11:23:27.103
ksedmp: internal or fatal error
ORA-00600: 内部错误代码, 参数: [4000], [9], [], [], [], [], [], []
Current SQL statement for this session:
select ctime, mtime, stime from obj$ where obj# = :1
----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
_ksedst+38           CALLrel  _ksedst1+0           0 1
_ksedmp+898          CALLrel  _ksedst+0            0
_ksfdmp+14           CALLrel  _ksedmp+0            3
603A816A             CALLreg  00000000             6AE8258 3
603A8550             CALLrel  603A80D8             6AE8258 4FB1090 FA0 1 8C886B0
__VInfreq__ktudba+1  CALLrel  _kgeasi+0            6AE8258 4FB1090 FA0 2 1 0 9 0
26                                                 
_ktrgcm+851          CALLrel  _ktudba+0            9 8C88B68 0 0
_ktrgtc+526          CALLrel  _ktrgcm+0            4FC753C
_kdsgrp+937          CALLrel  _ktrgtc+0            4FC753C 4FC74E0 1F1FB00
                                                   8C88CB4 240 2AED19C 1F1F71C
_kdsfbrcb+432        CALLrel  _kdsgrp+0            4FC7538 0 4FC7538
_qertbFetchByRowID+  CALLrel  _kdsfbrcb+0          4FC7538 4FDF004 0 1 0 0
4344                                               4FC74E0 22E8078 8C88E14 0
_opifch2+3104        CALL???  00000000             8DA05838 20135FC 8C890AC 1
_opifch+51           CALLrel  _opifch2+0           89 5 8C891F0
_opiodr+1099         CALLreg  00000000             5 2 8C89938
_rpidrus+178         CALLrel  _opiodr+0            5 2 8C89938 5
_rpidru+88           CALLrel  _rpidrus+0           8C894C4
_rpiswu2+426         CALLreg  00000000             8C8988C
_rpidrv+1461         CALLrel  _rpiswu2+0           90F3CCB4 0 8C89860 2 8C89880
                                                   0 8C89860 0 84DF58 84E018
                                                   8C8988C 8
_rpifch+44           CALLrel  _rpidrv+0            5 5 8C89938 8
_kqdpts+166          CALLrel  _rpifch+0            5 5 5 3 90AE2383 7 C 0 0 0 0
                                                   0 0 0 5 2 90AE237C 7 C 0 0 0
                                                   0 0 0 0
_kqrlfc+478          CALLrel  _kqdpts+0            90AE21F4
_kqlbplc+102         CALLrel  _kqrlfc+0            
_kqlblfc+240         CALLrel  _kqlbplc+0           0
_adbdrv+12911        CALLrel  _kqlblfc+0           0 8C8D034
_opiexe+11778        CALLrel  _adbdrv+0            
_opiosq0+6088        CALLrel  _opiexe+0            4 0 8C8D894
_kpooprx+232         CALLrel  _opiosq0+0           3 E 8C8D9AC A4
_kpoal8+775          CALLrel  _kpooprx+0           8C8F6CC 8C8E248 1D 1 0 A4
_opiodr+1099         CALLreg  00000000             5E 17 8C8F6C8
60FEFF8D             CALLreg  00000000             5E 17 8C8F6C8 0
_opitsk+1017         CALL???  00000000             
_opiino+1087         CALLrel  _opitsk+0            0 0
_opiodr+1099         CALLreg  00000000             3C 4 8C8FC60
_opidrv+819          CALLrel  _opiodr+0            3C 4 8C8FC60 0
_sou2o+45            CALLrel  _opidrv+0            3C 4 8C8FC60
_opimai_real+112     CALLrel  _sou2o+0             8C8FC54 3C 4 8C8FC60
_opimai+92           CALLrel  _opimai_real+0       2 8C8FC8C
_OracleThreadStart@  CALLrel  _opimai+0            
4+708                                              
74A93438             CALLptr  00000000             
76F09830             CALLreg  00000000             
76F09800             CALLrel  76F0980B             
 
--------------------- Binary Stack Dump ---------------------

通过一些技巧处理数据库正常open,导出数据完成本次恢复，参考以前恢复类似case:

重现ORA-600 4000异常
通过bbed解决ORA-00600[4000]案例
记录一次ORA-600 4000数据库故障恢复
ORACLE 8.1.7 数据库ORA-600 4000故障恢复

发表在 Oracle备份恢复 | 标签为 , | 评论关闭

ORA-07445: exception encountered: core dump [kdxlin()+4088]处理

发表于 2023 年 9 月 23 日 由 惜分飞

abort方式关闭数据库，启动报错

Tue Sep 19 21:52:56 2023
NOTE: dependency between database orcl and diskgroup resource ora.DATA.dg is established
Tue Sep 19 21:52:57 2023
Reconfiguration started (old inc 4, new inc 6)
List of instances:
 1 (myinst: 1) 
 Global Resource Directory frozen
 * dead instance detected - domain 0 invalid = TRUE 
 Communication channels reestablished
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
Tue Sep 19 21:52:57 2023
Tue Sep 19 21:52:57 2023
 LMS 3: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Tue Sep 19 21:52:57 2023
 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Tue Sep 19 21:52:57 2023
 LMS 2: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Post SMON to start 1st pass IR
 Submitted all GCS remote-cache requests
 Post SMON to start 1st pass IR
 Fix write in gcs resources
Reconfiguration complete
Tue Sep 19 21:53:05 2023
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_28917.trc  (incident=492333):
ORA-00600: internal error code, arguments: [2131], [33], [32], [], [], [], [], [], [], [], [], []
Incident details in:/u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_492333/orcl1_ora_28917_i492333.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORA-600 signalled during: ALTER DATABASE MOUNT /* db agent *//* {1:34652:2} */...

重建ctl之后，尝试recover数据库报错ORA-600 3020和ORA-07445 kdxlin等错误

SQL> recover database;
ORA-00600: internal error code, arguments: [3020], [41], [3142531],
[175108995], [], [], [], [], [], [], [], []
ORA-10567: Redo is inconsistent with data block (file# 41, block# 3142531, file
offset is 4268777472 bytes)
ORA-10564: tablespace XIFENFEI
ORA-01110: data file 41: '+DATA/orcl/datafile/xifenfei07.dbf'
ORA-10560: block type 'FIRST LEVEL BITMAP BLOCK'

Wed Sep 20 00:15:00 2023
ALTER DATABASE RECOVER  database  
Media Recovery Start
 started logmerger process
Parallel Media Recovery started with 64 slaves
Wed Sep 20 00:15:02 2023
Recovery of Online Redo Log: Thread 2 Group 6 Seq 67008 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_6.268.942097791
Recovery of Online Redo Log: Thread 1 Group 2 Seq 81767 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_2.262.942097651
Recovery of Online Redo Log: Thread 1 Group 5 Seq 81768 Reading mem 0
  Mem# 0: +DATA/orcl/onlinelog/group_5.263.942097651
Wed Sep 20 00:15:08 2023
Hex dump of (file 41, block 3142531) in trace file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr1m_45463.trc
Reading datafile '+DATA/orcl/datafile/ts_his3bz07.dbf' for corruption at rdba: 0x0a6ff383 (file 41, block 3142531)
Reread (file 41, block 3142531) found different corrupt data (logically corrupt)
Hex dump of (file 41, block 3142531) in trace file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr1m_45463.trc
Wed Sep 20 00:15:08 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC] [PC:0x95FB582, kdxlin()+4088][flags: 0x0,count:1]
Wed Sep 20 00:15:08 2023
Exception [type: SIGSEGV, Address not mapped to object][ADDR:0xC] [PC:0x95FB582, kdxlin()+4088][flags: 0x0,count:1]
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pr10_45419.trc  (incident=564584):
ORA-07445: exception encountered:core dump [kdxlin()+4088][SIGSEGV][ADDR:0xC][PC:0x95FB582][Address not mapped to object]
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_564640/orcl1_pr17_45433_i564640.trc

尝试随机恢复文件，也遭遇ORA-07445 kdxlin异常

SQL> recover datafile 34;
ORA-00283: recovery session canceled due to errors
ORA-10562: Error occurred while applying redo to data block (file# 34, block#
1999809)
ORA-10564: tablespace XIFENFEI
ORA-01110: data file 34: '+DATA/orcl/datafile/xifeifenfei06'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 97961
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kdxlin()+4088] [SIGSEGV]
[ADDR:0xC] [PC:0x95FB582] [Address not mapped to object] []

出现这种情况是由于redo和数据文件块不一致导致无法正常应用日志,人工对于异常的block进行处理,数据库open成功,然后遭遇undo回滚段异常,对其进行规避,数据库open并且稳定运行

发表在 Oracle备份恢复 | 标签为 , , | 评论关闭