标签云
asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-00742 ORA-01110 ORA-01555 ORA-01578 ORA-01595 ORA-08103 ORA-600 2131 ORA-600 2662 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (103)
- 数据库 (1,768)
- DB2 (22)
- MySQL (77)
- Oracle (1,609)
- Data Guard (52)
- EXADATA (8)
- GoldenGate (24)
- ORA-xxxxx (166)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (15)
- ORACLE 21C (3)
- Oracle 23ai (8)
- Oracle ASM (69)
- Oracle Bug (8)
- Oracle RAC (54)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (29)
- Oracle备份恢复 (591)
- Oracle安装升级 (97)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (86)
- PostgreSQL (30)
- pdu工具 (6)
- PostgreSQL恢复 (9)
- SQL Server (32)
- SQL Server恢复 (13)
- TimesTen (7)
- 达梦数据库 (3)
- 达梦恢复 (1)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (39)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (22)
-
最近发表
- ORA-600 kokiasg1故障分析(obj$中核心字典序列全部被恶意删除)
- ORA-00756 ORA-10567故障数据0丢失恢复
- 数据库文件变成32k故障恢复
- tcp连接过多导致监听TNS-12532 TNS-12560 TNS-00502错误
- 文件系统格式化MySQL数据库恢复
- .sstop勒索加密数据库恢复
- 解决一次硬件恢复之后数据文件0kb的故障恢复case
- Error in invoking target ‘libasmclntsh19.ohso libasmperl19.ohso client_sharedlib’问题处理
- ORA-01171: datafile N going offline due to error advancing checkpoint
- linux环境oracle数据库被文件系统勒索加密为.babyk扩展名溯源
- ORA-600 ksvworkmsgalloc: bad reaper
- ORA-600 krccfl_chunk故障处理
- Oracle Recovery Tools恢复案例总结—202505
- ORA-600 kddummy_blkchk 数据库循环重启
- 记录一次asm disk加入到vg通过恢复直接open库的案例
- CHECKDB 发现了 N 个分配错误和 M 个一致性错误
- 达梦数据库dm.ctl文件异常恢复
- Oracle Recovery Tools修复ORA-00742、ORA-600 ktbair2: illegal inheritance故障
- 可能是 tempdb 空间用尽或某个系统表不一致故障处理
- 11.2.0.4库中遇到ORA-600 kcratr_nab_less_than_odr报错
分类目录归档:ORA-xxxxx
重建控制文件丢失undo异常恢复—ORA-600 25025模拟与恢复
在现实的情况中,有些人因为某种原因重建控制文件(丢失undo[有意或者无意]),然后又resetlogs库尝试恢复,这样的操作可能导致各种比较麻烦的恢复,这里模拟ORA-600[25025]异常恢复
模拟ORA-600[25025]错误
SQL> select * from v$version; BANNER -------------------------------------------------------------------------------- Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - Production PL/SQL Release 11.2.0.3.0 - Production CORE 11.2.0.3.0 Production TNS for Linux: Version 11.2.0.3.0 - Production NLSRTL Version 11.2.0.3.0 - Production SQL> alter database backup controlfile to trace as '/tmp/ctl'; Database altered. SQL> create table chf.t_xifenfei_www as select * from dba_objects where 1=0; Table created. SQL> insert into chf.t_xifenfei_www select * from dba_objects; 74749 rows created. --另外一个会话abort SQL> shutdown abort; ORACLE instance shut down. SQL> STARTUP NOMOUNT ORACLE instance started. Total System Global Area 175775744 bytes Fixed Size 1343668 bytes Variable Size 117444428 bytes Database Buffers 50331648 bytes Redo Buffers 6656000 bytes SQL> !vi /tmp/ctl.sql CREATE CONTROLFILE REUSE DATABASE "ORA11G" NORESETLOGS ARCHIVELOG MAXLOGFILES 16 MAXLOGMEMBERS 3 MAXDATAFILES 100 MAXINSTANCES 8 MAXLOGHISTORY 292 LOGFILE GROUP 1 '/u01/oracle/oradata/ora11g/redo01.log' SIZE 50M BLOCKSIZE 512, GROUP 2 '/u01/oracle/oradata/ora11g/redo02.log' SIZE 50M BLOCKSIZE 512, GROUP 3 '/u01/oracle/oradata/ora11g/redo03.log' SIZE 50M BLOCKSIZE 512 DATAFILE '/u01/oracle/oradata/ora11g/system01.dbf', '/u01/oracle/oradata/ora11g/sysaux01.dbf', -- '/u01/oracle/oradata/ora11g/undo02.dbf', '/u01/oracle/oradata/ora11g/users01.dbf', '/u01/oracle/oradata/ora11g/dbfs01.dbf', '/u01/oracle/oradata/ora11g/tts_xifenfei02.dbf', '/u01/oracle/oradata/ora11g/tts_xifenfei01.dbf', '/u01/oracle/oradata/ora11g/system02.dbf', '/u01/oracle/oradata/ora11g/czum01.dbf', '/u01/oracle/oradata/ora11g/undotbs02.dbf', '/u01/oracle/oradata/sp2008', '/u01/oracle/oradata/sp_2009', '/u01/oracle/oradata/sp_2010', '/u01/oracle/oradata/sp_2011', '/u01/oracle/oradata/sp_2012', '/u01/oracle/oradata/sp_2013', '/u01/oracle/oradata/sp_2014', '/u01/oracle/oradata/sp_2015', '/u01/oracle/oradata/sp_2016', '/u01/oracle/oradata/sp_2017', '/u01/oracle/oradata/sp_2018', '/u01/oracle/oradata/sp_2019', '/u01/oracle/oradata/sp_2020', '/u01/oracle/oradata/sp_2021', '/u01/oracle/oradata/sp_2022', '/u01/oracle/oradata/sp_2023', '/u01/oracle/oradata/sp_2024', '/u01/oracle/oradata/sp_2025', '/u01/oracle/oradata/sp_20max' CHARACTER SET ZHS16GBK ; "/tmp/ctl.sql" [New] 43L, 1519C written SQL> @/tmp/ctl.sql Control file created. SQL> recover database using backup controlfile until cancel; ORA-00279: change 12696930343864 generated at 05/18/2013 01:17:54 needed for thread 1 ORA-00289: suggestion : /u01/oracle/oradata/ora11g/archivelog/1_38_805394597.dbf ORA-00280: change 12696930343864 for thread 1 is in sequence #38 Specify log: {<RET>=suggested | filename | AUTO | CANCEL} /u01/oracle/oradata/ora11g/redo01.log ORA-00310: archived log contains sequence 37; sequence 38 required ORA-00334: archived log: '/u01/oracle/oradata/ora11g/redo01.log' ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below ORA-01194: file 1 needs more recovery to be consistent ORA-01110: data file 1: '/u01/oracle/oradata/ora11g/system01.dbf' SQL> recover database using backup controlfile until cancel; ORA-00279: change 12696930343864 generated at 05/18/2013 01:17:54 needed for thread 1 ORA-00289: suggestion : /u01/oracle/oradata/ora11g/archivelog/1_38_805394597.dbf ORA-00280: change 12696930343864 for thread 1 is in sequence #38 Specify log: {<RET>=suggested | filename | AUTO | CANCEL} /u01/oracle/oradata/ora11g/redo03.log ORA-00310: archived log contains sequence 39; sequence 38 required ORA-00334: archived log: '/u01/oracle/oradata/ora11g/redo03.log' ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below ORA-01194: file 1 needs more recovery to be consistent ORA-01110: data file 1: '/u01/oracle/oradata/ora11g/system01.dbf' SQL> recover database using backup controlfile until cancel; ORA-00279: change 12696930343864 generated at 05/18/2013 01:17:54 needed for thread 1 ORA-00289: suggestion : /u01/oracle/oradata/ora11g/archivelog/1_38_805394597.dbf ORA-00280: change 12696930343864 for thread 1 is in sequence #38 Specify log: {<RET>=suggested | filename | AUTO | CANCEL} /u01/oracle/oradata/ora11g/redo02.log ORA-00279: change 12696930370956 generated at 08/26/2013 13:00:25 needed for thread 1 ORA-00289: suggestion : /u01/oracle/oradata/ora11g/archivelog/1_39_805394597.dbf ORA-00280: change 12696930370956 for thread 1 is in sequence #39 ORA-00278: log file '/u01/oracle/oradata/ora11g/redo02.log' no longer needed for this recovery Specify log: {<RET>=suggested | filename | AUTO | CANCEL} /u01/oracle/oradata/ora11g/redo03.log Log applied. Media recovery complete. SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [25025], [3], [], [], [], [], [], [], [], [], [], [] Process ID: 12603 Session ID: 125 Serial number: 3
alert日志信息
ORA-279 signalled during: ALTER DATABASE RECOVER database using backup controlfile until cancel ... ALTER DATABASE RECOVER LOGFILE '/u01/oracle/oradata/ora11g/redo02.log' Media Recovery Log /u01/oracle/oradata/ora11g/redo02.log Mon Aug 26 13:05:00 2013 ORA-279 signalled during: ALTER DATABASE RECOVER LOGFILE '/u01/oracle/oradata/ora11g/redo02.log' ... Mon Aug 26 13:05:10 2013 ALTER DATABASE RECOVER LOGFILE '/u01/oracle/oradata/ora11g/redo03.log' Media Recovery Log /u01/oracle/oradata/ora11g/redo03.log Mon Aug 26 13:05:10 2013 Incomplete recovery applied all redo ever generated. Recovery completed through change 12696930370973 time 08/26/2013 13:00:56 Media Recovery Complete (ora11g) Completed: ALTER DATABASE RECOVER LOGFILE '/u01/oracle/oradata/ora11g/redo03.log' alter database open resetlogs RESETLOGS after complete recovery through change 12696930370973 Archived Log entry 1 added for thread 1 sequence 37 ID 0xfa6fa6cb dest 1: Archived Log entry 2 added for thread 1 sequence 38 ID 0xfa6fa6cb dest 1: Archived Log entry 3 added for thread 1 sequence 39 ID 0xfa6fa6cb dest 1: Clearing online redo logfile 1 /u01/oracle/oradata/ora11g/redo01.log Clearing online log 1 of thread 1 sequence number 37 Mon Aug 26 13:05:22 2013 Clearing online redo logfile 1 complete Clearing online redo logfile 2 /u01/oracle/oradata/ora11g/redo02.log Clearing online log 2 of thread 1 sequence number 38 Clearing online redo logfile 2 complete Clearing online redo logfile 3 /u01/oracle/oradata/ora11g/redo03.log Clearing online log 3 of thread 1 sequence number 39 Clearing online redo logfile 3 complete Resetting resetlogs activation ID 4201621195 (0xfa6fa6cb) Online log /u01/oracle/oradata/ora11g/redo01.log: Thread 1 Group 1 was previously cleared Online log /u01/oracle/oradata/ora11g/redo02.log: Thread 1 Group 2 was previously cleared Online log /u01/oracle/oradata/ora11g/redo03.log: Thread 1 Group 3 was previously cleared Mon Aug 26 13:05:33 2013 Setting recovery target incarnation to 2 Mon Aug 26 13:05:33 2013 Using SCN growth rate of 16384 per second Mon Aug 26 13:05:33 2013 Assigning activation ID 4220644150 (0xfb91eb36) LGWR: STARTING ARCH PROCESSES Mon Aug 26 13:05:33 2013 ARC0 started with pid=20, OS id=12679 ARC0: Archival started LGWR: STARTING ARCH PROCESSES COMPLETE ARC0: STARTING ARCH PROCESSES Mon Aug 26 13:05:35 2013 ARC1 started with pid=21, OS id=12683 Mon Aug 26 13:05:35 2013 ARC2 started with pid=22, OS id=12687 Mon Aug 26 13:05:36 2013 ARC3 started with pid=24, OS id=12691 ARC1: Archival started ARC2: Archival started ARC1: Becoming the 'no FAL' ARCH ARC1: Becoming the 'no SRL' ARCH ARC2: Becoming the heartbeat ARCH Thread 1 opened at log sequence 1 Current log# 1 seq# 1 mem# 0: /u01/oracle/oradata/ora11g/redo01.log Successful open of redo thread 1 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Mon Aug 26 13:05:36 2013 SMON: enabling cache recovery ARC3: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_12603.trc (incident=146705): ORA-00600: internal error code, arguments: [25025], [3], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/oracle/diag/rdbms/ora11g/ora11g/incident/incdir_146705/ora11g_ora_12603_i146705.trc Mon Aug 26 13:05:45 2013 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Undo initialization errored: err:600 serial:0 start:57601994 end:57610584 diff:8590 (85 seconds) Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_12603.trc: ORA-00600: internal error code, arguments: [25025], [3], [], [], [], [], [], [], [], [], [], [] Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_12603.trc: ORA-00600: internal error code, arguments: [25025], [3], [], [], [], [], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 12603): terminating the instance due to error 600 Instance terminated by USER, pid = 12603 ORA-1092 signalled during: alter database open resetlogs... opiodr aborting process unknown ospid (12603) as a result of ORA-1092 Mon Aug 26 13:05:47 2013 ORA-1092 : opitsk aborting process
trace文件
*** 2013-08-26 13:05:38.945 dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0) ----- Current SQL Statement for this session (sql_id=7j16t46cacjt9) ----- alter database open resetlogs ----- Call Stack Trace ----- calling call entry argument values in hex location type point (? means dubious value) -------------------- -------- -------------------- ---------------------------- skdstdst()+41 call kgdsdst() BF8A34E4 ? 2 ? ksedst1()+77 call skdstdst() BF8A34E4 ? 0 ? 1 ? 8592C38 ? 8592AB6 ? 8592C38 ? ksedst()+33 call ksedst1() 0 ? 1 ? dbkedDefDump()+2704 call ksedst() 0 ? BF8A40E7 ? 47EF1FF4 ? BF8A3D66 ? 0 ? BF8A3BF4 ? ksedmp()+47 call dbkedDefDump() 3 ? 2 ? ksfdmp()+59 call ksedmp() 3EB ? BF8A5458 ? F1DADED ? 116056E0 ? 3EB ? 116056E0 ? dbgexPhaseII()+1725 call 00000000 116056E0 ? 3EB ? dbgexProcessError() call dbgexPhaseII() B6C515AC ? B6A0C890 ? +2550 BF8A8D30 ? dbgeExecuteForError call dbgexProcessError() B6C515AC ? B6A0C890 ? 1 ? 0 ? ()+65 B6C515AC ? B6A0C890 ? dbgePostErrorKGE()+ call dbgeExecuteForError B6C515AC ? B6A0C890 ? 0 ? 1 ? 1794 () 0 ? dbkePostKGE_kgsf()+ call dbgePostErrorKGE() 116056E0 ? B6C330D4 ? 258 ? 50 kgeade()+324 call 00000000 116056E0 ? B6C330D4 ? 258 ? kgeriv_int()+107 call kgeade() 116056E0 ? 11605808 ? B6C330D4 ? 258 ? 0 ? 61C1 ? kgeriv()+25 call kgeriv_int() 116056E0 ? B6C330D4 ? 61C1 ? 0 ? 1 ? BF8A96B8 ? kgesiv()+98 call kgeriv() 116056E0 ? B6C330D4 ? 61C1 ? 1 ? BF8A96B8 ? ksesic1()+48 call kgesiv() 116056E0 ? B6C330D4 ? 61C1 ? 1 ? BF8A96B8 ? 61C1 ? 1 ?krta2rfn()+78 call ksesic1() 61C1 ? 0 ? 3 ? 0 ? kttsinfo()+496 call krta2rfn() 3 ? 0 ? 0 ? 0 ? 0 ? 0 ? ktusmout_online_ut( call kttsinfo() 9 ? 0 ? 0 ? BF8A9948 ? )+810 ktusmiut_init_ut()+ call ktusmout_online_ut( B000 ? 89E57F8 ? 9 ? BF8A96B8 ? 348 ) ktuini()+518 call ktusmiut_init_ut() 0 ? 0 ? 0 ? 0 ? 1 ? 0 ? adbdrv()+46345 call ktuini() 0 ? BF8A9DE0 ? 1004BF42 ? BF8AA258 ? B6A0BFFC ? 10EA1F20 ? opiexe()+16835 call adbdrv() 25C8F1F8 ? 0 ? 0 ? 2A34F528 ? 2A0400C8 ? BF8AF07C ? opiosq0()+3437 call opiexe() 4 ? 0 ? BF8AFE8C ? kpooprx()+239 call opiosq0() 3 ? E ? BF8B0184 ? A4 ? 0 ? kpoal8()+607 call kpooprx() BF8B2D6C ? BF8B10AC ? 1D ? 1 ? 0 ? A4 ? opiodr()+962 call 00000000 5E ? 1C ? BF8B2D68 ? ttcpip()+1930 call 00000000 5E ? 1C ? BF8B2D68 ? 0 ? opitsk()+1355 call ttcpip() 11616580 ? 5E ? BF8B2D68 ? 0 ? BF8B29F8 ? BF8B2E90 ? FDEBA80 ? 0 ? BF8B2EBC ? opiino()+827 call opitsk() 0 ? 0 ? opiodr()+962 call 00000000 3C ? 4 ? BF8B3E2C ? opidrv()+479 call opiodr() 3C ? 4 ? BF8B3E2C ? 0 ? sou2o()+80 call opidrv() 3C ? 4 ? BF8B3E2C ? opimai_real()+109 call sou2o() BF8B3E10 ? 3C ? 4 ? BF8B3E2C ? ssthrdmain()+212 call 00000000 2 ? BF8B3F58 ? 0 ? 47DA6F14 ? BF8B3F14 ? 47D9A670 ? main()+147 call ssthrdmain() 2 ? BF8B3F58 ? __libc_start_main() call 00000000 2 ? BF8B4054 ? BF8B4060 ? +220 47D9A828 ? 0 ? 1 ? _start()+33 call __libc_start_main() 856F3B4 ? 2 ? BF8B4054 ? BCC1EA0 ? BCC1E90 ? 47D8C790 ? --------------------- Binary Stack Dump ---------------------
MOS中有类似描述ORA-600 [25025] [25] While Opening the Clone Database in Resetlog Mode (Doc ID 603100.1),该解决方案是重建控制文件增加所有数据文件,在本次测试中,我就是人为除掉了undo,模拟undo丢失[其实数据库已经resetlogs过了,就算加入undo重建控制文件也不会成功(人工修改undo文件头除外)],又做了不正确的重建控制文件操作的故障,我提供解决方案如下
解决办法
--参数文件修改 undo_management='manual' --尝试open数据库 recover database; alter database open; --新建undo create undo tablespace undo_new datafile '' size 100m autoextend on next 10m maxsize 30G; --屏蔽需要恢复回滚段 select tablespace_name,segment_name,status from dba_rollback_segs; _corrupted_rollback_segments --重启数据库使得_corrupted_rollback_segments生效 shutdown immediate; startup --删除老undo drop tablespace old_undo --修改参数 shutdonw immediate undo_management='auto' undo_tablespace='unod_new' --启动数据库 startup --导出数据,导入新库
ksuapc : ORA-1033 foreground process starts before PMON
在11.2.0.1数据库中启动出现ksuapc : ORA-1033 foreground process starts before PMON错误
Starting ORACLE instance (normal) LICENSE_MAX_SESSION = 0 LICENSE_SESSIONS_WARNING = 0 Picked latch-free SCN scheme 3 Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST Autotune of undo retention is turned on. IMODE=BR ILAT =27 LICENSE_MAX_USERS = 0 SYS auditing is disabled Starting up: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options. Using parameter settings in server-side spfile /opt/oracle/product/11.2.0/dbhome_1/dbs/spfileora11bak.ora System parameters with non-default values: processes = 150 nls_language = "SIMPLIFIED CHINESE" nls_territory = "CHINA" sga_target = 1536M control_files = "/opt/oracle/oradata/ora11bak/control01.ctl" control_files = "/opt/oracle/flash_recovery_area/ora11bak/control02.ctl" db_block_size = 8192 compatible = "11.2.0.0.0" db_recovery_file_dest = "/opt/oracle/flash_recovery_area" db_recovery_file_dest_size= 3882M undo_tablespace = "UNDOTBS1" remote_login_passwordfile= "EXCLUSIVE" db_domain = "" dispatchers = "(PROTOCOL=TCP) (SERVICE=ora11bakXDB)" audit_file_dest = "/opt/oracle/admin/ora11bak/adump" audit_trail = "DB" db_name = "ora11bak" open_cursors = 300 pga_aggregate_target = 1595M diagnostic_dest = "/opt/oracle" Fri May 17 05:03:15 2013 ksuapc : ORA-1033 foreground process starts before PMON Fri May 17 05:03:15 2013 ksuapc : ORA-1033 foreground process starts before PMON Fri May 17 05:03:16 2013 ksuapc : ORA-1033 foreground process starts before PMON Fri May 17 05:03:16 2013 ksuapc : ORA-1033 foreground process starts before PMON Fri May 17 05:03:16 2013 ksuapc : ORA-1033 foreground process starts before PMON Fri May 17 05:03:16 2013 ksuapc : ORA-1033 foreground process starts before PMON …………
该错误的原因是数据库在启动过程中有前台进程连接数据库导致,该现象是数据库bug 8991997,该bug影响版本为:11.2.0.1/11.1.0.7,在11.2.0.1.1开始修复
ORA-600[2037]与ORA-07445[kcbs_dump_adv_state]错误
一台win oracle 数据库,重启后发现数据库无法访问,检查发现是Bug 4899479,但是oracle未提供完整的解决方法,这里根据自己对于数据库启动过程的理解,通过屏蔽前滚和回滚,拉起来数据库
数据库版本平台信息
ORACLE:11.1.0.7 OS:WIN 2008 R2 X64
数据库启动报错
Tue Apr 16 12:36:31 2013 alter database open Beginning crash recovery of 1 threads parallel recovery started with 7 processes Started redo scan Completed redo scan 28878 redo blocks read, 7353 data blocks need recovery Started redo application at Thread 1: logseq 7960, block 14132 Recovery of Online Redo Log: Thread 1 Group 1 Seq 7960 Reading mem 0 Mem# 0: D:\APP\SDWLJG-DB101\ORADATA\WLJG\REDO01.LOG Tue Apr 16 12:36:32 2013 RECOVERY OF THREAD 1 STUCK AT BLOCK 915068 OF FILE 9 Hex dump of (file 9, block 1698691) in trace file c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\trace\wljg_p001_1500.trc Corrupt block relative dba: 0x0259eb83 (file 9, block 1698691) Bad header found during crash/instance recovery Data in bad block: type: 0 format: 0 rdba: 0x0000a206 last change scn: 0x2359.0259eb83 seq: 0xf7 flg: 0x0b spare1: 0x0 spare2: 0x0 spare3: 0x601 consistency value in tail: 0x02c10243 check value in block header: 0x0 block checksum disabled Reread of rdba: 0x0259eb83 (file 9, block 1698691) found valid data Slave exiting with ORA-1172 exception Errors in file c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\trace\wljg_p001_1500.trc: ORA-01172: recovery of thread 1 stuck at block 915068 of file 9 ORA-01151: use media recovery to recover block, restore backup if needed Tue Apr 16 12:36:32 2013 Errors in file c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\trace\wljg_p003_4088.trc (incident=187558): ORA-00600: internal error code, arguments: [2037], [12619645], [41474], [6], [1], [247], [12619645], [0], [], [], [], [] Incident details in: c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\incident\incdir_187558\wljg_p003_4088_i187558.trc ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+1352] [ACCESS_VIOLATION] [ADDR:0xFFFFFFFFFFFFFFFF] [PC:0x16BFD20] [UNABLE_TO_READ] [] ORA-00600: internal error code, arguments: [2037], [12619645], [41474], [6], [1], [247], [12619645], [0], [], [], [], [] Incident details in: c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\incident\incdir_187559\wljg_p003_4088_i187559.trc Errors in file c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\trace\wljg_p006_1216.trc (incident=187567):
这里提示file 9 block 915068异常,但是通过dbv检查发现file 9无任何坏块.
trace文件内容
Dump continued from file: c:\app\sdwljg-db101\diag\rdbms\wljg\wljg\trace\wljg_p003_4088.trc ORA-00600: internal error code, arguments: [2037], [12620930], [41474], [2], [1], [247], [12619645], [0], [], [], [], [] ** DBGRL Error: ARB Alert Log ** DBGRL Error: <msg time='2013-04-16T11:05:58.522+08:00' org_id='oracle' comp_id='rdbms' msg_id='dbgexProcessError:1097:3370026720' type='TRACE' level='16' host_id='SDWLSCJG-DB' host_addr='172.18.1.15'> <txt>Incident details in: c:\app\sdwljg-db101\diag\rdbms\wljg\wlj ========= Dump for incident 129879 (ORA 600 [2037]) ======== *** 2013-04-16 11:05:58.522 ----- SQL Statement (None) ----- Current SQL information unavailable - no cursor. ----- Call Stack Trace ----- calling call entry argument values in hex location type point (? means dubious value) -------------------- -------- -------------------- ---------------------------- ksedst1()+111 CALL??? skdstdst()+0 000000000 000000000 01CFC9B80 000000200 ksedst()+63 CALL??? ksedst1()+0 000000005 021B00600 005D30C80 000002004 dbkedDefDump()+1012 CALL??? ksedst()+0 000000000 000000000 000000000 000000000 ksedmp()+51 CALL??? dbkedDefDump()+0 000000003 000000002 021AF92C0 000405038 __PGOSF184_ksfdmp() CALL??? ksedmp()+0 000000000 000000000 000000000 +27 27F00000000 dbgexPhaseII()+266 CALL??? __PGOSF184_ksfdmp() 00000000D 0082FAE50 000000000 +0 000000004 dbgexProcessError() CALL??? dbgexPhaseII()+0 021B00600 021AFCA50 000000201 +1313 000000000 dbgeExecuteForError CALL??? dbgexProcessError() 021B00600 021B07590 000000001 ()+55 +0 000000000 dbgePostErrorKGE()+ CALL??? dbgeExecuteForError 021AFCA30 021AFCA80 00000002E 1608 ()+0 000000005 dbkePostKGE_kgsf()+ CALL??? dbgePostErrorKGE()+ 01CFC99D0 021B0E080 000000258 65 0 021B0E080 kgeade()+556 CALL??? dbkePostKGE_kgsf()+ 000002000 000000000 000000009 0 000000004 kgeriv_int()+105 CALL??? kgeade()+0 3A4F00000003 000C09482 0FFFFFFFF 000000000 kgeriv()+27 CALL??? kgeriv_int()+0 3A9A024E0 000000000 01CFC9410 000000000 kgesiv()+102 CALL??? kgeriv()+0 0000008D5 0000008C3 021AFD9A0 000AFDC73 ksesic7()+125 CALL??? kgesiv()+0 006371F20 000000007 27F912000 200000004 kcoexam()+248 CALL??? ksesic7()+0 2000007F5 000000000 000C09482 000000000 kcbtema()+2154 CALL??? kcoexam()+0 27FFC22C8 39E113470 3A940BBB8 000000000 kcrpap()+355 CALL??? kcbtema()+0 27FFC22C8 28BFC2628 000000000 021B10200 kcrpdv()+1655 CALL??? kcrpap()+0 021B101A0 000000002 000000004 000000512 kxfprdp()+1384 CALL??? kcrpdv()+0 3A7AD3098 000000000 00000000C 00757CF00 opirip()+1396 CALL??? kxfprdp()+0 00000001E 005CDB518 021AFF9E0 000000000 opidrv()+855 CALL??? opirip()+0 000000032 000000004 021AFFD30 000000000 sou2o()+52 CALL??? opidrv()+213 000000032 000000004 021AFFD30 021AFFDB0 opimai_real()+295 CALL??? sou2o()+0 000000000 7FEFD9819B5 000000000 000000000 opimai()+96 CALL??? opimai_real()+0 000000000 000000000 000000000 000000000 BackgroundThreadSta CALL??? opimai()+0 021AFFE98 000000001 000000000 rt()+695 000000000 00000000775AF56D CALL??? BackgroundThreadSta 00A26B7A0 000000000 000000000 rt()+0 000000000 0000000077923281 CALL??? 00000000775AF560 000000000 000000000 000000000 000000000 --------------------- Binary Stack Dump ---------------------
查询mos发现During Startup (Open Database) Alert Log Shows ORA-600[2037] and ORA-7445[kcbs_dump_adv_state] [ID 551993.1]和我们这里展示的错误相符,引起该问题的原因主要是因为:The database may crash and fail to open due to undo/redo corruption if you are using distributed transactions.因为使用分布式事务的时候,数据库crash导致undo/redo corruption,从而使得数据库无法正常启动.
故障处理思路
因为通过数据库alert日志可以知道,数据库是在做前滚的时候并发进程失败,设置fast_start_parallel_rollback=false,禁止数据库实例恢复并发,可以恢复依然失败.因为前滚过不去,那就通过设置隐含参数禁止数据库前滚,在open数据库的过程中发现ora-600[2662]错误,推进scn,继续open数据库发现ora-600[4194],通过设置undo管理模式,屏蔽事务,屏蔽回滚段等方法,终于重新open库并重建undo,然后重建库算是完成恢复任务