修改11.2 RAC 的 SCAN IP

发表于 2016 年 3 月 25 日 由 惜分飞

在某些情况下,由于是C/S架构,从以前的单机系统迁移到现在11.2的rac中,如果修改客户端ip地址工作量太大,而且也不现实,一般建议直接修改scan ip地址和以前一样,从而实现业务直接访问scan ip实现应用不用一个个单独配置.这里通过简单演示,实现修改scan ip的过程(网段不变),主要是把scan名字为scan-xff的ip地址从192.168.137.245修改为192.168.137.248
查看当前scan ip信息

[root-www.xifenfei.com@xff1 ~]# ping xff-scan
PING xff-scan (192.168.137.245) 56(84) bytes of data.
^C
--- xff-scan ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1738ms
[root-www.xifenfei.com@xff1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.LISTENER.lsnr
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.asm
               ONLINE  ONLINE       xff1                     Started             
               ONLINE  ONLINE       xff2                     Started             
ora.gsd
               OFFLINE OFFLINE      xff1                                         
               OFFLINE OFFLINE      xff2                                         
ora.net1.network
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.ons
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       xff1                                         
ora.cvu
      1        ONLINE  ONLINE       xff1                                         
ora.oc4j
      1        ONLINE  ONLINE       xff1                                         
ora.scan1.vip
      1        ONLINE  ONLINE       xff1                                         
ora.xff1.vip
      1        ONLINE  ONLINE       xff1                                         
ora.xff2.vip
      1        ONLINE  ONLINE       xff2                                         
ora.xffdb.db
      1        ONLINE  ONLINE       xff1                     Open                
      2        ONLINE  ONLINE       xff2                     Open   
[root-www.xifenfei.com@xff1 ~]# srvctl config scan
SCAN name: xff-scan, Network: 1/192.168.137.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /xff-scan/192.168.137.245

修改scan ip

[root-www.xifenfei.com@xff1 ~]# srvctl stop scan_listener
[root-www.xifenfei.com@xff1 ~]# srvctl stop scan
[root-www.xifenfei.com@xff1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.LISTENER.lsnr
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.asm
               ONLINE  ONLINE       xff1                     Started             
               ONLINE  ONLINE       xff2                     Started             
ora.gsd
               OFFLINE OFFLINE      xff1                                         
               OFFLINE OFFLINE      xff2                                         
ora.net1.network
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.ons
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        OFFLINE OFFLINE                                                   
ora.cvu
      1        ONLINE  ONLINE       xff1                                         
ora.oc4j
      1        ONLINE  ONLINE       xff1                                         
ora.scan1.vip
      1        OFFLINE OFFLINE                                                   
ora.xff1.vip
      1        ONLINE  ONLINE       xff1                                         
ora.xff2.vip
      1        ONLINE  ONLINE       xff2                                         
ora.xffdb.db
      1        ONLINE  ONLINE       xff1                     Open                
      2        ONLINE  ONLINE       xff2                     Open 

--如果是dns,注意修改dns中scan ip信息,如果是hosts文件注意多个节点都需要修改
[root-www.xifenfei.com@xff1 ~]# ping xff-scan
PING xff-scan (192.168.137.248) 56(84) bytes of data.
^C
--- xff-scan ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1738ms

[root-www.xifenfei.com@xff1 ~]# srvctl modify scan -n xff-scan
[root-www.xifenfei.com@xff1 ~]# srvctl config scan
SCAN name: xff-scan, Network: 1/192.168.137.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /xff-scan/192.168.137.248
[root-www.xifenfei.com@xff1 ~]# srvctl start scan
[root-www.xifenfei.com@xff1 ~]# srvctl start scan_listener
[root-www.xifenfei.com@xff1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.LISTENER.lsnr
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.asm
               ONLINE  ONLINE       xff1                     Started             
               ONLINE  ONLINE       xff2                     Started             
ora.gsd
               OFFLINE OFFLINE      xff1                                         
               OFFLINE OFFLINE      xff2                                         
ora.net1.network
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
ora.ons
               ONLINE  ONLINE       xff1                                         
               ONLINE  ONLINE       xff2                                         
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       xff2                                         
ora.cvu
      1        ONLINE  ONLINE       xff1                                         
ora.oc4j
      1        ONLINE  ONLINE       xff1                                         
ora.scan1.vip
      1        ONLINE  ONLINE       xff2                                         
ora.xff1.vip
      1        ONLINE  ONLINE       xff1                                         
ora.xff2.vip
      1        ONLINE  ONLINE       xff2                                         
ora.xffdb.db
      1        ONLINE  ONLINE       xff1                     Open                
      2        ONLINE  ONLINE       xff2                     Open

查看修改后的scan listener状态

xff2:/home/grid> lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 12-MAR-2016 17:02:32

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                12-MAR-2016 16:59:05
Uptime                    0 days 0 hr. 3 min. 27 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/oracle/app/grid/network/admin/listener.ora
Listener Log File         /u01/oracle/app/grid/log/diag/tnslsnr/xff2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.137.248)(PORT=1521)))
The listener supports no services
The command completed successfully

[root-www.xifenfei.com@xff2 ~]# su - oracle
xff2:/home/oracle> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Sat Mar 12 17:01:11 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> alter system register;

System altered.

SQL> /

System altered.

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
xff2:/home/oracle> exit
logout
[root-www.xifenfei.com@xff2 ~]# su - grid
xff2:/home/grid> lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 12-MAR-2016 17:01:24

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                12-MAR-2016 16:59:05
Uptime                    0 days 0 hr. 2 min. 18 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/oracle/app/grid/network/admin/listener.ora
Listener Log File         /u01/oracle/app/grid/log/diag/tnslsnr/xff2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.137.248)(PORT=1521)))
The listener supports no services
The command completed successfully

这里发现修改scan ip之后,scan listener没有正确或者到监听的动态注册信息,哪怕人工执行了alter system register;也不行.

通过重启数据库,解决修改scan ip后的动态监听注册问题

[root-www.xifenfei.com@xff2 ~]# su - oracle
xff2:/home/oracle> srvctl stop database -d xffdb
xff2:/home/oracle> srvctl start database -d xffdb
xff2:/home/oracle> exit
logout
[root-www.xifenfei.com@xff2 ~]# su - grid
xff2:/home/grid> lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 12-MAR-2016 17:06:17

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                12-MAR-2016 16:59:05
Uptime                    0 days 0 hr. 7 min. 11 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/oracle/app/grid/network/admin/listener.ora
Listener Log File         /u01/oracle/app/grid/log/diag/tnslsnr/xff2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.137.248)(PORT=1521)))
Services Summary...
Service "xffdb" has 2 instance(s).
  Instance "xffdb1", status READY, has 1 handler(s) for this service...
  Instance "xffdb2", status READY, has 1 handler(s) for this service...
Service "xffdbXDB" has 2 instance(s).
  Instance "xffdb1", status READY, has 1 handler(s) for this service...
  Instance "xffdb2", status READY, has 1 handler(s) for this service...
The command completed successfully
发表在 Oracle RAC | 标签为 , | 评论关闭

ORA-01172 ORA-01151 故障恢复

发表于 2016 年 3 月 24 日 由 惜分飞

有客户存储异常断电,导致数据库启动报ORA-01172错,导致数据库无法open
数据库启动报ORA-01172错误

Wed Mar 23 14:16:23 2016
ALTER DATABASE OPEN
Wed Mar 23 14:16:24 2016
Beginning crash recovery of 1 threads
 parallel recovery started with 15 processes
Wed Mar 23 14:16:24 2016
Started redo scan
Wed Mar 23 14:16:25 2016
Completed redo scan
 62588 redo blocks read, 15 data blocks need recovery
Wed Mar 23 14:16:25 2016
Started redo application at
 Thread 1: logseq 15050, block 2, scn 2439828667
Wed Mar 23 14:16:25 2016
Recovery of Online Redo Log: Thread 1 Group 1 Seq 15050 Reading mem 0
  Mem# 0 errs 0: /oracle/oradata/orcl/redo01.log
Wed Mar 23 14:16:25 2016
Completed redo application
Wed Mar 23 14:16:25 2016
RECOVERY OF THREAD 1 STUCK AT BLOCK 26185 OF FILE 3
Wed Mar 23 14:16:25 2016
RECOVERY OF THREAD 1 STUCK AT BLOCK 69385 OF FILE 3
Wed Mar 23 14:16:25 2016
RECOVERY OF THREAD 1 STUCK AT BLOCK 566 OF FILE 2
Wed Mar 23 14:16:25 2016
RECOVERY OF THREAD 1 STUCK AT BLOCK 89 OF FILE 2
Wed Mar 23 14:16:25 2016
RECOVERY OF THREAD 1 STUCK AT BLOCK 53769 OF FILE 3
Wed Mar 23 14:16:26 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p012_6540.trc:
ORA-01172: recovery of thread 1 stuck at block 566 of file 2
ORA-01151: use media recovery to recover block, restore backup if needed
Wed Mar 23 14:16:26 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p008_6532.trc:
ORA-01172: recovery of thread 1 stuck at block 53769 of file 3
ORA-01151: use media recovery to recover block, restore backup if needed
Wed Mar 23 14:16:26 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p011_6538.trc:
ORA-01172: recovery of thread 1 stuck at block 69385 of file 3
ORA-01151: use media recovery to recover block, restore backup if needed
Wed Mar 23 14:16:26 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p005_6526.trc:
ORA-01172: recovery of thread 1 stuck at block 26185 of file 3
ORA-01151: use media recovery to recover block, restore backup if needed
Wed Mar 23 14:16:27 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p014_6544.trc:
ORA-01172: recovery of thread 1 stuck at block 89 of file 2
ORA-01151: use media recovery to recover block, restore backup if needed
Wed Mar 23 14:16:27 2016
Aborting crash recovery due to slave death, attempting serial crash recovery
Wed Mar 23 14:16:27 2016
Beginning crash recovery of 1 threads
Wed Mar 23 14:16:27 2016
Started redo scan
Wed Mar 23 14:16:27 2016
Completed redo scan
 62588 redo blocks read, 15 data blocks need recovery
Wed Mar 23 14:16:27 2016
Started redo application at
 Thread 1: logseq 15050, block 2, scn 2439828667
Wed Mar 23 14:16:27 2016
Recovery of Online Redo Log: Thread 1 Group 1 Seq 15050 Reading mem 0
  Mem# 0 errs 0: /oracle/oradata/orcl/redo01.log
RECOVERY OF THREAD 1 STUCK AT BLOCK 566 OF FILE 2
Wed Mar 23 14:16:27 2016
Aborting crash recovery due to error 1172
Wed Mar 23 14:16:27 2016
Errors in file /oracle/admin/orcl/udump/orcl_ora_6514.trc:
ORA-01172: recovery of thread 1 stuck at block 566 of file 2
ORA-01151: use media recovery to recover block, restore backup if needed
ORA-1172 signalled during: ALTER DATABASE OPEN...

ALTER DATABASE RECOVER datafile 1 报错
尝试recover datafile 1之后报ORA-600 kcbrapply_4,ORA-600 kcfrbd_3,ORA-600 kcbrapply_12等错误,从报错信息看,出现这些错误的原因,是由于断电导致坏块引起.

Thu Mar 24 21:50:18 2016
ALTER DATABASE RECOVER  datafile 1  
Thu Mar 24 21:50:18 2016
Media Recovery Start
 parallel recovery started with 15 processes
Thu Mar 24 21:50:18 2016
Recovery of Online Redo Log: Thread 1 Group 1 Seq 15050 Reading mem 0
  Mem# 0 errs 0: /oracle/oradata/orcl/redo01.log
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p004_13391.trc:
ORA-00600: internal error code, arguments: [kcbrapply_4], [2], [], [], [], [], [], []
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p010_13403.trc:
ORA-00600: internal error code, arguments: [kcbrapply_4], [0], [], [], [], [], [], []
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p000_13383.trc:
ORA-00600: internal error code, arguments: [kcbrapply_4], [0], [], [], [], [], [], []
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p009_13401.trc:
ORA-00600: internal error code, arguments: [kcbrapply_4], [3], [], [], [], [], [], []
Thu Mar 24 21:50:19 2016
Hex dump of (file 1, block 61562) in trace file /oracle/admin/orcl/bdump/orcl_p001_13385.trc
Corrupt block relative dba: 0x0040f07a (file 1, block 61562)
Bad header found during media recovery
Data in bad block:
 type: 0 format: 0 rdba: 0xf07a0000
 last change scn: 0x916c.dc4b0040 seq: 0x0 flg: 0x00
 spare1: 0x6 spare2: 0xa2 spare3: 0xb088
 consistency value in tail: 0x06010fc1
 check value in block header: 0x601
 block checksum disabled
Thu Mar 24 21:50:19 2016
Hex dump of (file 1, block 55706) in trace file /oracle/admin/orcl/bdump/orcl_p014_13411.trc
Corrupt block relative dba: 0x0040d99a (file 1, block 55706)
Bad header found during media recovery
Data in bad block:
 type: 0 format: 0 rdba: 0xd99a0000
 last change scn: 0x916c.e1ad0040 seq: 0x0 flg: 0x00
 spare1: 0x6 spare2: 0xa2 spare3: 0xa520
 consistency value in tail: 0x06012222
 check value in block header: 0x601
 block checksum disabled
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p006_13395.trc:
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [3342335], [1], [0], [64000], [], []
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p003_13389.trc:
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [3932159], [1], [0], [64000], [], []
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p002_13387.trc:
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [2293759], [1], [0], [64000], [], []
Reread of rdba: 0x0040d99a (file 1, block 55706) found valid data
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p014_13411.trc:
ORA-00600: internal error code, arguments: [kcbrapply_12], [], [], [], [], [], [], []
Thu Mar 24 21:50:19 2016
Reread of rdba: 0x0040f07a (file 1, block 61562) found valid data
Thu Mar 24 21:50:19 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p001_13385.trc:
ORA-00600: internal error code, arguments: [kcbrapply_12], [], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p014_13411.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9782BF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_12], [], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p006_13395.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9C82BF4] [] []
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [3342335], [1], [0], [64000], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p009_13401.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9A02BF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_4], [3], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p003_13389.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9F02AF4] [] []
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [3932159], [1], [0], [64000], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p004_13391.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xBA182AF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_4], [2], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p010_13403.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xBA402AF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_4], [0], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p000_13383.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9282AF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_4], [0], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p001_13385.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+450] [SIGSEGV] [Address not mapped to object] [0xB9C82AF4] [] []
ORA-00600: internal error code, arguments: [kcbrapply_12], [], [], [], [], [], [], []
Thu Mar 24 21:50:23 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p002_13387.trc:
ORA-10562: Error occurred while applying redo to data block (file# 1, block# 11042)
ORA-10564: tablespace SYSTEM
ORA-01110: data file 1: '/oracle/oradata/orcl/system01.dbf'
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [kcfrbd_3], [1], [2293759], [1], [0], [64000], [], []

ALTER DATABASE RECOVER datafile 3 报错
该文件恢复主要报ORA-600 kcbrsearchflist_2,ORA-600 kdxlin:psno out of range,ORA-600 kcbs_dump_adv_state等错误

Thu Mar 24 21:52:04 2016
ALTER DATABASE RECOVER  datafile 3  
Thu Mar 24 21:52:04 2016
Media Recovery Start
 parallel recovery started with 15 processes
Thu Mar 24 21:52:04 2016
Recovery of Online Redo Log: Thread 1 Group 1 Seq 15050 Reading mem 0
  Mem# 0 errs 0: /oracle/oradata/orcl/redo01.log
Thu Mar 24 21:52:05 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p007_13462.trc:
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], []
Thu Mar 24 21:52:05 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p001_13450.trc:
ORA-00600: internal error code, arguments: [kcbrsearchflist_2], [], [], [], [], [], [], []
Thu Mar 24 21:52:05 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p007_13462.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9F076F4] [] []
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], []
Thu Mar 24 21:52:05 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p001_13450.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9C874F4] [] []
ORA-00600: internal error code, arguments: [kcbrsearchflist_2], [], [], [], [], [], [], []
Thu Mar 24 21:52:06 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p007_13462.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9F066F4] [] []
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9F076F4] [] []
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], []
Thu Mar 24 21:52:06 2016
Errors in file /oracle/admin/orcl/bdump/orcl_p001_13450.trc:
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9C864F4] [] []
ORA-07445: exception encountered: core dump [kcbs_dump_adv_state()+464] [SIGSEGV] [Address not mapped to object] [0xB9C874F4] [] []
ORA-00600: internal error code, arguments: [kcbrsearchflist_2], [], [], [], [], [], [], []

恢复过程

SQL> startup mount
ORACLE instance started.

Total System Global Area 2147483648 bytes
Fixed Size          1220432 bytes
Variable Size             369098928 bytes
Database Buffers         1761607680 bytes
Redo Buffers               15556608 bytes
Database mounted.
SQL> select file# from v$datafile;

     FILE#
----------
         1
         2
         3
         4
         5
         6

6 rows selected.

SQL> recover datafile 1;
ORA-03113: end-of-file on communication channel

SQL> startup mount
ORACLE instance started.

Total System Global Area 2147483648 bytes
Fixed Size          1220432 bytes
Variable Size             369098928 bytes
Database Buffers         1761607680 bytes
Redo Buffers               15556608 bytes
Database mounted.

SQL> recover datafile 3;
ORA-03113: end-of-file on communication channel

SQL> startup mount
ORACLE instance started.

Total System Global Area 2147483648 bytes
Fixed Size          1220432 bytes
Variable Size             369098928 bytes
Database Buffers         1761607680 bytes
Redo Buffers               15556608 bytes
Database mounted.
SQL> recover datafile 5;
Media recovery complete.
SQL> recover datafile 6;
Media recovery complete.
SQL> recover datafile 4;
Media recovery complete.
SQL> recover datafile 2;
Media recovery complete.
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-00607: Internal error occurred while making a change to a data block
ORA-00600: internal error code, arguments: [4194], [57], [11], [], [], [], [],
[]

SQL> select open_mode from v$database;

OPEN_MODE
----------
READ WRITE

这次运气不错,system坏的是mon_mods$,undo异常可以重建,基本上可以说没有数据丢失,数据库恢复完成.
重要的库,通过open过程报错信息,分析可能的坏块所属对象,然后确定处理方法,以免造成永久性数据块损坏.

发表在 Oracle备份恢复 | 标签为 , , , , , , , | 评论关闭

insert into aud$引起高版本问题导致ORA-600[17059]

发表于 2016 年 3 月 20 日 由 惜分飞

昨天晚上有朋友咨询我,他数据库(win2008 11.2.0.1 单机)出现大量ORA-00600[17059],如下错误,让帮忙分析原因
alert日志报错

Sat Mar 19 21:31:02 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_3336.trc  (incident=45304):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:06 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_4168.trc  (incident=45166):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
Sat Mar 19 21:31:09 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2928.trc  (incident=45342):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
Sat Mar 19 21:31:12 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_5812.trc  (incident=45399):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:17 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_5884.trc  (incident=45255):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:21 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2976.trc  (incident=45305):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:24 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_6068.trc  (incident=45256):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:28 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_6044.trc  (incident=45351):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-28001: 口令已经失效
Sat Mar 19 21:31:32 2016
Errors in file d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_2928.trc  (incident=45343):
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []
ORA-02002: 写入审计线索时出错

这里虽然报了ORA-00600[17059],ORA-28001,ORA-02002但是根据经验感觉很可能是由于ORA-00600[17059]错误导致后面的其他两个错误.

trace文件信息

Dump continued from file: d:\app\administrator\diag\rdbms\orcl\orcl\trace\orcl_ora_5484.trc
ORA-00600: 内部错误代码, 参数: [17059], [0x49BE74CB0], [0x49BE75548], [0x4A9A0FFD0], [], [], [], [], [], [], [], []

========= Dump for incident 45253 (ORA 600 [17059]) ========

*** 2016-03-19 21:28:34.244
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=4vs91dcv7u1p6) -----
insert into sys.aud$( sessionid,entryid,statement,ntimestamp#, userid,userhost,terminal,action#,returncode,
 obj$creator,obj$name,auth$privileges,auth$grantee, new$owner,new$name,ses$actions,ses$tid,logoff$pread, 
logoff$lwrite,logoff$dead,comment$text,spare1,spare2,  priv$used,clientid,sessioncpu,proxy$sid,user$guid, 
instance#,process#,xid,scn,auditid,  sqlbind,sqltext,obj$edition,dbid)  values(:1,:2,:3,SYS_EXTRACT_UTC(SYSTIMESTAMP),   
  :4,:5,:6,:7,:8,     :9,:10,:11,:12,     :13,:14,:15,:16,:17,     :18,:19,:20,:21,:22,     :23,:24,:25,:26,:27,  
   :28,:29,:30,:31,:32,     :33,:34,:35,:36)

----- Call Stack Trace -----
calling              call     entry                argument values in hex      
location             type     point                (? means dubious value)     
-------------------- -------- -------------------- ----------------------------
ksedst1()+129        CALL???  skdstdst()           009173DA2 000000000 000000000
                                                   000000000
ksedst()+69          CALL???  ksedst1()            000000002 000000000 006F605E0
                                                   000000000
dbkedDefDump()+4536  CALL???  ksedst()             000000287 000000000 000000000
                                                   000000000
ksedmp()+43          CALL???  dbkedDefDump()       000000003 000000002 000000000
                                                   000468E71
ksfdmp()+87          CALL???  ksedmp()             000000000 000000000 000000000
                                                   000000000
dbgexPhaseII()+1819  CALL???  ksfdmp()             000000000 000000000 000000000
                                                   000000000
dbgexProcessError()  CALL???  dbgexPhaseII()       021160570 02116D448 00000B0C5
+2563                                              000000002
dbgeExecuteForError  CALL???  dbgexProcessError()  021160570 021167540 000000001
()+65                                              000000000
dbgePostErrorKGE()+  CALL???  dbgeExecuteForError  025455460 00000000C 000000001
1726                          ()                   4A9A0FFD0
dbkePostKGE_kgsf()+  CALL???  dbgePostErrorKGE()   025455460 024690040 000000258
75                                                 7F883243A05D
kgeade()+560         CALL???  dbkePostKGE_kgsf()   000000001 000000000 00A684E58
                                                   00A5553E2
kgeriv_int()+111     CALL???  kgeade()             024444C49 000000000 000000000
                                                   000000000
kgeriv()+29          CALL???  kgeriv_int()         000000000 000000001 02B474DD0
                                                   000000001
kgesiv()+105         CALL???  kgeriv()             461A8AD00 7F883243A05D
                                                   025455460 000000001
kgesic3()+60         CALL???  kgesiv()             401122000 000000000 009B4310C
                                                   4A9A0EC80
kgltba()+739         CALL???  kgesic3()            000000200 6236313231656434
                                                   4000042A3 000000002
kglhdgc()+384        CALL???  kgltba()             4A095BC50 461A31858 02B476418
                                                   000000001
kglLock()+3063       CALL???  kglhdgc()            000000000 000020C68 4ADE08E18
                                                   000000008
kglget()+403         CALL???  kglLock()            02B476008 02B475360 02B475360
                                                   0004D112F
kxsGetLookupLock()+  CALL???  kglget()             025455460 02B4756A0 000000001
327                                                000000003
kksfbc()+14464       CALL???  kxsGetLookupLock()   4AC4EDFB0 0004FD4CA 0246CC6B8
                                                   4AC4EDFB0
kkspsc0()+2117       CALL???  kksfbc()             0246CC6B8 000000003 000000008
                                                   0070D7F80
kksParseCursor()+18  CALL???  kkspsc0()            0246B19B8 0070D7F80 000000256
1                                                  000000003
opiosq0()+2538       CALL???  kksParseCursor()     000000000 025454EA0 000000000
                                                   0033917DF
opiosq()+23          CALL???  opiosq0()            000000003 00000000F 02B478A28
                                                   025450000
opiodr()+1662        CALL???  opiosq()             0746E6172 000000002 06567656C
                                                   000000000
rpidrus()+862        CALL???  opiodr()             00000004A 00000000F 02B478A28
                                                   000000001
rpidru()+154         CALL???  rpidrus()            02B478028 000000001 000000000
                                                   000400000
rpiswu2()+2757       CALL???  rpidru()             02B4788B0 000000000
                                                   1D181E32DAE2234 000000000
rpidrv()+6105        CALL???  rpiswu2()            4AC4EC300 000000000 02B478680
                                                   000000002
rpisplu()+1607       CALL???  rpidrv()             400000001 7F880000004A
                                                   02B478A28 000000008
audins()+2562        CALL???  rpisplu()            000000001 000000000 000000000
                                                   000000000
audlon()+1286        CALL???  audins()             000005028 0070D9274 0070D9280
                                                   000000036
auddft()+2140        CALL???  audlon()             006F79DE0 0092ACAF1 000000000
                                                   000000000
kpolnb()+4007        CALL???  auddft()             0246AF458 000000064 000000000
                                                   000000000
kpolon()+237         CALL???  kpolnb()             1D181E300000051 02B47AF20
                                                   000000000 000026161
opiodr()+1662        CALL???  kpolon()             000000001 000000000 000000000
                                                   00A42F224
ttcpip()+1325        CALL???  opiodr()             4800000000000051 40000001A
                                                   02B47E100 000000000
opitsk()+2040        CALL???  ttcpip()             02546F180 000000000 000000000
                                                   000000000
opiino()+1258        CALL???  opitsk()             000000000 000000000 000000000
                                                   02B47F9F8
opiodr()+1662        CALL???  opiino()             00000003C 000000004 02B47FAB0
                                                   000000000
opidrv()+864         CALL???  opiodr()             00000003C 000000004 02B47FAB0
                                                   615C3A6400000000
sou2o()+98           CALL???  opidrv()+150         00000003C 000000004 02B47FAB0
                                                   000000000
opimai_real()+158    CALL???  sou2o()              1D181E32DAE2234 000000000
                                                   150013000307E0 601B80009001C
opimai()+191         CALL???  opimai_real()        000000000 02B47FC68 01D3200A0
                                                   02B47FC68
OracleThreadStart()  CALL???  opimai()             000000000 006DF0B34 0000000E0
+724                                               0000027CC
0000000076E1652D     CALL???  OracleThreadStart()  02957FF18 000000000 000000000
                                                   000000000
0000000076F4C521     CALL???  0000000076E16520     000000000 000000000 000000000
                                                   000000000
 

--------------------- Binary Stack Dump ---------------------

LibraryObject:  Address=9be74cb0 HeapMask=0000-0001-0001 Flags=EXS[0000] Flags2=[0000] PublicFlags=[0000] 
  ChildTable:  size='32768' 
    Child:  id='0' Table=9be75b60 Reference=9be75600 Handle=a9a0eb20 
    Child:  id='1' Table=9be75b60 Reference=9be61ed8 Handle=a9a06700 
    Child:  id='2' Table=9be75b60 Reference=9be621f0 Handle=a9a06500 
    Child:  id='3' Table=9be75b60 Reference=9be62528 Handle=a9997200 
    Child:  id='4' Table=9be75b60 Reference=9be62818 Handle=a99597a8 
    Child:  id='5' Table=9be75b60 Reference=9be62b50 Handle=a99cd210 
    Child:  id='6' Table=9be75b60 Reference=9bd0a418 Handle=a994a308 
    Child:  id='7' Table=9be75b60 Reference=9bd0a750 Handle=a9920980 
    Child:  id='8' Table=9be75b60 Reference=9bd0aa40 Handle=a99a6d48 
    Child:  id='9' Table=9be75b60 Reference=9bd0ad78 Handle=a99a6918 
    Child:  id='10' Table=9be75b60 Reference=9bd0b068 Handle=a9993388 
    …………
    Child:  id='32764' Table=becbd118 Reference=bec882d8 Handle=522c1118 
    Child:  id='32765' Table=becbd118 Reference=bec7c708 Handle=52319608 
    Child:  id='32766' Table=becbd118 Reference=bec7c9f8 Handle=522c0f18 
    Child:  id='32767' Table=becbd118 Reference=bec7cd30 Handle=522a7858

结合trace信息,我们可以发现高版本数量已经到了32768,理论最大值.因此数据库报了ORA-600[17059]错误(以前写过类似文章:因为高版本引起ORA-00600[17059]),并且引起了其他的ORA-错误.通过查询数据库高版本信息,在重启一会儿的库中,发现
insert-aud$

过几分钟后
insert-aud$-高版本

通过这里,进一步诊断,引起高版本的sql，是由于aud$表相关的插入时绑定参数导致（如果需要可以进一步分析是由于什么原因导致了高版本）.这里根据经验具体原因已经不再重要，对于11.2.0.1版本,本身bug比较多,且暂时无法升级
处理方法
1.因为本库的本地审计意义不大直接从数据库层面关闭

audit_trail='none'

2.在11g的初始版本中,acs特性导致子游标过多,建议关闭

_optimizer_adaptive_cursor_sharing=false 
_optimizer_extended_cursor_sharing=none 
_optimizer_extended_cursor_sharing_rel=none

3.限制数据库sql游标数量，当游标超过该数量直接抛弃重新硬解析

_cursor_features_enabled=34
event='106001 trace name context forever,level 100'

通过以上配置，重启数据库之后,运行一天alert日志未再出现任何错误,通过查询mos匹配Bug 10196339 : ORA600[17059] OCCURS DUE TO BIND_MISMATCH

发表在 Oracle Bug | 标签为 , , , | 2 条评论