某医院存储掉线导致Oracle数据库故障恢复

发表于 2015 年 11 月 9 日 由 惜分飞

xx医院存储突然掉线,导致数据库异常,现场工程师折腾了一天,问题依旧没有解决,无奈之下找到我们,希望我们能够帮忙恢复数据库.
启动报ORA-00600[2131]错误

Fri Nov 06 14:50:59 2015
ALTER DATABASE   MOUNT
This instance was first to mount
Fri Nov 06 14:50:59 2015
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.4.4)(PORT=1521))' SCOPE=MEMORY SID='xifenfei1';
NOTE: Loaded library: System 
SUCCESS: diskgroup DATA was mounted
NOTE: dependency between database xifenfei and diskgroup resource ora.DATA.dg is established
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13221.trc  (incident=191085):
ORA-00600: internal error code, arguments: [2131], [33], [32], [], [], [], [], [], [], [], [], []
Incident details in: /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_191085/xifenfei1_ora_13221_i191085.trc
Fri Nov 06 14:51:10 2015
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
ORA-600 signalled during: ALTER DATABASE   MOUNT...

出现该错误的原因是由于：We are attempting to write a controlfile checkpoint progress record, but find we do not have the progress record generating this exception.由于控制文件异常导致,出现此类情况,我们一般使用单个控制文件一次尝试,如果都不可以考虑重建控制文件

由于坏块(逻辑/物理)导致数据库实例恢复无法进行

Beginning crash recovery of 2 threads
Started redo scan
kcrfr_rnenq: use log nab 393216
kcrfr_rnenq: use log nab 2
Completed redo scan
 read 4427 KB redo, 500 data blocks need recovery
Started redo application at
 Thread 1: logseq 5731, block 391398
 Thread 2: logseq 4252, block 520815
Recovery of Online Redo Log: Thread 1 Group 2 Seq 5731 Reading mem 0
  Mem# 0: +DATA/xifenfei/onlinelog/group_2.266.835331047
Recovery of Online Redo Log: Thread 2 Group 8 Seq 4252 Reading mem 0
  Mem# 0: +DATA/xifenfei/onlinelog/group_8.331.835330421
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc  (incident=197486):
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], [], [], [], [], []
Incident details in:/home/app/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_197486/xifenfei1_ora_13770_i197486.trc
Fri Nov 06 15:03:09 2015
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc  (incident=197487):
ORA-01578: ORACLE data block corrupted (file # 2, block # 65207)
ORA-01110: data file 2: '+DATA/xifenfei/datafile/sysaux.257.835324753'
ORA-10564: tablespace SYSAUX
ORA-01110: data file 2: '+DATA/xifenfei/datafile/sysaux.257.835324753'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 81045
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], [], [], [], [], []
Incident details in:/home/app/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_197487/xifenfei1_ora_13770_i197487.trc
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc:
ORA-01578: ORACLE data block corrupted (file # 2, block # 65207)
ORA-01110: data file 2: '+DATA/xifenfei/datafile/sysaux.257.835324753'
ORA-10564: tablespace SYSAUX
ORA-01110: data file 2: '+DATA/xifenfei/datafile/sysaux.257.835324753'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 81045
ORA-00600: internal error code, arguments: [kdxlin:psno out of range], [], [], [], [], [], [], [], [], [], [], []
Recovery of Online Redo Log: Thread 2 Group 3 Seq 4253 Reading mem 0
  Mem# 0: +DATA/xifenfei/onlinelog/group_3.332.835330505
Hex dump of (file 14, block 62536) in trace file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc
Reading datafile '+DATA/xifenfei/datafile/ht01.dbf' for corruption at rdba: 0x0380f448 (file 14, block 62536)
Reread (file 14, block 62536) found same corrupt data (logically corrupt)
RECOVERY OF THREAD 1 STUCK AT BLOCK 62536 OF FILE 14
Fri Nov 06 15:03:13 2015
Abort recovery for domain 0
Aborting crash recovery due to error 1172
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc:
ORA-01172: recovery of thread 1 stuck at block 62536 of file 14
ORA-01151: use media recovery to recover block, restore backup if needed
Abort recovery for domain 0
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13770.trc:
ORA-01172: recovery of thread 1 stuck at block 62536 of file 14
ORA-01151: use media recovery to recover block, restore backup if needed
ORA-1172 signalled during: ALTER DATABASE OPEN...

查看资料发现和Bug 14301592 – Several errors by corrupt blocks shifted by 2 bytes in buffer cache during recovery caused by INDEX redo apply，可以通过ALLOW 1 CORRUPTION临时解决

使用ALLOW 1 CORRUPTION进行恢复,出现ORA-07445[kdxlin]错误

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
+DATA/xifenfei/onlinelog/group_3.332.835330505     
ORA-00279: change 700860458 generated at 11/05/2015 21:20:15 needed for thread
1
ORA-00289: suggestion : +ARCHIVE/xifenfei/xifenfei_1_5731_835324843.arc
ORA-00280: change 700860458 for thread 1 is in sequence #5731


Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
+DATA/xifenfei/onlinelog/group_2.266.835331047
ORA-00283: recovery session canceled due to errors
ORA-10562: Error occurred while applying redo to data block (file# 2, block#
70104)
ORA-10564: tablespace SYSAUX
ORA-01110: data file 2: '+DATA/xifenfei/datafile/sysaux.257.835324753'
ORA-10561: block type 'TRANSACTION MANAGED INDEX BLOCK', data object# 82289
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kdxlin()+4088] [SIGSEGV]
[ADDR:0xC] [PC:0x95FB572] [Address not mapped to object] []


ORA-01112: media recovery not started

ORA-07445[kdxlin()+4088]未找到类似说明,到了这一步,无法简单的恢复成功,只能通过设置隐含参数跳过实例恢复,尝试resetlog库

通过设置_allow_resetlogs_corruption参数继续恢复

SQL> startup pfile='/tmp/pfile.ora' mount;
ORACLE instance started.

Total System Global Area 7315603456 bytes
Fixed Size                  2267384 bytes
Variable Size            2566915848 bytes
Database Buffers         4731174912 bytes
Redo Buffers               15245312 bytes
Database mounted.
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-00600: internal error code, arguments: [kclchkblk_4], [0], [700869927],
[0], [700860464], [], [], [], [], [], [], []
Process ID: 13563
Session ID: 157 Serial number: 3

alert日志报错

Fri Nov 06 19:26:39 2015
SMON: enabling cache recovery
Instance recovery: looking for dead threads
Instance recovery: lock domain invalid but no dead threads
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13563.trc  (incident=319140):
ORA-00600: internal error code, arguments: [kclchkblk_4], [0], [700869927], [0], [700860464], [], [], [], [], [], [], []
Incident details in:/home/app/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_319140/xifenfei1_ora_13563_i319140.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Redo thread 2 internally disabled at seq 1 (CKPT)
ARC1: Archiving disabled thread 2 sequence 1
Archived Log entry 9956 added for thread 2 sequence 1 ID 0x0 dest 1:
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13563.trc:
ORA-00600: internal error code, arguments: [kclchkblk_4], [0], [700869927], [0], [700860464], [], [], [], [], [], [], []
Errors in file /home/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_13563.trc:
ORA-00600: internal error code, arguments: [kclchkblk_4], [0], [700869927], [0], [700860464], [], [], [], [], [], [], []
Error 600 happened during db open, shutting down database
USER (ospid: 13563): terminating the instance due to error 600
Fri Nov 06 19:26:42 2015
Instance terminated by USER, pid = 13563
ORA-1092 signalled during: alter database open resetlogs...
opiodr aborting process unknown ospid (13563) as a result of ORA-1092
Fri Nov 06 19:26:42 2015
ORA-1092 : opitsk aborting process

这里是比较熟悉的ora-600[kclchkblk_4]错误,和ora-600[2662]错误类似,需要调整scn,由于数据库版本为11.2.0.4,无法使用常规方法调整scn,在修改控制文件,oradebug,bbed方法可供选择

使用oradebug方法处理
因为是asm环境,其他方法处理起来都相对麻烦

[oracle@wisetop1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Fri Nov 6 19:30:59 2015

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup pfile='/tmp/pfile.ora' mount;
ORACLE instance started.

Total System Global Area 7315603456 bytes
Fixed Size                  2267384 bytes
Variable Size            2566915848 bytes
Database Buffers         4731174912 bytes
Redo Buffers               15245312 bytes
Database mounted.
SQL> oradebug setmypid
Statement processed.
SQL> oradebug poke 0x06001AE70 4 0x2FAF0800
BEFORE: [06001AE70, 06001AE74) = 00000000
AFTER:  [06001AE70, 06001AE74) = 2FAF0800
SQL> alter database open;

Database altered.

至此数据库open成功,后续就是处理一些坏块的工作,并建议客户逻辑重建库.

发表在 非常规恢复 | 标签为 , , , | 2 条评论

Oracle 12C Active Data Guard Far Sync 配置

发表于 2015 年 11 月 5 日 由 惜分飞

Active Data Guard Far Sync是Oracle 12c的新功能（也称为Far Sync Standby），Far Sync功能的实现是通过在距离主库(Primary Database)相对较近的地点配置Far Sync实例，主库(Primary Database) 同步(synchronous)传输redo到Far Sync实例，然后Far Sync实例再将redo异步(asynchronous)传输到终端备库(Standby Database)。这样既可以保证零数据丢失又可以降低主库压力。Far Sync实例只有密码文件，init参数文件和控制文件，而没有数据文件。 如果redo 传输采用Maximum Availability模式，我们可以在距离生产中心(Primary Database)相对较近的地点配置Far Sync实例，主库(Primary Database)同步(synchronous)传输redo到Far Sync实例，保证零数据丢失（zero data loss），同时主库和Far Sync距离较近，网络延时很小，因此对主库性能影响很小。然后Far Sync实例再将redo异步(asynchronous)发送到终端备库(Standby Database)。 如果redo 传输采用Maximum Performance模式，我们可以在距离生产中心(Primary Database)相对较近的地点配置Far Sync实例，主库(Primary Database) 异步传输redo到Far Sync实例，然后Far Sync实例再负责传输redo到其他多个终端备库(Standby Database)。这样可以减少主库向多个终端备库(Standby Database)传输redo的压力（offload）。 Far Sync配置对于Data Guard 角色转换(role transitions)是透明的，即switchover/failover命令方式与12c之前相同。
1.主库配置fra

SQL> alter system set DB_RECOVERY_FILE_DEST_SIZE=200G;

System altered.

SQL> alter system set db_recovery_file_dest='/u01/app/oracle/fast_recovery_area';

System altered.

2.启用归档模式和强制日志

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.

Total System Global Area 2516582400 bytes
Fixed Size                  4585912 bytes
Variable Size             671090248 bytes
Database Buffers         1828716544 bytes
Redo Buffers               12189696 bytes
Database mounted.
SQL> alter database archivelog;

Database altered.

SQL> alter database force logging;

Database altered.

SQL> alter database open;

Database altered.

3.主机规划

192.168.137.121  ora1221   --->主库(ORCL12C)
192.168.137.122  ora1222   --->Far Sync实例(ORCL12CFS)
192.168.137.123  ora1223   --->备库(ORCL12CDG)

4.tns配置

ORCL12C =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ora1221)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = orcl12c)
    )
  )

ORCL12CDG =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ora1223)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = orcl12c)
    )
  )

ORCL12CFS =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = ora1222)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = orcl12c)
    )
  )

5.参数文件配置

--主库
db_unique_name='orcl12c'
service_names='orcl12c'
log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST valid_for=(all_logfiles,all_roles) db_unique_name=orcl12c'
log_archive_dest_2='service=orcl12cfs lgwr  sync  AFFIRM MAX_FAILURE=1 ALTERNATE=LOG_ARCHIVE_DEST_3 valid_for=(online_logfiles,primary_role) db_unique_name=orcl12cfs'
LOG_ARCHIVE_DEST_3='SERVICE=orcl12cdg  lgwr ASYNC ALTERNATE=LOG_ARCHIVE_DEST_2 VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=orcl12cdg'
LOG_ARCHIVE_DEST_STATE_3=ALTERNATE
log_archive_config='dg_config=(orcl12c,orcl12cdg,orcl12cfs)'
standby_file_management=auto
db_file_name_convert='/u01/app/oracle/oradata/orcl12c/','/u01/app/oracle/oradata/orcl12c/'
log_file_name_convert='/u01/app/oracle/oradata/orcl12c/','/u01/app/oracle/oradata/orcl12c/'
fal_server='orcl12cdg','orcl12cfs'

--Far Sync实例
db_unique_name='orcl12cfs'
service_names='orcl12c'
log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST valid_for=(all_logfiles,all_roles) db_unique_name=orcl12cfs'
log_archive_dest_2='service=orcl12cdg lgwr  async valid_for=(STANDBY_LOGFILES,STANDBY_ROLE) db_unique_name=orcl12cdg'
log_archive_config='dg_config=(orcl12c,orcl12cdg,orcl12cfs)'
standby_file_management=manual
fal_server='orcl12c'

--备库
db_unique_name='orcl12cdg'
service_names='orcl12c'
log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST valid_for=(all_logfiles,all_roles) db_unique_name=orcl12cdg'
log_archive_dest_2='service=orcl12c lgwr  async   valid_for=(online_logfiles,primary_role) db_unique_name=orcl12c'
log_archive_config='dg_config=(orcl12c,orcl12cdg,orcl12cfs)'
standby_file_management=auto
db_file_name_convert='/u01/app/oracle/oradata/orcl12c/','/u01/app/oracle/oradata/orcl12c/'
log_file_name_convert='/u01/app/oracle/oradata/orcl12c/','/u01/app/oracle/oradata/orcl12c/'
fal_server='orcl12c','orcl12cfs'

6.密码文件
拷贝主库的密码文件到Far Sync实例和备库$ORACLE_HOME/dbs下(如果是win在%ORACLE_HOME%/database中)

7.创建Far Sync实例
创建和参数文件匹配的fra,adump目录

--主库
ALTER DATABASE CREATE FAR SYNC INSTANCE CONTROLFILE AS '/tmp/controlfs01.ctl';
拷贝到Far Sync实例对应的控制文件位置

--Far Sync实例
startup pfile='/tmp/pfile' mount

ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 10 '/u01/app/oracle/oradata/orcl12c/std_redo10.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 11 '/u01/app/oracle/oradata/orcl12c/std_redo11.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 12 '/u01/app/oracle/oradata/orcl12c/std_redo12.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 13 '/u01/app/oracle/oradata/orcl12c/std_redo13.log' size 50m reuse;

ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION;

8.创建备库
创建和参数文件匹配的fra,adump目录

--主库备份（传输至备库）
backup database format '/tmp/ora12c_%U';

--主库创建standby ctl（传输至备库和备库参数文件路径一致）
alter database create standby controlfile as '/tmp/controlst.ctl';

--启动备库至mount
startup pfile='/tmp/pfile' mount;

--备库注册备份
catalog start with '/tmp/xifenfei/';

--备库还原数据文件并恢复
restore database;
recover database;

--增加standby redo
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 10 '/u01/app/oracle/oradata/orcl12c/std_redo10.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 11 '/u01/app/oracle/oradata/orcl12c/std_redo11.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 12 '/u01/app/oracle/oradata/orcl12c/std_redo12.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 13 '/u01/app/oracle/oradata/orcl12c/std_redo13.log' size 50m reuse;

--备库开启mrp进程
ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT  LOGFILE DISCONNECT FROM SESSION;

--主库增加standby redo
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 10 '/u01/app/oracle/oradata/orcl12c/std_redo10.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 11 '/u01/app/oracle/oradata/orcl12c/std_redo11.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 12 '/u01/app/oracle/oradata/orcl12c/std_redo12.log' size 50m reuse;
ALTER DATABASE ADD STANDBY LOGFILE thread 1  GROUP 13 '/u01/app/oracle/oradata/orcl12c/std_redo13.log' size 50m reuse;

9.配置结果

SQL> select * from V$DATAGUARD_CONFIG;

DB_UNIQUE_NAME                 PARENT_DBUN                    DEST_ROLE         CURRENT_SCN     CON_ID
------------------------------ ------------------------------ ----------------- ----------- ----------
orcl12c                        NONE                           PRIMARY DATABASE      1950551          0
orcl12cfs                      orcl12c                        FAR SYNC INSTANCE     1950390          0
orcl12cdg                      orcl12cfs                      PHYSICAL STANDBY      1950390          0

SQL> select group#, status, thread#, sequence#, first_change#, next_change# from v$standby_log;

    GROUP# STATUS        THREAD#  SEQUENCE# FIRST_CHANGE# NEXT_CHANGE#
---------- ---------- ---------- ---------- ------------- ------------
        10 ACTIVE              1         27       1863140
        11 UNASSIGNED          1          0
        12 UNASSIGNED          1          0
        13 UNASSIGNED          1          0

SQL> select PROTECTION_MODE,PROTECTION_LEVEL from v$database;

PROTECTION_MODE      PROTECTION_LEVEL
-------------------- --------------------
MAXIMUM PERFORMANCE  MAXIMUM PERFORMANCE

10.调整保护级别
从MAXIMUM PERFORMANCE调整为MAXIMUM AVAILABILITY

--主库
SQL> startup mount;
ORACLE instance started.

Total System Global Area 2516582400 bytes
Fixed Size                  4585912 bytes
Variable Size             671090248 bytes
Database Buffers         1828716544 bytes
Redo Buffers               12189696 bytes
Database mounted.
SQL> alter database set standby database to maximize availability;

Database altered.

SQL>  select PROTECTION_MODE,PROTECTION_LEVEL from v$database;

PROTECTION_MODE      PROTECTION_LEVEL
-------------------- --------------------
MAXIMUM AVAILABILITY UNPROTECTED

SQL> alter database open;

Database altered.

SQL>  select PROTECTION_MODE,PROTECTION_LEVEL from v$database;

PROTECTION_MODE      PROTECTION_LEVEL
-------------------- --------------------
MAXIMUM AVAILABILITY MAXIMUM AVAILABILITY

--Far Sync实例
SQL>  select PROTECTION_MODE,PROTECTION_LEVEL from v$database;

PROTECTION_MODE      PROTECTION_LEVEL
-------------------- --------------------
MAXIMUM AVAILABILITY MAXIMUM AVAILABILITY

--备库
SQL> select PROTECTION_MODE,PROTECTION_LEVEL from v$database;

PROTECTION_MODE      PROTECTION_LEVEL
-------------------- --------------------
MAXIMUM AVAILABILITY RESYNCHRONIZATION
发表在 Data Guard, ORACLE 12C | 标签为 , , | 评论关闭

dns解析导致opiodr aborting process unknown ospid (7266) as a result of ORA-609类似错误

发表于 2015 年 10 月 29 日 由 惜分飞

数据库alert日志中出现大量类似opiodr aborting process unknown ospid (7266) as a result of ORA-609错误

Tue Oct 27 07:51:10 2015
opiodr aborting process unknown ospid (7266) as a result of ORA-609
Tue Oct 27 08:16:56 2015
opiodr aborting process unknown ospid (7660) as a result of ORA-609
Tue Oct 27 08:17:16 2015
opiodr aborting process unknown ospid (7666) as a result of ORA-609
Tue Oct 27 08:21:50 2015
opiodr aborting process unknown ospid (7725) as a result of ORA-609
Tue Oct 27 08:37:43 2015
opiodr aborting process unknown ospid (7940) as a result of ORA-609
Tue Oct 27 08:51:25 2015
opiodr aborting process unknown ospid (8126) as a result of ORA-609
Tue Oct 27 09:07:16 2015
opiodr aborting process unknown ospid (8359) as a result of ORA-609
Tue Oct 27 09:26:09 2015
opiodr aborting process unknown ospid (8600) as a result of ORA-609

测试tnsping

[oracle@Sql admin]$ tnsping orcl

TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 28-OCT-2015 01:40:55

Copyright (c) 1997, 2009, Oracle.  All rights reserved.

Used parameter files:
/opt/oracle/product/11.2.0/db_1/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.2.162)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl.Sql.xifenfei.com)))
OK (20120 msec)
[oracle@Sql admin]$ tnsping orcl

TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 28-OCT-2015 01:42:35

Copyright (c) 1997, 2009, Oracle.  All rights reserved.

Used parameter files:
/opt/oracle/product/11.2.0/db_1/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.2.162)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl.Sql.xifenfei.com)))
OK (41600 msec)

hosts文件配置

[oracle@Sql admin]$ vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 
192.168.2.162 Sql.xifenfei.com
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

同时测试lsnrctl start/stop/status都很慢,但是ping localhost非常快,证明问题在监听之上.search 关键字lsnrctl slow/tnsping slow关键字发现有相关文档:
Bug 8307164 : TNSPING 11G USING DNS AND NOT HOSTS FILE
Listener Startup or Connections Hang in 11g (Doc ID 803838.1)
11g: Remote Connections Take Very Long to Establish (Doc ID 856820.1)
通过这些文档,可以知道在oracle 10g中解析主机名使用gethostbyname()函数,11g中使用了使用getaddrinfo()函数.这个函数对于/etc/nsswitch.conf的调用和ethostbyname()不一样,从而出现此类问题.根据官方建议:

Linux
hosts: files [NOTFOUND=continue] dns OR  hosts:files/dns

HPUX and Solaris
ipnodes: files [NOTFOUND=continue] dns

本次解决方法
因为对于没有使用到dns服务器的数据库环境而言,没有必要配置这玩意,给自己埋雷，估计是当时安装yum之时没有清理掉

vi /etc/nsswitch.conf
hosts:      files nds  -->hosts:      files

再次测试tnsping,lsnrctl等命令,速度ok

[oracle@Sql"]$ tnsping orcl

TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 28-OCT-2015 01:58:30

Copyright (c) 1997, 2009, Oracle.  All rights reserved.

Used parameter files:
/opt/oracle/product/11.2.0/db_1/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.2.162)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl.Sql.xifenfei.com)))
OK (10 msec)

对于此类监听慢的问题，可以通过对client/server端trace进一步跟踪

TRACE_LEVEL_CLIENT = 16
TRACE_FILE_CLIENT = client/server
TRACE_DIRECTORY_CLIENT = [any valid directory path]
TRACE_TIMESTAMP_CLIENT = ON
DIAG_ADR_ENABLED=off
发表在 Oracle 监听 | 标签为 , , | 评论关闭