标签云
asm 恢复 asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 ORACLE恢复 Oracle 恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (100)
- 数据库 (1,597)
- DB2 (22)
- MySQL (70)
- Oracle (1,463)
- Data Guard (49)
- EXADATA (7)
- GoldenGate (21)
- ORA-xxxxx (158)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (13)
- ORACLE 21C (3)
- Oracle ASM (65)
- Oracle Bug (7)
- Oracle RAC (47)
- Oracle 安全 (6)
- Oracle 开发 (27)
- Oracle 监听 (27)
- Oracle备份恢复 (530)
- Oracle安装升级 (84)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (75)
- PostgreSQL (17)
- PostgreSQL恢复 (5)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (36)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (19)
-
最近发表
- Oracle 19c/21c最新patch信息-202404
- PostgreSQL恢复系列:pg_filedump批量处理
- PostgreSQL部分主要字典信息
- PostgreSQL恢复系列:pg_filedump恢复字典构造
- PostgreSQL 16 源码安装
- ORA-00742 ORA-00312 恢复
- 数据库open成功后报ORA-00353 ORA-00354错误引起的一系列问题(本质ntfs文件系统异常)
- ORA-600 ktsiseginfo1故障
- ORA-00600: internal error code, arguments: [16703], [1403], [4] 原因
- 最近遇到几起ORA-600 16703故障(tab$被清空),请引起重视
- ORA-600 2662快速恢复之Patch scn工具
- TNS-12518: TNS:listener could not hand off client connection
- ora.storage无法启动报ORA-12514故障处理
- 断电引起文件scn异常数据库恢复
- ORA-16188: LOG_ARCHIVE_CONFIG settings inconsistent with previously started instance
- .[hudsonL@cock.li].mkp勒索加密数据库完美恢复
- 模拟带库实现rman远程备份
- 又一例:ORA-600 kclchkblk_4和2662故障
- Oracle误删除数据文件恢复
- Oracle 19C 备库DML重定向—DML Redirection
月归档:八月 2016
hp平台rdisk中磁盘丢失导致asm启动报ORA-15042恢复
有老朋友找到我,说一个客户的数据库异常,问题是asm无法正常mount,提示是缺少两块磁盘.问我是否可以恢复.因为是内网环境,通过他那边发过来的零零散散的信息,大概分析如下
asm alert日志报错
ERROR: diskgroup DGROUP1 was not mounted
Fri Aug 12 16:03:12 EAT 2016 SQL> alter diskgroup DGROUP1 mount Fri Aug 12 16:03:12 EAT 2016 NOTE: cache registered group DGROUP1 number=1 incarn=0xf6781b5c Fri Aug 12 16:03:12 EAT 2016 NOTE: Hbeat: instance first (grp 1) Fri Aug 12 16:03:16 EAT 2016 NOTE: start heartbeating (grp 1) Fri Aug 12 16:03:16 EAT 2016 NOTE: cache dismounting group 1/0xF6781B5C (DGROUP1) NOTE: dbwr not being msg'd to dismount ERROR: diskgroup DGROUP1 was not mounted
前台尝试mount asm 磁盘组报错ORA-15042
从这里可以明显的看出来asm 磁盘组无法正常mount,是由于缺少asm disk 15,16.如果想恢复asm,最好的方法就是找出来这两个磁盘.通过kfed对现在的磁盘进行分析,最后我们发现asm disk 14对应的磁盘为disk160,,asm disk 17对应的disk163,根据第一感觉很可能是disk161和disk161两块盘异常,让机房检查硬件无任何告警
OS层面分析
省略和本次结论无关的记录
ls -l /dev/rdisk crw-rw---- 1 oracle dba 13 0x000070 Jan 1 2016 disk160 crw-rw---- 1 oracle dba 13 0x000073 Jan 1 2016 disk163 ls -l /dev/disk brw-r----- 1 bin sys 1 0x000070 Jan 13 2015 disk160 brw-r----- 1 bin sys 1 0x000071 Jan 13 2015 disk161 brw-r----- 1 bin sys 1 0x000072 Jan 13 2015 disk162 brw-r----- 1 bin sys 1 0x000073 Jan 13 2015 disk163
这里我们发现在hp unix中/dev/disk下面磁盘都存在,但是/dev/rdisk下面丢失,通过ioscan相关命令继续分析
ioscan -fNnkC disk disk 160 64000/0xfa00/0x70 esdisk CLAIMED DEVICE HP OPEN-V /dev/disk/disk160 /dev/rdisk/disk160 disk 161 64000/0xfa00/0x71 esdisk CLAIMED DEVICE HP OPEN-V /dev/disk/disk161 disk 162 64000/0xfa00/0x72 esdisk CLAIMED DEVICE HP OPEN-V /dev/disk/disk162 disk 163 64000/0xfa00/0x73 esdisk CLAIMED DEVICE HP OPEN-V /dev/disk/disk163 /dev/rdisk/disk163
这里我们基本上可以确定是/dev/rdisk下面的盘发生丢失.进一步分析,因为rdisk是聚合后的盘符,那我们分析聚合前的盘符是否正常
ioscan -m dsf /dev/rdisk/disk160 /dev/rdsk/c29t12d4 /dev/rdsk/c28t12d4 /dev/rdisk/disk163 /dev/rdsk/c29t12d7 /dev/rdsk/c28t12d7 ls -l /dev/rdsk crw-r----- 1 bin sys 188 0x1dc000 Apr 22 2014 c29t12d0 crw-r----- 1 bin sys 188 0x1dc100 Apr 22 2014 c29t12d1 crw-r----- 1 bin sys 188 0x1dc300 Jan 13 2015 c29t12d3 crw-r----- 1 bin sys 188 0x1dc400 Jan 13 2015 c29t12d4 crw-r----- 1 bin sys 188 0x1dc500 Jan 13 2015 c29t12d5 crw-r----- 1 bin sys 188 0x1dc600 Jan 13 2015 c29t12d6 crw-r----- 1 bin sys 188 0x1dc700 Jan 13 2015 c29t12d7 crw-r----- 1 bin sys 188 0x1cc100 Apr 22 2014 c28t12d1 crw-r----- 1 bin sys 188 0x1cc300 Jan 13 2015 c28t12d3 crw-r----- 1 bin sys 188 0x1cc400 Jan 13 2015 c28t12d4 crw-r----- 1 bin sys 188 0x1cc500 Jan 13 2015 c28t12d5 crw-r----- 1 bin sys 188 0x1cc600 Jan 13 2015 c28t12d6 crw-r----- 1 bin sys 188 0x1cc700 Jan 13 2015 c28t12d7
通过这里我们基本上可以大概判断出来/dev/rdsk/c28t12d5,/dev/rdsk/c28t12d6,/dev/rdsk/c29t12d5,/dev/rdsk/c29t12d6就是我们需要找的/dev/rdisk/disk161和disk162的聚合之前的盘符.也就是说,现在我们判断只有/dev/rdisk下面的字符设备有问题,其他均正常.
通过系统命令修复异常
insf -e -H 64000/0xfa00/0x71 insf -e -H 64000/0xfa00/0x72
现在已经可以正常看到/dev/rdisk/disk161和/dev/rdisk/disk162盘符,初步判断,os层面盘符已经恢复正常.修改磁盘权限和所属组
chmod 660 /dev/rdisk/disk161 chmod 660 /dev/rdisk/disk162 chown oracle:dba /dev/rdisk/disk161 chown oracle:dba /dev/rdisk/disk162
正常启动asm,mount磁盘组,open数据库
这次的恢复,主要是从操作系统层面判断解决问题,从而实现数据库完美恢复,数据0丢失.有类似恢复案例:分区无法识别导致asm diskgroup无法mount
如果您遇到此类情况,无法解决请联系我们,提供专业ORACLE数据库恢复技术支持
Phone:17813235971 Q Q:107644445 E-Mail:dba@xifenfei.com
ORA-600 kcbz_check_objd_typ_1 处理
客户数据库异常(ORA-600 kcbz_check_objd_typ_1),让我们远程给分析处理
ORA-600 kcbz_check_objd_typ_1异常
Mon Aug 8 12:19:28 2016 Completed: ALTER DATABASE OPEN Mon Aug 8 12:19:29 2016 db_recovery_file_dest_size of 20480 MB is 0.00% used. This is a user-specified limit on the amount of space that will be used by this database for recovery-related files, and does not reflect the amount of space available in the underlying filesystem or ASM diskgroup. Mon Aug 8 12:19:33 2016 Errors in file /home/oracle/admin/RT/bdump/rt_smon_1514.trc: ORA-00600: internal error code, arguments: [kcbz_check_objd_typ_1], [0], [0], [1], [], [], [], [] Mon Aug 8 12:20:21 2016 Shutting down archive processes Mon Aug 8 12:20:26 2016 ARCH shutting down ARC3: Archival stopped Mon Aug 8 13:12:25 2016 Thread 1 advanced to log sequence 13804 Current log# 3 seq# 13804 mem# 0: /home/oracle/product/10.2.0/oradata/RT/redo03a.log Mon Aug 8 14:01:37 2016 Thread 1 advanced to log sequence 13805 Current log# 2 seq# 13805 mem# 0: /home/oracle/product/10.2.0/oradata/RT/redo02a.log Mon Aug 8 14:20:51 2016 Errors in file /home/oracle/admin/RT/bdump/rt_smon_1514.trc: ORA-00600: internal error code, arguments: [kcbz_check_objd_typ_1], [0], [0], [1], [], [], [], [] Mon Aug 8 15:54:47 2016 Thread 1 advanced to log sequence 13808 Current log# 2 seq# 13808 mem# 0: /home/oracle/product/10.2.0/oradata/RT/redo02a.log Mon Aug 8 16:21:48 2016 Errors in file /home/oracle/admin/RT/bdump/rt_smon_1514.trc: ORA-00600: internal error code, arguments: [kcbz_check_objd_typ_1], [0], [0], [1], [], [], [], [] Mon Aug 8 16:22:05 2016 Errors in file /home/oracle/admin/RT/bdump/rt_pmon_1500.trc: ORA-00474: SMON process terminated with error
这里比较明显,数据库报大量ORA-600 kcbz_check_objd_typ_1错误之后,然后smon进程终止,实例crash.
smon trace文件
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production With the Partitioning, OLAP and Data Mining options ORACLE_HOME = /home/oracle/product/10.2.0/db_1 System name: SunOS Node name: st104 Release: 5.10 Version: Generic_141445-09 Machine: i86pc Instance name: RT Redo thread mounted by this instance: 1 Oracle process number: 12 Unix process pid: 1514, image: oracle@st104 (SMON) *** 2016-08-08 12:19:26.868 *** SERVICE NAME:() 2016-08-08 12:19:26.868 *** SESSION ID:(383.1) 2016-08-08 12:19:26.868 Dead transaction 0x003d.002.0000f964 recovered by SMON Dead transaction 0x0041.017.00004d55 recovered by SMON Dead transaction 0x0047.002.0000180c recovered by SMON Dead transaction 0x004c.01c.00001b57 recovered by SMON *** SESSION ID:(383.1) 2016-08-08 12:19:27.470 DATA seg.obj=0, on-disk obj=925949, dsflg=0, dsobj=923715, cls=4 Formatted dump of block: buffer tsn: 4 rdba: 0x0100336b (4/13163) scn: 0x09c6.b2c7f7a2 seq: 0x02 flg: 0x04 tail: 0xf7a20602 frmt: 0x02 chkval: 0x649b type: 0x06=trans data Hex dump of block: st=0, typ_found=1 *** SESSION ID:(383.1) 2016-08-08 12:19:34.244 DATA seg.obj=0, on-disk obj=925950, dsflg=0, dsobj=923671, cls=4 Formatted dump of block: buffer tsn: 4 rdba: 0x01003343 (4/13123) scn: 0x09c6.b2c7f7dc seq: 0x02 flg: 0x04 tail: 0xf7dc0602 frmt: 0x02 chkval: 0x8013 type: 0x06=trans data Hex dump of block: st=0, typ_found=1 *** SESSION ID:(383.1) 2016-08-08 12:19:35.197 DATA seg.obj=0, on-disk obj=925941, dsflg=0, dsobj=923657, cls=4 Formatted dump of block: buffer tsn: 7 rdba: 0x01c03d53 (7/15699) scn: 0x09c6.b2c7f570 seq: 0x02 flg: 0x04 tail: 0xf5700602 frmt: 0x02 chkval: 0xe5c5 type: 0x06=trans data Hex dump of block: st=0, typ_found=1 *** SESSION ID:(383.1) 2016-08-08 12:19:38.965 DATA seg.obj=0, on-disk obj=925948, dsflg=0, dsobj=923656, cls=4 Formatted dump of block: buffer tsn: 7 rdba: 0x01c03a6b (7/14955) scn: 0x09c6.b2c7f745 seq: 0x02 flg: 0x04 tail: 0xf7450602 frmt: 0x02 chkval: 0x58c5 type: 0x06=trans data Hex dump of block: st=0, typ_found=1
这里可以看出来有block中的obj和dataobj不匹配.
查询seg$.type=3
type=3为临时对象,由于异常原因导致smon在清理temp对象无法正常完成,从而使得smon终止,实例crash.
SQL> select file#, block#, ts# from seg$ where type# = 3; FILE# BLOCK# TS# ---------- ---------- ---------- 4 13163 4 4 13123 4 7 15699 7 7 14955 7
ORA-600 kcbz_check_objd_typ_1处理方法
1) Check tablespace bitmap SQL> oradebug setmypid SQL> exec dbms_space_admin.tablespace_verify('&TBSP_NAME') SQL> oradebug tracefile_name or if the tablespace involved is an ASSM tablespace: SQL> oradebug setmypid SQL> exec dbms_space_admin.assm_tablespace_verify ('&TBSP_NAME',dbms_space_admin.TS_VERIFY_BITMAPS) SQL> oradebug tracefile_name I am expecting to fail 2) Corrupt these temp segments SQL> exec dbms_space_admin.segment_corrupt('&TBSP_NAME', &FILE#, &BLOCK#) 3) Drop them SQL> exec dbms_space_admin.segment_drop_corrupt('&TBSP_NAME', &FILE#, &BLOCK#) 4) Rebuild tablespace bitmap exec DBMS_SPACE_ADMIN.TABLESPACE_REBUILD_BITMAPS('&TBSP_NAME') 5) Verify the tablespace again SQL> oradebug setmypid SQL> exec dbms_space_admin.tablespace_verify('&TBSP_NAME') SQL> oradebug tracefile_name or if the tablespace involved is an ASSM tablespace: SQL> oradebug setmypid SQL> exec dbms_space_admin.assm_tablespace_verify('&TBSP_NAME',dbms_space_admin.TS_VERIFY_BITMAPS) SQL> oradebug tracefile_name
linux 7(redhat,oracle linux,centos)中使用udev
慢慢的linux 7的使用人越来越多了,但是linux 7相对于5和6的版本,变动确实比较大,本文主要描写在linux 7中如何实现udev,实现设备持久化,权限和所属组的修改
linux版本
Oracle Linux Server release 7.1 [root@www.xifenfei.com ~]# uname -a Linux www.xifenfei.com 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 04:05:24 PST 2015 x86_64 x86_64 x86_64 GNU/Linux
VMware Workstation中显示uuid需要在vmx文件中增加
disk.enableUUID = "TRUE"
查看磁盘分区
[root@www.xifenfei.com ~]# fdisk -l Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0xf60fe217 Device Boot Start End Blocks Id System /dev/sdb1 2048 2099199 1048576 83 Linux Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x000bce7c Device Boot Start End Blocks Id System /dev/sda1 2048 4204543 2101248 8e Linux LVM /dev/sda2 * 4204544 79702015 37748736 83 Linux Disk /dev/sdc: 32.2 GB, 32212254720 bytes, 62914560 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/mapper/ol-swap: 2147 MB, 2147483648 bytes, 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
查看磁盘uuid
[root@www.xifenfei.com ~]# /usr/lib/udev/scsi_id -g -u /dev/sdb1 36000c29e91831cedbe69afe6cc08daf7 [root@www.xifenfei.com ~]# /usr/lib/udev/scsi_id -g -u /dev/sdc 36000c292495e9d9de6f21640cc7b53b9
udev绑定
[root@www.xifenfei.com ~]# more /etc/udev/rules.d/99-my-asmdevices.rules KERNEL=="sd*[!0-9]", ENV{DEVTYPE}=="disk", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode", RESULT=="36000c292495e9d9de6f21640cc7b53b9", RUN+="/bin/sh -c 'mknod /dev/xifenfei-sdc b $major $minor; chown oracle:dba /dev/xifenfei-sdc; chmod 0660 /dev/xifenfei-sdc'" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="36000c29e91831cedbe69afe6cc08daf7", SYMLINK+="xifenfei-sdb1", OWNER="oracle", GROUP="dba", MODE="0660"
绑定结果
[root@www.xifenfei.com ~]# ls -l /dev/xifenfei-* lrwxrwxrwx. 1 root root 4 Aug 7 22:49 /dev/xifenfei-sdb1 -> sdb1 brw-rw----. 1 oracle dba 8, 32 Aug 7 22:36 /dev/xifenfei-sdc [root@www.xifenfei.com ~]# ls -l /dev/sdb1 brw-rw----. 1 oracle dba 8, 17 Aug 7 22:49 /dev/sdb1
udev只修改磁盘权限
[root@www.xifenfei.com ~]# fdisk /dev/sdb Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): n Partition type: p primary (1 primary, 0 extended, 3 free) e extended Select (default p): p Partition number (2-4, default 2): First sector (2099200-41943039, default 2099200): Using default value 2099200 Last sector, +sectors or +size{K,M,G} (2099200-41943039, default 41943039): +1G Partition 2 of type Linux and of size 1 GiB is set Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@www.xifenfei.com ~]# more /etc/udev/rules.d/99-my-asmdevices.rules KERNEL=="sd?2", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="36000c29e91831cedbe69afe6cc08daf7", OWNER="oracle", GROUP="dba", MODE="0660" [root@www.xifenfei.com ~]# /sbin/udevadm trigger --type=devices --action=change [root@www.xifenfei.com ~]# ls -l /dev/sdb2 brw-rw----. 1 oracle dba 8, 18 Aug 7 23:14 /dev/sdb2
这里可以发现在linux 7中使用了两种方法绑定udev,一种是真实生成udev设备,另外一种是通过软连接实现.感谢lunar(Lunar的oracle实验室)在linux 7学习中的帮助