分类目录归档:Linux

Using mlock ulimits for SHM_HUGETLB is deprecated

Oracle 数据库运行在linux 6中,启用大页之后,我们经常在/var/log/messages里面会看到类似这样的记录:Mar 11 12:12:33 i-q2ghx82t kernel: oracle (3677): Using mlock ulimits for SHM_HUGETLB is deprecated,我这里的环境也有重现该问题
环境说明

--系统配置
[root@i-q2ghx82t ~]# more /etc/issue
CentOS release 6.8 (Final)
Kernel \r on an \m

[root@i-q2ghx82t ~]# ulimit  -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 128331
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 128331
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

[oracle@i-q2ghx82t ~]$ cat /proc/meminfo|grep Hu
AnonHugePages:         0 kB
HugePages_Total:   10752
HugePages_Free:    10752
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

[root@i-q2ghx82t ~]# more  /proc/sys/vm/hugetlb_shm_group
0

[root@i-q2ghx82t ~]# id oracle
uid=1000(oracle) gid=1000(oinstall) groups=1000(oinstall),1001(dba),1002(oper),1005(asmdba)

--数据库参数
use_large_pages=only

在本环境中数据库启动正常,大页也可以正常使用,但是在系统日志中有类似Mar 11 12:12:33 i-q2ghx82t kernel: oracle (3677): Using mlock ulimits for SHM_HUGETLB is deprecated 这样的告警.通过分析,是由于少配置了hugetlb_shm_group参数导致(vm.hugetlb_shm_group 参数设置为有权使用 HugePages 的操作系统组。默认情况下,此参数设置为 0,从而允许所有组使用 HugePages。可以将此参数设置为 Oracle 数据库进程所属的操作系统组,如 oinstall),在本系统中在sysctl.conf中增加vm.hugetlb_shm_group=1000,然后重启系统(测试中,如果只是重启数据库,非系统重启后第一次重启数据库,不会出现该告警),系统日志没有出现相关告警.
在Linux 6中配置大页建议加上对应的hugetlb_shm_group参数

发表在 Linux | 标签为 , | 评论关闭

redhat和oracle linux kernel对应关系

Red Hat Enterprise Linux Version / Update

Red Hat Enterprise Linux – Kernel version / redhat-release string

Oracle Linux – Kernel version / release strings

 Red Hat Enterprise Linux 7

 Red Hat Enterprise Linux 7 Update 6

 3.10.0-957.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo)

 4.14.35-1818.3.3.el7uek.x86_64 ^ * (x86_64 only)
3.10.0-957.el7.x86_64 ^ (x86_64 only)
redhat-release: Red Hat Enterprise Linux Server release 7.6 (Maipo)
system-release: Oracle Linux Server release 7.6
oracle-release: Oracle Linux Server release 7.6

 Red Hat Enterprise Linux 7 Update 5

 3.10.0-862.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.5 (Maipo)

 4.1.12-112.16.4.el7uek ^ * (x86_64 only)
3.10.0-862.el7.x86_64 ^ (x86_64 only)
redhat-release: Red Hat Enterprise Linux Server release 7.5 (Maipo)
system-release: Oracle Linux Server release 7.5
oracle-release: Oracle Linux Server release 7.5

 Red Hat Enterprise Linux 7 Update 4

 3.10.0-693.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.4 (Maipo)

 4.1.12-94.3.9.el7uek ^ * (x86_64 only)
3.10.0-693.el7.x86_64 ^ (x86_64 only)
redhat-release: Red Hat Enterprise Linux Server release 7.4 (Maipo)
system-release: Oracle Linux Server release 7.4
oracle-release: Oracle Linux Server release 7.4

 Red Hat Enterprise Linux 7 Update 3

 3.10.0-514.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.3 (Maipo)

 4.1.12-61.1.18.el7uek ^ * (x86_64 only)
3.10.0-514.el7.x86_64 ^ (x86_64 only)
redhat-release: Red Hat Enterprise Linux Server release 7.3 (Maipo)
system-release: Oracle Linux Server release 7.3
oracle-release: Oracle Linux Server release 7.3

 Red Hat Enterprise Linux 7 Update 2

 3.10.0-327.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.2 (Maipo)

 3.8.13-98.6.1.el7uek ^ * (x86_64 only)
3.10.0-327.el7.x86_64 ^ (x86_64 only)
redhat-release: Red Hat Enterprise Linux Server release 7.2 (Maipo)
system-release: Oracle Linux Server release 7.2
oracle-release: Oracle Linux Server release 7.2

 Red Hat Enterprise Linux 7 Update 1

 3.10.0-229.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.1 (Maipo)

 3.8.13-55.1.6.el7uek ^ * (x86_64 only)
3.10.0-229.el7.x86_64 ^ (x86_64 only)
redhat-release:Red Hat Enterprise Linux Server release 7.1 (Maipo)
system-release:Oracle Linux Server release 7.1
oracle-release: Oracle Linux Server release 7.1

 Red Hat Enterprise Linux 7 GA

 3.10.0-123.el7.x86_64
redhat-release: Red Hat Enterprise Linux Server release 7.0 (Maipo)

 3.8.13-35.3.1.el7uek ^ * (x86_64 only)
3.10.0-123.el7.x86_64 ^ (x86_64 only)
redhat-release:Red Hat Enterprise Linux Server release 7.0 (Maipo)
system-release:Oracle Linux Server release 7.0
oracle-release: Oracle Linux Server release 7.0

 Red Hat Enterprise Linux 6

 Red Hat Enterprise Linux Server 6 Update 10

 2.6.32-754.el6
 redhat-release: Red Hat Enterprise Linux Server release 6.10 (Santiago)

 4.1.12-124.16.4.el6uek ^ * (x86_64 only)
2.6.39-400.299.3.el6uek ^ * (x86 only)
2.6.32-754.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.10 (Santiago)
system-release: Oracle Linux Server release 6.10
oracle-release: Oracle Linux Server release 6.10

 Red Hat Enterprise Linux Server 6 Update 9

 2.6.32-696.el6
redhat-release: Red Hat Enterprise Linux Server release 6.9 (Santiago)

 4.1.12-61.1.28.el6uek ^ * (x86_64 only)
2.6.39-400.294.3.el6uek ^ * (x86 only)
2.6.32-696.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.9 (Santiago)
system-release: Oracle Linux Server release 6.9
oracle-release: Oracle Linux Server release 6.9

 Red Hat Enterprise Linux Server 6 Update 8

 2.6.32-642.el6
redhat-release: Red Hat Enterprise Linux Server release 6.8 (Santiago)

 4.1.12-37.3.1.el6uek ^ * (x86_64 only)
2.6.39-400.278.2.el6uek ^ * (x86 only)
2.6.32-642.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.8 (Santiago)
system-release: Oracle Linux Server release 6.8
oracle-release: Oracle Linux Server release 6.8

 Red Hat Enterprise Linux Server 6 Update 7

 2.6.32-573.el6
redhat-release: Red Hat Enterprise Linux Server release 6.7 (Santiago)

 3.8.13-68.3.4.el6uek ^ * (x86_64 only)
2.6.39-400.250.7.el6uek ^ * (x86 only)
2.6.32-573.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.7 (Santiago)
system-release: Oracle Linux Server release 6.7
oracle-release: Oracle Linux Server release 6.7

 Red Hat Enterprise Linux Server 6 Update 6

 2.6.32-504.el6
redhat-release: Red Hat Enterprise Linux Server release 6.6 (Santiago)

 3.8.13-44.1.1.el6uek ^ * (x86_64 only)
2.6.39-400.215.10.el6uek ^ * (x86 only)
2.6.32-504.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.6 (Santiago)
system-release: Oracle Linux Server release 6.6
oracle-release: Oracle Linux Server release 6.6

 Red Hat Enterprise Linux Server 6 Update 5

 2.6.32-431.el6
redhat-release: Red Hat Enterprise Linux Server release 6.5 (Santiago)

 3.8.13-16.121.el6uek ^ * (x86_64 only)
2.6.39-400.211.1.el6uek ^ * (x86 only)
2.6.32-431.el6 ^ (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 6.5 (Santiago)
system-release: Oracle Linux Server release 6.5
oracle-release: Oracle Linux Server release 6.5

 Red Hat Enterprise Linux Server 6 Update 4

 2.6.32-358.el6
redhat-release: Red Hat Enterprise Linux Server release 6.4 (Santiago)

 2.6.39-400.17.1.el6uek ^ *
2.6.32-358.el6 ^
redhat-release: Red Hat Enterprise Linux Server release 6.4 (Santiago)
system-release: Oracle Linux Server release 6.4
oracle-release: Oracle Linux Server release 6.4

 Red Hat Enterprise Linux Server 6 Update 3

 2.6.32-279.el6
redhat-release: Red Hat Enterprise Linux Server release 6.3 (Santiago)

 2.6.39-200.24.1.el6uek ^ *
2.6.32-279.el6 ^
redhat-release: Red Hat Enterprise Linux Server release 6.3 (Santiago)
system-release: Oracle Linux Server release 6.3
oracle-release: Oracle Linux Server release 6.3

 Red Hat Enterprise Linux Server 6 Update 2

 2.6.32-220.el6
redhat-release: Red Hat Enterprise Linux Server release 6.2 (Santiago)

 2.6.32-100.34.1.el6uek ^ *
2.6.32-220.el6 ^
redhat-release: Red Hat Enterprise Linux Server release 6.2 (Santiago)
system-release: Oracle Linux Server release 6.2
oracle-release: Oracle Linux Server release 6.2

 Red Hat Enterprise Linux Server 6 Update 1

 2.6.32-131.el6
redhat-release: Red Hat Enterprise Linux Server release 6.1 (Santiago)

 2.6.32-100.34.1.el6uek ^ *
2.6.32-131.0.15.el6 ^
redhat-release: Red Hat Enterprise Linux Server release 6.1 (Santiago)
system-release: Oracle Linux Server release 6.1
oracle-release: Oracle Linux Server release 6.1

 Red Hat Enterprise Linux Server 6 GA

 2.6.32-71.el6
redhat-release: Red Hat Enterprise Linux Server release 6.0 (Santiago)

 2.6.32-100.28.5.el6uek ^ *
2.6.32-71.el6 ^
redhat-release: Red Hat Enterprise Linux Server release 6.0 (Santiago)
system-release: Oracle Linux Server release 6.0
oracle-release: Oracle Linux Server release 6.0

 Red Hat Enterprise Linux 5

 Red Hat Enterprise Linux Server 5 Update 11

 2.6.18-398.el5
redhat-release: Red Hat Enterprise Linux Server release 5.11 (Tikanga)

 2.6.39-400.215.10.el5uek ^ * (x86, x86_64)
2.6.18-398.el5 ^ (x86, x86_64)
2.6.18-398.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.11 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.11 (Carthage)
oracle-release: Oracle Linux Server release 5.11

 Red Hat Enterprise Linux Server 5 Update 10

 2.6.18-371.el5
redhat-release: Red Hat Enterprise Linux Server release 5.10 (Tikanga)

 2.6.39-400.209.1.el5uek ^ * (x86, x86_64)
2.6.18-371.el5 ^ (x86, x86_64)
2.6.18-371.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.10 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.10 (Carthage)
oracle-release: Oracle Linux Server release 5.10

 Red Hat Enterprise Linux Server 5 Update 9

 2.6.18-348.el5
redhat-release: Red Hat Enterprise Linux Server release 5.9 (Tikanga)

 2.6.39-300.26.1.el5uek ^ * (x86, x86_64)
2.6.18-348.el5 ^ (x86, x86_64)
2.6.18-348.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.9 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.9 (Carthage)
oracle-release: Oracle Linux Server release 5.9

 Red Hat Enterprise Linux Server 5 Update 8

 2.6.18-308.el5
redhat-release: Red Hat Enterprise Linux Server release 5.8 (Tikanga)

 2.6.32-300.10.1.el5uek ^ * (x86, x86_64)
2.6.18-308.el5 ^ (x86, x86_64)
2.6.18-308.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.8 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.8 (Carthage)
oracle-release: Oracle Linux Server release 5.8

 Red Hat Enterprise Linux Server 5 Update 7

 2.6.18-274.el5
redhat-release: Red Hat Enterprise Linux Server release 5.7 (Tikanga)

 2.6.32-200.13.1.el5uek ^ * (x86, x86_64)
2.6.18-274.el5 ^ (x86, x86_64)
2.6.18-274.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.7 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.7 (Carthage)
oracle-release: Oracle Linux Server release 5.7

 Red Hat Enterprise Linux Server 5 Update 6

 2.6.18-238.el5
redhat-release: Red Hat Enterprise Linux Server release 5.6 (Tikanga)

 2.6.32-100.26.2.el5uek ^ * (x86_64 only)
2.6.18-238.el5 ^ (x86, x86_64)
2.6.18-238.0.0.0.1.el5 (x86, x86_64)
redhat-release: Red Hat Enterprise Linux Server release 5.6 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.6 (Carthage)
oracle-release: Oracle Linux Server release 5.6

 Red Hat Enterprise Linux Server 5 Update 5

 2.6.18-194.el5
redhat-release: Red Hat Enterprise Linux Server release 5.5 (Tikanga)

 2.6.18-194.el5 ^ * (x86, x86_64, ia64)
2.6.32-100.24.1.el5 # (x86_64 only)
2.6.18-194.0.0.0.3.el5 (x86, x86_64, ia64)
redhat-release: Red Hat Enterprise Linux Server release 5.5 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)
oracle-release: Oracle Linux Server release 5.5

 Red Hat Enterprise Linux Server 5 Update 4

 2.6.18-164.el5
redhat-release: Red Hat Enterprise Linux Server release 5.4 (Tikanga)

 2.6.18-164.el5 ^ * (x86, x86_64, ia64)
2.6.18-164.0.0.0.1.el5 (x86, x86_64, ia64)
UEK unavailable
redhat-release: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.4 (Carthage)

 Red Hat Enterprise Linux Server 5 Update 3

 2.6.18-128.el5
redhat-release: Red Hat Enterprise Linux Server release 5.3 (Tikanga)

 2.6.18-128.el5 ^ * (x86, x86_64, ia64)
2.6.18-128.0.0.0.2.el5 (x86, x86_64, ia64)
UEK unavailable
redhat-release: Red Hat Enterprise Linux Server release 5.3 (Tikanga)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.3 (Carthage)

 Red Hat Enterprise Linux Server 5 Update 2

 2.6.18-92.el5
redhat-release: Red Hat Enterprise Linux Server release 5.2 (Tikanga)

 2.6.18-92.el5 ^ * (x86, x86_64, ia64)
2.6.18-92.0.0.0.1.el5 (x86, x86_64, ia64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux Server release 5.2 (Carthage)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.2 (Carthage)

 Red Hat Enterprise Linux Server 5 Update 1

 2.6.18-53.el5
redhat-release: Red Hat Enterprise Linux Server release 5.1 (Tikanga)

 2.6.18-53.el5 ^ * (x86, x86_64, ia64)
2.6.18-53.0.0.0.1.el5 (x86, x86_64, ia64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux Server release 5.1 (Carthage)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5.1 (Carthage)

 Red Hat Enterprise Linux Server 5 GA

 2.6.18-8.el5
redhat-release: Red Hat Enterprise Linux Server release 5 (Tikanga)

 2.6.18-8.el5 ^ * (x86, x86_64, ia64)
2.6.18-8.0.0.4.1.el5 (x86, x86_64, ia64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux Server release 5 (Carthage)
enterprise-release: Enterprise Linux Enterprise Linux Server release 5 (Carthage)

 Red Hat Enterprise Linux 4

 Red Hat Enterprise Linux 4 Update 9

 2.6.9-100.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 9)

 2.6.9-100.0.0.0.1.EL ^ * (x86, x86_64, ia64)
 2.6.9-100.EL (x86, x86_64, ia64)
 UEK unavailable
 redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 9)
 enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 9)

 Red Hat Enterprise Linux 4 Update 8

 2.6.9-89.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 8)

 2.6.9-89.0.0.0.1.EL ^ * (x86, x86_64, ia64)
2.6.9-89.EL (x86, x86_64, ia64)
UEK unavailable
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 8)
enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 8)

 Red Hat Enterprise Linux 4 Update 7

 2.6.9-78.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 7)

 2.6.9-78.0.0.0.1.EL ^ * (x86, x86_64, ia64)
2.6.9-78.EL (x86, x86_64, ia64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 7)
enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 7)

 Red Hat Enterprise Linux 4 Update 6

 2.6.9-67.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 6)

 2.6.9-67.0.0.0.1.EL ^ * (x86, x86_64, ia64)
2.6.9-67.EL ^ * (x86, x86_64, ia64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 6)
enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 6)

 Red Hat Enterprise Linux 4 Update 5

 2.6.9-55.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 5)

 2.6.9-55.0.0.0.2.EL ^ * (x86, x86_64)
2.6.9-55.EL (x86, x86_64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 5)
enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 5)

 Red Hat Enterprise Linux 4 Update 4

 2.6.9-42.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 4

 2.6.9-42.0.0.0.1.EL ^ * (x86, x86_64)
2.6.9-42.EL (x86, x86_64)
UEK unavailable
redhat-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 4)
enterprise-release: Enterprise Linux Enterprise Linux AS release 4 (October Update 4)

 Red Hat Enterprise Linux 4 Update 3

 2.6.9-34.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 3)

No corresponding version

 Red Hat Enterprise Linux 4 Update 2

 2.6.9-22.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 2)

No corresponding version

 Red Hat Enterprise Linux 4 Update 1

 2.6.9-11.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant Update 1)

No corresponding version

 Red Hat Enterprise Linux 4 GA

 2.6.9-5.EL
redhat-release: Red Hat Enterprise Linux AS release 4 (Nahant)

No corresponding version

参考:Comparison of Red Hat and Oracle Linux kernel versions and release strings (文档 ID 560992.1)

发表在 Linux | 标签为 , , | 评论关闭

mount: wrong fs type, bad option, bad superblock恢复Oracle

有朋友找到我们,说对lv进行收缩操作之后,导致文件系统无法mount,提示超级块损坏.
尝试mount失败

[root@GZGSDB data]# mount /dev/vg_gzgsdb/lv_home /home
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vg_gzgsdb-lv_home,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

系统日志报错

Aug 16 21:23:24 GZGSDB kernel: EXT4-fs (dm-5): ext4_check_descriptors: Block bitmap for group 1 not in group (block 0)!
Aug 16 21:23:24 GZGSDB kernel: EXT4-fs (dm-5): group descriptors corrupted!

主要操作
cz


这里可以看出来,明显操作错误,在没有收缩文件系统之前,直接收缩的lv.
list_history


这里可以看出来,在发生故障之后,又做了很多操作,包括fsck和testdisk等,最终无法恢复,请求我们协助处理。

找出来备份超级块

[root@GZGSDB dul]# dumpe2fs /dev/vg_gzgsdb/lv_home |grep superblock
dumpe2fs 1.41.12 (17-May-2010)
ext2fs_read_bb_inode: A block group is missing an inode table 
  Primary superblock at 0, Group descriptors at 1-52
  Backup superblock at 32768, Group descriptors at 32769-32820
  Backup superblock at 98304, Group descriptors at 98305-98356
  Backup superblock at 163840, Group descriptors at 163841-163892
  Backup superblock at 229376, Group descriptors at 229377-229428
  Backup superblock at 294912, Group descriptors at 294913-294964
  Backup superblock at 819200, Group descriptors at 819201-819252
  Backup superblock at 884736, Group descriptors at 884737-884788
  Backup superblock at 1605632, Group descriptors at 1605633-1605684
  Backup superblock at 2654208, Group descriptors at 2654209-2654260
  Backup superblock at 4096000, Group descriptors at 4096001-4096052
  Backup superblock at 7962624, Group descriptors at 7962625-7962676
  Backup superblock at 11239424, Group descriptors at 11239425-11239476
  Backup superblock at 20480000, Group descriptors at 20480001-20480052
  Backup superblock at 23887872, Group descriptors at 23887873-23887924
  Backup superblock at 71663616, Group descriptors at 71663617-71663668
  Backup superblock at 78675968, Group descriptors at 78675969-78676020
  Backup superblock at 102400000, Group descriptors at 102400001-102400052
  Backup superblock at 214990848, Group descriptors at 214990849-214990900

使用fsck修复

[root@GZGSDB data]# fsck -y /dev/vg_gzgsdb/lv_home 
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
fsck.ext4: Group descriptors look bad... trying backup blocks...
fsck.ext4: The ext2 superblock is corrupt when using the backup blocks
fsck.ext4: going back to original superblock
fsck.ext4: Device or resource busy while trying to open /dev/mapper/vg_gzgsdb-lv_home
Filesystem mounted or opened exclusively by another program?
--指定超级块恢复
[root@GZGSDB data]# fsck -y -b 102400000 /dev/vg_gzgsdb/lv_home 
…………
Illegal block #0 (1296647770) in inode 354315.  CLEARED.
Illegal block #1 (1398362886) in inode 354315.  CLEARED.
Illegal block #3 (453538936) in inode 354315.  CLEARED.
Illegal block #5 (808333361) in inode 354315.  CLEARED.
Illegal block #6 (775434798) in inode 354315.  CLEARED.
Illegal block #8 (1180306180) in inode 354315.  CLEARED.
Illegal block #9 (1413893971) in inode 354315.  CLEARED.
Illegal block #10 (1229347423) in inode 354315.  CLEARED.
Illegal block #11 (1498613325) in inode 354315.  CLEARED.
Illegal indirect block (1296389203) in inode 354315.  CLEARED.
Inode 354315 is too big.  Truncate? yes

Block #1074301965 (69632) causes directory to be too big.  CLEARED.
Warning... fsck.ext4 for device /dev/mapper/vg_gzgsdb-lv_home exited with signal 11.

[root@GZGSDB data]# mount /dev/vg_gzgsdb/lv_home /home
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vg_gzgsdb-lv_home,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

至此基本上可以判断,直接修复该文件系统,让其正常mount的概率很小.直接通过lun层面来恢复.

通过工具解析lun
disk_size


比较明显,客户经过一系列操作现在的现象是lv 1.03T,文件系统只有821G,明显和客户给我反馈的操作不符(应该是lv 821G),证明客户做了大量操作,已经导致文件系统损坏
inode_recovery


由于文件系统严重异常,工具获取到的文件都是没有名称,直接从inode里面读取的数据.获取到这些数据之后,然后结合oracle的特性,判断出来对应的文件号关系(这里有大量文件重复);另外有个别文件通过inode恢复丢失,通过底层碎片重组进行恢复出来文件,然后恢复出来其中数据(asm disk header 彻底损坏恢复).这个客户比较幸运,system文件在另外一个分区中,不然工作量会大很多.
再次提醒:对lv操作一定要谨慎,特别是lvreduce操作,另外出现发生误操作之后,应该第一时间保护现场,而不是百度着去乱操作,可能导致故障更加悲剧

发表在 Linux, 非常规恢复 | 标签为 , | 评论关闭