标签归档:pg恢复

PostgreSQL oid文件替换实现数据访问

对于postgresql数据库,我们都知道他的数据在独立的oid文件里面,而字典主要在pg_type/1247,pg_class/1259,pg_attribute/1260等对象中,而数据是存储在独立的oid和toast oid文件中.如果通过提供oid文件,可以实现狸猫换太子的恢复数据.这里做一个简单测试,大概思路是这样的库里面有一个表a,然后创建一个空表b,然后把a表相关的oid文件复制成b表的oid文件,然后验证数据是否可以正常查询
创建一个空的b表,结构和a表一致

[postgres@xifenfei 13676]$ psql
psql (12.8)
Type "help" for help.

postgres=# CREATE TABLE "public"."AO_6384AB_DISCOVERED1" (
postgres(#   "DATE" "pg_catalog"."timestamp",
postgres(#   "ID" "pg_catalog"."int4",
postgres(#   "KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default",
postgres(#   "PLUGIN_KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default",
postgres(#   "USER_KEY" "pg_catalog"."varchar" COLLATE "pg_catalog"."default"
postgres(# )
postgres-# ;
CREATE TABLE

检查a/b的情况

postgres=# select count(1) from "AO_6384AB_DISCOVERED";
 count 
-------
   544
(1 row)

postgres=# select count(1) from "AO_6384AB_DISCOVERED1";
 count 
-------
     0
(1 row)

postgres=# \d "AO_6384AB_DISCOVERED"
                    Table "public.AO_6384AB_DISCOVERED"
   Column   |            Type             | Collation | Nullable | Default 
------------+-----------------------------+-----------+----------+---------
 DATE       | timestamp without time zone |           |          | 
 ID         | integer                     |           |          | 
 KEY        | character varying           |           |          | 
 PLUGIN_KEY | character varying           |           |          | 
 USER_KEY   | character varying           |           |          | 

postgres=# \d "AO_6384AB_DISCOVERED1"
                   Table "public.AO_6384AB_DISCOVERED1"
   Column   |            Type             | Collation | Nullable | Default 
------------+-----------------------------+-----------+----------+---------
 DATE       | timestamp without time zone |           |          | 
 ID         | integer                     |           |          | 
 KEY        | character varying           |           |          | 
 PLUGIN_KEY | character varying           |           |          | 
 USER_KEY   | character varying           |           |          | 

存在数据的a/b表相关oid信息



postgres=# select oid,relname,relfilenode from pg_class where relname like '%AO_6384AB_DISCOVERED%';
  oid  |        relname        | relfilenode 
-------+-----------------------+-------------
 16516 | AO_6384AB_DISCOVERED  |       16516
 17069 | AO_6384AB_DISCOVERED1 |       17069
(2 rows)

postgres=# select oid,relname,relfilenode from pg_class where relname like '%16516%' or  relname like '%17069%'  ;
  oid  |       relname        | relfilenode 
-------+----------------------+-------------
 16519 | pg_toast_16516       |       16519
 16521 | pg_toast_16516_index |       16521
 17072 | pg_toast_17069       |       17072
 17074 | pg_toast_17069_index |       17074
(4 rows)

关闭数据库把a表相关的oid文件拷贝替换b表的oid文件

[postgres@xifenfei 13676]$ pg_ctl stop -D /pgdata
waiting for server to shut down....2025-12-12 20:00:58.804 HKT [21445] LOG:  received fast shutdown request
2025-12-12 20:00:58.805 HKT [21445] LOG:  aborting any active transactions
2025-12-12 20:00:58.805 HKT [21471] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.805 HKT [21473] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.805 HKT [21856] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.806 HKT [21472] FATAL:  terminating connection due to administrator command
2025-12-12 20:00:58.806 HKT [21445] LOG: background worker"logical replication launcher"(PID 252)exited with exit code 1
2025-12-12 20:00:58.807 HKT [21447] LOG:  shutting down
2025-12-12 20:00:58.811 HKT [21445] LOG:  database system is shut down
 done
server stopped
[postgres@xifenfei 13676]$ cp  16516 17069
[postgres@xifenfei 13676]$ cp  16519 17072
[postgres@xifenfei 13676]$ cp  16521 17074
[postgres@xifenfei 13676]$ pg_ctl start -D /pgdata
waiting for server to start....2025-12-12 20:02:05.985 HKT [22241] LOG:  
starting PostgreSQL 12.8 on x86_64-pc-linux-gnu, 
compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-26.0.1), 64-bit
2025-12-12 20:02:05.985 HKT [22241] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2025-12-12 20:02:05.986 HKT [22241] LOG:  listening on IPv6 address "::", port 5432
2025-12-12 20:02:05.986 HKT [22241] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2025-12-12 20:02:05.993 HKT [22242] LOG:  database system was shut down at 2025-12-12 20:00:58 HKT
2025-12-12 20:02:05.995 HKT [22241] LOG:  database system is ready to accept connections
 done
server started

查询a/b表数据

[postgres@xifenfei 13676]$ psql
psql (12.8)
Type "help" for help.

postgres=# select count(1) from "AO_6384AB_DISCOVERED1";
 count 
-------
   544
(1 row)

postgres=# select count(1) from "AO_6384AB_DISCOVERED";
 count 
-------
   544
(1 row)

通过上述从测试,证明创建一个新的表结构,完全可以访问老的oid文件中的数据

发表在 PostgreSQL, PostgreSQL恢复 | 标签为 , , , | 评论关闭

PostgreSQL表文件损坏恢复—pdu恢复损坏的表文件

在某些情况下,由于PostgreSQL表文件损坏导致无法正常访问,可以通过pdu把好的block中的数据恢复出来
准备一张测试表,里面有97条记录

his5_dms=# \d hiscrm.t_sys_oper_log;
                                            Table "hiscrm.t_sys_oper_log"
       Column       |          Type          | Collation | Nullable |Default                     
--------------------+------------------------+-----------+----------+------------
 id                 | bigint                 |           | not null | 
 module             | character varying(50)  |           |          | 
 title              | character varying(50)  |           |          | 
 alias              | character varying(50)  |           |          | 
 business_type      | integer                |           |          | 0
 method             | character varying(200) |           |          | 
 request_method     | character varying(10)  |           |          | 
 operator_type      | integer                |           |          | 0
 oper_name          | character varying(50)  |           |          | 
 dept_name          | character varying(50)  |           |          | 
 oper_url           | character varying(255) |           |          | 
 oper_ip            | character varying(50)  |           |          | 
 oper_location      | character varying(255) |           |          | 
 oper_param         | text                   |           |          | 
 json_result        | text                   |           |          | 
 status             | integer                |           |          | 0
 error_msg          | text                   |           |          | 
 oper_time          | date                   |           |          | 
 create_id          | bigint                 |           |          | 
 create_time        | bigint                 |           |          | 
 clinic_id          | bigint                 |           |          | 
 group_id           | bigint                 |           |          | 
 patient_id         | bigint                 |           |          | 
 is_patient_related | integer                |           |          | 
 business_content   | json                   |           |          | 

his5_dms=# select count(1) from hiscrm.t_sys_oper_log;
 count 
-------
    97
(1 row)

查询表对应的具体文件

his5_dms=# SELECT oid,relfilenode FROM pg_class WHERE relname='t_sys_oper_log';
  oid  | relfilenode 
-------+-------------
 16850 |       16850
(1 row)

his5_dms=# SELECT pg_relation_filepath('hiscrm.t_sys_oper_log');
 pg_relation_filepath 
----------------------
 base/16386/16850
(1 row)

his5_dms=# SHOW data_directory;
     data_directory     
------------------------
 /var/lib/pgsql/12/data
(1 row)

使用dd对文件进行破坏

[postgres@xifenfeidg ~]$ ls -l  /var/lib/pgsql/12/data/base/16386/16850
-rw-------. 1 postgres postgres 90112 Sep  5 20:26 /var/lib/pgsql/12/data/base/16386/16850
[postgres@xifenfeidg ~]$ dd if=/dev/zero of=/var/lib/pgsql/12/data/base/16386/16850 bs=512 count=1 conv=notrunc
1+0 records in
1+0 records out
512 bytes copied, 0.000158756 s, 3.2 MB/s

重启pg库

[postgres@xifenfeidg bin]$ ./pg_ctl -m fast -D /var/lib/pgsql/12/data/ stop
waiting for server to shut down.... done
server stopped
[postgres@xifenfeidg bin]$ ./pg_ctl -D /var/lib/pgsql/12/data/ start
waiting for server to start....2025-03-02 19:02:11.395 HKT [64515] LOG:  
  starting PostgreSQL 12.20 on x86_64-pc-linux-gnu, 
  compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-22), 64-bit
2025-03-02 19:02:11.396 HKT [64515] LOG:  listening on IPv6 address "::1", port 5432
2025-03-02 19:02:11.396 HKT [64515] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2025-03-02 19:02:11.396 HKT [64515] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2025-03-02 19:02:11.397 HKT [64515] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2025-03-02 19:02:11.403 HKT [64515] LOG:  redirecting log output to logging collector process
2025-03-02 19:02:11.403 HKT [64515] HINT:  Future log output will appear in directory "log".
 done
server started

查询数据报错

[postgres@xifenfeidg bin]$ psql
psql (16.8, server 12.20)
Type "help" for help.

postgres=# \c his5_dms;
psql (16.8, server 12.20)
You are now connected to database "his5_dms" as user "postgres".
his5_dms=#  select count(1) from hiscrm.t_sys_oper_log;
ERROR:  invalid page in block 0 of relation base/16386/16850

通过pdu进行恢复
跳过了坏块,把好的block中数据均恢复出来

his5_dms.hiscrm=# unload tab t_sys_oper_log;
正在解析表 <t_sys_oper_log>. 已解析数据页: 0, 已解析数据: 0 条
        |-块号0 空页面或页面已损坏,已跳过
正在解析表 <t_sys_oper_log>. 已解析数据页: 11, 已解析数据: 86 条
表名<t_sys_oper_log>-</var/lib/pgsql/12/data/base/16386/16850> 
    解析完成, 11 个数据页 ,共计 86 条数据. 成功 86 条; 失败【0】条 
COPY文件路径为:<his5_dms/hiscrm/t_sys_oper_log.csv>

导入pg库中

his5_dms=# truncate table hiscrm.t_sys_oper_log;
TRUNCATE TABLE
his5_dms=# \i his5_dms/COPY/hiscrm_copy.sql
SET
COPY 86
his5_dms=# 
发表在 pdu工具 | 标签为 , , , , , | 评论关闭

PostgreSQL恢复工具—pdu恢复单个表文件

在某些情况下,比如我们需要对单个的PostgreSQL库的表文件进行恢复(比如文件系统损坏,drop库/表,truncate表等原因,然后找到了部分oid文件),可以使用pdu对其进行完美恢复(相比pg_filedump也方便很多),具体操作步骤:
1. 由于只有单个表文件,无法获取字典信息,因此需要应用厂商/客户提供具体表创建语句

his5_dms=#    CREATE TABLE t_xff (
his5_dms(#         id                       bigint,
his5_dms(#         hospital_id              bigint,
his5_dms(#         parent_id                bigint,
his5_dms(#         disease_code             varchar(60),
his5_dms(#         disease_name             varchar(60),
his5_dms(#         type                     smallint,
his5_dms(#         py                       varchar(60),
his5_dms(#         wb                       varchar(60),
his5_dms(#         sc                       varchar(20),
his5_dms(#         order_no                 int,
his5_dms(#         state                    smallint,
his5_dms(#         create_datetime          timestamp(6),
his5_dms(#         create_id                bigint,
his5_dms(#         edit_datetime            timestamp(6),
his5_dms(#         edit_id                  bigint,
his5_dms(#         search_path              varchar(300),
his5_dms(#         diagnosis_sort           int,
his5_dms(#         category_name            varchar(40),
his5_dms(#         input_option             varchar(40),
his5_dms(#         category_class           smallint,
his5_dms(#         memo1                    varchar(300),
his5_dms(#         memo2                    varchar(300),
his5_dms(#         other_code               varchar(60),
his5_dms(#         other_name               varchar(60),
his5_dms(#         special_disease_flag     smallint
his5_dms(#    );
CREATE TABLE

2. 把oid文件pdu放到restore库中

[root@xifenfeidg public]# pwd
/tmp/pdu/restore/public
[root@xifenfeidg public]# ls -l
total 7144
-rw-r--r--. 1 root root 7315456 Mar  2 21:04 123456
[root@xifenfeidg public]# 

3. 使用add语句在pdu加载数据类型

restore.public=# add 123456 t_xff bigint,bigint,bigint,varchar,varchar,smallint,varchar,varchar,varchar,
int,smallint,timestamp,bigint,timestamp,bigint,varchar,int,varchar,varchar,
smallint,varchar,varchar,varchar,varchar,smallint;
添加完成,请用\dt;查看可unload的表
restore.public=# \dt;
|--------------------------------------------------|
|               表名                  |  表大小    |
|--------------------------------------------------|
|    t_xff                            |  6.98 MB   |
|--------------------------------------------------|

        仅显示表大小排名前 1 的表名

4.使用pdu恢复表数据

restore.public=# unload t_xff;
正在解析表 <t_xff>. 已解析数据页: 893, 已解析数据: 46998 条
<t_xff>-<restore/public/123456> 解析完成, 894 个数据页 ,共计 46998 条数据. 成功 46998 条; 失败【0】条 
 COPY文件路径为:<restore/public/t_xff.csv>
restore.public=# unload COPY;

COPY命令导出完成, 文件路径: restore/COPY/public_copy.sql

5.导入数据到pg库中

his5_dms=# \i restore/COPY/public_copy.sql
SET
COPY 46998
his5_dms=# select count(1) from t_xff;
 count 
-------
 46998
(1 row)
his5_dms=# \x
Expanded display is on.
his5_dms=# select * from t_xff limit 1;
-[ RECORD 1 ]--------+---------------------------
id                   | 323839
hospital_id          | 0
parent_id            | 301
disease_code         | 57.8900x003
disease_name         | 腹腔镜下膀胱颈悬吊术
type                 | 2
py                   | fqjxpgjxds
wb                   | eeqgeeceks
sc                   | 
order_no             | 0
state                | 1
create_datetime      | 2022-09-29 15:22:58.588492
create_id            | 
edit_datetime        | 
edit_id              | 
search_path          | 301,
diagnosis_sort       | 
category_name        | 
input_option         | 
category_class       | 3
memo1                | 
memo2                | 
other_code           | 
other_name           | 
special_disease_flag | 0
发表在 pdu工具 | 标签为 , , , , | 评论关闭