PostgreSQL

08 September 2024 18:06

pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 13CF/8A000028 on timeline 37
pg_basebackup: starting background WAL receiver
pg_basebackup: created replication slot "sync_replica"
33/2797557515 kB (0%), 0/20 tablespaces (..._DDBB/PG_14_202107181/57375/1255)
65151/2797557515 kB (0%), 0/20 tablespaces (...DDBB/PG_14_202107181/57375/73762) <--- there was the error

PostgreSQL

08 September 2024 18:02

Yeah, the error was, i cannot create the replica slot first

PostgreSQL

08 September 2024 13:59

In thiscase I recommend to start thinking or hire an engineer.

PostgreSQL

08 September 2024 13:57

Why basebackup is deleting the Data directory? and Wal Logs?

PostgreSQL

08 September 2024 13:11

What puzzles me is that you're at 3TB and still do "manual" backups. At this size, you'd benefit massively from e.g. pgBackRest's parallel processing.

PostgreSQL

08 September 2024 12:17

The bottleneck is the wal transfer is slow

PostgreSQL

08 September 2024 11:55

First you have to identify bottlnecks in your setup.
Then a reasonable thoughts about what to do with it would be feasible.

PostgreSQL

08 September 2024 11:54

PostgreSQL: Documentation: 16: pg_basebackup
https://www.postgresql.org/docs/current/app-pgbasebackup.html#:~:text=Valid%20values%20are%20between%2032,transfer%20of%20the%20data%20directory.

PostgreSQL

08 September 2024 11:42

I meant that reply, sorry.

PostgreSQL

08 September 2024 11:41

Which is the parameter?

PostgreSQL

08 September 2024 11:38

Te versión are 14.10

PostgreSQL

08 September 2024 09:53

How to use postgres in k8s cluster or we should go for cloud services

PostgreSQL

07 September 2024 18:22

What? Wait! You mean pg_restore does not have a "--repair-corrupted-data" switch?

Well, now I am starting to question the "--from-the-dump-I-forgot-to-make" option!

LOL

PostgreSQL

07 September 2024 13:50

Patroni just tells Postgres what to do. When your replication fails, that has 0.0 to do with Patroni. Check the PG server logs for error messages, they should point you in the direction to look in (e.g. network disconnection).

PostgreSQL

07 September 2024 13:38

But gpt often allucinates or borrows stuff from others db

PostgreSQL

08 September 2024 18:02

But my basebackup already contains the -C

PostgreSQL

08 September 2024 17:20

Because you didn't get a backup, so keeping it is a waste 🤷

PostgreSQL

08 September 2024 13:57

pg_basebackup: error: child process exited with exit code 1
pg_basebackup: removing contents of data directory "/postgresql14/bbddpostgresql/"
pg_basebackup: removing contents of WAL directory "/postgresql14/xlogs"
pg_basebackup: changes to tablespace directories will not be undone

PostgreSQL

08 September 2024 13:15

i repeat, the pg_basebackup is for configure the replication

PostgreSQL

08 September 2024 12:24

This is for Data directory

PostgreSQL

08 September 2024 12:17

1163140159/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.828)
1163460127/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.828)
1163788031/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.828)
1164071807/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.829)
1164356511/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.829)
1164661311/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.829)
1164954143/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.829)
1165302208/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.830)
1165655872/2797506003 kB (41%), 6/20 tablespaces (...G_14_202107181/37580219/2613.830)

PostgreSQL

08 September 2024 11:55

here is u can maximise the max rate

PostgreSQL

08 September 2024 11:52

Oh, sorry for that guys

PostgreSQL

08 September 2024 11:42

For compress doesn't need more time?

PostgreSQL

08 September 2024 11:40

u can use, compression with tar & parallel workers also that should work way faster

PostgreSQL

08 September 2024 11:37

Guys, someone know any parameter of pg_basebackup to make faster? The size of around 3tb the server on same cpd and same subnet

PostgreSQL

07 September 2024 19:28

That will probably not help in the direction OP wants to replicate, though. Not that I understand nor support the desire to get data from PG into Mongo, but alas! 😏

PostgreSQL

07 September 2024 18:15

Let me second this. We did a lot of work, before I knew the power and efficiency of FDWs.
If I had to do it all over again... I would have worked with FDWs.

I would have created customized SELECT statements (like a virtualized view) to fix the tables I wanted fixed/changed.

From there you can do INSERT INTO ...
To pull the data into your actual tables.

You can also use COPY and \COPY against a (select from fdw) to generate a file you can easily import into another PG DB copy...

LESSON Learned. This is good advice!

PostgreSQL

07 September 2024 13:48

ChatGPT is very much like conservative politicians; it usually *sounds* reasonable, but more often than not it produces complete dogsh*t, albeit very eloquently. 😏

PostgreSQL

07 September 2024 12:51

There can be a lot of reasons for replication failure.