5.1. Cloning a standby from Barman

5.1.1. Prerequisites for cloning from Barman
5.1.2. Using Barman as a WAL file source
5.1.3. Using Barman through its API (pg-backup-api)

repmgr standby clone can use EDB's Barman application to clone a standby (and also as a fallback source for WAL files).

Tip

Barman (aka PgBarman) should be considered as an integral part of any PostgreSQL replication cluster. For more details see: https://www.pgbarman.org/.

Barman support provides the following advantages:

Note

Currently repmgr's support for cloning from Barman is implemented by using rsync to clone from the Barman server.

It is therefore not able to make use of Barman's parallel restore facility, which is executed on the Barman server and clones to the target server.

Barman's parallel restore facility can be used by executing it manually on the Barman server and configuring replication on the resulting cloned standby using repmgr standby clone --replication-conf-only.

5.1.1. Prerequisites for cloning from Barman

In order to enable Barman support for repmgr standby clone, following prerequisites must be met:

  • the Barman catalogue must include at least one valid backup for this server;

  • the barman_host setting in repmgr.conf is set to the SSH hostname of the Barman server;

  • the barman_server setting in repmgr.conf is the same as the server configured in Barman.

For example, assuming Barman is located on the host "barmansrv" under the "barman" user account, repmgr.conf should contain the following entries:

    barman_host='barman@barmansrv'
    barman_server='pg'

Here pg corresponds to a section in Barman's configuration file for a specific server backup configuration, which would look something like:

[pg]
description = "Main cluster"
...
     

More details on Barman configuration can be found in the Barman documentation's configuration section.

Note

To use a non-default Barman configuration file on the Barman server, specify this in repmgr.conf with barman_config:

      barman_config='/path/to/barman.conf'

We also recommend configuring the restore_command setting in repmgr.conf to use the barman-wal-restore script (see section Using Barman as a WAL file source below).

Tip

If you have a non-default SSH configuration on the Barman server, e.g. using a port other than 22, then you can set those parameters in a dedicated Host section in ~/.ssh/config corresponding to the value of barman_host in repmgr.conf. See the Host section in man 5 ssh_config for more details.

If you wish to place WAL files in a location outside the main PostgreSQL data directory, set --waldir (PostgreSQL 9.6 and earlier: --xlogdir) in pg_basebackup_options to the target directory (must be an absolute filepath). repmgr will create and symlink to this directory in exactly the same way pg_basebackup would.

It's now possible to clone a standby from Barman, e.g.:

    $ repmgr -f /etc/repmgr.conf -h node1 -U repmgr -d repmgr standby clone
    NOTICE: destination directory "/var/lib/postgresql/data" provided
    INFO: connecting to Barman server to verify backup for "test_cluster"
    INFO: checking and correcting permissions on existing directory "/var/lib/postgresql/data"
    INFO: creating directory "/var/lib/postgresql/data/repmgr"...
    INFO: connecting to Barman server to fetch server parameters
    INFO: connecting to source node
    DETAIL: current installation size is 30 MB
    NOTICE: retrieving backup from Barman...
    (...)
    NOTICE: standby clone (from Barman) complete
    NOTICE: you can now start your PostgreSQL server
    HINT: for example: pg_ctl -D /var/lib/postgresql/data start

Note

Barman support is automatically enabled if barman_server is set. Normally it is good practice to use Barman, for instance when fetching a base backup while cloning a standby; in any case, Barman mode can be disabled using the --without-barman command line option.

5.1.2. Using Barman as a WAL file source

As a fallback in case streaming replication is interrupted, PostgreSQL can optionally retrieve WAL files from an archive, such as that provided by Barman. This is done by setting restore_command in the replication configuration to a valid shell command which can retrieve a specified WAL file from the archive.

barman-wal-restore is a Python script provided as part of the barman-cli package (Barman 2.0 ~ 2.7) or as part of the core Barman distribution (Barman 2.8 and later).

To use barman-wal-restore with repmgr, assuming Barman is located on the host "barmansrv" under the "barman" user account, and that barman-wal-restore is located as an executable at /usr/bin/barman-wal-restore, repmgr.conf should include the following lines:

    barman_host='barman@barmansrv'
    barman_server='pg'
    restore_command='/usr/bin/barman-wal-restore barmansrv pg %f %p'

Note

barman-wal-restore supports command line switches to control parallelism (--parallel=N) and compression (--bzip2, --gzip).

5.1.3. Using Barman through its API (pg-backup-api)

You can find information on how to install and setup pg-backup-api in the pg-backup-api documentation.

This mode (`pg-backupapi`) was introduced in v5.4.0 as a way to further integrate with Barman letting Barman handle the restore. This also reduces the ssh keys that need to share between the backup and postgres nodes. As long as you have access to the API service by HTTP calls, you could perform recoveries right away. You just need to instruct Barman through the API which backup you need and on which node the backup needs to to be restored on.

In order to enable pg_backupapi mode support for repmgr standby clone, you need the following lines in repmgr.conf:

  • pg_backupapi_host: Where pg-backup-api is hosted

  • pg_backupapi_node_name: Name of the server as understood by Barman

  • pg_backupapi_remote_ssh_command: How Barman will be connecting as to the node

  • pg_backupapi_backup_id: ID of the existing backup you need to restore

This is an example of how repmgr.conf would look like:

        pg_backupapi_host = '192.168.122.154'
        pg_backupapi_node_name = 'burrito'
        pg_backupapi_remote_ssh_command = 'ssh john_doe@192.168.122.1'
        pg_backupapi_backup_id = '20230223T093201'
      

pg_backupapi_host is the variable name that enables this mode, and when you set it, all the rest of the above variables are required. Also, remember that this service is just an interface between Barman and repmgr, hence if something fails during a recovery, you should check Barman's logs upon why the process couldn't finish properly.

Note

Despite in Barman you can define shortcuts like "lastest" or "oldest", they are not supported for the time being in pg-backup-api. These shortcuts will be supported in a future release.

This is a real example of repmgr's output cloning with the API. Note that during this operation, we stopped the service for a little while and repmgr had to retry but that doesn't affect the final outcome. The primary is listening on localhost's port 6001:

$ repmgr -f ~/nodes/node_3/repmgr.conf standby clone -U repmgr -p 6001 -h localhost
NOTICE: destination directory "/home/mario/nodes/node_3/data" provided
INFO: Attempting to use `pg_backupapi` new restore mode
INFO: connecting to source node
DETAIL: connection string is: user=repmgr port=6001 host=localhost
DETAIL: current installation size is 8541 MB
DEBUG: 1 node records returned by source node
DEBUG: connecting to: "user=repmgr dbname=repmgr host=localhost port=6001 connect_timeout=2 fallback_application_name=repmgr options=-csearch_path="
DEBUG: upstream_node_id determined as 1
INFO: Attempting to use `pg_backupapi` new restore mode
INFO: replication slot usage not requested;  no replication slot will be set up for this standby
NOTICE: starting backup (using pg_backupapi)...
INFO: Success creating the task: operation id '20230309T150647'
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
Incorrect reply received for that operation ID.
INFO: Retrying...
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status IN_PROGRESS
INFO: status DONE
NOTICE: standby clone (from pg_backupapi) complete
NOTICE: you can now start your PostgreSQL server
HINT: for example: pg_ctl -D /home/mario/nodes/node_3/data start
HINT: after starting the server, you need to register this standby with "repmgr standby register"