linkend="populate-pitr">. If archiving or streaming replication were
turned on during execution of one of these statements, WAL would not
contain enough information for archive recovery. (Crash recovery is
- unaffected.) For this reason, wal_level can only be changed at
- server start. However, archive_command can be changed with a
+ unaffected.) For this reason, wal_levelvarname> can only be changed at
+ server start. However, archive_commandvarname> can be changed with a
configuration file reload. If you wish to temporarily stop archiving,
- one way to do it is to set archive_command to the empty
- string ('').
- This will cause WAL files to accumulate in pg_wal/ until a
- working archive_command is re-established.
+ one way to do it is to set archive_commandvarname> to the empty
+ string (''literal>).
+ This will cause WAL files to accumulate in pg_wal/filename> until a
+ working archive_commandvarname> is re-established.
It is not necessary to be concerned about the amount of time it takes
to make a base backup. However, if you normally run the
- server with full_page_writes disabled, you might notice a drop
- in performance while the backup runs since full_page_writes is
+ server with full_page_writesvarname> disabled, you might notice a drop
+ in performance while the backup runs since full_page_writesvarname> is
effectively forced on during backup mode.
To make use of the backup, you will need to keep all the WAL
segment files generated during and after the file system backup.
To aid you in doing this, the base backup process
- creates a backup history file that is immediately
+ creates a backup history filefirstterm> that is immediately
stored into the WAL archive area. This file is named after the first
WAL segment file that you need for the file system backup.
For example, if the starting WAL file is
- 0000000100001234000055CD the backup history file will be
+ 0000000100001234000055CDliteral> the backup history file will be
named something like
- 0000000100001234000055CD.007C9330.backup. (The second
+ 0000000100001234000055CD.007C9330.backupliteral>. (The second
part of the file name stands for an exact position within the WAL
file, and can ordinarily be ignored.) Once you have safely archived
the file system backup and the WAL segment files used during the
SELECT pg_start_backup('label', false, false);
- where label is any string you want to use to uniquely
+ where labelliteral> is any string you want to use to uniquely
identify this backup operation. The connection
- calling pg_start_backup must be maintained until the end of
+ calling pg_start_backupfunction> must be maintained until the end of
the backup, or the backup will be automatically aborted.
- By default, pg_start_backup can take a long time to finish.
+ By default, pg_start_backupfunction> can take a long time to finish.
This is because it performs a checkpoint, and the I/O
required for the checkpoint will be spread out over a significant
period of time, by default half your inter-checkpoint interval
). This is
usually what you want, because it minimizes the impact on query
processing. If you want to start the backup as soon as
- possible, change the second parameter to true, which will
+ possible, change the second parameter to trueliteral>, which will
issue an immediate checkpoint using as much I/O as available.
- The third parameter being false tells
- pg_start_backup to initiate a non-exclusive base backup.
+ The third parameter being falseliteral> tells
+ pg_start_backupfunction> to initiate a non-exclusive base backup.
Perform the backup, using any convenient file-system-backup tool
- such as
tar> or cpio> (not+ such as
tarapplication> or cpio> (not pg_dumpall). It is neither
necessary nor desirable to stop normal operation of the database
ready to archive.
- The pg_stop_backup will return one row with three
+ The pg_stop_backupfunction> will return one row with three
values. The second of these fields should be written to a file named
- backup_label in the root directory of the backup. The
+ backup_labelfilename> in the root directory of the backup. The
third field should be written to a file named
- tablespace_map unless the field is empty. These files are
+ tablespace_mapfilename> unless the field is empty. These files are
vital to the backup working, and must be written without modification.
Once the WAL segment files active during the backup are archived, you are
- done. The file identified by pg_stop_backup's first return
+ done. The file identified by pg_stop_backupfunction>'s first return
value is the last segment that is required to form a complete set of
- backup files. On a primary, if archive_mode is enabled and the
- wait_for_archive> parameter is true>,
- pg_stop_backup does not return until the last segment has
+ backup files. On a primary, if archive_modevarname> is enabled and the
+ wait_for_archiveliteral> parameter is true>,
+ pg_stop_backupfunction> does not return until the last segment has
been archived.
- On a standby, archive_mode> must be always> in order
- for pg_stop_backup to wait.
+ On a standby, archive_modevarname> must be always> in order
+ for pg_stop_backupfunction> to wait.
Archiving of these files happens automatically since you have
- already configured archive_command. In most cases this
+ already configured archive_commandvarname>. In most cases this
happens quickly, but you are advised to monitor your archive
system to ensure there are no delays.
If the archive process has fallen behind
because of failures of the archive command, it will keep retrying
until the archive succeeds and the backup is complete.
If you wish to place a time limit on the execution of
- pg_stop_backup, set an appropriate
+ pg_stop_backupfunction>, set an appropriate
statement_timeout value, but make note that if
- pg_stop_backup terminates because of this your backup
+ pg_stop_backupfunction> terminates because of this your backup
may not be valid.
If the backup process monitors and ensures that all WAL segment files
required for the backup are successfully archived then the
- wait_for_archive parameter (which defaults to true) can be set
+ wait_for_archiveliteral> parameter (which defaults to true) can be set
to false to have
- pg_stop_backup return as soon as the stop backup record is
- written to the WAL. By default, pg_stop_backup will wait
+ pg_stop_backupfunction> return as soon as the stop backup record is
+ written to the WAL. By default, pg_stop_backupfunction> will wait
until all WAL has been archived, which can take some time. This option
must be used with caution: if WAL archiving is not monitored correctly
then the backup might not include all of the WAL files and will
The process for an exclusive backup is mostly the same as for a
non-exclusive one, but it differs in a few key steps. This type of backup
can only be taken on a primary and does not allow concurrent backups.
- Prior to
PostgreSQL 9.6, this+ Prior to
PostgreSQLproductname> 9.6, this was the only low-level method available, but it is now recommended that
all users upgrade their scripts to use non-exclusive backups if possible.
SELECT pg_start_backup('label');
- where label is any string you want to use to uniquely
+ where labelliteral> is any string you want to use to uniquely
identify this backup operation.
- pg_start_backup> creates a backup label> file,
- called backup_label, in the cluster directory with
+ pg_start_backupfunction> creates a backup label> file,
+ called backup_labelfilename>, in the cluster directory with
information about your backup, including the start time and label string.
- The function also creates a tablespace map file,
- called tablespace_map, in the cluster directory with
- information about tablespace symbolic links in pg_tblspc/ if
+ The function also creates a tablespace mapfirstterm> file,
+ called tablespace_mapfilename>, in the cluster directory with
+ information about tablespace symbolic links in pg_tblspc/filename> if
one or more such link is present. Both files are critical to the
integrity of the backup, should you need to restore from it.
- By default, pg_start_backup can take a long time to finish.
+ By default, pg_start_backupfunction> can take a long time to finish.
This is because it performs a checkpoint, and the I/O
required for the checkpoint will be spread out over a significant
period of time, by default half your inter-checkpoint interval
Perform the backup, using any convenient file-system-backup tool
- such as
tar> or cpio> (not+ such as
tarapplication> or cpio> (not pg_dumpall). It is neither
necessary nor desirable to stop normal operation of the database
Note that if the server crashes during the backup it may not be
- possible to restart until the backup_label file has been
+ possible to restart until the backup_labelliteral> file has been
manually deleted from the PGDATA directory.
Once the WAL segment files active during the backup are archived, you are
- done. The file identified by pg_stop_backup's result is
+ done. The file identified by pg_stop_backupfunction>'s result is
the last segment that is required to form a complete set of backup files.
- If archive_mode is enabled,
- pg_stop_backup does not return until the last segment has
+ If archive_modevarname> is enabled,
+ pg_stop_backupfunction> does not return until the last segment has
been archived.
Archiving of these files happens automatically since you have
- already configured archive_command. In most cases this
+ already configured archive_commandvarname>. In most cases this
happens quickly, but you are advised to monitor your archive
system to ensure there are no delays.
If the archive process has fallen behind
because of failures of the archive command, it will keep retrying
until the archive succeeds and the backup is complete.
If you wish to place a time limit on the execution of
- pg_stop_backup, set an appropriate
+ pg_stop_backupfunction>, set an appropriate
statement_timeout value, but make note that if
- pg_stop_backup terminates because of this your backup
+ pg_stop_backupfunction> terminates because of this your backup
may not be valid.
When taking a base backup of an active database, this situation is normal
and not an error. However, you need to ensure that you can distinguish
complaints of this sort from real errors. For example, some versions
- of
rsync return a separate exit code for- vanished source files, and you can write a driver script to
+ of
rsyncapplication> return a separate exit code for+ vanished source filesquote>, and you can write a driver script to
accept this exit code as a non-error case. Also, some versions of
- GNU
tar return an error code indistinguishable from- a fatal error if a file was truncated while
tar was- copying it. Fortunately, GNU
tar versions 1.16 and+ GNU
tarapplication> return an error code indistinguishable from+ a fatal error if a file was truncated while
tarapplication> was+ copying it. Fortunately, GNU
tarapplication> versions 1.16 and later exit with 1 if a file was changed during the backup,
- and 2 for other errors. With GNU
tar version 1.23 and+ and 2 for other errors. With GNU
tarapplication> version 1.23 and later, you can use the warning options --warning=no-file-changed
--warning=no-file-removed to hide the related warning messages.
Be certain that your backup includes all of the files under
- the database cluster directory (e.g., /usr/local/pgsql/data).
+ the database cluster directory (e.g., /usr/local/pgsql/datafilename>).
If you are using tablespaces that do not reside underneath this directory,
be careful to include them as well (and be sure that your backup
archives symbolic links as links, otherwise the restore will corrupt
You should, however, omit from the backup the files within the
- cluster's pg_wal/ subdirectory. This
+ cluster's pg_wal/filename> subdirectory. This
slight adjustment is worthwhile because it reduces the risk
of mistakes when restoring. This is easy to arrange if
- pg_wal/ is a symbolic link pointing to someplace outside
+ pg_wal/filename> is a symbolic link pointing to someplace outside
the cluster directory, which is a common setup anyway for performance
- reasons. You might also want to exclude postmaster.pid
- and postmaster.opts, which record information
- about the running
postmaster, not about the-
postmaster which will eventually use this backup.- (These files can confuse
pg_ctl.)+ reasons. You might also want to exclude postmaster.pidfilename>
+ and postmaster.optsfilename>, which record information
+ about the running
postmasterapplication>, not about the+
postmasterapplication> which will eventually use this backup.+ (These files can confuse
pg_ctlapplication>.)
It is often a good idea to also omit from the backup the files
- within the cluster's pg_replslot/ directory, so that
+ within the cluster's pg_replslot/filename> directory, so that
replication slots that exist on the master do not become part of the
backup. Otherwise, the subsequent use of the backup to create a standby
may result in indefinite retention of WAL files on the standby, and
- The contents of the directories pg_dynshmem/,
- pg_notify/>, pg_serial/>,
- pg_snapshots/>, pg_stat_tmp/>,
- and pg_subtrans/ (but not the directories themselves) can be
+ The contents of the directories pg_dynshmem/filename>,
+ pg_notify/filename>, pg_serial/>,
+ pg_snapshots/filename>, pg_stat_tmp/>,
+ and pg_subtrans/filename> (but not the directories themselves) can be
omitted from the backup as they will be initialized on postmaster startup.
If is set and is under the data
directory then the contents of that directory can also be omitted.
The backup label
- file includes the label string you gave to pg_start_backup,
- as well as the time at which pg_start_backup was run, and
+ file includes the label string you gave to pg_start_backupfunction>,
+ as well as the time at which pg_start_backupfunction> was run, and
the name of the starting WAL file. In case of confusion it is therefore
possible to look inside a backup file and determine exactly which
backup session the dump file came from. The tablespace map file includes
the symbolic link names as they exist in the directory
- pg_tblspc/ and the full path of each symbolic link.
+ pg_tblspc/filename> and the full path of each symbolic link.
These files are not merely for your information; their presence and
contents are critical to the proper operation of the system's recovery
process.
It is also possible to make a backup while the server is
stopped. In this case, you obviously cannot use
- pg_start_backup> or pg_stop_backup>, and
+ pg_start_backupfunction> or pg_stop_backup>, and
you will therefore be left to your own devices to keep track of which
backup is which and how far back the associated WAL files go.
It is generally better to follow the continuous archiving procedure above.
location in case you need them later. Note that this precaution will
require that you have enough free space on your system to hold two
copies of your existing database. If you do not have enough space,
- you should at least save the contents of the cluster's pg_wal
+ you should at least save the contents of the cluster's pg_walfilename>
subdirectory, as it might contain logs which
were not archived before the system went down.
Restore the database files from your file system backup. Be sure that they
are restored with the right ownership (the database system user, not
- root!) and with the right permissions. If you are using
+ rootliteral>!) and with the right permissions. If you are using
tablespaces,
- you should verify that the symbolic links in pg_tblspc/
+ you should verify that the symbolic links in pg_tblspc/filename>
were correctly restored.
- Remove any files present in pg_wal/; these came from the
+ Remove any files present in pg_wal/filename>; these came from the
file system backup and are therefore probably obsolete rather than current.
- If you didn't archive pg_wal/ at all, then recreate
+ If you didn't archive pg_wal/filename> at all, then recreate
it with proper permissions,
being careful to ensure that you re-establish it as a symbolic link
if you had it set up that way before.
If you have unarchived WAL segment files that you saved in step 2,
- copy them into pg_wal/. (It is best to copy them,
+ copy them into pg_wal/filename>. (It is best to copy them,
not move them, so you still have the unmodified files if a
problem occurs and you have to start over.)
- Create a recovery command file recovery.conf in the cluster
+ Create a recovery command file recovery.conffilename> in the cluster
data directory (see ). You might
- also want to temporarily modify pg_hba.conf to prevent
+ also want to temporarily modify pg_hba.conffilename> to prevent
ordinary users from connecting until you are sure the recovery was successful.
recovery be terminated because of an external error, the server can
simply be restarted and it will continue recovery. Upon completion
of the recovery process, the server will rename
- recovery.conf> to recovery.done> (to prevent
+ recovery.conffilename> to recovery.done> (to prevent
accidentally re-entering recovery mode later) and then
commence normal database operations.
Inspect the contents of the database to ensure you have recovered to
the desired state. If not, return to step 1. If all is well,
- allow your users to connect by restoring pg_hba.conf to normal.
+ allow your users to connect by restoring pg_hba.conffilename> to normal.
The key part of all this is to set up a recovery configuration file that
describes how you want to recover and how far the recovery should
- run. You can use recovery.conf.sample (normally
- located in the installation's share/ directory) as a
+ run. You can use recovery.conf.samplefilename> (normally
+ located in the installation's share/filename> directory) as a
prototype. The one thing that you absolutely must specify in
- recovery.conf> is the restore_command>,
- which tells
PostgreSQL how to retrieve archived- WAL file segments. Like the archive_command, this is
- a shell command string. It can contain %f, which is
- replaced by the name of the desired log file, and %p,
+ recovery.conffilename> is the restore_command>,
+ which tells
PostgreSQLproductname> how to retrieve archived+ WAL file segments. Like the archive_commandvarname>, this is
+ a shell command string. It can contain %fliteral>, which is
+ replaced by the name of the desired log file, and %pliteral>,
which is replaced by the path name to copy the log file to.
(The path name is relative to the current working directory,
i.e., the cluster's data directory.)
- Write %%> if you need to embed an actual %>
+ Write %%literal> if you need to embed an actual %>
character in the command. The simplest useful command is
something like:
restore_command = 'cp /mnt/server/archivedir/%f %p'
which will copy previously archived WAL segments from the directory
- /mnt/server/archivedir. Of course, you can use something
+ /mnt/server/archivedirfilename>. Of course, you can use something
much more complicated, perhaps even a shell script that requests the
operator to mount an appropriate tape.
It is important that the command return nonzero exit status on failure.
- The command will be called requesting files that are not
+ The command willemphasis> be called requesting files that are not
present in the archive; it must return nonzero when so asked. This is not
an error condition. An exception is that if the command was terminated by
a signal (other than SIGTERM, which is used as
Not all of the requested files will be WAL segment
files; you should also expect requests for files with a suffix of
- .backup> or .history>. Also be aware that
- the base name of the %p path will be different from
- %f; do not expect them to be interchangeable.
+ .backupliteral> or .history>. Also be aware that
+ the base name of the %pliteral> path will be different from
+ %fliteral>; do not expect them to be interchangeable.
WAL segments that cannot be found in the archive will be sought in
- pg_wal/; this allows use of recent un-archived segments.
+ pg_wal/filename>; this allows use of recent un-archived segments.
However, segments that are available from the archive will be used in
- preference to files in pg_wal/.
+ preference to files in pg_wal/filename>.
Normally, recovery will proceed through all available WAL segments,
thereby restoring the database to the current point in time (or as
close as possible given the available WAL segments). Therefore, a normal
- recovery will end with a file not found message, the exact text
+ recovery will end with a file not foundquote> message, the exact text
of the error message depending upon your choice of
- restore_command. You may also see an error message
+ restore_commandvarname>. You may also see an error message
at the start of recovery for a file named something like
- 00000001.history. This is also normal and does not
+ 00000001.historyfilename>. This is also normal and does not
indicate a problem in simple recovery situations; see
for discussion.
If you want to recover to some previous point in time (say, right before
the junior DBA dropped your main transaction table), just specify the
- required stopping point in recovery.conf. You can specify
- the stop point, known as the recovery target, either by
+ required stopping point in recovery.conffilename>. You can specify
+ the stop point, known as the recovery targetquote>, either by
date/time, named restore point or by completion of a specific transaction
ID. As of this writing only the date/time and named restore point options
are very usable, since there are no tools to help you identify with any
The stop point must be after the ending time of the base backup, i.e.,
- the end time of pg_stop_backup. You cannot use a base backup
+ the end time of pg_stop_backupfunction>. You cannot use a base backup
to recover to a time when that backup was in progress. (To
recover to such a time, you must go back to your previous base backup
and roll forward from there.)
If recovery finds corrupted WAL data, recovery will
halt at that point and the server will not start. In such a case the
recovery process could be re-run from the beginning, specifying a
- recovery target before the point of corruption so that recovery
+ recovery targetquote> before the point of corruption so that recovery
can complete normally.
If recovery fails for an external reason, such as a system crash or
if the WAL archive has become inaccessible, then the recovery can simply
be restarted and it will restart almost from where it failed.
Recovery restart works much like checkpointing in normal operation:
the server periodically forces all its state to disk, and then updates
- the pg_control file to indicate that the already-processed
+ the pg_controlfilename> file to indicate that the already-processed
WAL data need not be scanned again.
suppose you dropped a critical table at 5:15PM on Tuesday evening, but
didn't realize your mistake until Wednesday noon.
Unfazed, you get out your backup, restore to the point-in-time 5:14PM
- Tuesday evening, and are up and running. In this history of
+ Tuesday evening, and are up and running. In thisemphasis> history of
the database universe, you never dropped the table. But suppose
you later realize this wasn't such a great idea, and would like
to return to sometime Wednesday morning in the original history.
- To deal with this problem,
PostgreSQL has a notion- of timelines. Whenever an archive recovery completes,
+ To deal with this problem,
PostgreSQLproductname> has a notion+ of timelinesfirstterm>. Whenever an archive recovery completes,
a new timeline is created to identify the series of WAL records
generated after that recovery. The timeline
ID number is part of WAL segment file names so a new timeline does
and so have to do several point-in-time recoveries by trial and error
until you find the best place to branch off from the old history. Without
timelines this process would soon generate an unmanageable mess. With
- timelines, you can recover to any prior state, including
+ timelines, you can recover to anyemphasis> prior state, including
states in timeline branches that you abandoned earlier.
- Every time a new timeline is created,
PostgreSQL creates- a timeline history file that shows which timeline it branched
+ Every time a new timeline is created,
PostgreSQLproductname> creates+ a timeline historyquote> file that shows which timeline it branched
off from and when. These history files are necessary to allow the system
to pick the right WAL segment files when recovering from an archive that
contains multiple timelines. Therefore, they are archived into the WAL
that was current when the base backup was taken. If you wish to recover
into some child timeline (that is, you want to return to some state that
was itself generated after a recovery attempt), you need to specify the
- target timeline ID in recovery.conf. You cannot recover into
+ target timeline ID in recovery.conffilename>. You cannot recover into
timelines that branched off earlier than the base backup.
Standalone Hot Backups
- It is possible to use
PostgreSQL's backup facilities to+ It is possible to use
PostgreSQLproductname>'s backup facilities to produce standalone hot backups. These are backups that cannot be used
for point-in-time recovery, yet are typically much faster to backup and
- restore than
pg_dump dumps. (They are also much larger- than
pg_dump dumps, so in some cases the speed advantage+ restore than
pg_dumpapplication> dumps. (They are also much larger+ than
pg_dumpapplication> dumps, so in some cases the speed advantage might be negated.)
As with base backups, the easiest way to produce a standalone
hot backup is to use the
- tool. If you include the -X parameter when calling
+ tool. If you include the -Xliteral> parameter when calling
it, all the write-ahead log required to use the backup will be
included in the backup automatically, and no special action is
required to restore the backup.
If more flexibility in copying the backup files is needed, a lower
level process can be used for standalone hot backups as well.
To prepare for low level standalone hot backups, make sure
- wal_level is set to
- replica> or higher, archive_mode> to
- on>, and set up an archive_command> that performs
- archiving only when a switch file exists. For example:
+ wal_levelvarname> is set to
+ replicaliteral> or higher, archive_mode> to
+ onliteral>, and set up an archive_command> that performs
+ archiving only when a switch fileemphasis> exists. For example:
archive_command = 'test ! -f /var/lib/pgsql/backup_in_progress || (test ! -f /var/lib/pgsql/archive/%f && cp %p /var/lib/pgsql/archive/%f)'
This command will perform archiving when
- /var/lib/pgsql/backup_in_progress exists, and otherwise
- silently return zero exit status (allowing
PostgreSQL+ /var/lib/pgsql/backup_in_progressfilename> exists, and otherwise
+ silently return zero exit status (allowing
PostgreSQLproductname> to recycle the unwanted WAL file).
rm /var/lib/pgsql/backup_in_progress
tar -rf /var/lib/pgsql/backup.tar /var/lib/pgsql/archive/
- The switch file /var/lib/pgsql/backup_in_progress is
+ The switch file /var/lib/pgsql/backup_in_progressfilename> is
created first, enabling archiving of completed WAL files to occur.
After the backup the switch file is removed. Archived WAL files are
then added to the backup so that both base backup and all required
- WAL files are part of the same
tar file.+ WAL files are part of the same
tarapplication> file. Please remember to add error handling to your backup scripts.
archive_command = 'gzip < %p > /var/lib/pgsql/archive/%f'
- You will then need to use
gunzip during recovery:+ You will then need to use
gunzipapplication> during recovery: restore_command = 'gunzip < /mnt/server/archivedir/%f > %p'
Many people choose to use scripts to define their
archive_command, so that their
- postgresql.conf entry looks very simple:
+ postgresql.conffilename> entry looks very simple:
archive_command = 'local_backup_script.sh "%p" "%f"'
more than a single command in the archiving process.
This allows all complexity to be managed within the script, which
can be written in a popular scripting language such as
+
bashapplication> or perl>.
When using an archive_command script, it's desirable
to enable .
- Any messages written to stderr from the script will then
+ Any messages written to stderrsystemitem> from the script will then
appear in the database server log, allowing complex configurations to
be diagnosed easily if they fail.
If a
command is executed while a base backup is being taken, and then
- the template database that the CREATE DATABASE copied
+ the template database that the CREATE DATABASEcommand> copied
is modified while the base backup is still in progress, it is
possible that recovery will cause those modifications to be
propagated into the created database as well. This is of course
before you do so.) Turning off page snapshots does not prevent
use of the logs for PITR operations. An area for future
development is to compress archived WAL data by removing
- unnecessary page copies even when full_page_writes is
+ unnecessary page copies even when full_page_writesvarname> is
on. In the meantime, administrators might wish to reduce the number
of page snapshots included in WAL by increasing the checkpoint
interval parameters as much as feasible.
projects / postgresql.git / commitdiff