-
+
Routine Database Maintenance Tasks
be performed regularly to achieve optimum performance. The tasks
discussed here are required, but they
are repetitive in nature and can easily be automated using standard
-
Unix tools such as
cron scripts or
+ tools such as
cron scripts or
Windows'
Task Scheduler. But it is the database administrator's responsibility to set up appropriate scripts, and to
check that they execute successfully.
- command has to run on
- a regular basis for several reasons:
+
PostgreSQL databases require periodic
+ maintenance known as vacuuming. For many installations, it
+ is sufficient to let vacuuming be performed by the autovacuum
+ daemon, which is described in . You might
+ need to adjust the autovacuuming parameters described there to obtain best
+ results for your situation. Some database administrators will want to
+ supplement or replace the daemon's activities with manually-managed
+ VACUUM commands, which typically are executed according to a
+ schedule by
cron or
Task+ Scheduler scripts. To set up manually-managed vacuuming properly,
+ it is essential to understand the issues discussed in the next few
+ subsections. Administrators who rely on autovacuuming may still wish
+ to skim this material to help them understand and adjust autovacuuming.
+
+
+
+
Vacuuming Basics
+
+ command has to
+ process each table on a regular basis for several reasons:
transaction ID wraparound.
-
- The standard form of VACUUM can run in parallel with production
- database operations. Commands such as SELECT,
- INSERT, UPDATE, and DELETE
- will continue to function as normal, though you will not be able to modify the
- definition of a table with commands such as ALTER TABLE ADD COLUMN
- while it is being vacuumed.
- Also, VACUUM requires a substantial amount of I/O
- traffic, which can cause poor performance for other active sessions.
- There are configuration parameters that can be adjusted to reduce the
- performance impact of background vacuuming — see
- .
-
+ Each of these reasons dictates performing VACUUM operations
+ of varying frequency and scope, as explained in the following subsections.
+
- Fortunately,
- monitors table
- activity and performs VACUUMs when necessary.
- Autovacuum works dynamically so it is often better
- administration-scheduled vacuuming.
-
+ There are two variants of VACUUM: standard VACUUM
+ and VACUUM FULL. VACUUM FULL can reclaim more
+ disk space but runs much more slowly. Also,
+ the standard form of VACUUM can run in parallel with production
+ database operations. (Commands such as SELECT,
+ INSERT, UPDATE, and
+ DELETE will continue to function as normal, though you
+ will not be able to modify the definition of a table with commands such as
+ ALTER TABLE while it is being vacuumed.)
+ VACUUM FULL requires exclusive lock on the table it is
+ working on, and therefore cannot be done in parallel with other use
+ of the table. Another disadvantage of VACUUM FULL is that
+ while it reduces table size, it does not reduce index size proportionally;
+ in fact it can make indexes larger. Generally, therefore,
+ administrators should strive to use standard VACUUM and
+ avoid VACUUM FULL.
+
+
+ VACUUM creates a substantial amount of I/O
+ traffic, which can cause poor performance for other active sessions.
+ There are configuration parameters that can be adjusted to reduce the
+ performance impact of background vacuuming — see
+ .
+
+
Recovering Disk Space
- In
normal PostgreSQL operation, an
UPDATE or DELETE of a row does not
immediately remove the old version of the row.
This approach is necessary to gain the benefits of multiversion
- concurrency control (see ): the row versions
+ concurrency control (see ): the row version
must not be deleted while it is still potentially visible to other
transactions. But eventually, an outdated or deleted row version is no
- longer of interest to any transaction. The space it occupies must be
+ longer of interest to any transaction. The space it occupies must then be
reclaimed for reuse by new rows, to avoid infinite growth of disk
space requirements. This is done by running VACUUM.
- There are two variants of the VACUUM
- command. The first form, known as lazy vacuum
or
- just VACUUM, marks dead data in tables and
- indexes for future reuse; it does not attempt
- to reclaim the space used by this dead data unless the space is
- at the end of the table and an exclusive table lock can be easily
- obtained. Unused space at the start or middle of the file does
- not result in the file being shortened and space returned to the
- operating system. This variant of VACUUM can be
- run concurrently with normal database operations.
-
-
- The second form is the VACUUM FULL
- command. This uses a more aggressive algorithm for reclaiming the
- space consumed by dead row versions. Any space that is freed by
- VACUUM FULL is immediately returned to the
- operating system, and the table data is physically compacted on
- the disk. Unfortunately, this variant of the
- VACUUM command acquires an exclusive lock on
- each table while VACUUM FULL is processing
- it. Therefore, frequently using VACUUM FULL can
- have an extremely negative effect on the performance of concurrent
- database queries.
+ The standard form of VACUUM removes dead row
+ versions in tables and indexes and marks the space available for
+ future reuse. However, it will not return the space to the operating
+ system, except in the special case where one or more pages at the
+ end of a table become entirely free and an exclusive table lock can be
+ easily obtained. In contrast, VACUUM FULL actively compacts
+ tables by moving row versions to earlier pages. It is thus able to
+ force pages at the end of the table to become entirely free, whereupon
+ it will return them to the operating system. However, if many rows
+ must be moved, this can take a long time. Also, moving a row requires
+ transiently making duplicate index entries for it (the entry pointing
+ to its new location must be made before the old entry can be removed);
+ so moving a lot of rows this way causes severe index bloat.
- Fortunately,
- monitors table
- activity and performs VACUUMs when necessary. This
- eliminates the need for administrators to worry about disk space
- recovery in all but the most unusual cases.
+ The usual goal of routine vacuuming is to do standard VACUUMs
+ often enough to avoid needing VACUUM FULL. The
+ autovacuum daemon attempts to work this way, and in fact will
+ never issue VACUUM FULL. In this approach, the idea
+ is not to keep tables at their minimum size, but to maintain steady-state
+ usage of disk space: each table occupies space equivalent to its
+ minimum size plus however much space gets used up between vacuumings.
+ Although VACUUM FULL can be used to shrink a table back
+ to its minimum size and return the disk space to the operating system,
+ there is not much point in this if the table will just grow again in the
+ future. Thus, moderately-frequent standard VACUUM runs are a
+ better approach than infrequent VACUUM FULL runs for
+ maintaining heavily-updated tables.
- For administrators who want to control VACUUM
- themselves, the standard form of VACUUM is best used to
- maintain a steady-state usage of disk space. If you need to return
- disk space to the operating system, you can use VACUUM
- FULL, but this is unwise if the table will just grow again in the
- future. Moderately-frequent standard VACUUM runs are a
- better approach than infrequent VACUUM FULL runs for
- maintaining heavily-updated tables. However, if some heavily-updated
- tables have gone too long with infrequent VACUUM, you can
- use VACUUM FULL or CLUSTER to get performance
- back (it is much slower to scan a table containing almost only dead
- rows).
+ Some administrators prefer to schedule vacuuming themselves, for example
+ doing all the work at night when load is low.
+ The difficulty with doing vacuuming according to a fixed schedule
+ is that if a table has an unexpected spike in update activity, it may
+ get bloated to the point that VACUUM FULL is really necessary
+ to reclaim space. Using the autovacuum daemon alleviates this problem,
+ since the daemon schedules vacuuming dynamically in response to update
+ activity. It is unwise to disable the daemon completely unless you
+ have an extremely predictable workload. One possible compromise is
+ to set the daemon's parameters so that it will only react to unusually
+ heavy update activity, thus keeping things from getting out of hand,
+ while scheduled VACUUMs are expected to do the bulk of the
+ work when the load is typical.
- For those not using autovacuum, one approach is to schedule a
- database-wide VACUUM once a day during low-usage period,
- supplemented by more frequent vacuuming of heavily-updated tables if
+ For those not using autovacuum, a typical approach is to schedule a
+ database-wide VACUUM once a day during a low-usage period,
+ supplemented by more frequent vacuuming of heavily-updated tables as
necessary. (Some installations with extremely high update rates vacuum
their busiest tables as often as once every few minutes.) If you have
multiple databases in a cluster, don't forget to
linkend="app-vacuumdb" endterm="app-vacuumdb-title"> might be helpful.
+
- VACUUM FULL is recommended for cases where you know
- you have deleted the majority of rows in a table, so that the
- steady-state size of the table can be shrunk substantially with
- VACUUM FULL's more aggressive approach. Use plain
- VACUUM, not VACUUM FULL, for routine
- vacuuming for space recovery.
+ Neither form of VACUUM is entirely satisfactory when
+ a table contains large numbers of dead row versions as a result of
+ massive update or delete activity. If you have such a table and
+ you need to reclaim the excess disk space it occupies, the best
+ way is to use
+ or one of the table-rewriting variants of
+ .
+ These commands rewrite an entire new copy of the table and build
+ new indexes for it. Like VACUUM FULL, they require
+ exclusive lock. Note that they also temporarily use extra disk
+ space, since the old copies of the table and indexes can't be
+ released until the new ones are complete. In the worst case where
+ your disk is nearly full, VACUUM FULL may be the only
+ workable alternative.
+
+
If you have a table whose entire contents are deleted on a periodic
- basis, consider doing it with TRUNCATE rather
+ basis, consider doing it with
+ rather
than using DELETE followed by
VACUUM. TRUNCATE removes the
entire content of the table immediately, without requiring a
subsequent VACUUM or VACUUM
FULL to reclaim the now-unused disk space.
+ The disadvantage is that strict MVCC semantics are violated.
+
degrade database performance.
+ The autovacuum daemon, if enabled, will automatically issue
+ ANALYZE commands whenever the content of a table has
+ changed sufficiently. However, administrators might prefer to rely
+ on manually-scheduled ANALYZE operations, particularly
+ if it is known that update activity on a table will not affect the
+ statistics of interesting columns. The daemon schedules
+ ANALYZE strictly as a function of the number of rows
+ inserted or updated; it has no knowledge of whether that will lead
+ to meaningful statistical changes.
+
+
As with vacuuming for space recovery, frequent updates of statistics
are more useful for heavily-updated tables than for seldom-updated
It is possible to run ANALYZE on specific tables and even
just specific columns of a table, so the flexibility exists to update some
statistics more frequently than others if your application requires it.
- In practice, however, it is usually best to just analyze the entire database
- because it is a fast operation. It uses a statistical random sampling of
- the rows of a table rather than reading every single row.
+ In practice, however, it is usually best to just analyze the entire
+ database, because it is a fast operation. ANALYZE uses a
+ statistical random sampling of the rows of a table rather than reading
+ every single row.
Although per-column tweaking of ANALYZE frequency might not be
very productive, you might well find it worthwhile to do per-column
adjustment of the level of detail of the statistics collected by
- ANALYZE. Columns that are heavily used in WHERE clauses
- and have highly irregular data distributions might require a finer-grain
- data histogram than other columns. See ALTER TABLE SET
- STATISTICS.
+ ANALYZE. Columns that are heavily used in WHERE
+ clauses and have highly irregular data distributions might require a
+ finer-grain data histogram than other columns. See ALTER TABLE
+ SET STATISTICS, or change the database-wide default using the
+ linkend="guc-default-statistics-target"> configuration parameter.
-
- Fortunately,
- monitors table
- activity and performs ANALYZEs when necessary. This
- eliminates the need for administrators to manually schedule
- ANALYZE.
-
-
- For those not using autovacuum, one approach is to schedule a
- database-wide ANALYZE once a day at a low-usage time of
- day; this can usefully be combined with a nightly VACUUM.
- However, sites with relatively slowly changing table statistics might
- find that this is overkill, and that less-frequent ANALYZE
- runs are sufficient.
-
The maximum time that a table can go unvacuumed is two billion
transactions minus the vacuum_freeze_min_age that was used
- when it was last vacuumed.
- If it were to go unvacuumed for longer than that,
- data loss could result. To ensure that this does not
- happen,
- is invoked on any table
- that might contain XIDs older than the age specified by the
- configuration parameter
- . (This will happen
- even if autovacuum is otherwise disabled.)
+ when it was last vacuumed. If it were to go unvacuumed for longer than
+ that, data loss could result. To ensure that this does not happen,
+ autovacuum is invoked on any table that might contain XIDs older than the
+ age specified by the configuration parameter
+ linkend="guc-autovacuum-freeze-max-age">. (This will happen even if
+ autovacuum is otherwise disabled.)
- The Auto-Vacuum Daemon
+ The Autovacuum Daemon
general information
- Beginning in
PostgreSQL 8.1, there is an
- optional feature called autovacuum,
+
PostgreSQL has an optional but highly
+ recommended feature called autovacuum,
whose purpose is to automate the execution of
VACUUM and ANALYZE commands.
When enabled, autovacuum checks for
- Beginning in
PostgreSQL 8.3, autovacuum has a
- multiprocess architecture: There is a daemon process, called the
+ The autovacuum daemon actually consists of multiple processes.
+ There is a persistent daemon process, called the
autovacuum launcher, which is in charge of starting
an autovacuum worker process on each database every
seconds. On each run, the worker
- process checks each table within that database, and VACUUM or
- ANALYZE commands are issued as needed.
+ process checks each table within that database, and executes
+ VACUUM and/or ANALYZE commands as needed.
The setting limits how many
workers may be running at any time. If several large tables all become
eligible for vacuuming in a short amount of time, all autovacuum workers
- may end up vacuuming those tables for a very long time. This would result
+ may become occupied with vacuuming those tables for a long period.
+ This would result
in other tables and databases not being vacuumed until a worker became
available. There is not a limit on how many workers might be in a
single database, but workers do try to avoid repeating work that has
Tables whose relfrozenxid value is more than
autovacuum_freeze_max_age transactions old are always
- vacuumed. Otherwise,
- two conditions are used to determine which operation(s)
- to apply. If the number of obsolete tuples since the last
+ vacuumed. Otherwise, if the number of tuples obsoleted since the last
VACUUM exceeds the vacuum threshold
, the
table is vacuumed. The vacuum threshold is defined as:
collector; it is a semi-accurate count updated by each
UPDATE and DELETE operation. (It
is only semi-accurate because some information might be lost under heavy
- load.) For analyze, a similar condition is used: the threshold, defined as:
+ load.)
+
+
+ For analyze, a similar condition is used: the threshold, defined as:
analyze threshold = analyze base threshold + analyze scale factor * number of tuples
The next two parameters, the vacuum cost delay
(pg_autovacuum.vac_cost_delay)
and the vacuum cost limit
- (pg_autovacuum.vac_cost_limit),
+ (pg_autovacuum.vac_cost_limit),
are used to set table-specific values for the
feature.
The last two parameters,
(pg_autovacuum.freeze_min_age)
and
- (pg_autovacuum.freeze_max_age),
+ (pg_autovacuum.freeze_max_age),
are used to set table-specific values for
and
respectively.
projects / postgresql.git / commitdiff