Update the documentation on recovering from (M)XID exhaustion.

The old documentation encourages entering single-user mode for no
reason, which is a bad plan in most cases. Instead, discourage users
from doing that, and explain the limited cases in which it may be
desirable.

The old documentation claims that running VACUUM as anyone but the
superuser can't possibly work, which is not really true, because it
might be that some other user has enough permissions to VACUUM all
the tables that matter. Weaken the language just a bit.

The old documentation claims that you can't run any commands
when near XID exhaustion, which is false because you can still
run commands that don't require an XID, like a SELECT without a
locking clause.

The old documentation doesn't clearly explain that it's a good idea
to get rid of prepared transactons, long-running transactions, and
replication slots that are preventing (M)XID horizon advancement.
Spell out the steps to do that.

Also, discourage the use of VACUUM FULL and VACUUM FREEZE in
this type of scenario.

Back-patch to v14. Much of this is good advice on all supported
versions, but before 60f1f09ff44308667ef6c72fbafd68235e55ae27
the chances of VACUUM failing in multi-user mode were much higher.

Alexander Alekseev, John Naylor, Robert Haas, reviewed at various
times by Peter Geoghegan, Hannu Krosing, and Andres Freund.

Discussion: http://postgr.es/m/CA+TgmoYtsUDrzaHcmjFhLzTk1VEv29mO_u-MT+XWHrBJ_4nD8A@mail.gmail.com

diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 3018d8f64cf33f2775529475c7136b05c178c82e..66f2c6a02e86ff277c0856250e0be3b9079f9426 100644 (file)
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -660,29 +660,79 @@ HINT:  To avoid a database shutdown, execute a database-wide VACUUM in that data
 
 
     (A manual VACUUM should fix the problem, as suggested by the
-    hint; but note that the VACUUM must be performed by a
-    superuser, else it will fail to process system catalogs and thus not
-    be able to advance the database's datfrozenxid.)
-    If these warnings are
-    ignored, the system will shut down and refuse to start any new
-    transactions once there are fewer than three million transactions left
-    until wraparound:
+    hint; but note that the VACUUM should be performed by a
+    superuser, else it will fail to process system catalogs, which prevent it from
+    being able to advance the database's datfrozenxid.)
+    If these warnings are ignored, the system will refuse to assign new XIDs once
+    there are fewer than three million transactions left until wraparound:
 
 
 ERROR:  database is not accepting commands to avoid wraparound data loss in database "mydb"
 HINT:  Stop the postmaster and vacuum that database in single-user mode.
 
 
-    The three-million-transaction safety margin exists to let the
-    administrator recover without data loss, by manually executing the
-    required VACUUM commands.  However, since the system will not
-    execute commands once it has gone into the safety shutdown mode,
-    the only way to do this is to stop the server and start the server in single-user
-    mode to execute VACUUM.  The shutdown mode is not enforced
-    in single-user mode.  See the  reference
-    page for details about using single-user mode.
+    In this condition any transactions already in progress can continue,
+    but only read-only transactions can be started. Operations that
+    modify database records or truncate relations will fail.
+    The VACUUM command can still be run normally.
+    Contrary to what the hint states, it is not necessary or desirable to stop the
+    postmaster or enter single user-mode in order to restore normal operation.
+    Instead, follow these steps:
+
+    
+     
+      Resolve old prepared transactions. You can find these by checking
+       pg_prepared_xacts for rows where
+       age(transactionid) is large. Such transactions should be
+       committed or rolled back.
+     
+     
+      End long-running open transactions. You can find these by checking
+       pg_stat_activity for rows where
+       age(backend_xid) or age(backend_xmin) is
+       large. Such transactions should be committed or rolled back, or the session
+       can be terminated using pg_terminate_backend.
+     
+     
+      Drop any old replication slots. Use
+       pg_stat_replication to
+       find slots where age(xmin) or age(catalog_xmin)
+       is large. In many cases, such slots were created for replication to servers that no
+       longer exist, or that have been down for a long time. If you drop a slot for a server
+       that still exists and might still try to connect to that slot, that replica may
+       need to be rebuilt.
+     
+     
+      Execute VACUUM in the target database. A database-wide
+       VACUUM is simplest; to reduce the time required, it as also possible
+       to issue manual VACUUM commands on the tables where
+       relminxid is oldest. Do not use VACUUM FULL
+       in this scenario, because it requires an XID and will therefore fail, except in super-user
+       mode, where it will instead consume an XID and thus increase the risk of transaction ID
+       wraparound. Do not use VACUUM FREEZE either, because it will do
+       more than the minimum amount of work required to restore normal operation.
+     
+     
+      Once normal operation is restored, ensure that autovacuum is properly configured
+       in the target database in order to avoid future problems.
+     
+    
    
 
+   
+    
+     In earlier versions, it was sometimes necessary to stop the postmaster and
+     VACUUM the database in a single-user mode. In typical scenarios, this
+     is no longer necessary, and should be avoided whenever possible, since it involves taking
+     the system down. It is also riskier, since it disables transaction ID wraparound safeguards
+     that are designed to prevent data loss.  The only reason to use single-user mode in this
+     scenario is if you wish to TRUNCATE or DROP unneeded
+     tables to avoid needing to VACUUM them.  The three-million-transaction
+     safety margin exists to let the administrator do this. See the
+      reference page for details about using single-user mode.
+    
+   
+
    
     Multixacts and Wraparound
 
@@ -747,6 +797,38 @@ HINT:  Stop the postmaster and vacuum that database in single-user mode.
      have the oldest multixact-age.  Both of these kinds of aggressive
      scans will occur even if autovacuum is nominally disabled.
     
+
+    
+     Similar to the XID case, if autovacuum fails to clear old MXIDs from a table, the
+     system will begin to emit warning messages when the database's oldest MXIDs reach forty
+     million transactions from the wraparound point.  And, just as an the XID case, if these
+     warnings are ignored, the system will refuse to generate new MXIDs once there are fewer
+     than three million left until wraparound.
+    
+
+    
+     Normal operation when MXIDs are exhausted can be restored in much the same way as
+     when XIDs are exhausted. Follow the same steps in the previous section, but with the
+     following differences:
+
+    
+     
+      Running transactions and prepared transactions can be ignored if there
+       is no chance that they might appear in a multixact.
+     
+     
+      MXID information is not directly visible in system views such as
+       pg_stat_activity; however, looking for old XIDs is still a good
+       way of determining which transactions are causing MXID wraparound problems.
+     
+     
+      XID exhaustion will block all write transactions, but MXID exhaustion will
+       only block a subset of write transactions, specifically those that involve
+       row locks that require an MXID.
+     
+    
+   
+