Prevent WAL corruption after a standby promotion.

When a PostgreSQL instance performing archive recovery but not using
standby mode is promoted, and the last WAL segment that it attempted
to read ended in a partial record, the previous code would create
invalid WAL on the new timeline. The WAL from the previously timeline
would be copied to the new timeline up until the end of the last valid
record, but instead of beginning to write WAL at immediately
afterwards, the promoted server would write an overwrite contrecord at
the beginning of the next segment. The end of the previous segment
would be left as all-zeroes, resulting in failures if anything tried
to read WAL from that file.

The root of the issue is that ReadRecord() decides whether to set
abortedRecPtr and missingContrecPtr based on the value of StandbyMode,
but ReadRecord() switches to a new timeline based on the value of
ArchiveRecoveryRequested. We shouldn't try to write an overwrite
contrecord if we're switching to a new timeline, so change the test in
ReadRecod() to check ArchiveRecoveryRequested instead.

Code fix by Dilip Kumar. Comments by me incorporating suggested
language from Álvaro Herrera. Further review from Kyotaro Horiguchi
and Sami Imseih.

Discussion: http://postgr.es/m/CAFiTN-t7umki=PK8dT1tcPV=mOUe2vNhHML6b3T7W7qqvvajjg@mail.gmail.com
Discussion: http://postgr.es/m/FB0DEA0B-E14E-43A0-811F-C1AE93D00FF3%40amazon.com

diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 6efc103c900176e05c23227c9b2a0b743473971a..8f4c8ae4df78535569e159b5639addf488baa860 100644 (file)
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -5444,6 +5444,14 @@ StartupXLOG(void)
     */
    if (!XLogRecPtrIsInvalid(missingContrecPtr))
    {
+       /*
+        * We should only have a missingContrecPtr if we're not switching to
+        * a new timeline. When a timeline switch occurs, WAL is copied from
+        * the old timeline to the new only up to the end of the last complete
+        * record, so there can't be an incomplete WAL record that we need to
+        * disregard.
+        */
+       Assert(newTLI == endOfRecoveryInfo->lastRecTLI);
        Assert(!XLogRecPtrIsInvalid(abortedRecPtr));
        EndOfLog = missingContrecPtr;
    }

diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c
index 013239d64e8fbf3a514fd972f482ece4a799d8c5..f6f894b36d19daaca2272935cfcdddba8020cad3 100644 (file)
--- a/src/backend/access/transam/xlogrecovery.c
+++ b/src/backend/access/transam/xlogrecovery.c
@@ -3022,12 +3022,18 @@ ReadRecord(XLogPrefetcher *xlogprefetcher, int emode,
        if (record == NULL)
        {
            /*
-            * When not in standby mode we find that WAL ends in an incomplete
-            * record, keep track of that record.  After recovery is done,
-            * we'll write a record to indicate to downstream WAL readers that
-            * that portion is to be ignored.
+            * When we find that WAL ends in an incomplete record, keep track
+            * of that record.  After recovery is done, we'll write a record to
+            * indicate to downstream WAL readers that that portion is to be
+            * ignored.
+            *
+            * However, when ArchiveRecoveryRequested = true, we're going to
+            * switch to a new timeline at the end of recovery. We will only
+            * copy WAL over to the new timeline up to the end of the last
+            * complete record, so if we did this, we would later create an
+            * overwrite contrecord in the wrong place, breaking everything.
             */
-           if (!StandbyMode &&
+           if (!ArchiveRecoveryRequested &&
                !XLogRecPtrIsInvalid(xlogreader->abortedRecPtr))
            {
                abortedRecPtr = xlogreader->abortedRecPtr;