Alexander Korotkov [Sat, 11 Jul 2020 00:20:46 +0000 (03:20 +0300)]
Improve error reporting for jsonpath .double() method
When jsonpath .double() method detects that numeric or string can't be
converted to double precision, it throws an error. This commit makes these
errors explicitly express the reason of failure.
Discussion: https://postgr.es/m/CAPpHfdtqJtiSXkP7tOXez18NxhLUH_-75bL8%3DOce4Ki%2Bbv7V6Q%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Tom Lane
Backpatch-through: 12
Tom Lane [Fri, 10 Jul 2020 17:16:00 +0000 (13:16 -0400)]
Doc: update or remove dead external links.
Re-point comp.ai.genetic FAQ link to a more stable address.
Remove stale links to AIX documentation; we don't really need to
tell AIX users how to use their systems.
Remove stale links to HP documentation about SSL. We've had to
update those twice before, making it increasingly obvious that
HP does not intend them to be stable landing points. They're
not particularly authoritative, either. (This change effectively
reverts
bbd3bdba3.)
Daniel Gustafsson and Álvaro Herrera, per a gripe from
Kyotaro Horiguchi. Back-patch, since these links are
just as dead in the back branches.
Discussion: https://postgr.es/m/
20200709.161226.
204639179120026914[email protected]
Alvaro Herrera [Fri, 10 Jul 2020 00:13:25 +0000 (20:13 -0400)]
Remove WARNING message from brin_desummarize_range
This message was being emitted on the grounds that only crashed
summarization could cause it, but in reality even an aborted vacuum
could do it ... which makes it way too noisy, particularly since it
shows up in regression tests and makes them die.
Reported by Tom Lane.
Discussion: https://postgr.es/m/489091.
1593534251@sss.pgh.pa.us
Tom Lane [Thu, 9 Jul 2020 21:38:52 +0000 (17:38 -0400)]
Tighten up Windows CRLF conversion in our TAP test scripts.
Back-patch commits
91bdf499b and
ffb4cee43, so that all branches
agree on when and how to do Windows CRLF conversion.
This should close the referenced thread. Thanks to Andrew Dunstan
for discussion/review.
Discussion: https://postgr.es/m/
412ae8da-76bb-640f-039a-
f3513499e53d@gmx.net
Tom Lane [Thu, 9 Jul 2020 20:02:23 +0000 (16:02 -0400)]
Fix pg_current_logfile() to not emit a carriage return on Windows.
Due to not having our signals straight about CRLF vs. LF line
termination, the output of pg_current_logfile() included a trailing
\r on Windows. To fix, force the file descriptor it uses into text
mode.
While here, move a couple of local variable declarations to make
the function's logic clearer.
In v12 and v13, also back-patch the test added by
1c4e88e2f so that
this function has some test coverage. However, the 004_logrotate.pl
test script doesn't exist before v12, and it didn't seem worth adding
to older branches just for this.
Per report from Thomas Kellerer. Back-patch to v10 where this
function was added.
Discussion: https://postgr.es/m/
412ae8da-76bb-640f-039a-
f3513499e53d@gmx.net
Fujii Masao [Thu, 9 Jul 2020 04:31:33 +0000 (13:31 +0900)]
doc: Correct the description about the length of pg_stat_activity.query.
pg_stat_activity.query text is truncated at 1024 bytes. But previously
the document described that it's truncated at 1024 characters.
This was not accurate when considering multibyte characters.
Back-patch to v10 where this inaccurate description was added.
Author: Atsushi Torikoshi
Reviewed-by: Daniel Gustafsson, Fujii Masao
Discussion: https://postgr.es/m/
cd5b49a5a14e887542f5f569c1c6bde2@oss.nttdata.com
Michael Paquier [Sun, 5 Jul 2020 10:36:18 +0000 (19:36 +0900)]
doc: Fix incorrect reference to textout in plpgsql examples
This error has survived for 22 years, and has been introduced by
da63386.
Reported-by: Erwin Brandstetter
Discussion: https://postgr.es/m/CAGHENJ57wogGOvGXo5LgWYcqswxafLck8ELqHDR+zrkTPgs_OQ@mail.gmail.com
Backpatch-through: 9.5
Joe Conway [Sat, 4 Jul 2020 17:47:21 +0000 (13:47 -0400)]
Fix "ignoring return value" complaints from commit
96d1f423f9
The cfbot and some BF animals are complaining about the previous
read_binary_file commit because of ignoring return value of ‘fread’.
So let's make everyone happy by testing the return value even though
not strictly needed.
Reported by Justin Pryzby, and suggested patch by Tom Lane. Backpatched
to v11 same as the previous commit.
Reported-By: Justin Pryzby
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/
969b8d82-5bb2-5fa8-4eb1-
f0e685c5d736%40joeconway.com
Backpatch-through: 11
Joe Conway [Sat, 4 Jul 2020 10:28:44 +0000 (06:28 -0400)]
Read until EOF vice stat-reported size in read_binary_file
read_binary_file(), used by SQL functions pg_read_file() and friends,
uses stat to determine file length to read, when not passed an explicit
length as an argument. This is problematic, for example, if the file
being read is a virtual file with a stat-reported length of zero.
Arrange to read until EOF, or StringInfo data string lenth limit, is
reached instead.
Original complaint and patch by me, with significant review, corrections,
advice, and code optimizations by Tom Lane. Backpatched to v11. Prior to
that only paths relative to the data and log dirs were allowed for files,
so no "zero length" files were reachable anyway.
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/flat/
969b8d82-5bb2-5fa8-4eb1-
f0e685c5d736%40joeconway.com
Backpatch-through: 11
Tom Lane [Fri, 3 Jul 2020 23:01:21 +0000 (19:01 -0400)]
Clamp total-tuples estimates for foreign tables to ensure planner sanity.
After running GetForeignRelSize for a foreign table, adjust rel->tuples
to be at least as large as rel->rows. This prevents bizarre behavior
in estimate_num_groups() and perhaps other places, especially in the
scenario where rel->tuples is zero because pg_class.reltuples is
(suggesting that ANALYZE has never been run for the table). As things
stood, we'd end up estimating one group out of any GROUP BY on such a
table, whereas the default group-count estimate is more likely to result
in a sane plan.
Also, clarify in the documentation that GetForeignRelSize has the option
to override the rel->tuples value if it has a better idea of what to use
than what is in pg_class.reltuples.
Per report from Jeff Janes. Back-patch to all supported branches.
Patch by me; thanks to Etsuro Fujita for review
Discussion: https://postgr.es/m/CAMkU=1xNo9cnan+Npxgz0eK7394xmjmKg-QEm8wYG9P5-CcaqQ@mail.gmail.com
Tom Lane [Fri, 3 Jul 2020 21:01:34 +0000 (17:01 -0400)]
Fix temporary tablespaces for shared filesets some more.
Commit
ecd9e9f0b fixed the problem in the wrong place, causing unwanted
side-effects on the behavior of GetNextTempTableSpace(). Instead,
let's make SharedFileSetInit() responsible for subbing in the value
of MyDatabaseTableSpace when the default tablespace is called for.
The convention about what is in the tempTableSpaces[] array is
evidently insufficiently documented, so try to improve that.
It also looks like SharedFileSetInit() is doing the wrong thing in the
case where temp_tablespaces is empty. It was hard-wiring use of the
pg_default tablespace, but it seems like using MyDatabaseTableSpace
is more consistent with what happens for other temp files.
Back-patch the reversion of PrepareTempTablespaces()'s behavior to
9.5, as
ecd9e9f0b was. The changes in SharedFileSetInit() go back
to v11 where that was introduced. (Note there is net zero code change
before v11 from these two patch sets, so nothing to release-note.)
Magnus Hagander and Tom Lane
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
Magnus Hagander [Fri, 3 Jul 2020 13:09:06 +0000 (15:09 +0200)]
Fix temporary tablespaces for shared filesets
A likely copy/paste error in
98e8b480532 from back in 2004 would
cause temp tablespace to be reset to InvalidOid if temp_tablespaces
was set to the same value as the primary tablespace in the database.
This would cause shared filesets (such as for parallel hash joins)
to ignore them, putting the temporary files in the default tablespace
instead of the configured one. The bug is in the old code, but it
appears to have been exposed only once we had shared filesets.
Reviewed-By: Daniel Gustafsson
Discussion: https://postgr.es/m/CABUevExg5YEsOvqMxrjoNvb3ApVyH+9jggWGKwTDFyFCVWczGQ@mail.gmail.com
Backpatch-through: 9.5
Michael Meskes [Tue, 30 Jun 2020 15:31:08 +0000 (17:31 +0200)]
Fix ecpg crash with bytea and cursor variables.
Author: Jehan-Guillaume de Rorthais
Bruce Momjian [Tue, 30 Jun 2020 16:26:51 +0000 (12:26 -0400)]
doc: clarify that storage parameter values are optional
In a few cases, the documented syntax specified storage parameter values
as required.
Reported-by: [email protected]
Discussion: https://postgr.es/m/
159283163235.684.
4482737698910467437@wrigleys.postgresql.org
Backpatch-through: 9.5
Bruce Momjian [Tue, 30 Jun 2020 15:55:53 +0000 (11:55 -0400)]
doc: change pg_upgrade wal_level to be not minimal
Previously it was specified to be only replica.
Discussion: https://postgr.es/m/
20200618180058[email protected]
Backpatch-through: 9.5
Noah Misch [Sun, 28 Jun 2020 05:05:04 +0000 (22:05 -0700)]
Fix documentation of "must be vacuumed within" warning.
Warnings start 10M transactions before xidStopLimit, which is 11M
transactions before wraparound. The sample WARNING output showed a
value greater than 11M, and its HINT message predated commit
25ec228ef760eb91c094cc3b6dea7257cc22ffb5. Hence, the sample was
impossible. Back-patch to 9.5 (all supported versions).
Bruce Momjian [Thu, 25 Jun 2020 22:33:28 +0000 (18:33 -0400)]
doc: mention trigger helper functions in CREATE TRIGGER docs
Reported-by: [email protected]
Discussion: https://postgr.es/m/
159195294959.673.
5752624528747900508@wrigleys.postgresql.org
Backpatch-through: 9.5
Bruce Momjian [Thu, 25 Jun 2020 22:22:44 +0000 (18:22 -0400)]
docs: clarify that CREATE DATABASE does not copy db permissions
That is, those database permissions set by GRANT.
Diagnosed-by: Joseph Nahmias
Discussion: https://postgr.es/m/
20200614072613[email protected]
Backpatch-through: 9.5
Tom Lane [Wed, 24 Jun 2020 19:47:30 +0000 (15:47 -0400)]
Fix compiler warning induced by commit
d8b15eeb8.
I forgot that INT64_FORMAT can't be used with sscanf on Windows.
Use the same trick of sscanf'ing into a temp variable as we do in
some other places in zic.c.
The upstream IANA code avoids the portability problem by relying on
's SCNdFAST64 macro. Once we're requiring C99 in all
branches, we should do likewise and drop this set of diffs from
upstream. For now, though, a hack seems fine, since we do not
actually care about leapseconds anyway.
Discussion: https://postgr.es/m/4e5d1a5b-143e-e70e-a99d-a3b01c1ae7c3@2ndquadrant.com
Alvaro Herrera [Wed, 24 Jun 2020 18:00:37 +0000 (14:00 -0400)]
Add parens to ConvertToXSegs macro
The current definition is dangerous. No bugs exist in our code at
present, but backpatch to 11 nonetheless where it was introduced.
Author: Álvaro Herrera
Tom Lane [Mon, 22 Jun 2020 15:46:41 +0000 (11:46 -0400)]
Undo double-quoting of index names in non-text EXPLAIN output formats.
explain_get_index_name() applied quote_identifier() to the index name.
This is fine for text output, but the non-text output formats all have
their own quoting conventions and would much rather start from the
actual index name. For example in JSON you'd get something like
"Index Name": "\"My Index\"",
which is surely not desirable, especially when the same does not
happen for table names. Hence, move the responsibility for applying
quoting out to the callers, where it can go into already-existing
special code paths for text format.
This changes the API spec for users of explain_get_index_name_hook:
before, they were supposed to apply quote_identifier() if necessary,
now they should not. Research suggests that the only publicly
available user of the hook is hypopg, and it actually forgot to
apply quoting anyway, so it's fine. (In any case, there's no
behavioral change for the output of a hook as seen in non-text
EXPLAIN formats, so this won't break any case that programs should
be relying on.)
Digging in the commit logs, it appears that quoting was included in
explain_get_index_name's duties when commit
604ffd280 invented it;
and that was fine at the time because we only had text output format.
This should have been rethought when non-text formats were invented,
but it wasn't.
This is a fairly clear bug for users of non-text EXPLAIN formats,
so back-patch to all supported branches.
Per bug #16502 from Maciek Sakrejda. Patch by me (based on
investigation by Euler Taveira); thanks to Julien Rouhaud for review.
Discussion: https://postgr.es/m/16502-
57bd1c9f913ed1d1@postgresql.org
Alexander Korotkov [Sat, 20 Jun 2020 14:34:51 +0000 (17:34 +0300)]
Fix masking of SP-GiST pages during xlog consistency check
spg_mask() didn't take into account that pd_lower equal to SizeOfPageHeaderData
is still valid value. This commit fixes that. Backpatch to 11, where
spg_mask() pg_lower check was introduced.
Reported-by: Michael Paquier
Discussion: https://postgr.es/m/
20200615131405.GM52676%40paquier.xyz
Backpatch-through: 11
Alvaro Herrera [Fri, 19 Jun 2020 20:46:07 +0000 (16:46 -0400)]
Ensure write failure reports no-disk-space
A few places calling fwrite and gzwrite were not setting errno to ENOSPC
when reporting errors, as is customary; this led to some failures being
reported as
"could not write file: Success"
which makes us look silly. Make a few of these places in pg_dump and
pg_basebackup use our customary pattern.
Backpatch-to: 9.5
Author: Justin Pryzby
Author: Tom Lane
Author: Álvaro Herrera
Discussion: https://postgr.es/m/20200611153753[email protected]
Tom Lane [Fri, 19 Jun 2020 17:55:21 +0000 (13:55 -0400)]
Future-proof regression tests against possibly-missing posixrules file.
The IANA time zone folk have deprecated use of a "posixrules" file in
the tz database. While for now it's our choice whether to keep
supplying one in our own builds, installations built with
--with-system-tzdata will soon be needing to cope with that file not
being present, at least on some platforms.
This causes a problem for the horology test, which expected the
nonstandard POSIX zone spec "CST7CDT" to apply pre-2007 US daylight
savings rules. That does happen if the posixrules file supplies such
information, but otherwise the test produces undesired results.
To fix, add an explicit transition date rule that matches 2005 practice.
(We could alternatively have switched the test to use some real time
zone, but it seems useful to have coverage of this type of zone spec.)
While at it, update a documentation example that also relied on
"CST7CDT"; use a real-world zone name instead. Also, document why
the zone names EST5EDT, CST6CDT, MST7MDT, PST8PDT aren't subject to
similar failures when "posixrules" is missing.
Back-patch to all supported branches, since the hazard is the same
for all.
Discussion: https://postgr.es/m/
1665379.
1592581287@sss.pgh.pa.us
Andres Freund [Mon, 8 Jun 2020 23:50:37 +0000 (16:50 -0700)]
Fix deadlock danger when atomic ops are done under spinlock.
This was a danger only for --disable-spinlocks in combination with
atomic operations unsupported by the current platform.
While atomics.c was careful to signal that a separate semaphore ought
to be used when spinlock emulation is active, spin.c didn't actually
implement that mechanism. That's my (Andres') fault, it seems to have
gotten lost during the development of the atomic operations support.
Fix that issue and add test for nesting atomic operations inside a
spinlock.
Author: Andres Freund
Discussion: https://postgr.es/m/
20200605023302[email protected]
Backpatch: 9.5-
Andres Freund [Mon, 8 Jun 2020 23:36:51 +0000 (16:36 -0700)]
Add basic spinlock tests to regression tests.
As s_lock_test, the already existing test for spinlocks, isn't run in
an automated fashion (and doesn't test a normal backend environment),
adding tests that are run as part of a normal regression run is a good
idea. Particularly in light of several recent and upcoming spinlock
related fixes.
Currently the new tests are run as part of the pre-existing
test_atomic_ops() test. That perhaps can be quibbled about, but for
now seems ok.
The only operations that s_lock_test tests but the new tests don't are
the detection of a stuck spinlock and S_LOCK_FREE (which is otherwise
unused, not implemented on all platforms, and will be removed).
This currently contains a test for more than INT_MAX spinlocks (only
run with --disable-spinlocks), to ensure the recent commit fixing a
bug with more than INT_MAX spinlock initializations is correct. That
test is somewhat slow, so we might want to disable it after a few
days.
It might be worth retiring s_lock_test after this. The added coverage
of a stuck spinlock probably isn't worth the added complexity?
Author: Andres Freund
Discussion: https://postgr.es/m/
20200606023103[email protected]
Tom Lane [Thu, 18 Jun 2020 20:27:18 +0000 (16:27 -0400)]
Doc: document POSIX-style time zone specifications in full.
We'd glossed over most of this complexity for years, but it's hard
to avoid writing it all down now, so that we can explain what happens
when there's no "posixrules" file in the IANA time zone database.
That was at best a tiny minority situation till now, but it's likely
to become quite common in the future, so we'd better explain it.
Nonetheless, we don't really encourage people to use POSIX zone specs;
picking a named zone is almost always what you really want, unless
perhaps you're stuck with an out-of-date zone database. Therefore,
let's shove all this detail into an appendix.
Patch by me; thanks to Robert Haas for help with some awkward wording.
Discussion: https://postgr.es/m/1390.
1562258309@sss.pgh.pa.us
Michael Paquier [Thu, 18 Jun 2020 07:35:32 +0000 (16:35 +0900)]
Fix oldest xmin and LSN computation across repslots after advancing
Advancing a replication slot did not recompute the oldest xmin and LSN
values across replication slots, preventing resource removal like
segments not recycled at checkpoint time. The original commit that
introduced the slot advancing in
9c7d06d never did the update of those
oldest values, and
b0afdca removed this code.
This commit adds a TAP test to check segment recycling with advancing
for physical slots, enforcing an extra segment switch before advancing
to check if the segment gets correctly recycled after a checkpoint.
Reported-by: Andres Freund
Reviewed-by: Alexey Kondratov, Kyptaro Horiguchi
Discussion: https://postgr.es/m/
20200609171904[email protected]
Backpatch-through: 11
Tom Lane [Wed, 17 Jun 2020 22:29:29 +0000 (18:29 -0400)]
Sync our copy of the timezone library with IANA release tzcode2020a.
This absorbs a leap-second-related bug fix in localtime.c, and
teaches zic to handle an expiration marker in the leapseconds file.
Neither are of any interest to us (for the foreseeable future
anyway), but we need to stay more or less in sync with upstream.
Also adjust some over-eager changes in the README from commit
957338418.
I have no intention of making changes that require C99 in this code,
until such time as all the live back branches require C99. Otherwise
back-patching will get too exciting.
For the same reason, absorb assorted whitespace and other cosmetic
changes from HEAD into the back branches; mostly this reflects use of
improved versions of pgindent.
All in all then, quite a boring update. But I figured I'd get it
done while I was looking at this code.
Andres Freund [Mon, 8 Jun 2020 22:25:49 +0000 (15:25 -0700)]
spinlock emulation: Fix bug when more than INT_MAX spinlocks are initialized.
Once the counter goes negative we ended up with spinlocks that errored
out on first use (due to check in tas_sema).
Author: Andres Freund
Reviewed-By: Robert Haas
Discussion: https://postgr.es/m/
20200606023103[email protected]
Backpatch: 9.5-
Tom Lane [Tue, 16 Jun 2020 20:41:11 +0000 (16:41 -0400)]
Doc: fix copy-and-pasteo in ecpg docs.
The synopsis for PGTYPESinterval_free() used the wrong name.
Discussion: https://postgr.es/m/
159231203030.679.
3061023914894071953@wrigleys.postgresql.org
Thomas Munro [Tue, 16 Jun 2020 01:50:56 +0000 (13:50 +1200)]
Fix buffile.c error handling.
Convert buffile.c error handling to use ereport. This fixes cases where
I/O errors were indistinguishable from EOF or not reported. Also remove
"%m" from error messages where errno would be bogus. While we're
modifying those strings, add block numbers and short read byte counts
where appropriate.
Back-patch to all supported releases.
Reported-by: Amit Khandekar
Reviewed-by: Melanie Plageman
Reviewed-by: Alvaro Herrera
Reviewed-by: Robert Haas
Reviewed-by: Ibrar Ahmed
Reviewed-by: Michael Paquier
Discussion: https://postgr.es/m/CA%2BhUKGJE04G%3D8TLK0DLypT_27D9dR8F1RQgNp0jK6qR0tZGWOw%40mail.gmail.com
Bruce Momjian [Tue, 16 Jun 2020 00:59:40 +0000 (20:59 -0400)]
pg_upgrade: set vacuum_defer_cleanup_age to zero
Non-zero vacuum_defer_cleanup_age values cause pg_upgrade freezing of
the system catalogs to be incomplete, or do nothing. This will cause
the upgrade to fail in confusing ways.
Reported-by: Laurenz Albe
Discussion: https://postgr.es/m/
7d6f6c22ba05ce0c526e9e8b7bfa8105e7da45e6[email protected]
Backpatch-through: 9.5
Thomas Munro [Sun, 14 Jun 2020 23:33:13 +0000 (11:33 +1200)]
Doc: Add references for SI and SSI.
Our documentation failed to point out that REPEATABLE READ is really
snapshot isolation, which might be important to some users. Point to
the standard reference paper for this complicated topic.
Likewise, add a reference to the VLDB paper about PostgreSQL SSI, for
technical information about our SSI implementation and how it compares
to S2PL.
While here, add a note about catalog access using a lower isolation
level, per recent user complaint.
Back-patch to all releases.
Reported-by: Kyle Kingsbury
Reviewed-by: Andres Freund
Reviewed-by: Peter Geoghegan
Reviewed-by: Tatsuo Ishii
Discussion: https://postgr.es/m/
db7b729d-0226-d162-a126-
8a8ab2dc4443%40jepsen.io
Discussion: https://postgr.es/m/16454-
9408996bb1750faf%40postgresql.org
Tom Lane [Sat, 13 Jun 2020 17:43:24 +0000 (13:43 -0400)]
Fix behavior of float aggregates for single Inf or NaN inputs.
When there is just one non-null input value, and it is infinity or NaN,
aggregates such as stddev_pop and covar_pop should produce a NaN
result, because the calculation is not well-defined. They used to do
so, but since we adopted Youngs-Cramer aggregation in commit
e954a727f,
they produced zero instead. That's an oversight, so fix it. Add tests
exercising these edge cases.
Affected aggregates are
var_pop(double precision)
stddev_pop(double precision)
var_pop(real)
stddev_pop(real)
regr_sxx(double precision,double precision)
regr_syy(double precision,double precision)
regr_sxy(double precision,double precision)
regr_r2(double precision,double precision)
regr_slope(double precision,double precision)
regr_intercept(double precision,double precision)
covar_pop(double precision,double precision)
corr(double precision,double precision)
Back-patch to v12 where the behavior change was accidentally introduced.
Report and patch by me; thanks to Dean Rasheed for review.
Discussion: https://postgr.es/m/353062.
1591898766@sss.pgh.pa.us
Bruce Momjian [Thu, 11 Jun 2020 22:19:25 +0000 (18:19 -0400)]
doc: remove xreflabels from commits
75fcdd2ae2 and
85af628da5
xreflabels prevent references to the chapter numbers of sections id's.
It should only be used in specific cases.
Discussion: https://postgr.es/m/
8315c0ca-7758-8823-fcb6-
f37f9413e6b6@2ndquadrant.com
Backpatch-through: 9.5
Tom Lane [Thu, 11 Jun 2020 21:38:42 +0000 (17:38 -0400)]
Fix mishandling of NaN counts in numeric_[avg_]combine.
When merging two NumericAggStates, the code missed adding the new
state's NaNcount unless its N was also nonzero; since those counts
are independent, this is wrong.
This would only have visible effect if some partial aggregate scans
found only NaNs while earlier ones found only non-NaNs; then we could
end up falsely deciding that there were no NaNs and fail to return a
NaN final result as expected. That's pretty improbable, so it's no
surprise this hasn't been reported from the field. Still, it's a bug.
I didn't try to produce a regression test that would show the bug,
but I did notice that these functions weren't being reached at all
in our regression tests, so I improved the tests to at least
exercise them. With these additions, I see pretty complete code
coverage on the aggregation-related functions in numeric.c.
Back-patch to 9.6 where this code was introduced. (I only added
the improved test case as far back as v10, though, since the
relevant part of aggregates.sql isn't there at all in 9.6.)
Peter Geoghegan [Thu, 11 Jun 2020 17:09:43 +0000 (10:09 -0700)]
Avoid update conflict out serialization anomalies.
SSI's HeapCheckForSerializableConflictOut() test failed to correctly
handle conditions involving a concurrently inserted tuple which is later
concurrently updated by a separate transaction . A SELECT statement
that called HeapCheckForSerializableConflictOut() could end up using the
same XID (updater's XID) for both the original tuple, and the successor
tuple, missing the XID of the xact that created the original tuple
entirely. This only happened when neither tuple from the chain was
visible to the transaction's MVCC snapshot.
The observable symptoms of this bug were subtle. A pair of transactions
could commit, with the later transaction failing to observe the effects
of the earlier transaction (because of the confusion created by the
update to the non-visible row). This bug dates all the way back to
commit
dafaa3ef, which added SSI.
To fix, make sure that we check the xmin of concurrently inserted tuples
that happen to also have been updated concurrently.
Author: Peter Geoghegan
Reported-By: Kyle Kingsbury
Reviewed-By: Thomas Munro
Discussion: https://postgr.es/m/
db7b729d-0226-d162-a126-
8a8ab2dc4443@jepsen.io
Backpatch: All supported versions
Amit Kapila [Thu, 11 Jun 2020 08:40:43 +0000 (14:10 +0530)]
Fix typos.
Reported-by: John Naylor
Author: John Naylor
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CACPNZCtRuvs6G+EYqejhVJgBq2AKeZdXRVJsbX4syhO9gn5SNQ@mail.gmail.com
Peter Eisentraut [Wed, 10 Jun 2020 09:57:41 +0000 (11:57 +0200)]
Update description of parameter password_encryption
The previous description string still described the pre-PostgreSQL
10 (pre
eb61136dc75a76caef8460fa939244d8593100f2) behavior of
selecting between encrypted and unencrypted, but it is now choosing
between encryption algorithms.
Andres Freund [Tue, 9 Jun 2020 02:52:19 +0000 (19:52 -0700)]
Avoid need for valgrind suppressions for pg_atomic_init_u64 on some platforms.
Previously we used pg_atomic_write_64_impl inside
pg_atomic_init_u64. That works correctly, but on platforms without
64bit single copy atomicity it could trigger spurious valgrind errors
about uninitialized memory, because we use compare_and_swap for atomic
writes on such platforms.
I previously suppressed one instance of this problem (
6c878edc1df),
but as Tom reports that wasn't enough. As the atomic variable cannot
yet be concurrently accessible during initialization, it seems better
to have pg_atomic_init_64_impl set the value directly.
Change pg_atomic_init_u32_impl for symmetry.
Reported-By: Tom Lane
Author: Andres Freund
Discussion: https://postgr.es/m/
1714601.
1591503815@sss.pgh.pa.us
Backpatch: 9.5-
Thomas Munro [Mon, 8 Jun 2020 01:57:24 +0000 (13:57 +1200)]
Fix locking bugs that could corrupt pg_control.
The redo routines for XLOG_CHECKPOINT_{ONLINE,SHUTDOWN} must acquire
ControlFileLock before modifying ControlFile->checkPointCopy, or the
checkpointer could write out a control file with a bad checksum.
Likewise, XLogReportParameters() must acquire ControlFileLock before
modifying ControlFile and calling UpdateControlFile().
Back-patch to all supported releases.
Author: Nathan Bossart
Author: Fujii Masao
Reviewed-by: Fujii Masao
Reviewed-by: Michael Paquier
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/70BF24D6-DC51-443F-B55A-95735803842A%40amazon.com
Thomas Munro [Mon, 8 Jun 2020 01:20:46 +0000 (13:20 +1200)]
Doc: Update example symptom of systemd misconfiguration.
In PostgreSQL 10, we stopped using System V semaphores on Linux
systems. Update the example we give of an error message from a
misconfigured system to show what people are most likely to see these
days.
Back-patch to 10, where PREFERRED_SEMAPHORES=UNNAMED_POSIX arrived.
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/CA%2BhUKGLmJUSwybaPQv39rB8ABpqJq84im2UjZvyUY4feYhpWMw%40mail.gmail.com
Noah Misch [Sun, 7 Jun 2020 23:27:13 +0000 (16:27 -0700)]
MSVC: Avoid warning when testing a TAP suite without PROVE_FLAGS.
Commit
7be5d8df1f74b78620167d3abf32ee607e728919 surfaced the logic
error, which had no functional implications, by adding "use warnings".
The buildfarm always customizes PROVE_FLAGS, so the warning did not
appear there. Back-patch to 9.5 (all supported versions).
Tom Lane [Sun, 7 Jun 2020 17:44:13 +0000 (13:44 -0400)]
Try to read data from the socket in pqSendSome's write_failed paths.
Even when we've concluded that we have a hard write failure on the
socket, we should continue to try to read data. This gives us an
opportunity to collect any final error message that the backend might
have sent before closing the connection; moreover it is the job of
pqReadData not pqSendSome to close the socket once EOF is detected.
Due to an oversight in
1f39a1c06, pqSendSome failed to try to collect
data in the case where we'd already set write_failed. The problem was
masked for ordinary query operations (which really only make one write
attempt anyway), but COPY to the server would continue to send data
indefinitely after a mid-COPY connection loss.
Hence, add pqReadData calls into the paths where pqSendSome drops data
because of write_failed. If we've lost the connection, this will
eventually result in closing the socket and setting CONNECTION_BAD,
which will cause PQputline and siblings to report failure, allowing
the application to terminate the COPY sooner. (Basically this restores
what happened before
1f39a1c06.)
There are related issues that this does not solve; for example, if the
backend sends an error but doesn't drop the connection, we did and
still will keep pumping COPY data as long as the application sends it.
Fixing that will require application-visible behavior changes though,
and anyway it's an ancient behavior that we've had few complaints about.
For now I'm just trying to fix the regression from
1f39a1c06.
Per a complaint from Andres Freund. Back-patch into v12 where
1f39a1c06 came in.
Discussion: https://postgr.es/m/
20200603201242[email protected]
Peter Eisentraut [Sun, 7 Jun 2020 11:18:36 +0000 (13:18 +0200)]
doc: Clean up title case use
Peter Eisentraut [Sun, 7 Jun 2020 09:16:51 +0000 (11:16 +0200)]
doc: Fix incorrect link target
Noah Misch [Sat, 6 Jun 2020 03:10:53 +0000 (20:10 -0700)]
Refresh function name in CRC-associated Valgrind suppressions.
Back-patch to 9.5, where commit
4f700bcd20c087f60346cb8aefd0e269be8e2157
first appeared.
Reviewed by Tom Lane. Reported by Andrew Dunstan.
Discussion: https://postgr.es/m/
4dfabec2-a3ad-0546-2d62-
f816c97edd0c@2ndQuadrant.com
Joe Conway [Fri, 5 Jun 2020 20:49:28 +0000 (16:49 -0400)]
Add unlikely() to CHECK_FOR_INTERRUPTS()
Add the unlikely() branch hint macro to CHECK_FOR_INTERRUPTS().
Backpatch to REL_10_STABLE where we first started using unlikely().
Discussion: https://www.postgresql.org/message-id/flat/
8692553c-7fe8-17d9-cbc1-
7cddb758f4c6%40joeconway.com
Tom Lane [Fri, 5 Jun 2020 20:18:50 +0000 (16:18 -0400)]
Use query collation, not column's collation, while examining statistics.
Commit
5e0928005 changed the planner so that, instead of blindly using
DEFAULT_COLLATION_OID when invoking operators for selectivity estimation,
it would use the collation of the column whose statistics we're
considering. This was recognized as still being not quite the right
thing, but it seemed like a good incremental improvement. However,
shortly thereafter we introduced nondeterministic collations, and that
creates cases where operators can fail if they're passed the wrong
collation. We don't want planning to fail in cases where the query itself
would work, so this means that we *must* use the query's collation when
invoking operators for estimation purposes.
The only real problem this creates is in ineq_histogram_selectivity, where
the binary search might produce a garbage answer if we perform comparisons
using a different collation than the column's histogram is ordered with.
However, when the query's collation is significantly different from the
column's default collation, the estimate we previously generated would be
pretty irrelevant anyway; so it's not clear that this will result in
noticeably worse estimates in practice. (A follow-on patch will improve
this situation in HEAD, but it seems too invasive for back-patch.)
The patch requires changing the signatures of mcv_selectivity and allied
functions, which are exported and very possibly are used by extensions.
In HEAD, I just did that, but an API/ABI break of this sort isn't
acceptable in stable branches. Therefore, in v12 the patch introduces
"mcv_selectivity_ext" and so on, with signatures matching HEAD, and makes
the old functions into wrappers that assume DEFAULT_COLLATION_OID should
be used. That does not match the prior behavior, but it should avoid risk
of failure in most cases. (In practice, I think most extension datatypes
aren't collation-aware, so the change probably doesn't matter to them.)
Per report from James Lucas. Back-patch to v12 where the problem was
introduced.
Discussion: https://postgr.es/m/CAAFmbbOvfi=wMM=3qRsPunBSLb8BFREno2oOzSBS=mzfLPKABw@mail.gmail.com
Michael Paquier [Fri, 5 Jun 2020 01:29:27 +0000 (10:29 +0900)]
Preserve pg_index.indisreplident across REINDEX CONCURRENTLY
If the flag value is lost, logical decoding would work the same way as
REPLICA IDENTITY NOTHING, meaning that no old tuple values would be
included in the changes anymore produced by logical decoding.
Author: Michael Paquier
Reviewed-by: Euler Taveira
Discussion: https://postgr.es/m/
20200603065340[email protected]
Backpatch-through: 12
Tom Lane [Thu, 4 Jun 2020 20:42:08 +0000 (16:42 -0400)]
Reject "23:59:60.nnn" in datetime input.
It's intentional that we don't allow values greater than 24 hours,
while we do allow "24:00:00" as well as "23:59:60" as inputs.
However, the range check was miscoded in such a way that it would
accept "23:59:60.nnn" with a nonzero fraction. For time or timetz,
the stored result would then be greater than "24:00:00" which would
fail dump/reload, not to mention possibly confusing other operations.
Fix by explicitly calculating the result and making sure it does not
exceed 24 hours. (This calculation is redundant with what will happen
later in tm2time or tm2timetz. Maybe someday somebody will find that
annoying enough to justify refactoring to avoid the duplication; but
that seems too invasive for a back-patched bug fix, and the cost is
probably unmeasurable anyway.)
Note that this change also rejects such input as the time portion
of a timestamp(tz) value.
Back-patch to v10. The bug is far older, but to change this pre-v10
we'd need to ensure that the logic behaves sanely with float timestamps,
which is possibly nontrivial due to roundoff considerations.
Doesn't really seem worth troubling with.
Per report from Christoph Berg.
Discussion: https://postgr.es/m/
20200520125807[email protected]
Michael Paquier [Thu, 4 Jun 2020 01:18:02 +0000 (10:18 +0900)]
Fix instance of elog() called while holding a spinlock
This broke the project rule to not call any complex code while a
spinlock is held. Issue introduced by
b89e151.
Discussion: https://postgr.es/m/
20200602.161518.
1399689010416646074[email protected]
Backpatch-through: 9.5
Tom Lane [Wed, 3 Jun 2020 16:36:00 +0000 (12:36 -0400)]
Don't call palloc() while holding a spinlock, either.
Fix some more violations of the "only straight-line code inside a
spinlock" rule. These are hazardous not only because they risk
holding the lock for an excessively long time, but because it's
possible for palloc to throw elog(ERROR), leaving a stuck spinlock
behind.
copy_replication_slot() had two separate places that did pallocs
while holding a spinlock. We can make the code simpler and safer
by copying the whole ReplicationSlot struct into a local variable
while holding the spinlock, and then referencing that copy.
(While that's arguably more cycles than we really need to spend
holding the lock, the struct isn't all that big, and this way seems
far more maintainable than copying fields piecemeal. Anyway this
is surely much cheaper than a palloc.) That bug goes back to v12.
InvalidateObsoleteReplicationSlots() not only did a palloc while
holding a spinlock, but for extra sloppiness then leaked the memory
--- probably for the lifetime of the checkpointer process, though
I didn't try to verify that. Fortunately that silliness is new
in HEAD.
pg_get_replication_slots() had a cosmetic violation of the rule,
in that it only assumed it's safe to call namecpy() while holding
a spinlock. Still, that's a hazard waiting to bite somebody, and
there were some other cosmetic coding-rule violations in the same
function, so clean it up. I back-patched this as far as v10; the
code exists before that but it looks different, and this didn't
seem important enough to adapt the patch further back.
Discussion: https://postgr.es/m/
20200602.161518.
1399689010416646074[email protected]
Fujii Masao [Wed, 3 Jun 2020 00:59:43 +0000 (09:59 +0900)]
doc: Move wal_init_zero and wal_recycle descriptions to proper section.
The group of wal_init_zero and wal_recycle is WAL_SETTINGS in guc.c,
but previously their documents were located in
"Replication"/"Sending Servers" section. This commit moves them to
the proper section "Write Ahead Log"/"Settings".
Back-patch to v12 where wal_init_zero and wal_recycle parameters
were introduced.
Author: Fujii Masao
Discussion: https://postgr.es/m/
b5190ab4-a169-6a42-0e49-
aed0807c8976@oss.nttdata.com
Michael Paquier [Mon, 1 Jun 2020 05:41:25 +0000 (14:41 +0900)]
Fix use-after-release mistake in currtid() and currtid2() for views
This issue has been present since the introduction of this code as of
a3519a2 from 2002, and has been found by buildfarm member prion that
uses RELCACHE_FORCE_RELEASE via the tests introduced recently in
e786be5.
Discussion: https://postgr.es/m/
20200601022055[email protected]
Backpatch-through: 9.5
Michael Paquier [Mon, 1 Jun 2020 01:32:53 +0000 (10:32 +0900)]
Fix crashes with currtid() and currtid2()
A relation that has no storage initializes rd_tableam to NULL, which
caused those two functions to crash because of a pointer dereference.
Note that in 11 and older versions, this has always failed with a
confusing error "could not open file".
These two functions are used by the Postgres ODBC driver, which requires
them only when connecting to a backend strictly older than 8.1. When
connected to 8.2 or a newer version, the driver uses a RETURNING clause
instead whose support has been added in 8.2, so it should be possible to
just remove both functions in the future. This is left as an issue to
address later.
While on it, add more regression tests for those functions as we never
really had coverage for them, and for aggregates of TIDs.
Reported-by: Jaime Casanova, via sqlsmith
Author: Michael Paquier
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/CAJGNTeO93u-5APMga6WH41eTZ3Uee9f3s8dCpA-GSSqNs1b=Ug@mail.gmail.com
Backpatch-through: 12
Andrew Dunstan [Sun, 31 May 2020 22:33:00 +0000 (18:33 -0400)]
Make install-tests target work with vpath builds
Also add a top-level install-tests target.
Backpatch to all live branches.
Craig Ringer, tweaked by me.
Michael Paquier [Sun, 31 May 2020 01:48:28 +0000 (10:48 +0900)]
Doc: Mention about caveats of --concurrently on reindexdb page
The documentation of REINDEX includes a complete description of
CONCURRENTLY and its advantages as well as its disadvantages, but
reindexdb was not really clear about all that.
From discussion with Tom Lane, based on a report from Andrey Klychkov.
Discussion: https://postgr.es/m/
1590486572.
205117372@f500.i.mail.ru
Backpatch-through: 12
Andres Freund [Thu, 28 May 2020 22:08:12 +0000 (15:08 -0700)]
llvmjit: Fix building against LLVM 11 by removing unnecessary include.
LLVM has removed this header, in the branch that will become llvm
11. But as it turns out we didn't actually need it, so just remove it.
Author: Jesse Zhang
Discussion: https://postgr.es/m/CAGf+fX7bvtP0YXMu7pOsu_NwhxW6dArTkxb=jt7M2-UJkyJ_3g@mail.gmail.com
Backpatch: 11, where JIT support using llvm was introduced.
Joe Conway [Thu, 28 May 2020 17:44:59 +0000 (13:44 -0400)]
Initialize dblink remoteConn struct in all cases
Two of the members of rconn were left uninitialized. When
dblink_open() is called without an outer transaction it
handles the initialization for us, but with an outer
transaction it does not. Arrange for initialization
in all cases. Backpatch to all supported versions.
Reported-by: Alexander Lakhin
Discussion: https://www.postgresql.org/message-id/flat/
9bd0744f-5f04-c778-c5b3-
809efe9c30c7%40joeconway.com#
c545909a41664991aca60c4d70a10ce7
Joe Conway [Thu, 28 May 2020 17:16:56 +0000 (13:16 -0400)]
Add CHECK_FOR_INTERRUPTS() to the repeat() function
The repeat() function loops for potentially a long time without
ever checking for interrupts. This prevents, for example, a query
cancel from interrupting until the work is all done. Fix by
inserting a CHECK_FOR_INTERRUPTS() into the loop.
Backpatch to all supported versions.
Discussion: https://www.postgresql.org/message-id/flat/
8692553c-7fe8-17d9-cbc1-
7cddb758f4c6%40joeconway.com
Heikki Linnakangas [Thu, 28 May 2020 09:37:00 +0000 (12:37 +0300)]
Add missing error code to "cannot attach index ..." error.
ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE was used in an ereport with the
same message but different errdetail a few lines earlier, so use that
here as well.
Backpatch-through: 11
Heikki Linnakangas [Thu, 28 May 2020 09:35:18 +0000 (12:35 +0300)]
Fix typo in test comment.
The same comment was copied to a few different places, with the same typo.
Backpatch down to v11, where this typo was introduced.
Peter Eisentraut [Tue, 26 May 2020 12:09:36 +0000 (14:09 +0200)]
Add lcov exclusion markers to jsonpath scanner
This was done for all scanners in
421167362242ce1fb46d6d720798787e7cd65aad but not added to the new one.
Bruce Momjian [Tue, 26 May 2020 00:19:28 +0000 (20:19 -0400)]
gss: add missing references to hostgssenc and hostnogssenc
These were missed when these were added to pg_hba.conf in PG 12;
updates docs and pg_hba.conf.sample.
Reported-by: Arthur Nascimento
Bug: 16380
Discussion: https://postgr.es/m/
20200421182736[email protected]
Backpatch-through: 12
Noah Misch [Mon, 25 May 2020 23:21:04 +0000 (16:21 -0700)]
Add a temp-install prerequisite to top-level "check-tests".
The target failed, tested $PATH binaries, or tested a stale temporary
installation. Commit
c66b438db62748000700c9b90b585e756dd54141 missed
this. Back-patch to 9.5 (all supported versions).
Alvaro Herrera [Fri, 22 May 2020 21:39:16 +0000 (17:39 -0400)]
Fix two typos in a comment
They were introduced in
898e5e3290a7; backpatch to 12.
Bruce Momjian [Fri, 22 May 2020 00:28:38 +0000 (20:28 -0400)]
doc: suggest 1.1 as a random_page_cost value for SSDs
Reported-by: yigong hu
Discussion: https://postgr.es/m/CAOxFffcourucFqSk+tZA13ErS3XRYkDy6EeaPff4AvHGiEEuug@mail.gmail.com
Backpatch-through: 9.5
Bruce Momjian [Thu, 21 May 2020 23:49:30 +0000 (19:49 -0400)]
doc: Simplify mention of unique indexes for NULL control
Discussion: https://postgr.es/m/2304.
1586532634@sss.pgh.pa.us
Backpatch-through: 9.5
Michael Paquier [Thu, 21 May 2020 05:41:30 +0000 (14:41 +0900)]
Fix MSVC installations with multiple "configure" files detected
When installing binaries and libraries using the MSVC installation
routines, the operation gets done after moving to the root folder, whose
location is detected by checking if "configure" exists two times in a
row. So, calling the installation script from src/tools/msvc/ with an
extra "configure" file four levels up the root path of the code tree
causes the execution to go further up, leading to a failure in finding
the builds. This commit fixes the issue by moving to the root folder of
the code tree only once, when necessary.
Author: Arnold Müller
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/16343-
f638f67e7e52b86c@postgresql.org
Backpatch-through: 9.5
Alvaro Herrera [Wed, 20 May 2020 18:35:39 +0000 (14:35 -0400)]
doc: Adding a partition does not require Access Exclusive lock
This doc update was missed in
898e5e3290a7. Backpatch to 12.
Pointed out by Pavel Luzanov
Discussion: https://postgr.es/m/
642e9fbc-b832-698b-9a8f-
d626afd7014d@postgrespro.ru
Michael Paquier [Wed, 20 May 2020 05:21:45 +0000 (14:21 +0900)]
Doc: Fix description of pg_class.relreplident
The description missed a comma and lacked an explanation of what happens
with REPLICA IDENTITY USING INDEX when the dependent index is dropped.
Author: Marina Polyakova
Reviewed-by: Daniel Gustafsson, Michael Paquier
Discussion: https://postgr.es/m/
ad1a0badc32658b1bbb07aa312346a1d@postgrespro.ru
Backpatch-through: 9.5
Amit Kapila [Mon, 18 May 2020 02:45:05 +0000 (08:15 +0530)]
Fix comment in slot.c.
Reported-by: Sawada Masahiko
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CA+fd4k4Ws7M7YQ8PqSym5WB1y75dZeBTd1sZJUQdfe0KJQ-iSA@mail.gmail.com
Michael Paquier [Sat, 16 May 2020 09:16:31 +0000 (18:16 +0900)]
Fix assertion with relation using REPLICA IDENTITY FULL in subscriber
In a logical replication subscriber, a table using REPLICA IDENTITY FULL
which has a primary key would try to use the primary key's index
available to scan for a tuple, but an assertion only assumed as correct
the case of an index associated to REPLICA IDENTITY USING INDEX. This
commit corrects the assertion so as the use of a primary key index is a
valid case.
Reported-by: Dilip Kumar
Analyzed-by: Dilip Kumar
Author: Euler Taveira
Reviewed-by: Michael Paquier, Masahiko Sawada
Discussion: https://postgr.es/m/CAFiTN-u64S5bUiPL1q5kwpHNd0hRnf1OE-bzxNiOs5zo84i51w@mail.gmail.com
Backpatch-through: 10
Tom Lane [Fri, 15 May 2020 23:05:39 +0000 (19:05 -0400)]
Fix bogus initialization of replication origin shared memory state.
The previous coding zeroed out offsetof(ReplicationStateCtl, states)
more bytes than it was entitled to, as a consequence of starting the
zeroing from the wrong pointer (or, if you prefer, using the wrong
calculation of how much to zero).
It's unsurprising that this has not caused any reported problems,
since it can be expected that the newly-allocated block is at the end
of what we've used in shared memory, and we always make the shmem
block substantially bigger than minimally necessary. Nonetheless,
this is wrong and it could bite us someday; plus it's a dangerous
model for somebody to copy.
This dates back to the introduction of this code (commit
5aa235042),
so back-patch to all supported branches.
Alvaro Herrera [Fri, 15 May 2020 20:50:34 +0000 (16:50 -0400)]
Avoid killing btree items that are already dead
_bt_killitems marks btree items dead when a scan leaves the page where
they live, but it does so with only share lock (to improve concurrency).
This was historicall okay, since killing a dead item has no
consequences. However, with the advent of data checksums and
wal_log_hints, this action incurs a WAL full-page-image record of the
page. Multiple concurrent processes would write the same page several
times, leading to WAL bloat. The probability of this happening can be
reduced by only killing items if they're not already dead, so change the
code to do that.
The problem could eliminated completely by having _bt_killitems upgrade
to exclusive lock upon seeing a killable item, but that would reduce
concurrency so it's considered a cure worse than the disease.
Backpatch all the way back to 9.5, since wal_log_hints was introduced in
9.4.
Author: Masahiko Sawada
Discussion: https://postgr.es/m/CA+fd4k6PeRj2CkzapWNrERkja5G0-6D-YQiKfbukJV+qZGFZ_Q@mail.gmail.com
Bruce Momjian [Fri, 15 May 2020 16:38:40 +0000 (12:38 -0400)]
docs: add xreflabel entries for autovacuum, SP-GiST, and TOAST
This is for use by the PG 13 release notes, but might be used for minor
release notes in the future.
Backpatch-through: 9.5
Bruce Momjian [Fri, 15 May 2020 16:05:43 +0000 (12:05 -0400)]
doc: add missing xreflabels to the main docs (not refs)
Add missing xreflabels for index types, geqo, libpq, spi, server-side
languages, ecpg, and vaacuumlo.
Backpatch-through: 9.5
Bruce Momjian [Fri, 15 May 2020 13:55:43 +0000 (09:55 -0400)]
doc: remove extra blank line at the top of SGML files
Backpatch-through: 9.5
Bruce Momjian [Fri, 15 May 2020 12:52:24 +0000 (08:52 -0400)]
doc: make ref/*.sgml file header comment layout consistent
Alexander Korotkov [Wed, 6 May 2020 12:35:27 +0000 (15:35 +0300)]
Fix amcheck for page checks concurrent to replay of btree page deletion
amcheck expects at least hikey to always exist on leaf page even if it is
deleted page. But replica reinitializes page during replay of page deletion,
causing deleted page to have no items. Thus, replay of page deletion can
cause an error in concurrent amcheck run.
This commit relaxes amcheck expectation making it tolerate deleted page with
no items.
Reported-by: Konstantin Knizhnik
Discussion: https://postgr.es/m/CAPpHfdt_OTyQpXaPJcWzV2N-LNeNJseNB-K_A66qG%3DL518VTFw%40mail.gmail.com
Author: Alexander Korotkov
Reviewed-by: Peter Geoghegan
Backpatch-through: 11
Heikki Linnakangas [Thu, 14 May 2020 05:39:26 +0000 (08:39 +0300)]
Move check for fsync=off so that pendingOps still gets cleared.
Commit
3eb77eba5a moved the loop and refactored it, and inadvertently
changed the effect of fsync=off so that it also skipped removing entries
from the pendingOps table. That was not intentional, and leads to an
assertion failure if you turn fsync on while the server is running and
reload the config.
Backpatch-through: 12-
Reviewed-By: Thomas Munro
Discussion: https://www.postgresql.org/message-id/
3cbc7f4b-a5fa-56e9-9591-
c886deb07513%40iki.fi
Amit Kapila [Thu, 14 May 2020 04:04:46 +0000 (09:34 +0530)]
Fix the MSVC build for versions 2015 and later.
Visual Studio 2015 and later versions should still be able to do the same
as Visual Studio 2012, but the declaration of locale_name is missing in
_locale_t, causing the code compilation to fail, hence this falls back
instead on to enumerating all system locales by using EnumSystemLocalesEx
to find the required locale name. If the input argument is in Unix-style
then we can get ISO Locale name directly by using GetLocaleInfoEx() with
LCType as LOCALE_SNAME.
In passing, change the documentation references of the now obsolete links.
Note that this problem occurs only with NLS enabled builds.
Author: Juan José Santamaría Flecha, Davinder Singh and Amit Kapila
Reviewed-by: Ranier Vilela and Amit Kapila
Backpatch-through: 9.5
Discussion: https://postgr.es/m/CAHzhFSFoJEWezR96um4-rg5W6m2Rj9Ud2CNZvV4NWc9tXV7aXQ@mail.gmail.com
Noah Misch [Thu, 14 May 2020 03:42:09 +0000 (20:42 -0700)]
Fix pg_recvlogical avoidance of superfluous Standby Status Update.
The defect suppressed a Standby Status Update message when bytes flushed
to disk had changed but bytes received had not changed. If
pg_recvlogical then exited with no intervening Standby Status Update,
the next pg_recvlogical repeated already-flushed records. The defect
could also cause superfluous messages, which are functionally harmless.
Back-patch to 9.5 (all supported versions).
Discussion: https://postgr.es/m/
20200502221647[email protected]
Noah Misch [Thu, 14 May 2020 03:42:09 +0000 (20:42 -0700)]
In successful pg_recvlogical, end PGRES_COPY_OUT cleanly.
pg_recvlogical merely called PQfinish(), so the backend sent messages
after the disconnect. When that caused EPIPE in internal_flush(),
before a LogicalConfirmReceivedLocation(), the next pg_recvlogical would
repeat already-acknowledged records. Whether or not the defect causes
EPIPE, post-disconnect messages could contain an ErrorResponse that the
user should see. One properly ends PGRES_COPY_OUT by repeating
PQgetCopyData() until it returns a negative value. Augment one of the
tests to cover the case of WAL past --endpos. Back-patch to v10, where
commit
7c030783a5bd07cadffc2a1018bc33119a4c7505 first appeared. Before
that commit, pg_recvlogical never reached PGRES_COPY_OUT.
Reported by Thomas Munro.
Discussion: https://postgr.es/m/CAEepm=1MzM2Z_xNe4foGwZ1a+MO_2S9oYDq3M5D11=JDU_+0Nw@mail.gmail.com
Tom Lane [Mon, 11 May 2020 21:08:58 +0000 (17:08 -0400)]
Stamp 12.3.
Peter Eisentraut [Mon, 11 May 2020 11:19:18 +0000 (13:19 +0200)]
Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash:
60bf9b5caac08d0483f6f92ebf9ef2e0eef5b6bb
Tom Lane [Sun, 10 May 2020 19:05:59 +0000 (15:05 -0400)]
Release notes for 12.3, 11.8, 10.13, 9.6.18, 9.5.22.
Fujii Masao [Tue, 7 Apr 2020 15:49:29 +0000 (00:49 +0900)]
Prevent archive recovery from scanning non-existent WAL files.
Previously when there were multiple timelines listed in the history file
of the recovery target timeline, archive recovery searched all of them,
starting from the newest timeline to the oldest one, to find the segment
to read. That is, archive recovery had to continuously fail scanning
the segment until it reached the timeline that the segment belonged to.
These scans for non-existent segment could be harmful on the recovery
performance especially when archival area was located on the remote
storage and each scan could take a long time.
To address the issue, this commit changes archive recovery so that
it skips scanning the timeline that the segment to read doesn't belong to.
Per discussion, back-patch to all supported versions.
Author: Kyotaro Horiguchi, tweaked a bit by Fujii Masao
Reviewed-by: David Steele, Pavel Suderevsky, Grigory Smolkin
Discussion: https://postgr.es/m/16159-
f5a34a3a04dc67e0@postgresql.org
Discussion: https://postgr.es/m/
20200129.120222.
1476610231001551715[email protected]
Alvaro Herrera [Fri, 8 May 2020 23:38:46 +0000 (19:38 -0400)]
pg_restore: Provide file name with one failure message
Almost all error messages already include file name where relevant, but
this one had been overlooked. Repair.
Backpatch to 9.5.
Author: Euler Taveira
Discussion: https://postgr.es/m/CAH503wA_VOrcKL_43p9atRejCDYmOZ8MzfK9S6TJrQqBqNeAXA@mail.gmail.com
Reviewed-by: Álvaro Herrera
Tom Lane [Fri, 8 May 2020 20:40:14 +0000 (16:40 -0400)]
First-draft release notes for 12.3.
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
Peter Eisentraut [Wed, 6 May 2020 14:25:54 +0000 (16:25 +0200)]
Fix several DDL issues of generated columns versus inheritance
Several combinations of generated columns and inheritance in CREATE
TABLE were not handled correctly. Specifically:
- Disallow a child column specifying a generation expression if the
parent column is a generated column. The child column definition
must be unadorned and the parent column's generation expression will
be copied.
- Prohibit a child column of a generated parent column specifying
default values or identity.
- Allow a child column of a not-generated parent column specifying
itself as a generated column. This previously did not work, but it
was possible to arrive at the state via other means (involving ALTER
TABLE), so it seems sensible to support it.
Add tests for each case. Also add documentation about the rules
involving generated columns and inheritance.
Discussion:
https://www.postgresql.org/message-id/flat/15830.
1575468847%40sss.pgh.pa.us
https://www.postgresql.org/message-id/flat/
2678bad1-048f-519a-ef24-
b12962f41807%40enterprisedb.com
https://www.postgresql.org/message-id/flat/CAJvUf_u4h0DxkCMCeEKAWCuzGUTnDP-G5iVmSwxLQSXn0_FWNQ%40mail.gmail.com
Peter Eisentraut [Thu, 9 Apr 2020 12:10:01 +0000 (14:10 +0200)]
Propagate ALTER TABLE ... SET STORAGE to indexes
When creating a new index, the attstorage setting of the table column
is copied to regular (non-expression) index columns. But a later
ALTER TABLE ... SET STORAGE is not propagated to indexes, thus
creating an inconsistent and undumpable state.
Discussion: https://www.postgresql.org/message-id/flat/
9765d72b-37c0-06f5-e349-
2a580aafd989%402ndquadrant.com
Amit Kapila [Fri, 8 May 2020 03:17:51 +0000 (08:47 +0530)]
Fix inconsistency in pg_buffercache docs.
Commit
6e654546fb avoids locking bufmgr partitions to make pg_buffercache
less disruptive on production systems but forgot to update the docs.
Reported-by: Sawada Masahiko
Author: Sawada Masahiko
Reviewed-by: Amit Kapila
Backpatch-through: 10
Discussion: https://postgr.es/m/CA+fd4k6sD8oeP1qJbFAor=rCpYckU9DsywHiYx3x5Hz5Z8Ua_w@mail.gmail.com
Fujii Masao [Fri, 8 May 2020 01:36:40 +0000 (10:36 +0900)]
Report missing wait event for timeline history file.
TimelineHistoryRead and TimelineHistoryWrite wait events are reported
during waiting for a read and write of a timeline history file, respectively.
However, previously, TimelineHistoryRead wait event was not reported
while readTimeLineHistory() was reading a timeline history file. Also
TimelineHistoryWrite was not reported while writeTimeLineHistory() was
writing one line with the details of the timeline split, at the end.
This commit fixes these issues.
Back-patch to v10 where wait events for a timeline history file was added.
Author: Masahiro Ikeda
Reviewed-by: Michael Paquier, Fujii Masao
Discussion: https://postgr.es/m/
d11b0c910b63684424e06772eb844ab5@oss.nttdata.com
Tom Lane [Thu, 7 May 2020 19:59:51 +0000 (15:59 -0400)]
Fix YA text phrase search bug.
checkcondition_str() failed to report multiple matches for a prefix
pattern correctly: it would dutifully merge the match positions, but
then after exiting that loop, if the last prefix-matching word had
had no suitable positions, it would report there were no matches.
The upshot would be failing to recognize a match that the query
should match.
It looks like you need all of these conditions to see the bug:
* a phrase search (else we don't ask for match position details)
* a prefix search item (else we don't get to this code)
* a weight restriction (else checkclass_str won't fail)
Noted while investigating a problem report from Pavel Borisov,
though this is distinct from the issue he was on about.
Back-patch to 9.6 where phrase search was added.
Alvaro Herrera [Wed, 6 May 2020 16:29:41 +0000 (12:29 -0400)]
Heed lock protocol in DROP OWNED BY
We were acquiring object locks then deleting objects one by one, instead
of acquiring all object locks first, ignoring those that did not exist,
and then deleting all objects together. The latter is the correct
protocol to use, and what this commits changes to code to do. Failing
to follow that leads to "cache lookup failed for relation XYZ" error
reports when DROP OWNED runs concurrently with other DDL -- for example,
a session termination that removes some temp tables.
Author: Álvaro Herrera
Reported-by: Mithun Chicklore Yogendra (Mithun CY)
Reviewed-by: Ahsan Hadi, Tom Lane
Discussion: https://postgr.es/m/CADq3xVZTbzK4ZLKq+dn_vB4QafXXbmMgDP3trY-GuLnib2Ai1w@mail.gmail.com
Michael Paquier [Wed, 6 May 2020 12:08:22 +0000 (21:08 +0900)]
Handle spaces for Python install location in MSVC scripts
Attempting to use an installation path of Python that includes spaces
caused the MSVC builds to fail. This fixes the issue by using the same
quoting method as
ad7595b for OpenSSL.
Author: Victor Wagner
Discussion: https://postgr.es/m/
20200430150608.
6dc6b8c4@antares.wagner.home
Backpatch-through: 9.5
Tom Lane [Tue, 5 May 2020 20:06:49 +0000 (16:06 -0400)]
Doc: warn that timezone abbreviations don't work in recovery_target_time.
Moving this setting into the main configuration file was ill-considered,
perhaps, because that typically causes it to be set before
timezone_abbreviations has been set. Which in turn means that zone
abbreviations don't work, only full zone names.
We could imagine hacking things so that such cases do work, but the
stability of the hack would be questionable, and the value isn't really
that high. Instead just document that you should use a numeric zone
offset or a full zone name.
Per bug #16404 from Reijo Suhonen.
Back-patch to v12 where this was changed.
Discussion: https://postgr.es/m/16404-
4603a99603fbd04c@postgresql.org