Remove race condition in 022_crash_temp_files.pl test.

It's possible for the query that "waits for restart" to complete a
successful iteration before the postmaster has noticed its SIGKILL'd
child and begun the restart cycle.  (This is a bit hard to believe
perhaps, but it's been seen at least twice in the buildfarm, mainly
on ancient platforms that likely have quirky schedulers.)

To provide a more secure interlock, wait for the other session
we're using to report that it's been forcibly shut down.

Patch by me, based on a suggestion from Andres Freund.
Back-patch to v14 where this test case came in.

Discussion: https://postgr.es/m/1801850.1649047827@sss.pgh.pa.us

diff --git a/src/test/recovery/t/022_crash_temp_files.pl b/src/test/recovery/t/022_crash_temp_files.pl
index 74965348791079a1438ad3c307de9032d1d226cb..5ddc00fb05f306916d38d8766878679b1560bb69 100644 (file)
--- a/src/test/recovery/t/022_crash_temp_files.pl
+++ b/src/test/recovery/t/022_crash_temp_files.pl
@@ -16,7 +16,7 @@ if ($Config{osname} eq 'MSWin32')
 }
 else
 {
-   plan tests => 9;
+   plan tests => 11;
 }
 
 
@@ -130,11 +130,23 @@ $killme_stderr2 = '';
 my $ret = TestLib::system_log('pg_ctl', 'kill', 'KILL', $pid);
 is($ret, 0, 'killed process with KILL');
 
-# Close psql session
+# Close that psql session
 $killme->finish;
+
+# Wait till the other session reports failure, ensuring that the postmaster
+# has noticed its dead child and begun a restart cycle.
+$killme_stdin2 .= qq[
+SELECT pg_sleep($TestLib::timeout_default);
+];
+ok( pump_until(
+       $killme2,
+       \$killme_stderr2,
+       qr/WARNING:  terminating connection because of crash of another server process|server closed the connection unexpectedly|connection to server was lost|could not send data to server/m
+   ),
+   "second psql session died successfully after SIGKILL");
 $killme2->finish;
 
-# Wait till server restarts
+# Wait till server finishes restarting
 $node->poll_query_until('postgres', undef, '');
 
 # Check for temporary files
@@ -219,11 +231,23 @@ $killme_stderr2 = '';
 $ret = TestLib::system_log('pg_ctl', 'kill', 'KILL', $pid);
 is($ret, 0, 'killed process with KILL');
 
-# Close psql session
+# Close that psql session
 $killme->finish;
+
+# Wait till the other session reports failure, ensuring that the postmaster
+# has noticed its dead child and begun a restart cycle.
+$killme_stdin2 .= qq[
+SELECT pg_sleep($TestLib::timeout_default);
+];
+ok( pump_until(
+       $killme2,
+       \$killme_stderr2,
+       qr/WARNING:  terminating connection because of crash of another server process|server closed the connection unexpectedly|connection to server was lost|could not send data to server/m
+   ),
+   "second psql session died successfully after SIGKILL");
 $killme2->finish;
 
-# Wait till server restarts
+# Wait till server finishes restarting
 $node->poll_query_until('postgres', undef, '');
 
 # Check for temporary files -- should be there