+ linkend="regress-evaluation"> below for more.
- and were extensively revised/repackaged by Marc Fournier and Thomas Lockhart.
- The regression testing notes below assume the following (except where noted):
- Commands are Unix-compatible. See note below.
- Defaults are used except where noted.
- The source path is /usr/src/pgsql (other paths are possible).
- The runtime path is /usr/local/pgsql (other paths are possible).
+ when you are the root user (the server will not start as root).
+ some other user, log in as that user, and restart the tests.
- 'src/test/regress' directory tree must be writeable by that user.
+ Alternatively, run the tests after installation.
- tests will fail.
+ To run the tests after installation (see
+ linkend="installation">), initialize a data area and start the
+ server, as explained in , then type
+
+
+ The server is expected to be running on the local host with the
+ default port number.
+
+
+
+
Test Evaluation
- The "date" command above should have returned the current system time
- in the PST8PDT time zone. If the PST8PDT database is not available, then
- your system may have returned the time in GMT. If the PST8PDT time zone
- is not available, you can set the time zone rules explicitly:
-setenv PGTZ PST8PDT7,M04.01.0,M10.05.03
-
+ Some properly installed and fully functional
+
PostgreSQL installations can
+ fail
some of these regression tests due to
+ artifacts of floating point representation and time zone
+ support. The tests are currently evaluated using a simple
+
diff comparison against the outputs
+ generated on a reference system, so the results are sensitive to
+ small system differences. When a test is reported as
+ failed
, always examine the differences between
+ expected and actual results; you may well find that the
+ differences are not significant. Nonetheless, we still strive to
+ maintain accurate reference files across all supported platforms,
+ so it can be expected that all tests pass.
- The directory layout for the regression test area is:
-
-
Directory Layout
-
- Kerberos
-
-
-
- |
- Directory
- Description
-
-
-
- |
- Directory
- Description
-
- |
- input
-
- Source files that are converted using
- make all into
- some of the .sql files in the
- sql subdirectory.
-
-
-
- |
- output
-
- Source files that are converted using
- make all into
- .out files in the
- expected subdirectory.
-
-
-
- |
- sql
-
- .sql files used to perform the
- regression tests.
-
-
-
- |
- expected
-
- .out files that represent what we
- expect the results to
- look like.
-
-
-
- |
- results
-
- .out files that contain what the results
- actually look
- like. Also used as temporary storage for table copy testing.
-
-
-
- |
- tmp_check
-
- Temporary installation created by parallel testing script.
-
-
-
-
-
+ The actual outputs of the regression tests are in files in the
+ src/test/regress/results directory. The test
+ script uses
diff to compare each output
+ file against the reference outputs stored in the
+ src/test/regress/expected directory. Any
+ differences are saved for your inspection in
+ src/test/regress/regression.diffs. (Or you
+ can run
diff yourself, if you prefer.)
-
-
-
-
Regression Test Procedure
-
+
+
+
Error message differences
+
- Commands were tested on RedHat Linux version 4.2 using the bash shell.
- Except where noted, they will probably work on most systems. Commands
- like ps and tar vary
- wildly on what options you should use on each
- platform. Use common sense before typing in these commands.
+ Some of the regression tests involve intentional invalid input
+ values. Error messages can come from either the
+
PostgreSQL code or from the host
+ platform system routines. In the latter case, the messages may
+ vary between platforms, but should reflect similar
+ information. These differences in messages will result in a
+ failed
regression test which can be validated by
+ inspection.
+
-
<productname>Postgres</productname> Regression Test
-
-
- Prepare the files needed for the regression test with:
- cd /usr/src/pgsql/src/test/regress
- gmake clean
- gmake all
-
- You can skip "gmake clean" if this is the first time you
- are running the tests.
-
- program with PostgreSQL extension functions into a shared library.
- Localized SQL scripts and output-comparison files are also created
- for the tests that need them. The localization replaces macros in
- the source files with absolute pathnames and user names.
-
-
-
-
- If you intend to use the "sequential" test procedure, which tests
- an already-installed postmaster, be sure that the postmaster
- is running. If it isn't already running,
- start the postmaster in an available window by typing
- postmaster
-
- or start the postmaster daemon running in the background by typing
- cd
- nohup postmaster > regress.log 2>&1 &
-
- The latter is probably preferable, since the regression test log
- will be quite lengthy (60K or so, in
-
Postgres 7.0) and you might want to
- review it for clues if things go wrong.
-
-
- Do not run postmaster from the root account.
-
-
-
-
-
-
- Run the regression tests. For a sequential test, type
- cd /usr/src/pgsql/src/test/regress
- gmake runtest
-
- For a parallel test, type
- cd /usr/src/pgsql/src/test/regress
- gmake runcheck
-
- The sequential test just runs the test scripts using your
- already-running postmaster.
- The parallel test will perform a complete installation of
-
Postgres into a temporary directory,
- start a private postmaster therein, and then run the test scripts.
- Finally it will kill the private postmaster (but the temporary
- directory isn't removed automatically).
-
-
+
+
Date and time differences
-
- You should get on the screen (and also written to file ./regress.out)
- a series of statements stating which tests passed and which tests
- failed. Please note that it can be normal for some of the tests to
- "fail" due to platform-specific variations. See the next section
- for details on determining whether a "failure" is significant.
-
- Some of the tests, notably "numeric", can take a while, especially
- on slower platforms. Have patience.
-
-
-
-
- After running the tests and examining the results, type
- cd /usr/src/pgsql/src/test/regress
- gmake clean
-
- to recover the temporary disk space used by the tests.
- If you ran a sequential test, also type
- dropdb regression
-
-
-
-
-
-
-
-
Regression Analysis
-
- The actual outputs of the regression tests are in files in the
- ./results directory. The test script
- uses
diff to compare each output file
- against the reference outputs stored in the
- ./expected directory. Any differences are
- saved for your inspection in
- ./regression.diffs. (Or you can run
-
diff yourself, if you prefer.)
-
+ Most of the date and time results are dependent on the time zone
+ environment. The reference files are generated for time zone
+ PST8PDT (Berkeley, California) and there will be apparent
+ failures if the tests are not run with that time zone setting.
+ The regression test driver sets environment variable
+ PGTZ to PST8PDT to ensure
+ proper results. However, your system must provide library
+ support for the PST8PDT time zone, or the time zone-dependent
+ tests will fail. To verify that your machine does have this
+ support, type the following:
+
+
+ The command above should have returned the current system time in
+ the PST8PDT time zone. If the PST8PDT database is not available,
+ then your system may have returned the time in GMT. If the
+ PST8PDT time zone is not available, you can set the time zone
+ rules explicitly:
+PGTZ='PST8PDT7,M04.01.0,M10.05.03'; export PGTZ
+
+
- The files might not compare exactly. The test script will report
- any difference as a "failure", but the difference might be due
- to small cross-system differences in error message wording,
- math library behavior, etc.
- "Failures" of this type do not indicate a problem with
+ There appear to be some systems which do not accept the
+ recommended syntax for explicitly setting the local time zone
+ rules; you may need to use a different PGTZ
+ setting on such machines.
-
+
- Thus, it is necessary to examine the actual differences for each
- "failed" test to determine whether there is really a problem.
- The following paragraphs attempt to provide some guidance in
- determining whether a difference is significant or not.
+ Some systems using older time zone libraries fail to apply
+ daylight-savings corrections to dates before 1970, causing
+ pre-1970 PDT times to be displayed in PST instead. This will
+ result in localized differences in the test results.
-
-
-
Error message differences
- Some of the regression tests involve intentional invalid input values.
- Error messages can come from either the Postgres code or from the host
- platform system routines. In the latter case, the messages may vary
- between platforms, but should reflect similar information. These
- differences in messages will result in a "failed" regression test which
- can be validated by inspection.
-
-
-
+ Some of the queries in the timestamp
test will
+ fail if you run the test on the day of a daylight-savings time
+ changeover, or the day before or after one. These queries assume
+ that the intervals between midnight yesterday, midnight today and
+ midnight tomorrow are exactly twenty-four hours -- which is wrong
+ if daylight-savings time went into or out of effect meanwhile.
+
+
-
- Date and time</span> differences
+
+ Floating point</span> differences
- Most of the date and time results are dependent on timezone environment.
- The reference files are generated for timezone PST8PDT (Berkeley,
- California) and there will be apparent failures if the tests are not
- run with that timezone setting. The regression test driver sets
- environment variable PGTZ to PST8PDT to ensure proper results.
-
-
- Some of the queries in the "timestamp" test will fail if you run
- the test on the day of a daylight-savings time changeover, or the
- day before or after one. These queries assume that the intervals
- between midnight yesterday, midnight today and midnight tomorrow are
- exactly twenty-four hours ... which is wrong if daylight-savings time
- went into or out of effect meanwhile.
-
-
- There appear to be some systems which do not accept the recommended syntax
- for explicitly setting the local time zone rules; you may need to use
- a different PGTZ setting on such machines.
-
+ Some of the tests involve computing 64-bit (double
+ precision) numbers from table columns. Differences in
+ results involving mathematical functions of double
+ precision columns have been observed. The float8 and
+ geometry tests are particularly prone to small differences across
+ platforms, or even with different compiler optimization options.
+ Human eyeball comparison is needed to determine the real
+ significance of these differences which are usually 10 places to
+ the right of the decimal point.
+
- Some systems using older timezone libraries fail to apply daylight-savings
- corrections to pre-1970 dates, causing pre-1970 PDT times to be displayed
- in PST instead. This will result in localized differences in the test
- results.
-
-
-
+ Some systems signal errors from pow() and
+ exp() differently from the mechanism
+ expected by the current
PostgreSQL+ code.
+
+
-
- Floating point</span> differences
+
+ Polygon</span> differences
- Some of the tests involve computing 64-bit (float8) numbers from table
- columns. Differences in results involving mathematical functions of
- float8 columns have been observed. The float8
- and geometry tests are particularly prone to small differences
- across platforms.
- Human eyeball comparison is needed to determine the real significance
- of these differences which are usually 10 places to the right of
- the decimal point.
-
+ Several of the tests involve operations on geographic data about
+ the Oakland/Berkeley, CA street map. The map data is expressed as
+ polygons whose vertices are represented as pairs of double
+ precision numbers (decimal latitude and
+ longitude). Initially, some tables are created and loaded with
+ geographic data, then some views are created which join two
+ tables using the polygon intersection operator
+ (##), then a select is done on the view.
+
- Some systems signal errors from pow() and exp() differently from
- the mechanism expected by the current Postgres code.
-
-
-
-
-
-
Polygon differences
-
- Several of the tests involve operations on geographic date about the
- Oakland/Berkley CA street map. The map data is expressed as polygons
- whose vertices are represented as pairs of float8 numbers (decimal
- latitude and longitude). Initially, some tables are created and
- loaded with geographic data, then some views are created which join
- two tables using the polygon intersection operator (##), then a select
- is done on the view.
-
- When comparing the results from different platforms, differences occur
- in the 2nd or 3rd place to the right of the decimal point. The SQL
- statements where these problems occur are the following:
-
- QUERY: SELECT * from street;
- QUERY: SELECT * from iexit;
-
-
-
-
-
-
-
Random differences
-
- There is at least one case in the "random" test script that is
- intended to produce
- random results. This causes random to fail the regression test
- once in a while (perhaps once in every five to ten trials).
- Typing
- diff results/random.out expected/random.out
-
- should produce only one or a few lines of differences. You need
- not worry unless the random test always fails in repeated attempts.
- (On the other hand, if the random test is never
- reported to fail even in many trials of the regress tests, you
- probably should worry.)
-
-
-
+ When comparing the results from different platforms, differences
+ occur in the 2nd or 3rd place to the right of the decimal
+ point. The SQL statements where these problems occur are the
+ following:
+SELECT * from street;
+SELECT * from iexit;
+
+
+
-
- The "expected" files</span>
+
+ The <quote>random</quote> test</span>
- The ./expected/*.out files were adapted from the original monolithic
- expected.input file provided by Jolly Chen et al. Newer versions of these
- files generated on various development machines have been substituted after
- careful (?) inspection. Many of the development machines are running a
- Unix OS variant (FreeBSD, Linux, etc) on Ix86 hardware.
-
- The original expected.input file was created on a SPARC Solaris 2.4
- system using the postgres5-1.02a5.tar.gz source tree. It was compared
- with a file created on an I386 Solaris 2.4 system and the differences
- were only in the floating point polygons in the 3rd digit to the right
- of the decimal point.
-
- The original sample.regress.out file was from the postgres-1.01 release
- constructed by Jolly Chen. It may
- have been created on a DEC ALPHA machine as the Makefile.global
- in the postgres-1.01 release has PORTNAME=alpha.
-
-
-
-
+ There is at least one case in the random
test
+ script that is intended to produce random results. This causes
+ random to fail the regression test once in a while (perhaps once
+ in every five to ten trials). Typing
+diff results/random.out expected/random.out
+
+ should produce only one or a few lines of differences. You need
+ not worry unless the random test always fails in repeated
+ attempts. (On the other hand, if the random test is
+ never reported to fail even in many trials
+ of the regress tests, you probably should
+ worry.)
+
+
+
Platform-specific comparison files
-
-Introduction
-
- The PostgreSQL regression tests are a comprehensive set of tests for the
- SQL implementation embedded in PostgreSQL. They test standard SQL
- operations as well as the extended capabilities of PostgreSQL.
-
- The regression tests were originally developed by Jolly Chen and Andrew Yu,
- and were extensively revised/repackaged by Marc Fournier and Thomas Lockhart.
- From PostgreSQL v6.1 onward the regression tests are current for every
- official release.
-
- Some properly installed and fully functional PostgreSQL installations
- can fail some of these regression tests due to artifacts of floating point
- representation and time zone support. The current tests are evaluated
- using a simple "diff" algorithm, and are sensitive to small system
- differences. For apparently failed tests, examining the differences
- may reveal that the differences are not significant.
-
-Preparation
-
- To prepare for regression testing, do "make all" in the regression test
- directory. This compiles a 'C' program with PostgreSQL extension functions
- into a shared library. Localized SQL scripts and output-comparison
- files are also created for the tests that need them. The localization
- replaces macros in the source files with absolute pathnames and user names.
-
- Normally, the regression tests should be run as the postgres user since
- the 'src/test/regress' directory and sub-directories are owned by the
- postgres user. If you run the regression test as another user the
- 'src/test/regress' directory tree must be writeable to that user.
-
- It was formerly necessary to run the postmaster with system time zone
- set to PST, but this is no longer required. You can run the regression
- tests under your normal postmaster configuration. The test script will
- set the PGTZ environment variable to ensure that timezone-dependent tests
- produce the expected results.
-
-Directory Layout
-
- input/ .... .source files that are converted using 'make all' into
- some of the .sql files in the 'sql' subdirectory
-
- output/ ... .source files that are converted using 'make all' into
- .out files in the 'expected' subdirectory
-
- sql/ ...... .sql files used to perform the regression tests
-
- expected/ . .out files that represent what we *expect* the results to
- look like
-
- results/ .. .out files that contain what the results *actually* look
- like. Also used as temporary storage for table copy testing.
-
-Running the regression test
-
- If you have previously run the regression test for a different Postgres
- release, make sure you have up-to-date comparison files by doing
-
- make clean all
-
- The regression test is invoked with the command:
-
- make runtest
-
- or you can do
-
- make runcheck
-
- which invokes a parallel form of the regress tests, and does not
- need an already-installed postmaster. Instead, runcheck creates
- a temporary installation under the regress directory.
-
-Comparing expected/actual output
-
- The results are in files in the ./results directory. These results
- can be compared with results in the ./expected directory using 'diff'.
- (The test script now does this for you, and leaves the differences
- in ./regression.diffs.)
-
- The files might not compare exactly. The following paragraphs attempt
- to explain the differences.
-
- Once the output files have been verified for a particular platform,
- it is possible to provide new platform-specific comparison files,
- so that future test runs won't report bogus "failures". See
- 'Platform-specific comparison files', below.
+REGRESSION TESTS
+
+The regression tests are a comprehensive set of tests for the SQL
+implementation in PostgreSQL. They test standard SQL operations as
+well as the extended capabilities of PostgreSQL. The test suite was
+originally developed by Jolly Chen and Andrew Yu, and was extensively
+revised and repackaged by Marc Fournier and Thomas Lockhart. From
+PostgreSQL 6.1 onward the regression tests are current for every
+official release.
+
+The regression test can be run against an already installed and
+running server, or using a temporary installation within the build
+tree. Furthermore, there is a "parallel" and a "sequential" mode for
+running the tests. The sequential method runs each test script in
+turn, whereas the parallel method starts up multiple server processes
+to run groups of tests in parallel. Parallel testing gives confidence
+that interprocess communication and locking are working correctly. For
+historical reasons, the sequential test is usually run against an
+existing installation and the parallel method "stand-alone", but there
+are technical reasons for this.
+
+To run the regression tests after building but before installation,
+type
+
+$ gmake check
+
+in the top-level directory. (Or you can change to src/test/regress and
+run the command there.) This will first build several auxiliary files,
+such as platform-dependent "expected" files and some sample
+user-defined trigger functions, and then run the test driver
+script. At the end you should see something like
+
+======================
+ All 75 tests passed.
+======================
+
+or otherwise a note about what tests failed. See the section called
+Test Evaluation below for more.
+
+ Note: Because this test method runs a temporary server, it will
+ not work when you are the root user (the server will not start as
+ root). If you already did the build as root, you do not have to
+ start all over. Instead, make the regression test directory
+ writable by some other user, log in as that user, and restart the
+ tests.
+
+ root# chmod -R a+w src/test/regress
+ root# su - joeuser
+ joeuser$ gmake check
+
+ (The only possible "security risk" here is that other users might
+ be able to alter the regression test results behind your back. Use
+ common sense when managing user permissions.)
+
+ Alternatively, run the tests after installation.
+
+ Tip: On some systems, the default Bourne-compatible shell
+ (/bin/sh) gets confused when it has to manage too many child
+ processes in parallel. This may cause the parallel test run to
+ lock up or fail. In such cases, specify a different
+ Bourne-compatible shell on the command line, for example:
+
+ $ gmake SHELL=/bin/ksh check
+
+To run the tests after installation, initialize a data area and start
+the server, then type
+
+$ gmake installcheck
+
+The server is expected to be running on the local host with the
+default port number.
+
+Test Evaluation
+
+Some properly installed and fully functional PostgreSQL installations
+can "fail" some of these regression tests due to artifacts of floating
+point representation and time zone support. The tests are currently
+evaluated using a simple diff comparison against the outputs generated
+on a reference system, so the results are sensitive to small system
+differences. When a test is reported as "failed", always examine the
+differences between expected and actual results; you may well find
+that the differences are not significant. Nonetheless, we still strive
+to maintain accurate reference files across all supported platforms,
+so it can be expected that all tests pass.
+
+The actual outputs of the regression tests are in files in the
+src/test/regress/results directory. The test script uses diff to
+compare each output file against the reference outputs stored in the
+src/test/regress/expected directory. Any differences are saved for
+your inspection in src/test/regress/regression.diffs. (Or you can run
+diff yourself, if you prefer.)
Error message differences
- Some of the regression tests involve intentional invalid input values.
- Error messages can come from either the Postgres code or from the host
- platform system routines. In the latter case, the messages may vary
- between platforms, but should reflect similar information. These
- differences in messages will result in a "failed" regression test which
- can be validated by inspection.
-
-DATE/TIME differences
-
- Most of the date and time results are dependent on timezone environment.
- The reference files are generated for timezone PST8PDT (Berkeley,
- California) and there will be apparent failures if the tests are not
- run with that timezone setting. The regression test driver sets
- environment variable PGTZ to PST8PDT to ensure proper results.
-
- There appear to be some systems which do not accept the recommended syntax
- for explicitly setting the local time zone rules; you may need to use
- a different PGTZ setting on such machines.
+Some of the regression tests involve intentional invalid input
+values. Error messages can come from either the PostgreSQL code or
+from the host platform system routines. In the latter case, the
+messages may vary between platforms, but should reflect similar
+information. These differences in messages will result in a "failed"
+regression test which can be validated by inspection.
+
+Date and time differences
+
+Most of the date and time results are dependent on the time zone
+environment. The reference files are generated for time zone PST8PDT
+(Berkeley, California) and there will be apparent failures if the
+tests are not run with that time zone setting. The regression test
+driver sets environment variable PGTZ to PST8PDT to ensure proper
+results. However, your system must provide library support for the
+PST8PDT time zone, or the time zone-dependent tests will fail. To
+verify that your machine does have this support, type the following:
+
+$ env TZ=PST8PDT date
+
+The command above should have returned the current system time in the
+PST8PDT time zone. If the PST8PDT database is not available, then your
+system may have returned the time in GMT. If the PST8PDT time zone is
+not available, you can set the time zone rules explicitly:
+
+PGTZ='PST8PDT7,M04.01.0,M10.05.03'; export PGTZ
+
+There appear to be some systems which do not accept the recommended
+syntax for explicitly setting the local time zone rules; you may need
+to use a different PGTZ setting on such machines.
+
+Some systems using older time zone libraries fail to apply
+daylight-savings corrections to dates before 1970, causing pre-1970
+PDT times to be displayed in PST instead. This will result in
+localized differences in the test results.
+
+Some of the queries in the "timestamp" test will fail if you run the
+test on the day of a daylight-savings time changeover, or the day
+before or after one. These queries assume that the intervals between
+midnight yesterday, midnight today and midnight tomorrow are exactly
+twenty-four hours -- which is wrong if daylight-savings time went into
+or out of effect meanwhile.
+
+Floating point differences
+
+Some of the tests involve computing 64-bit (double precision) numbers
+from table columns. Differences in results involving mathematical
+functions of double precision columns have been observed. The float8
+and geometry tests are particularly prone to small differences across
+platforms, or even with different compiler optimization options. Human
+eyeball comparison is needed to determine the real significance of
+these differences which are usually 10 places to the right of the
+decimal point.
+
+Some systems signal errors from pow() and exp() differently from the
+mechanism expected by the current PostgreSQL code.
+
+Polygon differences
+
+Several of the tests involve operations on geographic data about the
+Oakland/Berkeley, CA street map. The map data is expressed as polygons
+whose vertices are represented as pairs of double precision numbers
+(decimal latitude and longitude). Initially, some tables are created
+and loaded with geographic data, then some views are created which
+join two tables using the polygon intersection operator (##), then a
+select is done on the view.
+
+When comparing the results from different platforms, differences occur
+in the 2nd or 3rd place to the right of the decimal point. The SQL
+statements where these problems occur are the following:
+
+SELECT * from street;
+SELECT * from iexit;
+
+The "random" test
+
+There is at least one case in the "random" test script that is
+intended to produce random results. This causes random to fail the
+regression test once in a while (perhaps once in every five to ten
+trials). Typing
+
+diff results/random.out expected/random.out
+
+should produce only one or a few lines of differences. You need not
+worry unless the random test always fails in repeated attempts. (On
+the other hand, if the random test is never reported to fail even in
+many trials of the regress tests, you probably should worry.)
- Some systems using older timezone libraries fail to apply daylight-savings
- corrections to pre-1970 dates, causing pre-1970 PDT times to be displayed
- in PST instead. This will result in localized differences in the test
- results.
-
-FLOATING POINT differences
-
- Some of the tests involve computing 64-bit (FLOAT8) numbers from table
- columns. Differences in results involving mathematical functions of
- FLOAT8 columns have been observed. These differences occur where
- different operating systems are used on the same platform ie:
- BSDI and SOLARIS on Intel/86, and where the same operating system is
- used used on different platforms, ie: SOLARIS on SPARC and Intel/86.
-
- Human eyeball comparison is needed to determine the real significance
- of these differences which are usually 10 places to the right of
- the decimal point.
+Platform-specific comparison files
- Some systems signal errors from pow() and exp() differently from
- the mechanism expected by the current Postgres code.
+Since some of the tests inherently produce platform-specific results,
+we have provided a way to supply platform-specific result comparison
+files. Frequently, the same variation applies to multiple platforms;
+rather than supplying a separate comparison file for every platform,
+there is a mapping file that defines which comparison file to use. So,
+to eliminate bogus test "failures" for a particular platform, you must
+choose or make a variant result file, and then add a line to the
+mapping file, which is "resultmap".
-POLYGON differences
+Each line in the mapping file is of the form
- Several of the tests involve operations on geographic data about the
- Oakland/Berkley CA street map. The map data is expressed as polygons
- whose vertices are represented as pairs of FLOAT8 numbers (decimal
- latitude and longitude). Initially, some tables are created and
- loaded with geographic data, then some views are created which join
- two tables using the polygon intersection operator (##), then a select
- is done on the view.
+ testname/platformnamepattern=comparisonfilename
- When comparing the results from different platforms, differences occur
- in the 2nd or 3rd place to the right of the decimal point. The SQL
- statements where these problems occur are the following:
+The test name is just the name of the particular regression test
+module. The platform name pattern is a pattern in the style of expr(1)
+(that is, a regular expression with an implicit ^ anchor at the
+start). It is matched against the platform name as printed by
+config.guess. The comparison file name is the name of the substitute
+result comparison file.
- QUERY: SELECT * from street;
- QUERY: SELECT * from iexit;
+For example: the int2 regress test includes a deliberate entry of a
+value that is too large to fit in int2. The specific error message
+that is produced is platform-dependent; our reference platform emits
-Random differences
+ ERROR: pg_atoi: error reading "100000": Numerical result out of range
- There is at least one test case in random.out which is intended to produce
- random results. This causes random to fail the regression testing.
- Typing "diff results/random.out expected/random.out" should produce only
- one or a few lines of differences for this reason, but other floating
- point differences on dissimilar architectures might cause many more
- differences. See the release notes below.
+but a fair number of other Unix platforms emit
-The 'expected' files
+ ERROR: pg_atoi: error reading "100000": Result too large
- The ./expected/*.out files were adapted from the original monolithic
- 'expected.input' file provided by Jolly Chen et al. Newer versions of these
- files generated on various development machines have been substituted after
- careful (?) inspection. Many of the development machines are running a
- Unix OS variant (FreeBSD, Linux, etc) on Ix86 hardware.
+Therefore, we provide a variant comparison file, int2-too-large.out,
+that includes this spelling of the error message. To silence the bogus
+"failure" message on HPPA platforms, resultmap includes
-Platform-specific comparison files
+ int2/hppa=int2-too-large
- Since some of the tests inherently produce platform-specific results,
- we have provided a way to supply platform-specific result comparison
- files. Frequently, the same variation applies to multiple platforms;
- rather than supplying a separate comparison file for every platform,
- there is a mapping file that defines which comparison file to use.
- So, to eliminate bogus test "failures" for a particular platform,
- you must choose or make a variant result file, and then add a line
- to the mapping file, which is "resultmap".
-
- Each line in the mapping file is of the form
- testname/platformnamepattern=comparisonfilename
- The test name is just the name of the particular regression test module.
- The platform name pattern is a pattern in the style of expr(1) (that is,
- a regular expression with an implicit ^ anchor at the start). It is matched
- against the platform name as printed by config.guess. The comparison
- file name is the name of the substitute result comparison file.
-
- For example: the int2 regress test includes a deliberate entry of a value
- that is too large to fit in int2. The specific error message that is
- produced is platform-dependent; our reference platform emits
- ERROR: pg_atoi: error reading "100000": Numerical result out of range
- but a fair number of other Unix platforms emit
- ERROR: pg_atoi: error reading "100000": Result too large
- Therefore, we provide a variant comparison file, int2-too-large.out,
- that includes this spelling of the error message. To silence the
- bogus "failure" message on HPPA platforms, resultmap includes
- int2/hppa=int2-too-large
- which will trigger on any machine for which config.guess's output
- begins with 'hppa'. Other lines in resultmap select the variant
- comparison file for other platforms where it's appropriate.
-
-
- The regression tests have been adapted and extensively modified for the
- v6.1 release of PostgreSQL.
-
- Three new data types (datetime, timespan, and circle) have been added to
- the native set of PostgreSQL types. Points, boxes, paths, and polygons
- have had their output formats made consistant across the data types.
- The polygon output in misc.out has only been spot-checked for correctness
- relative to the original regression output.
-
- PostgreSQL v6.1 introduces a new, alternate optimizer which uses "genetic"
- algorithms. These algorithms introduce a random behavior in the ordering
- of query results when the query contains multiple qualifiers or multiple
- tables (giving the optimizer a choice on order of evaluation). Several
- regression tests have been modified to explicitly order the results, and
- hence are insensitive to optimizer choices. A few regression tests are
- for data types which are inherently unordered (e.g. points and time
- intervals) and tests involving those types are explicitly bracketed with
- "set geqo to 'off'" and "reset geqo".
-
- The interpretation of array specifiers (the curly braces around atomic
- values) appears to have changed sometime after the original regression
- tests were generated. The current ./expected/*.out files reflect this
- new interpretation, which may not be correct!
-
- The float8 regression test fails on at least some platforms. This is due
- to differences in implementations of pow() and exp() and the signaling
- mechanisms used for overflow and underflow conditions.
-
- The "random" results in the random test should cause the "random" test
- to be "failed", since the regression tests are evaluated using a simple
- diff. However, "random" does not seem to produce random results on my
- test machine (Linux/gcc/i686).
-
-Sample timing results
-
- Timing under Linux 2.0.27 seems to have a roughly 5% variation from run
- to run, presumably due to the timing vagaries of multitasking systems.
-
- Time System
- 06:12 Pentium Pro 180, 32MB, Linux 2.0.30, gcc 2.7.2 -O2 -m486
- 12:06 P-100, 48MB, Linux 2.0.29, gcc
- 39:58 Sparc IPC 32MB, Solaris 2.5, gcc 2.7.2.1 -O -g
+which will trigger on any machine for which config.guess's output
+begins with 'hppa'. Other lines in resultmap select the variant
+comparison file for other platforms where it's appropriate.
projects / postgresql.git / commitdiff