Update pgcvslog

diff --git a/doc/TODO b/doc/TODO
index 168adf66a67d7cad054653a68b151a1971f9fa6f..91a8f554b9618e204f6d4f2b461c03c95c9e16ff 100644 (file)
--- a/doc/TODO
+++ b/doc/TODO
@@ -192,7 +192,8 @@ PERFORMANCE
 
 FSYNC
 
-* Allow transaction commits with rollback with no-fsync performance [fsync](Vadim)
+* Allow transaction commits with rollback with no-fsync performance
+  [fsync] (Vadim)
 
 INDEXES
 
@@ -231,6 +232,7 @@ MISC
 * Remove pg_listener index
 * Remove ANALYZE from VACUUM so it can be run separately without locks
 * Gather more accurate statistics using indexes
+* Improve statistics storage in pg_class [performance]
 
 SOURCE CODE
 -----------

diff --git a/doc/TODO.detail/performance b/doc/TODO.detail/performance
index abe1f676192076f4acea4a5acabe86db4019582c..e73550bb5aea85abccbb93444b612bcb1882914d 100644 (file)
--- a/doc/TODO.detail/performance
+++ b/doc/TODO.detail/performance
@@ -341,3 +341,214 @@ Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
  good, you'll have to ram them down people's throats." -- Howard Aiken
 
 
+From [email protected] Tue Oct 19 10:31:10 1999
+Received: from renoir.op.net ([email protected] [209.152.193.4])
+   by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id KAA29087
+   for ; Tue, 19 Oct 1999 10:31:08 -0400 (EDT)
+Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.2 $) with ESMTP id KAA27535 for ; Tue, 19 Oct 1999 10:19:47 -0400 (EDT)
+Received: from localhost (majordom@localhost)
+   by hub.org (8.9.3/8.9.3) with SMTP id KAA30328;
+   Tue, 19 Oct 1999 10:12:10 -0400 (EDT)
+   (envelope-from owner-pgsql-hackers)
+Received: by hub.org (bulk_mailer v1.5); Tue, 19 Oct 1999 10:11:55 -0400
+Received: (from majordom@localhost)
+   by hub.org (8.9.3/8.9.3) id KAA30030
+   for pgsql-hackers-outgoing; Tue, 19 Oct 1999 10:11:00 -0400 (EDT)
+   (envelope-from [email protected])
+Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2])
+   by hub.org (8.9.3/8.9.3) with ESMTP id KAA29914
+   for ; Tue, 19 Oct 1999 10:10:33 -0400 (EDT)
+   (envelope-from [email protected])
+Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1])
+   by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA09038;
+   Tue, 19 Oct 1999 10:09:15 -0400 (EDT)
+To: "Hiroshi Inoue" 
+cc: "Vadim Mikheev" , [email protected]
+Subject: Re: [HACKERS] mdnblocks is an amazing time sink in huge relations 
+In-reply-to: Your message of Tue, 19 Oct 1999 19:03:22 +0900 
+             <[email protected]> 
+Date: Tue, 19 Oct 1999 10:09:15 -0400
+Message-ID: <[email protected]>
+From: Tom Lane 
+Sender: [email protected]
+Status: OR
+
+"Hiroshi Inoue"  writes:
+> 1. shared cache holds committed system tuples.
+> 2. private cache holds uncommitted system tuples.
+> 3. relpages of shared cache are updated immediately by
+>     phisical change and corresponding buffer pages are
+>     marked dirty.
+> 4. on commit, the contents of uncommitted tuples except
+>    relpages,reltuples,... are copied to correponding tuples
+>    in shared cache and the combined contents are
+>    committed.
+> If so,catalog cache invalidation would be no longer needed.
+> But synchronization of the step 4. may be difficult.
+
+I think the main problem is that relpages and reltuples shouldn't
+be kept in pg_class columns at all, because they need to have
+very different update behavior from the other pg_class columns.
+
+The rest of pg_class is update-on-commit, and we can lock down any one
+row in the normal MVCC way (if transaction A has modified a row and
+transaction B also wants to modify it, B waits for A to commit or abort,
+so it can know which version of the row to start from).  Furthermore,
+there can legitimately be several different values of a row in use in
+different places: the latest committed, an uncommitted modification, and
+one or more old values that are still being used by active transactions
+because they were current when those transactions started.  (BTW, the
+present relcache is pretty bad about maintaining pure MVCC transaction
+semantics like this, but it seems clear to me that that's the direction
+we want to go in.)
+
+relpages cannot operate this way.  To be useful for avoiding lseeks,
+relpages *must* change exactly when the physical file changes.  It
+matters not at all whether the particular transaction that extended the
+file ultimately commits or not.  Moreover there can be only one correct
+value (per relation) across the whole system, because there is only one
+length of the relation file.
+
+If we want to take reltuples seriously and try to maintain it
+on-the-fly, then I think it needs still a third behavior.  Clearly
+it cannot be updated using MVCC rules, or we lose all writer
+concurrency (if A has added tuples to a rel, B would have to wait
+for A to commit before it could update reltuples...).  Furthermore
+"updating" isn't a simple matter of storing what you think the new
+value is; otherwise two transactions adding tuples in parallel would
+leave the wrong answer after B commits and overwrites A's value.
+I think it would work for each transaction to keep track of a net delta
+in reltuples for each table it's changed (total tuples added less total
+tuples deleted), and then atomically add that value to the table's
+shared reltuples counter during commit.  But that still leaves the
+problem of how you use the counter during a transaction to get an
+accurate answer to the question "If I scan this table now, how many tuples
+will I see?"  At the time the question is asked, the current shared
+counter value might include the effects of transactions that have
+committed since your transaction started, and therefore are not visible
+under MVCC rules.  I think getting the correct answer would involve
+making an instantaneous copy of the current counter at the start of
+your xact, and then adding your own private net-uncommitted-delta to
+the saved shared counter value when asked the question.  This doesn't
+look real practical --- you'd have to save the reltuples counts of
+*all* tables in the database at the start of each xact, on the off
+chance that you might need them.  Ugh.  Perhaps someone has a better
+idea.  In any case, reltuples clearly needs different mechanisms than
+the ordinary fields in pg_class do, because updating it will be a
+performance bottleneck otherwise.
+
+If we allow reltuples to be updated only by vacuum-like events, as
+it is now, then I think keeping it in pg_class is still OK.
+
+In short, it seems clear to me that relpages should be removed from
+pg_class and kept somewhere else if we want to make it more reliable
+than it is now, and the same for reltuples (but reltuples doesn't
+behave the same as relpages, and probably ought to be handled
+differently).
+
+           regards, tom lane
+
+************
+
+From [email protected] Tue Oct 19 21:25:30 1999
+Received: from renoir.op.net ([email protected] [209.152.193.4])
+   by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id VAA28130
+   for ; Tue, 19 Oct 1999 21:25:26 -0400 (EDT)
+Received: from hub.org (hub.org [216.126.84.1]) by renoir.op.net (o1/$Revision: 1.2 $) with ESMTP id VAA10512 for ; Tue, 19 Oct 1999 21:15:28 -0400 (EDT)
+Received: from localhost (majordom@localhost)
+   by hub.org (8.9.3/8.9.3) with SMTP id VAA50745;
+   Tue, 19 Oct 1999 21:07:23 -0400 (EDT)
+   (envelope-from owner-pgsql-hackers)
+Received: by hub.org (bulk_mailer v1.5); Tue, 19 Oct 1999 21:07:01 -0400
+Received: (from majordom@localhost)
+   by hub.org (8.9.3/8.9.3) id VAA50644
+   for pgsql-hackers-outgoing; Tue, 19 Oct 1999 21:06:06 -0400 (EDT)
+   (envelope-from [email protected])
+Received: from sd.tpf.co.jp (sd.tpf.co.jp [210.161.239.34])
+   by hub.org (8.9.3/8.9.3) with ESMTP id VAA50584
+   for ; Tue, 19 Oct 1999 21:05:26 -0400 (EDT)
+   (envelope-from [email protected])
+Received: from cadzone ([126.0.1.40] (may be forged))
+          by sd.tpf.co.jp (2.5 Build 2640 (Berkeley 8.8.6)/8.8.4) with SMTP
+   id KAA01715; Wed, 20 Oct 1999 10:05:14 +0900
+From: "Hiroshi Inoue" 
+To: "Tom Lane" 
+Cc: 
+Subject: RE: [HACKERS] mdnblocks is an amazing time sink in huge relations 
+Date: Wed, 20 Oct 1999 10:09:13 +0900
+Message-ID: <[email protected]>
+MIME-Version: 1.0
+Content-Type: text/plain;
+   charset="iso-8859-1"
+Content-Transfer-Encoding: 7bit
+X-Priority: 3 (Normal)
+X-MSMail-Priority: Normal
+X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
+X-Mimeole: Produced By Microsoft MimeOLE V4.72.2106.4
+Importance: Normal
+Sender: [email protected]
+Status: ORr
+
+> -----Original Message-----
+> From: Hiroshi Inoue [mailto:[email protected]]
+> Sent: Tuesday, October 19, 1999 6:45 PM
+> To: Tom Lane
+> Cc: [email protected]
+> Subject: RE: [HACKERS] mdnblocks is an amazing time sink in huge
+> relations 
+> 
+> 
+> > 
+> > "Hiroshi Inoue"  writes:
+> 
+> [snip]
+>  
+> > 
+> > > Deletion is necessary only not to consume disk space.
+> > >
+> > > For example vacuum could remove not deleted files.
+> > 
+> > Hmm ... interesting idea ... but I can hear the complaints
+> > from users already...
+> >
+> 
+> My idea is only an analogy of PostgreSQL's simple recovery
+> mechanism of tuples.
+> 
+> And my main point is
+>  "delete fails after commit" doesn't harm the database
+>  except that not deleted files consume disk space.
+> 
+> Of cource,it's preferable to delete relation files immediately
+> after(or just when) commit.
+> Useless files are visible though useless tuples are invisible.
+>
+
+Anyway I don't need "DROP TABLE inside transactions" now
+and my idea is originally for that issue.
+
+After a thought,I propose the following solution.
+
+1. mdcreate() couldn't create existent relation files.
+    If the existent file is of length zero,we would overwrite
+    the file.(seems the comment in md.c says so but the
+    code doesn't do so). 
+    If the file is an Index relation file,we would overwrite
+    the file.
+
+2. mdunlink() couldn't unlink non-existent relation files.
+    mdunlink() doesn't call elog(ERROR) even if the file
+    doesn't exist,though I couldn't find where to change
+    now.
+    mdopen() doesn't call elog(ERROR) even if the file
+    doesn't exist and leaves the relation as CLOSED. 
+
+Comments ?
+
+Regards. 
+
+Hiroshi Inoue
[email protected]
+
+************
+

diff --git a/doc/src/FAQ.html b/doc/src/FAQ.html
index 4285490eeb3b4a00dd2c6cc89026414d7c101627..a4f6a57d86bc2254360deeb2511266d764ba5ad9 100644 (file)
--- a/doc/src/FAQ.html
+++ b/doc/src/FAQ.html
@@ -28,109 +28,109 @@ HREF="http://www.PostgreSQL.org/docs/faq-hpux.shtml">http://www.PostgreSQL.org/d
 
 General Questions
 
-1.1)    What is PostgreSQL?
-1.2)    What's the copyright on PostgreSQL?
-1.3)    What Unix platforms does PostgreSQL run on?
+1.1) What is PostgreSQL?
+1.2) What's the copyright on PostgreSQL?
+1.3) What Unix platforms does PostgreSQL run on?
 1.4) What non-unix ports are available?
-1.5)    Where can I get PostgreSQL?
-1.6)    Where can I get support for PostgreSQL?
-1.7)    What is the latest release of PostgreSQL?
-1.8)    What documentation is available for PostgreSQL?
-1.9)    How do I find out about known bugs or missing features?
-1.10)  How can I learn SQL?
-1.11)  Is PostgreSQL Y2K compliant?
-1.12)  How do I join the development team?
-1.13)  How do I submit a bug report?
-1.14)  How does PostgreSQL compare to other DBMS's?
+1.5) Where can I get PostgreSQL?
+1.6) Where can I get support for PostgreSQL?
+1.7) What is the latest release of PostgreSQL?
+1.8) What documentation is available for PostgreSQL?
+1.9) How do I find out about known bugs or missing features?
+1.10) How can I learn SQL?
+1.11) Is PostgreSQL Y2K compliant?
+1.12) How do I join the development team?
+1.13) How do I submit a bug report?
+1.14) How does PostgreSQL compare to other DBMS's?
+1.15) What are the maximum size limits?
 
 
 User Client Questions
 
-2.1)    Are there ODBC drivers for
+2.1) Are there ODBC drivers for
 PostgreSQL?
-2.2)    What tools are available for hooking
+2.2) What tools are available for hooking
 PostgreSQL to Web pages?
-2.3)    Does PostgreSQL have a graphical user interface?
+2.3) Does PostgreSQL have a graphical user interface?
 A report generator? An embedded query language interface?
-2.4)    What languages are available to communicate
+2.4) What languages are available to communicate
 with PostgreSQL?
 
 
 Administrative Questions
 
-3.1)    Why does initdb fail?
-3.2)    How do I install PostgreSQL somewhere other than
+3.1) Why does initdb fail?
+3.2) How do I install PostgreSQL somewhere other than
 /usr/local/pgsql?
-3.3)    When I start the postmaster, I get a
+3.3) When I start the postmaster, I get a
 Bad System Call or core dumped message.  Why?
-3.4)    When I try to start the postmaster, I get
+3.4) When I try to start the postmaster, I get
 IpcMemoryCreate errors3.  Why?
 3.5) When I try to start the postmaster, I get
 IpcSemaphoreCreate errors.  Why?
-3.6)    How do I prevent other hosts from accessing my
+3.6) How do I prevent other hosts from accessing my
 PostgreSQL database?
-3.7)    Why can't I connect to my database from
+3.7) Why can't I connect to my database from
 another machine?
-3.8)    Why can't I access the database as the
+3.8) Why can't I access the database as the
 root user?
-3.9)    All my servers crash under concurrent
+3.9) All my servers crash under concurrent
 table access.  Why?
-3.10)  How do I tune the database engine for
+3.10) How do I tune the database engine for
 better performance?
-3.11)  What debugging features are available in
+3.11) What debugging features are available in
 PostgreSQL?
 3.12) I get 'Sorry, too many clients' when trying to
 connect.  Why?
-3.13)  What are the pg_psort.XXX files in my
+3.13) What are the pg_psort.XXX files in my
 database directory?
-3.14)  How do I set up a pg_group?
 
 Operational Questions
 
-4.1)    The system seems to be confused about commas,
+4.1) The system seems to be confused about commas,
 decimal points, and date formats.
-4.2)    What is the exact difference between
+4.2) What is the exact difference between
 binary cursors and normal cursors?
-4.3)    How do I select only the first few rows of
+4.3) How do I select only the first few rows of
 a query?
 
-4.4)    How do I get a list of tables, or other
+4.4) How do I get a list of tables, or other
 things I can see in psql?
-4.5)    How do you remove a column from a table?
+4.5) How do you remove a column from a table?
 
-4.6)    What is the maximum size for a
+4.6) What is the maximum size for a
 row, table, database?
-4.7)    How much database disk space is required
+4.7) How much database disk space is required
 to store data from a typical flat file?
 
-4.8)    How do I find out what indices or
+4.8) How do I find out what indices or
 operations are defined in the database?
-4.9)    My queries are slow or don't make use of the
+4.9) My queries are slow or don't make use of the
 indexes.  Why?
-4.10)  How do I see how the query optimizer is
+4.10) How do I see how the query optimizer is
 evaluating my query?
-4.11)  What is an R-tree index?
-4.12)  What is Genetic Query Optimization?
+4.11) What is an R-tree index?
+4.12) What is Genetic Query Optimization?
 
-4.13)  How do I do regular expression searches
+4.13) How do I do regular expression searches
 and case-insensitive regexp searching?
-4.14)  In a query, how do I detect if a field
+4.14) In a query, how do I detect if a field
 is NULL?
-4.15)  What is the difference between the
+4.15) What is the difference between the
 various character types?
-4.16.1)  How do I create a serial/auto-incrementing field?
-4.16.2)  How do I get the value of a serial insert?
-4.16.3)  Wouldn't use of currval() and nextval() lead to a race condition with other concurrent backend processes?
+4.16.1) How do I create a serial/auto-incrementing field?
+4.16.2) How do I get the value of a serial insert?
+4.16.3) Wouldn't use of currval() and nextval() lead to a race condition with other concurrent backend processes?
 
-4.17)  What is an oid?  What is a tid?
-4.18)  What is the meaning of some of the terms
+4.17) What is an oid?  What is a tid?
+4.18) What is the meaning of some of the terms
 used in PostgreSQL?
 
-4.19)  Why do I get the error "FATAL:  palloc
+4.19) Why do I get the error "FATAL:  palloc
 failure: memory exhausted?"
-4.20)  How do I tell what PostgreSQL version I
+4.20) How do I tell what PostgreSQL version I
 am running? 
-4.21)  My large-object operations get invalid

diff --git a/src/tools/pgcvslog b/src/tools/pgcvslog
index 51eae013db6c3c2ebad25fce41f29583c2114f61..83d7e3bbf6f66fcf7a62ba95380b907e4687c297 100755 (executable)
--- a/src/tools/pgcvslog
+++ b/src/tools/pgcvslog
@@ -2,16 +2,17 @@
 # This utility is used to generate a compact list of changes for each
 # release, bjm 2000-02-22
 
-# Usage $0 [-r 'revision pattern'] file
+# Usage $0 file
 
+# no branches: 
 # cvs log -d '>1999-06-14 00:00:00 GMT' . > log
-# pgcvslog -r '\.2\.[0-9]*$' log
+#
+# pre and post-branch logs:
+# cvs log -d'2000-05-08 00:00:00 GMT<2000-05-29 00:00:00 GMT'
+# cvs log -d'>2000-05-29 00:00:00 GMT' -rREL7_0_PATCHES 
+#
 
-if [ "X$1" = "X-r" ]
-then   REV="$2"
-   shift 2
-else   REV=".*"
-fi
+# pgcvslog -r '\.2\.[0-9]*$' log
 
 cat "$@" |
 
@@ -21,8 +22,6 @@ cat "$@" |
 awk '
    $0 ~ /^Working file:/   {workingfile = $0}
 
-   $1 == "revision" && $2 !~ /'"$REV"'/ {skip = "Y"}
-
    ($0 ~ /^====*$/ || $0 ~ /^----*$/) && skip == "N" \
    {
        /* print blank line separating entries */