E.29. Release 9.0

Allow a standby server to accept read-only queries (Simon Riggs, Heikki Linnakangas)

This feature is called Hot Standby. There are new postgresql.conf and recovery.conf settings to control this feature, as well as extensive documentation.

Allow write-ahead log (WAL) data to be streamed to a standby server (Fujii Masao, Heikki Linnakangas)

This feature is called Streaming Replication. Previously WAL data could be sent to standby servers only in units of entire WAL files (normally 16 megabytes each). Streaming Replication eliminates this inefficiency and allows updates on the master to be propagated to standby servers with very little delay. There are new postgresql.conf and recovery.conf settings to control this feature, as well as extensive documentation.

Add pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), which can be used to monitor standby server WAL activity (Simon Riggs, Fujii Masao, Heikki Linnakangas)

Allow per-tablespace values to be set for sequential and random page cost estimates (seq_page_cost/random_page_cost) via ALTER TABLESPACE ... SET/RESET (Robert Haas)

Improve performance and reliability of EvalPlanQual rechecks in join queries (Tom Lane)

UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries that involve joins will now behave much better when encountering freshly-updated rows.

Improve performance of TRUNCATE when the table was created or truncated earlier in the same transaction (Tom Lane)

Improve performance of finding inheritance child tables (Tom Lane)

Remove unnecessary outer joins (Robert Haas)

Outer joins where the inner side is unique and not referenced above the join are unnecessary and are therefore now removed. This will accelerate many automatically generated queries, such as those created by object-relational mappers (ORMs).

Allow IS NOT NULL restrictions to use indexes (Tom Lane)

This is particularly useful for finding MAX()/MIN() values in indexes that contain many null values.

Improve the optimizer's choices about when to use materialize nodes, and when to use sorting versus hashing for DISTINCT (Tom Lane)

Improve the optimizer's equivalence detection for expressions involving boolean <> operators (Tom Lane)

Use the same random seed every time GEQO plans a query (Andres Freund)

While the Genetic Query Optimizer (GEQO) still selects random plans, it now always selects the same random plans for identical queries, thus giving more consistent performance. You can modify geqo_seed to experiment with alternative plans.

Improve GEQO plan selection (Tom Lane)

This avoids the rare error « failed to make a valid plan », and should also improve planning speed.

Improve ANALYZE to support inheritance-tree statistics (Tom Lane)

This is particularly useful for partitioned tables. However, autovacuum does not yet automatically re-analyze parent tables when child tables change.

Improve autovacuum's detection of when re-analyze is necessary (Tom Lane)

Improve optimizer's estimation for greater/less-than comparisons (Tom Lane)

When looking up statistics for greater/less-than comparisons, if the comparison value is in the first or last histogram bucket, use an index (if available) to fetch the current actual column minimum or maximum. This greatly improves the accuracy of estimates for comparison values near the ends of the data range, particularly if the range is constantly changing due to addition of new data.

Allow setting of number-of-distinct-values statistics using ALTER TABLE (Robert Haas)

This allows users to override the estimated number or percentage of distinct values for a column. This statistic is normally computed by ANALYZE, but the estimate can be poor, especially on tables with very large numbers of rows.

Add support for RADIUS (Remote Authentication Dial In User Service) authentication (Magnus Hagander)

Allow LDAP (Lightweight Directory Access Protocol) authentication to operate in « search/bind » mode (Robert Fleming, Magnus Hagander)

This allows the user to be looked up first, then the system uses the DN (Distinguished Name) returned for that user.

Add samehost and samenet designations to pg_hba.conf (Stef Walter)

These match the server's IP address and subnet address respectively.

Pass trusted SSL root certificate names to the client so the client can return an appropriate client certificate (Craig Ringer)

Add the ability for clients to set an application name, which is displayed in pg_stat_activity (Dave Page)

This allows administrators to characterize database traffic and troubleshoot problems by source application.

Add a SQLSTATE option (%e) to log_line_prefix (Guillaume Smet)

This allows users to compile statistics on errors and messages by error code number.

Write to the Windows event log in UTF16 encoding (Itagaki Takahiro)

Now there is true multilingual support for PostgreSQL log messages on Windows.

Add pg_stat_reset_shared('bgwriter') to reset the cluster-wide shared statistics for the background writer (Greg Smith)

Add pg_stat_reset_single_table_counters() and pg_stat_reset_single_function_counters() to allow resetting the statistics counters for individual tables and functions (Magnus Hagander)

Allow setting of configuration parameters based on database/role combinations (Alvaro Herrera)

Previously only per-database and per-role settings were possible, not combinations. All role and database settings are now stored in the new pg_db_role_setting system catalog. A new psql command \drds shows these settings. The legacy system views pg_roles, pg_shadow, and pg_user do not show combination settings, and therefore no longer completely represent the configuration for a user or database.

Add server parameter bonjour, which controls whether a Bonjour-enabled server advertises itself via Bonjour™ (Tom Lane)

The default is off, meaning it does not advertise. This allows packagers to distribute Bonjour-enabled builds without worrying that individual users might not want the feature.

Add server parameter enable_material, which controls the use of materialize nodes in the optimizer (Robert Haas)

The default is on. When off, the optimizer will not add materialize nodes purely for performance reasons, though they will still be used when necessary for correctness.

Change server parameter log_temp_files to use default file size units of kilobytes (Robert Haas)

Previously this setting was interpreted in bytes if no units were specified.

Log changes of parameter values when postgresql.conf is reloaded (Peter Eisentraut)

This lets administrators and security staff audit changes of database settings, and is also very convenient for checking the effects of postgresql.conf edits.

Properly enforce superuser permissions for custom server parameters (Tom Lane)

Non-superusers can no longer issue ALTER ROLE/DATABASE SET for parameters that are not currently known to the server. This allows the server to correctly check that superuser-only parameters are only set by superusers. Previously, the SET would be allowed and then ignored at session start, making superuser-only custom parameters much less useful than they should be.

Support Unicode surrogate pairs (dual 16-bit representation) in U& strings and identifiers (Peter Eisentraut)

Support Unicode escapes in E'...' strings (Marko Kreen)

Implement IF EXISTS for ALTER TABLE DROP COLUMN and ALTER TABLE DROP CONSTRAINT (Andres Freund)

Allow ALTER TABLE commands that rewrite tables to skip WAL logging (Itagaki Takahiro)

Such operations either produce a new copy of the table or are rolled back, so WAL archiving can be skipped, unless running in continuous archiving mode. This reduces I/O overhead and improves performance.

Fix failure of ALTER TABLE table ADD COLUMN col serial when done by non-owner of table (Tom Lane)

Add support for copying COMMENTS and STORAGE settings in CREATE TABLE ... LIKE commands (Itagaki Takahiro)

Add a shortcut for copying all properties in CREATE TABLE ... LIKE commands (Itagaki Takahiro)

Add the SQL-standard CREATE TABLE ... OF type command (Peter Eisentraut)

This allows creation of a table that matches an existing composite type. Additional constraints and defaults can be specified in the command.

Add deferrable unique constraints (Dean Rasheed)

This allows mass updates, such as UPDATE tab SET col = col + 1, to work reliably on columns that have unique indexes or are marked as primary keys. If the constraint is specified as DEFERRABLE it will be checked at the end of the statement, rather than after each row is updated. The constraint check can also be deferred until the end of the current transaction, allowing such updates to be spread over multiple SQL commands.

Add exclusion constraints (Jeff Davis)

Exclusion constraints generalize uniqueness constraints by allowing arbitrary comparison operators, not just equality. They are created with the CREATE TABLE CONSTRAINT ... EXCLUDE clause. The most common use of exclusion constraints is to specify that column entries must not overlap, rather than simply not be equal. This is useful for time periods and other ranges, as well as arrays. This feature enhances checking of data integrity for many calendaring, time-management, and scientific applications.

Improve uniqueness-constraint violation error messages to report the values causing the failure (Itagaki Takahiro)

For example, a uniqueness constraint violation might now report Key (x)=(2) already exists.

Add the ability to make mass permission changes across a whole schema using the new GRANT/REVOKE IN SCHEMA clause (Petr Jelinek)

This simplifies management of object permissions and makes it easier to utilize database roles for application data security.

Add ALTER DEFAULT PRIVILEGES command to control privileges of objects created later (Petr Jelinek)

This greatly simplifies the assignment of object privileges in a complex database application. Default privileges can be set for tables, views, sequences, and functions. Defaults may be assigned on a per-schema basis, or database-wide.

Add the ability to control large object (BLOB) permissions with GRANT/REVOKE (KaiGai Kohei)

Formerly, any database user could read or modify any large object. Read and write permissions can now be granted and revoked per large object, and the ownership of large objects is tracked.

Accept COPY ... CSV FORCE QUOTE * (Itagaki Takahiro)

Now * can be used as shorthand for « all columns » in the FORCE QUOTE clause.

Add new COPY syntax that allows options to be specified inside parentheses (Robert Haas, Emmanuel Cecchet)

This allows greater flexibility for future COPY options. The old syntax is still supported, but only for pre-existing options.

Allow EXPLAIN to output in XML, JSON, or YAML format (Robert Haas, Greg Sabino Mullane)

The new output formats are easily machine-readable, supporting the development of new tools for analysis of EXPLAIN output.

Add new BUFFERS option to report query buffer usage during EXPLAIN ANALYZE (Itagaki Takahiro)

This allows better query profiling for individual queries. Buffer usage is no longer reported in the output for log_statement_stats and related settings.

Add hash usage information to EXPLAIN output (Robert Haas)

Add new EXPLAIN syntax that allows options to be specified inside parentheses (Robert Haas)

This allows greater flexibility for future EXPLAIN options. The old syntax is still supported, but only for pre-existing options.

Change VACUUM FULL to rewrite the entire table and rebuild its indexes, rather than moving individual rows around to compact space (Itagaki Takahiro, Tom Lane)

The previous method was usually slower and caused index bloat. Note that the new method will use more disk space transiently during VACUUM FULL; potentially as much as twice the space normally occupied by the table and its indexes.

Add new VACUUM syntax that allows options to be specified inside parentheses (Itagaki Takahiro)

This allows greater flexibility for future VACUUM options. The old syntax is still supported, but only for pre-existing options.

Allow an index to be named automatically by omitting the index name in CREATE INDEX (Tom Lane)

By default, multicolumn indexes are now named after all their columns; and index expression columns are now named based on their expressions (Tom Lane)

Reindexing shared system catalogs is now fully transactional and crash-safe (Tom Lane)

Formerly, reindexing a shared index was only allowed in standalone mode, and a crash during the operation could leave the index in worse condition than it was before.

Add point_ops operator class for GiST (Teodor Sigaev)

This feature permits GiST indexing of point columns. The index can be used for several types of queries such as point <@ polygon (point is in polygon). This should make many PostGIS™ queries faster.

Use red-black binary trees for GIN index creation (Teodor Sigaev)

Red-black trees are self-balancing. This avoids slowdowns in cases where the input is in nonrandom order.

Add prefix support in synonym dictionaries (Teodor Sigaev)

Add filtering dictionaries (Teodor Sigaev)

Filtering dictionaries allow tokens to be modified then passed to subsequent dictionaries.

Allow underscores in email-address tokens (Teodor Sigaev)

Use more standards-compliant rules for parsing URL tokens (Tom Lane)

Allow aggregate functions to use ORDER BY (Andrew Gierth)

For example, this is now supported: array_agg(a ORDER BY b). This is useful with aggregates for which the order of input values is significant, and eliminates the need to use a nonstandard subquery to determine the ordering.

Multi-argument aggregate functions can now use DISTINCT (Andrew Gierth)

Add the string_agg() aggregate function to combine values into a single string (Pavel Stehule)

Aggregate functions that are called with DISTINCT are now passed NULL values if the aggregate transition function is not marked as STRICT (Andrew Gierth)

For example, agg(DISTINCT x) might pass a NULL x value to agg(). This is more consistent with the behavior in non-DISTINCT cases.

Add get_bit() and set_bit() functions for bit strings, mirroring those for bytea (Leonardo F)

Implement OVERLAY() (replace) for bit strings and bytea (Leonardo F)

Add pg_table_size() and pg_indexes_size() to provide a more user-friendly interface to the pg_relation_size() function (Bernd Helmle)

Add has_sequence_privilege() for sequence permission checking (Abhijit Menon-Sen)

Update the information_schema views to conform to SQL:2008 (Peter Eisentraut)

Make the information_schema views correctly display maximum octet lengths for char and varchar columns (Peter Eisentraut)

Speed up information_schema privilege views (Joachim Wieland)

Support execution of anonymous code blocks using the DO statement (Petr Jelinek, Joshua Tolley, Hannu Valtonen)

This allows execution of server-side code without the need to create and delete a temporary function definition. Code can be executed in any language for which the user has permissions to define a function.

Implement SQL-standard-compliant per-column triggers (Itagaki Takahiro)

Such triggers are fired only when the specified column(s) are affected by the query, e.g. appear in an UPDATE's SET list.

Add the WHEN clause to CREATE TRIGGER to allow control over whether a trigger is fired (Itagaki Takahiro)

While the same type of check can always be performed inside the trigger, doing it in an external WHEN clause can have performance benefits.

Install PL/pgSQL by default (Bruce Momjian)

The language can still be removed from a particular database if the administrator has security or performance concerns about making it available.

Improve handling of cases where PL/pgSQL variable names conflict with identifiers used in queries within a function (Tom Lane)

The default behavior is now to throw an error when there is a conflict, so as to avoid surprising behaviors. This can be modified, via the configuration parameter plpgsql.variable_conflict or the per-function option #variable_conflict, to allow either the variable or the query-supplied column to be used. In any case PL/pgSQL will no longer attempt to substitute variables in places where they would not be syntactically valid.

Make PL/pgSQL use the main lexer, rather than its own version (Tom Lane)

This ensures accurate tracking of the main system's behavior for details such as string escaping. Some user-visible details, such as the set of keywords considered reserved in PL/pgSQL, have changed in consequence.

Avoid throwing an unnecessary error for an invalid record reference (Tom Lane)

An error is now thrown only if the reference is actually fetched, rather than whenever the enclosing expression is reached. For example, many people have tried to do this in triggers:

if TG_OP = 'INSERT' and NEW.col1 = ... then

This will now actually work as expected.

Improve PL/pgSQL's ability to handle row types with dropped columns (Pavel Stehule)

Allow input parameters to be assigned values within PL/pgSQL functions (Steve Prentice)

Formerly, input parameters were treated as being declared CONST, so the function's code could not change their values. This restriction has been removed to simplify porting of functions from other DBMSes that do not impose the equivalent restriction. An input parameter now acts like a local variable initialized to the passed-in value.

Improve error location reporting in PL/pgSQL (Tom Lane)

Add count and ALL options to MOVE FORWARD/BACKWARD in PL/pgSQL (Pavel Stehule)

Allow PL/pgSQL's WHERE CURRENT OF to use a cursor variable (Tom Lane)

Allow PL/pgSQL's OPEN cursor FOR EXECUTE to use parameters (Pavel Stehule, Itagaki Takahiro)

This is accomplished with a new USING clause.

Add new PL/Perl functions: quote_literal(), quote_nullable(), quote_ident(), encode_bytea(), decode_bytea(), looks_like_number(), encode_array_literal(), encode_array_constructor() (Tim Bunce)

Add server parameter plperl.on_init to specify a PL/Perl initialization function (Tim Bunce)

plperl.on_plperl_init and plperl.on_plperlu_init are also available for initialization that is specific to the trusted or untrusted language respectively.

Support END blocks in PL/Perl (Tim Bunce)

END blocks do not currently allow database access.

Allow use strict in PL/Perl (Tim Bunce)

Perl strict checks can also be globally enabled with the new server parameter plperl.use_strict.

Allow require in PL/Perl (Tim Bunce)

This basically tests to see if the module is loaded, and if not, generates an error. It will not allow loading of modules that the administrator has not preloaded via the initialization parameters.

Allow use feature in PL/Perl if Perl version 5.10 or later is used (Tim Bunce)

Verify that PL/Perl return values are valid in the server encoding (Andrew Dunstan)

Add Unicode support in PL/Python (Peter Eisentraut)

Strings are automatically converted from/to the server encoding as necessary.

Improve bytea support in PL/Python (Caleb Welton)

Bytea values passed into PL/Python are now represented as binary, rather than the PostgreSQL bytea text format. Bytea values containing null bytes are now also output properly from PL/Python. Passing of boolean, integer, and float values was also improved.

Support arrays as parameters and return values in PL/Python (Peter Eisentraut)

Improve mapping of SQL domains to Python types (Peter Eisentraut)

Add Python 3 support to PL/Python (Peter Eisentraut)

The new server-side language is called plpython3u. This cannot be used in the same session with the Python 2 server-side language.

Improve error location and exception reporting in PL/Python (Peter Eisentraut)

Add support for quoting/escaping the values of psql variables as SQL strings or identifiers (Pavel Stehule, Robert Haas)

For example, :'var' will produce the value of var quoted and properly escaped as a literal string, while :"var" will produce its value quoted and escaped as an identifier.

Ignore a leading UTF-8-encoded Unicode byte-order marker in script files read by psql (Itagaki Takahiro)

This is enabled when the client encoding is UTF-8. It improves compatibility with certain editors, mostly on Windows, that insist on inserting such markers.

Fix psql --file - to properly honor --single-transaction (Bruce Momjian)

Avoid overwriting of psql's command-line history when two psql sessions are run concurrently (Tom Lane)

Improve psql's tab completion support (Itagaki Takahiro)

Show \timing output when it is enabled, regardless of « quiet » mode (Peter Eisentraut)

Make pg_dump/pg_restore --clean also remove large objects (Itagaki Takahiro)

Fix pg_dump to properly dump large objects when standard_conforming_strings is enabled (Tom Lane)

The previous coding could fail when dumping to an archive file and then generating script output from pg_restore.

pg_restore now emits large-object data in hex format when generating script output (Tom Lane)

This could cause compatibility problems if the script is then loaded into a pre-9.0 server. To work around that, restore directly to the server, instead.

Allow pg_dump to dump comments attached to columns of composite types (Taro Minowa (Higepon))

Make pg_dump --verbose output the pg_dump and server versions in text output mode (Jim Cox, Tom Lane)

These were already provided in custom output mode.

pg_restore now complains if any command-line arguments remain after the switches and optional file name (Tom Lane)

Previously, it silently ignored any such arguments.

Allow pg_ctl to be used safely to start the postmaster during a system reboot (Tom Lane)

Previously, pg_ctl's parent process could have been mistakenly identified as a running postmaster based on a stale postmaster lock file, resulting in a transient failure to start the database.

Give pg_ctl the ability to initialize the database (by invoking initdb) (Zdenek Kotala)

Add new libpq functions PQconnectdbParams() and PQconnectStartParams() (Guillaume Lelarge)

These functions are similar to PQconnectdb() and PQconnectStart() except that they accept a null-terminated array of connection options, rather than requiring all options to be provided in a single string.

Add libpq functions PQescapeLiteral() and PQescapeIdentifier() (Robert Haas)

These functions return appropriately quoted and escaped SQL string literals and identifiers. The caller is not required to pre-allocate the string result, as is required by PQescapeStringConn().

Add support for a per-user service file (.pg_service.conf), which is checked before the site-wide service file (Peter Eisentraut)

Properly report an error if the specified libpq service cannot be found (Peter Eisentraut)

Add TCP keepalive settings in libpq (Tollef Fog Heen, Fujii Masao, Robert Haas)

Keepalive settings were already supported on the server end of TCP connections.

Avoid extra system calls to block and unblock SIGPIPE in libpq, on platforms that offer alternative methods (Jeremy Kerr)

When a .pgpass-supplied password fails, mention where the password came from in the error message (Bruce Momjian)

Load all SSL certificates given in the client certificate file (Tom Lane)

This improves support for indirectly-signed SSL certificates.

Add SQLDA (SQL Descriptor Area) support to ecpg (Boszormenyi Zoltan)

Add the DESCRIBE [ OUTPUT ] statement to ecpg (Boszormenyi Zoltan)

Add an ECPGtransactionStatus function to return the current transaction status (Bernd Helmle)

Add the string data type in ecpg Informix-compatibility mode (Boszormenyi Zoltan)

Allow ecpg to use new and old variable names without restriction (Michael Meskes)

Allow ecpg to use variable names in free() (Michael Meskes)

Make ecpg_dynamic_type() return zero for non-SQL3 data types (Michael Meskes)

Previously it returned the negative of the data type OID. This could be confused with valid type OIDs, however.

Support long long types on platforms that already have 64-bit long (Michael Meskes)

New Makefile targets world, install-world, and installcheck-world (Andrew Dunstan)

These are similar to the existing all, install, and installcheck targets, but they also build the HTML documentation, build and test contrib, and test server-side languages and ecpg.

Add data and documentation installation location control to PGXS Makefiles (Mark Cave-Ayland)

Add Makefile rules to build the PostgreSQL™ documentation as a single HTML file or as a single plain-text file (Peter Eisentraut, Bruce Momjian)

Support compiling on 64-bit Windows™ and running in 64-bit mode (Tsutomu Yamada, Magnus Hagander)

This allows for large shared memory sizes on Windows™.

Support server builds using Visual Studio 2008™ (Magnus Hagander)

Require Autoconf 2.63 to build configure (Peter Eisentraut)

Require Flex 2.5.31 or later to build from a CVS checkout (Tom Lane)

Require Perl version 5.8 or later to build from a CVS checkout (John Naylor, Andrew Dunstan)

Use a more modern API for Bonjour (Tom Lane)

Bonjour support now requires OS X™ 10.3 or later. The older API has been deprecated by Apple.

Add spinlock support for the SuperH™ architecture (Nobuhiro Iwamatsu)

Allow non-GCC compilers to use inline functions if they support them (Kurt Harriman)

Remove support for platforms that don't have a working 64-bit integer data type (Tom Lane)

Restructure use of LDFLAGS to be more consistent across platforms (Tom Lane)

LDFLAGS is now used for linking both executables and shared libraries, and we add on LDFLAGS_EX when linking executables, or LDFLAGS_SL when linking shared libraries.

Make backend header files safe to include in C++™ (Kurt Harriman, Peter Eisentraut)

These changes remove keyword conflicts that previously made C++™ usage difficult in backend code. However, there are still other complexities when using C++™ for backend functions. extern "C" { } is still necessary in appropriate places, and memory management and error handling are still problematic.

Add AggCheckCallContext() for use in detecting if a C™ function is being called as an aggregate (Hitoshi Harada)

Change calling convention for SearchSysCache() and related functions to avoid hard-wiring the maximum number of cache keys (Robert Haas)

Existing calls will still work for the moment, but can be expected to break in 9.1 or later if not converted to the new style.

Require calls of fastgetattr() and heap_getattr() backend macros to provide a non-NULL fourth argument (Robert Haas)

Custom typanalyze functions should no longer rely on VacAttrStats.attr to determine the type of data they will be passed (Tom Lane)

This was changed to allow collection of statistics on index columns for which the storage type is different from the underlying column data type. There are new fields that tell the actual datatype being analyzed.

Add parser hooks for processing ColumnRef and ParamRef nodes (Tom Lane)

Add a ProcessUtility hook so loadable modules can control utility commands (Itagaki Takahiro)

Add contrib/pg_upgrade to support in-place upgrades (Bruce Momjian)

This avoids the requirement of dumping/reloading the database when upgrading to a new major release of PostgreSQL, thus reducing downtime by orders of magnitude. It supports upgrades to 9.0 from PostgreSQL 8.3 and 8.4.

Add support for preserving relation relfilenode values during binary upgrades (Bruce Momjian)

Add support for preserving pg_type and pg_enum OIDs during binary upgrades (Bruce Momjian)

Move data files within tablespaces into PostgreSQL™-version-specific subdirectories (Bruce Momjian)

This simplifies binary upgrades.