EDB Postgres Distributed 5.6.0 release notes v5.6

One of the main advantages of streaming is that the WAL sender sends the partial transaction before it commits, which reduces replication lag. Now, with streaming support, the WAL decoder does the same thing, but it streams to the LCRs segments. Eventually, the WAL sender will read the LCRs and mimic the same behavior of streaming large transactions before they commit. This provides the benefits of decoding worker, such as reduced CPU and disk space, as well as the benefits of streaming, such as reduced lag and disk space, since ".spill" files are not generated. The WAL decoder always streams the transaction to LCRs, but based on downstream requests, the WAL sender either streams the transaction or just mimics the normal BEGIN..COMMIT scenario. In addition to the normal LCRs segment files, we create streaming files with the starting names TR_TXN_<file-name-format> and CAS_TXN_<file-name-format> for each streamed transaction.

There are several view providing new information as well as making some existing information easier to discover:

• bdr.stat_commit_scope : Cumulative statistics for commit scopes.
• bdr.stat_commit_scope_state : Information about current use of commit scopes by backends.
• bdr.stat_receiver : Per subscription receiver statistics.
• bdr.stat_writer : Per writer statistics. There can be multiple writers for each subscription. This also includes additional information about the currently applied transaction.
• bdr.stat_raft_state : The state of the Raft consensus on the local node.
• bdr.stat_raft_followers_state : The state of the followers on the Raft leader node (empty on other nodes), also includes approximate clock drift between nodes.
• bdr.stat_worker : Detailed information about PGD workers, including what the operation manager worker is currently doing.
• bdr.stat_routing_state : The state of the connection routing which PGD Proxy uses to route the connections.
• bdr.stat_routing_candidate_state : Information about routing candidate nodes on the Raft leader node (empty on other nodes).

This allows defining EXCLUDE constraint on table replicated by PGD either withCREATE TABLE or with ALTER TABLE and uses similar conflict detection to resolve conflicts as for UNIQUE constraints.

This will cancel synchronous replication wait on disconnected sessions if it deadlocks against replication, preventing deadlocks on failovers when using synchronous replication. This only affects commit scopes, not synchronous replication configured via synchronous_standby_names.

Fix: Correct privileges for bdr_superuser. Creating wrapper SECURITY DEFINER functions in the bdr schema and granting access to bdr_superuser to use those:

• bdr.bdr_show_all_file_settings
• bdr.bdr_file_settings

Add functions for creating and dropping commit scopes that will eventually deprecate the non-standard functions for adding and removing commit scopes. Notify the user that these will be deprecated in a future version, suggesting the use of the new versions.

Permissions for the following objects have been updated to include SELECT permissions for role "bdr_monitor": bdr.node_config

This update introduces GUCs to regulate the frequency of automatic vacuuming on the specified catalogs. The GUC bdr.raft_vacuum_interval determines the frequency at which tables are examined for VACUUM and ANALYZE. Autovacuum GUCs and table reloptions are utilized to ascertain the necessity of VACUUM/ANALYZE. The bdr.raft_vacuum_full_interval initiates VACUUM FULL on the tables. Users have the ability to deactivate VACUUM FULL if regular VACUUM suffices to manage bloat.

Add "node_name" to the view "bdr.node_config_summary". This makes it consistent with other summary views, which report the name of the object (node, group, etc.) for which the summary is being generated.

Ensure that bdr_init_physical emits details about connection failure if the "--local-dsn" parameter is syntactically correct but invalid, e.g., due to an incorrect host or port setting.

bdr_config now shows the PostgreSQL "flavor" which BDR was built against, one of:

• COMMUNITY
• EPAS
• EXTENDED
• BDRPG

Enhance messages issued during DML and DDL lock acquisition.

Avoid sending Raft snapshots too frequently as it can slow down follower nodes. Limit the snapshot rate to once in every election timeout, unless there is no other communication between the nodes, in which case send a snapshot every 1/3rd of the election timeout. This will help all nodes keep pace with the leader and improve CPU utilization.

It is now possible to set akk top-level and subgroup level options. The following options are available for top-both groups:

• check_constraints
• enable_wal_decoder
• num_writers
• streaming_mode
• enable_raft Subgroups inherit settings from their parent group, but can override them if set in the subgroup.

Subscriber-only node groups have a leader elected by top-level Raft. There is now a bdr.leader catalog that tracks leadership of subgroups and subscriber-only nodes. If the node that is the leader of a subscriber-only node group goes down or becomes unreachable, a new leader is elected from that group.

Subscriber-only nodes earlier used to have subscriptions to each data node. Now if optimized topology is enabled, only the leaders of subscriber-only node groups have subscriptions to routing leaders of data node subsgroups. The subscriber only nodegroup leaders route data to other nodes of that subscriber-only nodegroup. This reduces the load on all data nodes so they do not have to send data to all subscriber-only nodes. The GUC bdr.force_full_mesh=false enables this optimized topology. It is off by default.

New subscription types that forward data from all nodes of the subgroup via a routing leader (mode: l), and those that forward data from the entire cluster via a subscriber-only group leader (mode: w) are introduced.

A write leader has a version. Every time a new leader is elected, the version is incremented and timestamp noted via Raft. This is to build a foundation for better conflict resolution.

Using column references in default expressions is now supported, this is particularly useful with generated columns, for example:ALTER TABLE gtest_tableoid ADD COLUMN c regclass GENERATED ALWAYS AS (tableoid) STORED;

Both REINDEX and REINDEX CONCURRENTLY are now replicated commands.

Receiver worker could get stuck when exiting, waiting for a writer that never actually started. This could on rare occasions break replication after configuration changes until Postgres was restarted.

Changes to values of variables bdr.last_committed_lsn, transaction_idand bdr.local_node_id are automatically reported to clients when using CAMO or GROUP COMMIT. This has now been optimized to use less resources.

When there is a commit scope defined for top-level group, it can be used by any node in a subgroup and does not need to be redefined for every subgroup anymore. This is particularly useful when combined with ORIGIN\_GROUPkeyword to reduce the complexity of commit scope setup.

Use bdr.bdr_file_settings view to get the current settings for the proxy.

Include an extra zero byte at the end of a column value allocation in shared memory queue insert/update/delete messages.

When ALTER TABLE .. DROP COLUMN is used, the object_access_hook is fired with classId set to RelationRelationId, but the subId is set to the attribute number to differentiate it from the DROP TABLE command.

Therefore, we need to check the subId field to make sure that we are not performing actions that should only be triggered when a table is dropped.

Ensure that the propagation of bdr.alter_table_conflict_detection() (as well as the related, deprecated bdr.column_timestamps_(en|dis)able() functions) is carried out correctly to all logical standbys. Previously, this propagation did not occur if the logical standby was not directly attached to the node on which the functions were executed.

Ensure that a nodegroup is not inadvertently created with the same name as an existing nodegroup. Failure to do so may result in a complete shutdown of the top-level Raft on all nodes, with no possibility of recovery.

Handle the absence of the expected node record gracefully when a node is being removed, the local node record might have already been deleted, but an attempt could be made to update it anyway. This resulted in harmless "BDR node local info for node ... not found" errors.

Improve Raft error messages to handle cases where nodes may not be correctly participating in Raft.

When processing RAFT entries, it's crucial to handle duplicate requests properly to prevent Raft protocol issues. Duplicate requests can occur when a client retries a request that has already been accepted and applied by the Raft leader. The problem arose when the leader failed to detect the duplicate request due to historical evidence being pruned.

When installing or importing a Raft snapshot, discard the consensus log unless it contains an entry matching the snapshot's last included entry and term.

This fixes various index related errors during replication like: 'could not lookup equality operator for type, optype in opfamily' or 'function "amgettuple" is not defined for index "brinidx"'

The createrole_self_grant configuration option affects inherited grants by newly created roles. In previous versions CREATE ROLE/CREATE USERreplication would not take this into consideration, resulting in different role privileges on different nodes.

Previously, this would throw "cannot change current role within security-restricted operation" error

This prevents errors when replicating UPDATEs for domains defined as NOT VALID where tables contain data which would not be allowed by current definition of such domain.