Postgres FM

Tip: watch on YouTube. Since episode 193, the video shows charts and illustrations as we talk. https://www.youtube.com/watch?v=gAdhpYewLZ8 Nik and Kirk discuss the hidden costs of CREATE TEMP TABLE: dead tuples and catalog bloat, and the shared invalidation queue that every session has to read. They cover why thousands of idle sessions make it much worse, what it looks like in wait events, and what to use instead. Here are some links to things they mentioned: Times are podcast times of the first mention.Main link[00:34:15] Temp tables in the request path: where the cliff is, and what LWLock:SInvalWrite really tells you (DBLab Forge brief, PostgresAI) https://console.postgres.ai/shared/brief/dfe99713-f5a8-483b-9aab-1606a883b75dMichael[00:00:44] Michael's post about taking a few months off (X) https://x.com/michristofides/status/2117283823314674021Episodes[00:06:12] UUIDv4 vs UUIDv7 https://postgres.fm/episodes/uuidv4-vs-uuidv7Postgres docs[00:01:32] CREATE TABLE, Compatibility: temporary tables vs the SQL standard's global temporary tables (docs) https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-COMPATIBILITY[00:04:34] CREATE TABLE, TEMPORARY: autovacuum cannot vacuum or analyze temporary tables (docs) https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-TEMPORARY[00:15:40] LWLock wait events, including SInvalWrite and SInvalRead (docs) https://www.postgresql.org/docs/current/monitoring-stats.html#WAIT-EVENT-LWLOCK-TABLE[00:19:16] max_locks_per_transaction (docs) https://www.postgresql.org/docs/current/runtime-config-locks.html#GUC-MAX-LOCKS-PER-TRANSACTION[00:27:14] CREATE TABLE, ON COMMIT: PRESERVE ROWS, DELETE ROWS, DROP (docs) https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-PARMS-ON-COMMIT[00:32:24] CREATE TABLE, UNLOGGED (docs) https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-UNLOGGED[00:37:45] DISCARD (docs) https://www.postgresql.org/docs/current/sql-discard.htmlPostgres source and history[00:18:11] sinvaladt.c: MAXNUMMESSAGES, the 4,096-message invalidation queue https://github.com/postgres/postgres/blob/3d342deeb433060dbea02fdd276656a5563275d5/src/backend/storage/ipc/sinvaladt.c[00:18:49] Increase the number of fast-path lock slots (Tomas Vondra, 2024, Postgres 18) https://github.com/postgres/postgres/commit/c4d5cb71d229095a39fda1121a75ee40e6069a2a[00:23:38] Postgres 14 release notes: better scalability with many connections (Andres Freund's work) https://www.postgresql.org/docs/release/14.0/[00:35:43] Rewrite the sinval messaging mechanism to reduce contention (Tom Lane, 2008) https://github.com/postgres/postgres/commit/fad153ec45299bd4d4f29dec8d9e04e2f1c08148Tools[00:13:10] pg_repack https://github.com/reorg/pg_repack[00:20:24] PgBouncer, server_reset_query (default DISCARD ALL) https://www.pgbouncer.org/config.html#server_reset_query[00:24:06] Amazon RDS Performance Insights https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PerfInsights.html[00:24:12] pg_ash, active session history for Postgres https://github.com/NikolayS/pg_ashOther[00:24:39] Linux Performance Tools, talk (Brendan Gregg) https://youtu.be/FJW8nGV4jxY[00:36:41] Don't Do This: Don't create temporary tables in frequently executed code (PostgreSQL wiki) https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_create_temporary_tables_in_frequently_executed_codeCorrections (checked against the Postgres source after recording)[00:13:58] Two temp tables and an index are not "three records in, three out". Each temp table adds at least 13 catalog rows (pg_class, pg_type, pg_attribute, pg_depend), about 31 if it has a text column, and the index adds more. So 10M queries a day means at least 600 million catalog row changes, not 60M.[00:15:47] sinval means shared invalidation (of the catalog and relation caches), not "snapshot invalidation".[00:18:49] LWLock:LockManager: up to Postgres 17 a session has 16 fast-path lock slots; Postgres 18 has 64 by default and scales them with max_locks_per_transaction. But temp-table DDL takes AccessExclusiveLock, which never uses the fast path, so raising the setting does not make this wait go away (the brief measured 6.6 to 12.7% LockManager with it set to 1024).[00:25:09] The invalidation queue is a circular buffer of 4,096 messages, but writers never wait for readers. When it is half full (2,048 messages), the session that is furthest behind gets a catch-up signal, and it passes the signal on, one session at a time. When it is full, sessions that have not read are reset and must rebuild all their caches. Writers wait on SInvalWriteLock because there is one such lock for the whole cluster, and both a write (which flags every session) and a queue cleanup (which scans every session) hold it.[00:27:26] ON COMMIT DELETE ROWS does not make dead tuples: at commit it truncates the table and its indexes. And ON COMMIT DROP is not slow mostly because of more messages: without idle sessions the cost is catalog changes and file create and unlink; sinval is under 1%.Clarifications[00:04:02] Temp table data is invisible to other sessions, but the definition and statistics are visible (pg_class, pg_stat_all_tables).[00:05:32] The docs glossary defines bloat, and the definition includes dead row versions.[00:14:56] Wraparound: load is not the condition. Autovacuum never freezes temp tables, so any long-lived temp table holds back datfrozenxid.[00:17:05] "Catalog is global": the catalog is per database and ...

Tip: watch on YouTube. Since episode 193, the video shows charts and illustrations as we talk. https://www.youtube.com/watch?v=7oeAAqU00zk Nik and Kirk are joined by Álvaro Hernández to discuss how Postgres in containers gets its extensions: a fat image with all of them, extensions downloaded at run time, or StackGres Containers, built on DOCIR (Dynamic OCI Registry), which composes an image with only the extensions you ask for, on demand. They also discuss what this means for security, pinning versions, and major upgrades. Here are some links to things they mentioned: Times are podcast times of the first mention.Main link[00:18:54] Dynamic containers for PostgreSQL: what we measured (DBLab Forge brief, PostgresAI) https://console.postgres.ai/shared/brief/c501d438-b872-4705-89a6-bdab30ea1680Álvaro and StackGres[00:00:25] Álvaro Hernández https://postgres.fm/people/alvaro-hernandez-tortosa[00:01:16] StackGres Containers (public beta; the images come from the registry host sgcr.io, and the demo used sgcr.dev, a dev instance) https://stackgres.io/containers[00:14:52] StackGres https://stackgres.io[00:20:55] PostgreSQL extensions in StackGres, loaded into running pods (docs) https://stackgres.io/doc/latest/features/extensions/[00:24:28] StackGres extensions cache, experimental (docs) https://stackgres.io/doc/latest/administration/extensions/cache/[01:04:08] Why Postgres extensions should be packaged and distributed as OCI images (Álvaro Hernández, 2024) https://ongres.com/blog/why-postgres-extensions-should-be-distributed-and-packaged-as-oci-images/Episodes[00:00:25] Our previous episode with Álvaro: Citus + StackGres https://postgres.fm/episodes/citus-stackgresHacking sessions[00:31:33] Hacking Postgres sessions with Andrey, Kirk and Nik (YouTube playlist, PostgresTV) https://www.youtube.com/playlist?list=PLH8y1BNPAKjKDdJA7sDmFWUkbbYux3TRzPostgres and extensions[00:38:51] extension_control_path, new in Postgres 18 (docs) https://www.postgresql.org/docs/18/runtime-config-client.html#GUC-EXTENSION-CONTROL-PATH[00:50:56] pg_background https://github.com/vibhorkum/pg_background[00:50:58] plsh (Peter Eisentraut) https://github.com/petere/plsh[01:02:58] PGXN, the PostgreSQL Extension Network https://pgxn.org[01:04:40] POC: Distributing Trunk binaries via OCI (David E. Wheeler, 2024) https://justatheory.com/2024/06/trunk-oci-poc/[01:05:12] Anarchy in the Database: a survey and evaluation of DBMS extensibility (Kim, Slot, Andersen, Pavlo, VLDB 2025; tests extensions in pairs) https://www.vldb.org/pvldb/vol18/p1962-kim.pdfContainers[00:01:22] OCI image format specification https://github.com/opencontainers/image-spec[00:41:19] Spilo (Zalando), one Postgres image with several major versions https://github.com/zalando/spilo[01:05:43] Testcontainers, PostgreSQL module https://testcontainers.com/modules/postgresql/Other[00:36:32] Mitchell Hashimoto on never updating dependencies unless it breaks (post on X, 2026-05-20) https://x.com/mitchellh/status/2057171518027887035~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of PostgresAIWith credit to:Jessie Draws for the elephant artworkIntro and outro music: our own arrangement of Apple Loops from GarageBand (royalty-free under the GarageBand licence)

Tip: watch on YouTube. Since episode 193, the video shows charts and illustrations as we talk. https://www.youtube.com/watch?v=gf8e3JkQ_8M Nik is joined by Kirk Wolak to discuss what a random UUIDv4 primary key really costs compared with a time-ordered UUIDv7 or an int8 key on Postgres 18 — index size, WAL and full-page writes, insert rate once the index outgrows memory — and what DBLab Forge found when it ran the experiment. Here are some links to things they mentioned: Times are podcast times of the first mention.Main link[00:00:34] UUIDv4 vs UUIDv7 vs int8 primary keys on Postgres 18 (DBLab Forge brief, PostgresAI) https://console.postgres.ai/shared/brief/9fffe2da-00c1-470b-a79d-a210ae2306bcHacking sessions[00:00:51] Hacking Postgres 101: ULID function (Postgres.TV, 2023) https://www.youtube.com/watch?v=YPq_hiOE-N8[00:00:51] Hacking Postgres sessions with Andrey, Kirk and Nik (YouTube playlist, PostgresTV) https://www.youtube.com/playlist?list=PLH8y1BNPAKjKDdJA7sDmFWUkbbYux3TRzEpisodes[00:05:53] Our previous episode with Kirk, also built on DBLab Forge: Dump/restore done. Now what? https://postgres.fm/episodes/dump-restore-done-now-whatPostgres[00:01:34] Commit adding uuidv7() to Postgres 18 (78c5e141e9c1, 2024-12-11) https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=78c5e141e9c1[00:04:16] UUID functions, uuidv7() and uuid_extract_timestamp() (docs) https://www.postgresql.org/docs/18/functions-uuid.html[00:20:42] full_page_writes (docs) https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-FULL-PAGE-WRITES[00:25:01] wal_compression, lz4 and zstd (docs) https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-WAL-COMPRESSION[00:31:07] pg_stat_io (docs) https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-IO-VIEWStandards and ID schemes[00:01:20] RFC 9562, Universally Unique IDentifiers (UUIDv7) https://www.rfc-editor.org/rfc/rfc9562[00:28:05] ULID spec https://github.com/ulid/spec[00:41:29] Snowflake ID (Twitter) https://github.com/twitter-archive/snowflake[00:41:29] Sharding & IDs at Instagram https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5cArticles, books and tools[00:05:34] Sequential UUID generators (Tomas Vondra, 2018) https://www.enterprisedb.com/blog/sequential-uuid-generators[00:05:59] Systems Performance, 2nd edition (Brendan Gregg) https://www.brendangregg.com/systems-performance-2nd-edition-book.html[00:12:25] fio https://github.com/axboe/fio[00:26:17] Amazon Aurora pricing (I/O charges) https://aws.amazon.com/rds/aurora/pricing/[00:32:53] UUIDv7 doesn't have the properties you think it does (Robert Alexander, 2026) https://alexsci.com/blog/uuidv7-is-weird/[00:32:53] Hacker News discussion of that article https://news.ycombinator.com/item?id=50211746~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of PostgresAIWith credit to:Jessie Draws for the elephant artworkIntro and outro music: our own arrangement of Apple Loops from GarageBand (royalty-free under the GarageBand licence)

Nik is joined by Kirk Wolak, Software Architect at KiraSoft, to discuss what a database still needs after pg_restore finishes and before it takes production traffic — what pg_dump and pg_restore leave behind, why autovacuum may not catch up in time, and how Postgres itself could do better. Here are some links to things they mentioned: Times are podcast times of the first mention.Hacking sessions[00:00:27] Hacking Postgres sessions with Andrey, Kirk and Nik (YouTube playlist, PostgresTV) https://www.youtube.com/playlist?list=PLH8y1BNPAKjKDdJA7sDmFWUkbbYux3TRz[00:00:27] Hacking Postgres website https://hacking.postgres.tv[00:00:27] Hacking Postgres Telegram group https://t.me/postgres_hackingEpisodes[00:00:27] Our previous episode with Kirk: GSoC B-Tree Merge https://postgres.fm/episodes/gsoc-btree-merge[00:03:23] Is pg_dump a backup tool? https://postgres.fm/episodes/is-pg_dump-a-backup-toolpgsql-hackers[00:15:39] pg_restore --include-analyze proposal (Jian He, 2025) https://www.postgresql.org/message-id/flat/CACJufxGx_YLLxmnZVMuUApiVYqjKrydvEQqD3qxhwM6e0_8jrQ%40mail.gmail.com[00:30:13] ANALYZE after restore (Christopher Kings-Lynne, 2002) https://www.postgresql.org/message-id/GNELIHDDFBOCMGBFGEFOOEOPCBAA.chriskl%40familyhealth.com.au[00:31:55] Statistics Import and Export (PG18 statistics transfer) https://www.postgresql.org/message-id/flat/CADkLM%3DfzX7QX6r78fShWDjNN3Vcr4PVAnvXxQ4DiGy6V%3D0bCUA%40mail.gmail.comCommits[00:04:39] Transfer statistics during pg_upgrade (adds statistics to pg_dump, PG18) https://github.com/postgres/postgres/commit/1fd1bd871012732e3c6c482667d2f2c56f1a9395[00:30:13] vacuumdb --analyze-only processes partitioned tables (Laurenz Albe, PG19) https://github.com/postgres/postgres/commit/6429e5b771dbb98aa56441677b79cdb58f6e1675[00:42:38] Set pd_prune_xid on insert (Melanie Plageman, PG19) https://github.com/postgres/postgres/commit/378a216187aea1b488ce60ed07dd1ac5c14a9984~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of PostgresAIWith credit to:Jessie Draws for the elephant artworkIntro and outro music: our own arrangement of Apple Loops from GarageBand (royalty-free under the GarageBand licence)

Nik and Michael are joined by Álvaro Hernández to discuss sharding Postgres with Citus and StackGres, especially the new query router support that has been added very recently. Here are some links to things they mentioned: Álvaro Hernández https://postgres.fm/people/alvaro-hernandez-tortosaOngres https://ongres.comScaling Citus without limits: announcing Query Routers in StackGres https://stackgres.io/blog/scaling-citus-beyond-one-coordinator-announcing-query-routers-stackgres/Open-source Citus matches PlanetScale's Neki at 10M QPS on half the data nodes https://stackgres.io/blog/open-source-citus-matches-planetscales-neki-at-10m-qps-on-half-the-data-nodes/Postgres TV discussion about licenses with Álvaro https://www.youtube.com/watch?v=1rcbyIjA4gIOur episode with Sugu from Multigres https://postgres.fm/episodes/multigresOur (first) episode with Lev from PgDog https://postgres.fm/episodes/pgdogCitus https://github.com/citusdata/citusCitus goes fully open source (2022) https://www.citusdata.com/blog/2022/06/17/citus-11-goes-fully-open-source/6M requests per second using ClickHouse (blog post by Alex Bocharov / Cloudflare) https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse/pg_ash https://github.com/NikolayS/pg_ash118 million queries per second on Neki (blog post by Florent Poinsard and Hirad Pourtahmasbi / PlanetScale) https://planetscale.com/blog/118-million-queries-per-second-on-nekiAnalyzing the Limits of Connection Scalability in Postgres (blog post by Andres Freund) https://www.citusdata.com/blog/2020/10/08/analyzing-connection-scalability/~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of Postgres.aiWith credit to:Jessie Draws for the elephant artwork