Oct 12, 2026/8 min readCore Platform
pg_lake 3.5: Universal Iceberg REST Catalog Support for Postgres
pg_lake 3.5 lets Postgres query Apache Iceberg™ tables through REST catalogs beyond Apache Polaris including Lakekeeper and Nessie, and use storage credentials vended by the catalog. Writable tables are currently supported on Apache Polaris only. The next pg_lake release adds automatic prefix discovery, clearer errors for a misconfigured catalog, unauthenticated catalogs for dev/test, and Unity Catalog support.
Apache Iceberg makes table data portable: Write from one engine, read from any other. The Iceberg REST Catalog specification extends that portability to metadata: a single HTTP contract for discovering namespaces, loading table metadata, and committing new snapshots.
pg_lake, Snowflake's open source Postgres extension, brings Postgres into the Iceberg ecosystem. It lets Postgres read and write Iceberg tables in object storage, participating in a lakehouse rather than sitting beside it. pg_lake has included an Iceberg REST catalog client from the start, and its URLs assume the Apache Polaris layout.
We’re excited to announce that the pg_lake 3.5, released in September 2026, and its REST catalog client are no longer tied to the Polaris URL layout.
Once Postgres can talk to the same REST catalog as the rest of your lakehouse, it stops being a separate island. A few things follow from that:
Zero-ETL between operational and analytical data
If Postgres speaks to the central Iceberg catalog, an application can write a transaction directly to the lakehouse format. Analytical engines can instantly query that exact same data without it ever being duplicated or moved. Writing through the catalog is supported on Apache Polaris today; other catalogs are read-only for now (tracked in #690).
No more reverse ETL
Postgres can now directly query the central Iceberg catalog using standard SQL. The application can instantly display advanced analytics to the user right alongside their live transactional data, without data teams having to build pipelines to sync the data back.
Ecosystem interoperability and no vendor lock-in
Developers aren't locked into a specific vendor's ecosystem. A standard Postgres application can natively query tables created and analyzed by other systems, all securely mediated by the shared catalog API.
Centralized security and governance
Instead of managing security rules in Postgres and in the data warehouse, governance can be handled at the catalog layer. When pg_lake requests access to an Iceberg table, the catalog decides whether Postgres can see it and which storage credentials it gets, the same way it does for every other engine.
One Iceberg spec, many mount paths
The Iceberg REST spec defines API paths like /v1/{prefix}/namespaces and /v1/{prefix}/tables, but it does not prescribe where these paths are mounted on a server. Each catalog implementation chooses its own base path:
| Catalog | Mount path |
|---|---|
| Apache Polaris | /api/catalog |
| Lakekeeper | /catalog |
| Nessie | /iceberg |
| Unity Catalog | /api/2.1/unity-catalog/iceberg * |
* Unity Catalog support requires the next pg_lake release (#715).
pg_lake's original implementation hardcoded /api/catalog the Polaris convention into every URL template. This meant that pointing pg_lake at a Lakekeeper or Nessie instance required either a reverse proxy or manual URL rewriting. In practice, pg_lake could only talk to catalogs that used Polaris's URL layout.
Figure 1. With a configurable base URI, pg_lake connects to Polaris, Lakekeeper and Nessie, and to Unity Catalog from the next release, using the same CREATE SERVER syntax.
Three changes that open pg_lake to every catalog
Three patches, merged between August and September 2026, removed the single-catalog assumption. PR #507 shipped in pg_lake 3.5.0; PRs #625 and #649 will ship in the next tagged release.
1. Configurable REST catalog base URI
PR #507: merged August 2026
This patch removed the hardcoded /api/catalog prefix from all URL templates. The rest_endpoint server option now accepts a full base URI including the mount path, and pg_lake appends only the spec-defined /v1/... paths.
Before:
-- Only Polaris worked; the /api/catalog prefix was implicit
CREATE SERVER my_catalog TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (
rest_endpoint 'https://polaris.example.com'
);
After:
-- Polaris: include /api/catalog in the endpoint
CREATE SERVER polaris_catalog TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (
rest_endpoint 'https://polaris.example.com/api/catalog'
);
-- Lakekeeper: include /catalog
CREATE SERVER lakekeeper_catalog TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (
rest_endpoint 'https://lakekeeper.example.com/catalog'
);
-- Nessie: include /iceberg
CREATE SERVER nessie_catalog TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (
rest_endpoint 'https://nessie.example.com/iceberg'
);This is a breaking change starting in pg_lake 3.5.0: Existing configurations that relied on the implicit /api/catalog prefix must update rest_endpoint to include it explicitly. The built-in GUC default for pg_lake_iceberg.rest_catalog_host was updated from http://localhost:8181 to http://localhost:8181/api/catalog to match.
2. Automatic catalog prefix discovery via /v1/config
PR #625: merged September 21, 2026 · next release
Iceberg REST API paths include a {prefix} segment that identifies which catalog on the server is being addressed (for example, /v1/my-catalog/namespaces/...). pg_lake exposed this as the catalog_name option and fell back to the Postgres database name when unset.
That works for Polaris, where the prefix is a human-readable catalog name. It does not work for Lakekeeper (which uses the warehouse UUID) or Nessie (which uses the branch name, such as main). Neither maps naturally to a Postgres database name.
The Iceberg REST spec already solved this problem. The GET /v1/config endpoint exists so clients can discover the server's prefix without manual configuration. Starting in the next release, pg_lake calls it automatically when catalog_name is not set:
| Situation | Behavior |
|---|---|
| catalog_name explicitly set on table or server | Used as-is (no change) |
| Not set, /v1/config returns overrides.prefix | Auto-detected |
| Not set, /v1/config returns only defaults.prefix | Auto-detected |
| Not set, no prefix in response or non-200 | Falls back to database name |
Explicit configuration always wins. Auto-detection works when the catalog returns a prefix from a plain GET /v1/config; Nessie for example, returns main. Polaris and Lakekeeper only return a prefix when the request names a warehouse, which pg_lake doesn’t send yet (#719). So for now, set catalog_name to the catalog name on Polaris and to the warehouse UUID on Lakekeeper.
3. Clear diagnostics for misconfigured catalogs
PR #649: merged September 22, 2026 · next release
With multiple catalogs now reachable, misconfiguration became more common, and pg_lake's error messages pointed at the wrong thing.
When a namespace lookup (GET /v1/{prefix}/namespaces/{ns}) returned HTTP 404, pg_lake assumed the namespace did not exist. But a 404 can also mean the catalog prefix itself is wrong. The previous behavior led to confusing outcomes:
Before (read-only table with wrong catalog name):
ERROR: namespace "my_schema" does not exist in the rest catalog while
creating on catalog "wrong_name"On a writable table it was worse: pg_lake caught the 404 from the namespace lookup and silently tried to create the namespace, which also 404'd because the catalog itself was absent, producing a confusing double failure.
After:
ERROR: catalog "wrong_name" does not exist in the rest catalog server
HINT: Create the catalog in the rest catalog server, or check the
catalog_name option.
The fix uses the spec's own semantics: GET /v1/{prefix}/namespaces (without a specific namespace) returns 200 if the catalog exists, or 404 if it does not. A new RestCatalogExists() helper runs only on the 404 code path, adding zero overhead in steady state. Non-404 errors (401, 403, 500) pass through unchanged so that authentication or server errors are never masked by an existence check.
Connect Postgres to a Lakekeeper catalog
Here's what it looks like against Lakekeeper. You need the right rest_endpoint, and, because Lakekeeper doesn't advertise its prefix to pg_lake yet, the warehouse UUID as catalog_name:
-- 1. Connect to a Lakekeeper instance
CREATE SERVER lakekeeper TYPE 'rest'
FOREIGN DATA WRAPPER iceberg_catalog
OPTIONS (
rest_endpoint 'https://lakekeeper.corp.example.com/catalog',
rest_auth_type 'oauth2'
);
CREATE USER MAPPING FOR CURRENT_USER
SERVER lakekeeper
OPTIONS (
client_id 'pg_lake_client',
client_secret 'secret'
);
-- 2. Attach an existing table (catalog_name is the Lakekeeper warehouse UUID)
CREATE TABLE iot.sensor_readings (
device_id TEXT,
reading DOUBLE PRECISION,
recorded_at TIMESTAMPTZ
)
USING iceberg WITH (
catalog = 'lakekeeper',
catalog_name = '<warehouse-uuid>',
read_only = 'true'
);
-- 3. Query it like any Postgres table
SELECT device_id, avg(reading)
FROM iot.sensor_readings
WHERE recorded_at > now() - interval '1 hour'
GROUP BY device_id;
No reverse proxy required. In the next release, a wrong catalog_name produces an error that says so, and catalogs that advertise their prefix, like Nessie, won't need catalog_name at all.
Figure 2. Connection flow from CREATE SERVER to query execution. Steps marked "next release" (prefix discovery via /v1/config and the catalog-exists check) are not in 3.5.0.
What's next for pg_lake
Three related changes are on main and will ship in the next release, alongside #625 and #649:
- Unity Catalog support (#715, merged October 8, 2026). Unity's prefix contains a slash (catalogs/<name>), and pg_lake was percent-encoding it, so Unity returned 404. The fix sends the prefix as-is, the way the Iceberg Java client does.
- Unauthenticated catalog access (#662, merged October 1, 2026) adds rest_auth_type 'none' for dev/test setups that skip authentication, which is common with Lakekeeper, Nessie and open source Unity Catalog.
- Deferred metadata deletion (#397, under review) fixes a transaction-ordering bug in writable REST catalog tables, where metadata cleanup was queued before the remote commit and caused duplicate-key errors on retries.
Two open issues cover the gaps called out above: sending warehouse on /v1/config so Polaris and Lakekeeper prefixes are auto-detected (#719), and writable tables on catalogs other than Polaris (#690).
Key takeaways
- pg_lake 3.5 works with Iceberg REST catalogs beyond Polaris, including Lakekeeper and Nessie, through a configurable rest_endpoint. Unity Catalog support follows in the next release, and writes are Polaris-only for now.
- Vended credentials (pg_lake 3.5): Read-only tables can use short-lived S3 credentials issued by the catalog.
- Coming next: Automatic prefix discovery. pg_lake will call GET /v1/config to detect the prefix on catalogs that advertise it, such as Nessie. Polaris and Lakekeeper still need catalog_name until #719 lands.
- Coming next: Actionable misconfiguration errors. A wrong catalog name will produce a clear "catalog does not exist" message with a hint, rather than a misleading "namespace does not exist" error.
- Breaking change starting in pg_lake 3.5.0: The rest_endpoint option must now include the full mount path (for example, https://host/api/catalog for Polaris). Existing bare-host configurations must be updated.
Put together, this means a Postgres application can query the same Iceberg tables your other engines use, through the Iceberg REST catalog your organization already runs.
To get started, see the pg_lake documentation and the Iceberg REST catalog specification. pg_lake is open source; issues and contributions are welcome. Thanks to Mark Nefedov for his contributions to this release.
