Partitioning
Declarative partitioning lets you split a large table into smaller physical pieces while presenting a single logical table to your application. Postgres handles routing inserts and queries to the correct partitions automatically. This is one of the most effective performance and scaling strategies for databases with largae tables.
There are three partition strategies, each suited to different data patterns.
Time Based Partitioning
Range partitioning is the most common strategy, ideal for time-series data where queries filter by date ranges.
CREATE TABLE payments ( payment_id bigint GENERATED ALWAYS AS IDENTITY, customer_id int NOT NULL, amount numeric(10,2) NOT NULL, payment_date date NOT NULL, CONSTRAINT payments_pk PRIMARY KEY (payment_id, payment_date) ) PARTITION BY RANGE (payment_date); -- Create yearly partitions CREATE TABLE payments_2023 PARTITION OF payments FOR VALUES FROM ('2023-01-01') TO ('2024-01-01'); CREATE TABLE payments_2024 PARTITION OF payments FOR VALUES FROM ('2024-01-01') TO ('2025-01-01'); CREATE TABLE payments_2025 PARTITION OF payments FOR VALUES FROM ('2025-01-01') TO ('2026-01-01'); -- Catch anything that doesn't match a defined range CREATE TABLE payments_default PARTITION OF payments DEFAULT;
Note that the partition key (payment_date) must be included in the primary key. This is a Postgres requirement — any unique constraint or primary key on a partitioned table must include the partition key columns.
List Partitioning
List partitioning works well for categorical data with a known set of values.
CREATE TABLE orders ( order_id bigint GENERATED ALWAYS AS IDENTITY, region text NOT NULL, product_id int NOT NULL, quantity int NOT NULL, order_date timestamp NOT NULL, CONSTRAINT orders_pk PRIMARY KEY (order_id, region) ) PARTITION BY LIST (region); CREATE TABLE orders_us PARTITION OF orders FOR VALUES IN ('us-east', 'us-west', 'us-central'); CREATE TABLE orders_eu PARTITION OF orders FOR VALUES IN ('eu-west', 'eu-central', 'eu-north'); CREATE TABLE orders_apac PARTITION OF orders FOR VALUES IN ('apac-east', 'apac-south'); CREATE TABLE orders_default PARTITION OF orders DEFAULT;
Hash Partitioning
Hash partitioning distributes rows evenly when there's no natural range or list to partition by. This is useful for breaking up large tables where queries always filter on a specific key.
CREATE TABLE user_sessions ( session_id uuid NOT NULL, user_id bigint NOT NULL, started_at timestamp NOT NULL, data jsonb, CONSTRAINT user_sessions_pk PRIMARY KEY (session_id, user_id) ) PARTITION BY HASH (user_id); CREATE TABLE user_sessions_p0 PARTITION OF user_sessions FOR VALUES WITH (MODULUS 4, REMAINDER 0); CREATE TABLE user_sessions_p1 PARTITION OF user_sessions FOR VALUES WITH (MODULUS 4, REMAINDER 1); CREATE TABLE user_sessions_p2 PARTITION OF user_sessions FOR VALUES WITH (MODULUS 4, REMAINDER 2); CREATE TABLE user_sessions_p3 PARTITION OF user_sessions FOR VALUES WITH (MODULUS 4, REMAINDER 3);
Quick Comparison
| Strategy | Best For | Example Use Case |
|---|---|---|
| RANGE | Time-series, sequential data | Logs, payments, IoT readings |
| LIST | Known categories | Regional data, status codes |
| HASH | Even distribution, no natural key | User sharding, session data |
Postgres automatically routes INSERT statements to the correct partition. If a row doesn't match any partition and no DEFAULT partition exists, the insert fails with an error.
Partition Pruning and Query Performance
The primary performance benefit of partitioning comes from partition pruning — Postgres eliminates irrelevant partitions during query planning so it only scans the data it needs.
Pruning in Action
When your WHERE clause filters on the partition key, Postgres skips partitions that can't contain matching rows:
EXPLAIN (COSTS OFF) SELECT * FROM payments WHERE payment_date BETWEEN '2024-06-01' AND '2024-06-30';
Append -> Seq Scan on payments_2024 Filter: (payment_date >= '2024-06-01' AND payment_date <= '2024-06-30')
Only payments_2024 is scanned. The planner knows that no rows in payments_2023 or payments_2025 can match, so those partitions are excluded entirely.
Without Pruning
If your query doesn't filter on the partition key, Postgres must scan all partitions:
EXPLAIN (COSTS OFF) SELECT * FROM payments WHERE customer_id = 42;
Append -> Seq Scan on payments_2023 Filter: (customer_id = 42) -> Seq Scan on payments_2024 Filter: (customer_id = 42) -> Seq Scan on payments_2025 Filter: (customer_id = 42) -> Seq Scan on payments_default Filter: (customer_id = 42)
This scans every partition — you lose the benefit of partitioning. If queries frequently filter by customer_id alone, an index on that column is more appropriate than partitioning.
Verifying Pruning Settings
Partition pruning is enabled by default, but verify these settings if you're not seeing expected behavior:
SHOW enable_partition_pruning; -- should be 'on'
Dynamic partition pruning (Postgres 11+) works with prepared statements and parameterized queries, pruning partitions at execution time when the filter value isn't known at plan time:
PREPARE payment_lookup (date, date) AS SELECT * FROM payments WHERE payment_date BETWEEN $1 AND $2; EXPLAIN (COSTS OFF) EXECUTE payment_lookup('2024-03-01', '2024-03-31');
Append -> Seq Scan on payments_2024 Filter: (payment_date >= '2024-03-01' AND payment_date <= '2024-03-31')
Tips for Effective Pruning
- Always include the partition key in
WHEREclauses when possible - Create indexes on non-partition-key columns that you frequently filter by
- Use
EXPLAINto confirm pruning is happening before going to production - For join queries, ensure the partition key is part of the join condition or filter
Partition Maintenance with pg_partman
Managing partitions by hand — creating future partitions, dropping expired ones — is tedious and error-prone in production. The pg_partman extension automates this entirely.
Initial Setup
CREATE EXTENSION pg_partman;
Configuring Automated Partitioning
Use create_parent() to configure a partitioned table for automatic management. This works on a table that's already partitioned:
SELECT partman.create_parent( p_parent_table => 'public.payments', p_control => 'payment_date', p_interval => '1 month', p_premake => 3 );
This tells pg_partman to:
- Manage
public.paymentspartitioned onpayment_date - Create monthly partitions
- Pre-create 3 future partitions so inserts never fail
Setting Up Retention
Configure how long to keep old partitions:
UPDATE partman.part_config SET retention = '12 months', retention_keep_table = false WHERE parent_table = 'public.payments';
With retention_keep_table = false, expired partitions are dropped entirely. Set it to true to detach partitions without dropping them (useful if you want to archive the data first).
Dropping a partition is nearly instantaneous compared to DELETE FROM on millions of rows — no dead tuples, no vacuum needed.
Running Maintenance
pg_partman needs a periodic maintenance call to create new partitions and handle retention:
SELECT partman.run_maintenance();
Schedule this with pg_cron to run automatically:
SELECT cron.schedule( 'partition-maintenance', '0 * * * *', -- every hour $$SELECT partman.run_maintenance()$$ );
Complete IoT Example
Here's a full setup for an IoT sensor readings table with daily partitions and 90-day retention:
-- Create the parent table CREATE TABLE sensor_readings ( reading_id bigint GENERATED ALWAYS AS IDENTITY, sensor_id int NOT NULL, reading_time timestamptz NOT NULL, temperature numeric(5,2), humidity numeric(5,2), CONSTRAINT sensor_readings_pk PRIMARY KEY (reading_id, reading_time) ) PARTITION BY RANGE (reading_time); -- Let pg_partman manage it SELECT partman.create_parent( p_parent_table => 'public.sensor_readings', p_control => 'reading_time', p_interval => '1 day', p_premake => 7 ); -- Configure 90-day retention UPDATE partman.part_config SET retention = '90 days', retention_keep_table = false WHERE parent_table = 'public.sensor_readings'; -- Schedule maintenance SELECT cron.schedule( 'sensor-partition-maint', '15 * * * *', $$SELECT partman.run_maintenance()$$ );
With this setup, pg_partman creates daily partitions a week ahead and drops them after 90 days. Your application just inserts into sensor_readings and queries filter by reading_time — everything else is handled automatically.
Automatic Archiving with Partition Detach
A common pattern is to archive old data rather than delete it — detach expired partitions, move them to an archive schema, then optionally offload them to cheaper storage. This keeps your active table fast while preserving historical data for compliance or analytics.
-- Create an archive schema CREATE SCHEMA IF NOT EXISTS archive; -- Configure pg_partman to detach (not drop) expired partitions UPDATE partman.part_config SET retention = '6 months', retention_keep_table = true, -- detach, don't drop retention_schema = 'archive' -- move detached partitions here WHERE parent_table = 'public.payments';
When run_maintenance() fires, partitions older than 6 months are detached from the parent table and moved to the archive schema. They remain queryable as standalone tables:
-- Query archived data directly SELECT * FROM archive.payments_p2024_01 WHERE customer_id = 42;
For a fully automated archiving pipeline, combine this with a scheduled function that compresses or exports the detached partitions:
-- Example: compress archived partitions after detach CREATE OR REPLACE FUNCTION archive_and_compress() RETURNS void LANGUAGE plpgsql AS $$ DECLARE tbl text; BEGIN -- Find tables in archive schema that haven't been compressed FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = 'archive' AND tablename LIKE 'payments_p%' AND tablename NOT IN ( SELECT tablename FROM pg_tables WHERE schemaname = 'archive_compressed' ) LOOP -- Export to compressed format via pg_dump or COPY EXECUTE format( 'COPY (SELECT * FROM archive.%I) TO ''/archive/%s.csv.gz'' WITH (FORMAT csv, HEADER)', tbl, tbl ); -- Drop after successful export EXECUTE format('DROP TABLE archive.%I', tbl); END LOOP; END; $$; -- Schedule weekly archival compression SELECT cron.schedule( 'archive-compress', '0 2 * * 0', -- Sundays at 2am $$SELECT archive_and_compress()$$ );
Archiving to Snowflake with pg_lake
For time-series data that grows beyond your operational needs, pg_lake lets you automatically push archived partitions to Snowflake's analytics engine as Iceberg tables — keeping Postgres lean while retaining full SQL access to historical data from Snowflake.
-- Enable pg_lake on the partitioned table SELECT pg_lake.enable_table( 'public.payments', schedule => '0 3 * * *' -- sync daily at 3am );
Once enabled, pg_lake writes Iceberg-format data to object storage on each sync. You can then query years of historical payment data from Snowflake while Postgres only holds the recent active partitions:
-- In Snowflake: query historical data from pg_lake SELECT customer_id, SUM(amount) as total_payments FROM payments_iceberg WHERE payment_date BETWEEN '2020-01-01' AND '2023-12-31' GROUP BY customer_id;
This gives you the best of both worlds — sub-millisecond OLTP queries on recent data in Postgres, and scalable analytics over the full history in Snowflake.
aside positive Snowflake Postgres supports native partitioning and pg_partman. For time-series data that grows beyond your operational needs, use pg_lake to automatically archive old partitions to Snowflake's analytics engine — keeping your Postgres lean while retaining full query access to historical data.
When to Partition (and When Not To)
Partitioning adds complexity. It's a powerful tool, but only when applied to the right problems.
Partition When
- Tables exceed 100M+ rows — query planning and index maintenance benefit significantly
- Time-series or append-only data — natural range boundaries, easy retention management
- Data retention requirements — dropping partitions is instant vs. slow bulk deletes
- Large maintenance overhead — VACUUM and REINDEX run per-partition, reducing lock contention
- IoT and high-volume ingestion — daily/hourly partitions keep each piece manageable
Don't Partition When
- Tables have fewer than 10M rows — overhead exceeds benefit, indexes are sufficient
- Queries frequently span all partitions — if you rarely filter on the partition key, you won't get pruning
- A simple index solves the problem — try an index first, partition only when indexes aren't enough
- You'd create too many small partitions — hundreds of partitions with < 1M rows each adds planner overhead
Common Mistakes
Partitioning small tables. If your table has 5M rows and a proper index, queries are already fast. Partitioning adds planning overhead and complicates schema management for no gain.
Choosing the wrong partition key. If most queries filter by customer_id but you partition by created_at, pruning won't help your most common query patterns.
Too many partitions. Each partition is a separate table internally. Thousands of tiny partitions slow down planning. A good rule of thumb: each partition should contain at least 10M rows, and the total partition count should stay manageable (under a few hundred).
Forgetting to pre-create partitions. Without pg_partman or a DEFAULT partition, inserts fail when they target a range with no matching partition. Always have a safety net.
Decision Framework
Ask yourself:
- Is the table large enough that maintenance (VACUUM, REINDEX) is painful?
- Do queries naturally filter on a column that makes a good partition key?
- Do you need to efficiently drop or archive old data?
If you answer yes to two or more, partitioning is likely worth the complexity.
Conclusion
Partitioning transforms large-table management from a liability into a strength. Use range partitioning for time-series workloads, list for categorical splits, and hash for even distribution. Combine with pg_partman to automate the operational overhead, and always verify with EXPLAIN that partition pruning is working in your favor.
Related Resources
Complete guide to native partitioning with automated management.
Combine partitioning with Iceberg for scalable time-series workloads.
Official docs on declarative partitioning, pruning, and constraints.
How to use and manage the DEFAULT partition effectively.
This content is provided as is, and is not maintained on an ongoing basis. It may be out of date with current Snowflake instances