Tom Lane on the Architectural Decisions That Shaped 30 Years of Postgres

We’re celebrating 30 years of Postgres. We took a moment to sit down with Tom Lane, one of the project’s main developers, to talk about the architectural bets that paid off, the chaos of the early days and what’s next for Postgres in the cloud and AI era.

This interview is also available as a video.

PostgreSQL is celebrating 30 years as an open source project. I sat down with Tom Lane, core team member and committer for 25 of those years. We talked about the technical and architectural decisions that shaped the project over the last three decades.

As you’ll see in this interview, Postgres’ architecture and community have held steady over the last three decades. While things have improved and features refined, the project is overall the same database project we started with. The main goals have always remained the same: don't lose data, recover from crashes as quickly as possible and meet the demands of modern business with extensions and new features.

Elizabeth: I know you've been a core committer on Postgres for about 25 of the 30 years. For folks who don't know you, can you describe what your day-to-day involvement in the project looks like today in 2026?

Tom: Well, it doesn't really look that much different from what it looked like in the '90s. I mean, this has always been a pretty much email-driven project. Still is. So I guess I spend maybe a third of my time reading and writing email, a third of my time reviewing other people's patches, another third of my time working on my own patches. Obviously it varies a lot from day to day, but that's kind of the general flow of things.

Elizabeth: And you're 100% committed just on core Postgres, right? You don't have other activities beyond that involvement in the core project?

Tom: That's right.

Elizabeth: Well, I have some deep technical questions, but just to get us talking, let's do a quick couple of questions that you can answer with really short answers just for fun. What is your IDE? What are you doing your core development work in?

Tom: Emacs.

Elizabeth: Do you use Postgres documentation or code comments when you're working? Or do you kind of have everything already in your head?

Tom: No, I consult the docs a lot. I consult the code even more because there's an awful lot of information in the code comments that never worked its way into the docs. I do kind of have an idea in my head of where everything is in the code tree, so I can frequently find stuff a lot faster that way than I can in the docs.

Elizabeth: When you connect to Postgres, the system spawns an entirely separate operating system process just for you. Your own private copy of the query executor, your own memory space, isolated from other connections. Other databases like MySQL or SQL Server or Oracle use threads where all the connections share processes and share memory space. This is a common discussion point for folks deep into Postgres internals. What does Postgres gain from this isolation that you can't get with a threaded model, and what are the trade-offs?

Tom: Well, I think the thing that we get from it mostly is code simplicity. When you write a straight-line piece of code, you don't have to worry about whether some other thread is messing with your data structure halfway through. There's stuff in shared memory where we do have to worry about that, but that's really a pretty tiny part of the system's overall data set.

There are also some benefits in crash resistance, because if a particular session process goes crazy and crashes, we don't have to worry that it corrupted some of the postmaster's state — so we can go ahead and restart the system and have confidence that we're still working with good data there.

So there are definitely advantages to it, but there are also disadvantages as I think we're going to get into later on. There are people looking at converting to a threaded model. But there's a huge amount of work involved there. And I don't really know if it's going to work out or not.

Elizabeth: Do you have a particular vantage point on that discussion? Are you liking the model that we currently have, or how are you approaching the question now?

Tom: I'm not particularly involved in the work that's being done to try to make things threaded; some other people are doing most of the heavy lifting on that. Obviously, it's going to be a different set of trade-offs and  a different set of strengths and weaknesses, but certainly we should push forward on that and see if we can get to a place that's better than where we are.

Elizabeth: Because each connection is its own process, each backend has its own piece of memory. And then we've got the shared memory that all the backends see — the buffer cache and lock tables and transaction stuff. Can you dive a little bit deeper into the shared region versus what stays private, and how this shapes what the system does with different parts of its memory?

Tom: Yeah. Well, for one thing — as you were just mentioning — the shared buffer cache. All data from regular tables, when it's being worked on, it's in that shared cache. And so every session sees the same view of any particular page, which is really critical for correctness for obvious reasons.

On the other hand, temporary tables live in buffers that are local to each session, which is why you can't do anything with another session's temporary tables. But in return for that, we don't have to worry about locking when we're dealing with temporary table data. So there are some performance advantages and also some disadvantages, such as that you can't rely on background vacuuming to clean up your temporary tables.

Another example worth mentioning is that each session has its own caches — retained copies of catalog data that it needs at the moment. That's really helpful for us because it allows us to deal in a pretty straightforward fashion with the fact that different sessions can have different views of what's in the catalogs. You may, for example, have an uncommitted DDL modification to a table — like adding a column. That would be fairly difficult to deal with in a shared catalog cache. But in our world, the session that has that yet uncommitted change can see it in its own caches and nobody else can see it.

Elizabeth: That's super interesting. Tell me about temporary tables — what they are and how they get created. I've used those more in data loading situations and less than in a transaction.

Tom: I don't think we allow you to convert temporary tables to non-temporary or vice versa. But it's very simple — you just say CREATE TEMP TABLE and it's otherwise exactly like CREATE TABLE. The contents of the table live as long as your session does unless you drop it. When you exit the session, anything that may have been there gets automatically dropped. It's basically just for transient data that you're not going to care about after your session finishes.

Elizabeth: One thing that surprises folks about Postgres is that while you can have lots of processes writing data concurrently, every INSERT, UPDATE and DELETE generates write-ahead log (WAL) records in parallel. There’s also only a single process that can perform crash recovery and replay those records. Can you talk about the architecture underneath — how the recovery process and the write-ahead log work, and what that means for how you design the WAL system?

Tom: Well, the point of this basically is to reduce the amount of disk writes that have to happen before you can commit a transaction. The idea is that anytime you go and update a page in a table or an index or whatever, before you actually make that change in a shared buffer, you're required to emit a record to the write-ahead log that says what you're about to do.

Then you go and do it. And if the system crashes before that change gets out to disk, then during recovery, we can replay it and reapply the change from the WAL log. The point of this is that instead of having pending writes scattered possibly all across your tables and all across your indexes, you've got this one stream of data that has to be written out and committed.

As soon as you know that the WAL data is on disk, you can safely commit your transaction, even though there may be a whole bunch of other data still waiting in shared buffers that hasn't been written out. So it's all about making that more friendly to the disk I/O system, basically.

We do have to sync those table writes eventually, but we don't do it until checkpoint, which doesn't happen too often. And it's a background operation, your transaction commit is not waiting around for that to happen.

Elizabeth: I hear a lot of people tuning that checkpoint timeout as a way to manage I/O.

Tom: Yeah. Again, that is a practicality that I don't have deep expertise in. I know how it works, but I don't know where the best spots are to set the settings.

You mentioned the point about the replay being just a single process. I think that is basically partly a matter of nobody's gotten around to it, but also partly a matter of wanting to keep that mechanism as simple as possible. We want to be certain the system will recover. So we don't want any lurking bugs in that replay. Maybe it will be parallelized at some point, but I don't think anyone's actively working on that.

Elizabeth: We've been talking about process-per-connection. If there's some kind of issue in a single process, it dies on its own and doesn't take down any other session. That has great isolation. But I live in a world where you have lots of busy applications, fleets of application servers, thousands of connections all talking to a single Postgres database. The community has largely solved this with external connection poolers like PgBouncer. What do you think about this world where we have crash isolation that's beneficial, but overhead from connection scaling? Is the current pooler situation the best way, or is Postgres working on some long-term architectural change?

Tom: Yeah, it's certainly being thought about. The dirty little secret here is that if one process crashes, we take down all the rest anyway, because we aren't totally certain that that one process didn't manage to corrupt anything in shared memory before it actually failed.

So what actually happens is the postmaster kills all its other children and spawns a fresh version of shared memory before it allows any new children to start. So from a point of view of crash recovery, all we really have to do is keep the postmaster as a separate process. We could have all the actual work being done by threads within one big process, and it would be just the same from a crash recovery standpoint.

The trick that we would have to solve to get to that is things like what I alluded to earlier — it'd be nice if we could support a lot more sessions, but the difficulty there is that each session has got some catalog caches. Until you have spun up a reasonable amount of data in those caches, the session isn't very speedy because every time it wants to discover some new fact, it has to go look in the catalog.

And because of that, once you have a session that's populated its caches pretty well, it's ready to get some work done. It's kind of a large heavyweight object and you don't want to just kill it immediately. So that's why people have landed on this pooling thing where you successively reuse that same backend process for different queries belonging to different application threads.

In order to get past that, we would have to deal with the problem of how do we share caches across sessions that may have different ideas of what the relevant data actually is. People are working on that. But it's a pretty gnarly problem. And I'm not sure that we're close to having solutions for it.

Elizabeth: There are such large servers now. It's common to see 128 or 256 CPU cores in a large instance. What do you think the end goal or direction is with threading and modern hardware?

Tom: Well, certainly you can spin up a system with 100 or 200 active backend processes, and it works reasonably well as long as you've got enough memory, because again, each of those sessions wants a fair amount of working memory. But servers these days are so huge that that's not too much of a problem.

The thing that gets you, if you're trying to run more sessions than that, tends to be the context swap time. Because with each session having its own memory map, you've got to load that into the CPU's caches.

The hope with switching to a threaded model is that we could reduce that overhead a little bit because the threads all share the same address space. So it's definitely interesting to get there, but it's going to take us a while.

Elizabeth: Stepping back, what do you think in these architectural choices really made such a huge difference in Postgres going from a pretty small academic database project to the most popular relational database on the market?

Tom: One thing I would point out is simply the relative simplicity of the system, and that arises out of choices like not having tried to do threads. It's not something we consciously chose. When we were first working with the system in the late '90s, it was simply not reasonable to think about using threads because it wasn't well standardized. Every platform that we were interested in did it a little differently. Obviously that's changed in the last 20 years, but at the time that was a big obstacle to even thinking about the idea. But it did buy us a lot of simplicity, which meant that people could come in and work on the system without too much background, and that's helped us a lot.

I think the other thing I would point out as a major step function in what we could do: when we got the code from Berkeley, it did not have a write-ahead log at all. If you crashed, you were frequently looking at having to reinitialize your database from whatever backups you had. So putting that in I think made a huge difference in terms of our reliability and ability to convince people that we're usable for production purposes.

Elizabeth: Postgres handles concurrent reads and writes with multi-version concurrency control (MVCC). When a row is updated, Postgres doesn't overwrite it. It writes a brand new copy of the row into the table with transaction metadata in the row header. A reader looks at the old version while the write is happening. You end up with old dead rows that are cleaned up with VACUUM. Other databases work differently. They have some kind of undo log. What do you think about MVCC architecturally? Do you feel like that was a good approach, or are there talks about doing something different?

Tom: No, I think it's fundamentally a good design. And the reason is precisely that that maintenance work that you have to do does get shoved off into a background process. It's not interfering with your ability to perform the actual work that people want to get done.

With an undo-based system, you have to go back and replay that undo before you can abort your transaction, for example. So as long as you commit everything, that's not too much of a penalty. But you still have the question of how do you present consistent snapshots of data without blocking people?

So I think it's a good design because it pushes a lot of that maintenance effort into background processes. Obviously we've spent a ton of engineering effort on vacuum and continue to spend a ton of engineering effort on it. But I think if we didn't have that, we'd just be needing to solve those same problems in a different way somewhere else.

Elizabeth: I think Postgres 19 is getting parallel vacuum, right?

Tom: That's new. I haven't been paying close attention to that. But yeah, Melanie, for instance, has been doing a lot of work on vacuum.

But I remember a conversation I had back around 2000 — some guy came up to me at a trade show and he was very obviously a deep, deep Oracle expert. And he said to me, "You guys did it right. We did it wrong." I always like to hear that.

Maybe he was leading me, who knows? But yeah, I'm not unhappy with this direction. It's what we've got and it works pretty well. And we know how to continue to improve it.

Elizabeth: I think autovacuum has been huge too — not relying on folks to set up their own vacuum processes. Postgres pretty much does all of this for you now, unless you have a very large database or something unusual.

Tom: Yeah, that's another area where we've done a ton of work on tuning the autovacuum heuristics to know when to vacuum. That work continues. But yeah, I think that's overall been pretty successful.

Elizabeth: I know you do a good amount of work in the query planner yourself. When you write a SQL query, the Postgres query planner goes through lots of different execution strategies — it decides whether to scan the whole table or use an index, whether it wants a nested loop or a hash join or a merge join, the order of tables in joins. There's lots of internal statistics and cost modeling to pick the right query plan. This is one of the things that makes Postgres what it is. Can you walk folks through how the query planner works at a high level, what makes it complicated, and where you see the future engineering work?

Tom: Well, as you said, basically it considers a whole bunch of different execution strategies and tries to pick the one that's cheapest according to its cost model. So the first hole, of course, is the cost model doesn't always correspond to reality. Then the second hole in it is that as soon as you get past a pretty small number of joined tables, the number of possible execution plans increases exponentially.

So at that point, we have to fall back on heuristically searching a portion of the plan space — which sometimes causes us to fail to find a good plan. So that means we're always interested in improving the cost model, and we're always looking at ways to reduce the planner's execution time. And those goals are kind of in conflict. So basically there's a whole lot of complex engineering trade-offs involved — getting good query plans in a reasonable amount of time. And that complexity is what I find interesting about it.

Elizabeth: Do you have things on the near-term or long-term horizon for major updates to the query planner, or is it smaller iterative work?

Tom: I'm not doing big things in it right at the moment. Some other people have stepped up — David Rowley, Richard Guo, for instance — are doing significant things in that area, and that's great. But yeah, it's still my first love and if I got a chance to work on it, I would.

Elizabeth: Postgres is about 1.5 million lines of C — one big giant codebase, not separate modules. Instead of standard C patterns, the project uses memory contexts where you allocate into a named context tree tied to a query or a transaction, and when that unit of work finishes, the entire tree is freed all together. There's a lot of conversation in engineering about Rust and memory-safe languages. What do you think about how the memory context pattern works as an alternative to compiler-enforced memory safety? And do you see Postgres continuing as one big C project, or is there interest in rewriting pieces or decomposing it?

Tom: I'm not seeing a lot of interest in rewriting it. We had some discussions like 20 years ago about whether to rewrite it in C++, for example. We explored that a little bit, and decided that the cost-benefit just wasn't there. And I'm not sure it's any better today with what's available now.

On the question of memory contexts — the thing to understand is we've got different contexts all over the system with different lifespans. There are some that live for the entire life of a session. There are some that live for as long as a query. There are some that are reset for each row processed within a query.

The name of the game generally is to try to allocate stuff in the shortest-lived context that you can. Assuming you can do that, you can basically stop worrying about memory leaks because whatever you may have forgotten to clean up will go away automatically in a reasonable amount of time anyway.

In fact, retail freeing stuff is kind of an anti-pattern in our code, because the context cleanup is much more efficient than individually freeing each thing that may have been in that context — and just a lot more robust also, because if you forget to free something specifically, it will still get cleaned up.

The downside of that mindset is that if you have to work in a long-lived context, it does matter whether you forget to free something. And frequently we carry over that mindset into a place where it's not applicable, and then we have a memory leak bug that we have to fix. So it's not without downsides, for sure, but it does fit what we need to do pretty darn well.

I think I invented the concept, or big chunks of it. So I may be biased. But I think it works well.

Aside from the problem of leaks in places where it actually matters, there's the problem of building a data structure in one context that has pointers into some other data structure that is shorter-lived in a different context. Then you have dangling pointer problems. So it's not all peaches and cream for sure, but it fits our problem very well and I'm not really convinced a language-imposed memory safety model would work equally well for us. I haven't spent time actually looking into that, so I can't say that for positive — but I'm suspicious about it.

Elizabeth: I want to talk a little bit about the world of Postgres extensions. There's just a huge trend right now in developers using Postgres for everything — "just use Postgres." You can use it as a key-value store. You can use the messaging queue with LISTEN/NOTIFY. You can make it into a job scheduler. You can make it into a document database. You can do tons of stuff. And the pitch is why run five different data systems when one Postgres can do it all. As someone who's focused on this core database, are you loving that versatility? Do you find it exciting, or are you worried that people are pushing Postgres past what it was designed for?

Tom: Well, extensibility has been part of the project's mindset from the very beginning. The Berkeley guys built it so you could add on new data types. They also had the idea that you could add on new index access methods, build new kinds of indexes.

And since then we've kind of taken that and run with it and added other ways you can extend the system. So for example, if a proposed feature only works with the core data types and there is no way to extend it to work with extension data types, we're probably not going to accept it in that form. We're going to say, go away and think about how this could be extended and then come back.

And I think that's served us really well in terms of attracting people who want to build things around and on top of core Postgres. So I think it's really been a big part of the project's success that we have that mindset. So yeah, I'm definitely in the camp that says it's a good idea. Certainly there are tasks that relational databases are ill-suited for, but frequently people find that we're good enough. And then that saves them having to integrate with some other toolset — it's just nice in a lot of ways.

So for example, you mentioned LISTEN/NOTIFY — it's a good simple signaling mechanism but does not scale terribly well. So if you've got to pass an enormous amount of messages, you're going to have to go find something else. But for a lot of applications it's good enough.

Elizabeth: Let's talk about how extensions work in Postgres itself. You can add new data types. You can do new index types, new operators. You can even put procedural languages in Postgres. And you can do that without touching that core Postgres engine and building an extension. But as far as I know, extensions can't modify the SQL parser. So if you want a new keyword, a new clause, that's a patch to core Postgres. And getting a patch into core Postgres is a pretty big project — potentially even a multi-year review cycle in a major version release. Something like pgvector, the AI embedding extension, is doing everything through existing SQL grammar. How are folks feeling about this line? Are there discussions about making some of that parser extensible?

Tom: Yeah. So that basically goes back to the fact that we use Flex and Bison to build the parser. Those tools read in a static description of the grammar, and they build some parsing tables, and those get compiled into the server. And that is what it is. And you can't change it on the fly.

I don't have anything against extensible parsers in principle. In fact, I worked on systems like that back in the '70s before Postgres was even a thing. But what I know from that experience is that extensible parsers are harder to build than you might guess. The great thing about Bison is that if your grammar is ambiguous, it will tell you so and it will make you fix it. And then there are no runtime surprises like, well, you can't actually get at that behavior because it's masked by some other grammar that it's ambiguous with.

So if we did find another tool that we wanted to look at that was extensible, I'd be asking hard questions about how we can get similar correctness guarantees for that.

The other thing about Bison is that it's out there — it's well documented, well debugged. It's free. It's readily available on any platform. And I don't know that there are any competitive tools that share those attributes. But as you say, this is certainly one place where the extensibility story falls down. So if we could find something better, I'd be all for that.

Elizabeth: I did a little research prior to chatting with you today, and it looks like the internet believes you started working on Postgres because you needed a database to store stock trading information, and you started out as a user and a bug reporter and then submitted a few patches and then made your way to core committer. Feel free to correct that if that story is not true. And how did that evolution happen? Did you know that this project, once you first saw it, would be something you'd spend 30 years working on?

Tom: Nope. I had no idea. Yeah, the story is pretty accurate. I needed a database and we did not want to pay for Oracle. At the time the two reasonable choices were Postgres and MySQL, and I looked at the MySQL codebase and decided I didn't want to have anything to do with it.

Elizabeth: I'll have to circle back to that one in a second.

Tom: I looked at the Postgres codebase and it seemed much better structured. So we started working with that one and immediately ran into some bugs we had to send in fixes for, and it kind of grew from there.

Up to that point, I knew zero about databases. When I was a graduate student, none of my professors cared at all about databases. So I was interested to discover that there were actually interesting problems under the hood and just got sucked into doing more and more with it.

Elizabeth: Oh, that's so interesting. You started on the project just a few years after it came out of UC Berkeley as an open source project. When you first got into this, what was Postgres like at that time? Was there major stuff you knew needed to be changed?

Tom: The bones of the system were good. We would never have gotten anywhere if that were not the case. But yeah, there were a ton of niggling little bugs that we had to identify and fix. And that period took years to really get to the point where we could say the system was pretty bug-free.

In terms of major functionality, I think the only thing that I felt was missing in those years was crash recovery. We already talked about that, the write-ahead log mechanism was what dealt with that. Otherwise, the other theme that's been ongoing through the whole time I've been involved is getting more and more performance out of it.

Elizabeth: Sure. Yeah, but it must have been pretty performant, so you must have used it. Did you use Postgres then in the early days with the project you were working on?

Tom: Yeah. We did actually use it for a certain amount of stock trading. We had some market models that worked until they stopped working. And after that we got out of the business.

Elizabeth: Lots of stuff has happened in databases in the last 30 years. There's been lots of other popular databases from that era that have faded away or just got absorbed into other projects. Thirty years is pretty old for any software project to be as popular as Postgres is. What do you think was one of those early decisions that made Postgres survive so long as an independent project?

Tom: In my mind, there's no question that it was the liberal license terms that Berkeley put on it that allowed people to use it any way they wanted, and even to start proprietary forks if they wanted on top of it. And a lot of people did. But the thing about it was because it was entirely legal, and everybody understood that, they still kept in touch with the community project and would contribute bits of work back because they would rather have a solid foundation to build on.

People would go back and forth between working on proprietary stuff and working on the community project. I think a lot of the work that was done in the early years was basically funded by that kind of model, where you were selling something on top of community Postgres.

I guess another project that you can look at of a similar vintage is Linux. They went in a different direction. Obviously they went with GPL, which has a different set of trade-offs, but again, it's something people understood how to work with, and it didn't stop people from contributing to the project.

Elizabeth: I agree, I talk about Postgres a lot and I do point out the license because I think it's been a big deal. So many people not only were able to contribute to it and build entire businesses on it. I think while Linux and Postgres have slightly different operating models and licenses, the two have helped each other along the way. Linux hasn't gone anywhere in the last 30 years. And the popularity of that as an operating system and Postgres working so well in that world has also been a factor.

Elizabeth: Were you involved in any of the licensing? As far as I know, Postgres has its own license, the PostgreSQL license, which I think is very similar to the FreeBSD license.

Tom: Yeah, in our opinion — well, actually the people who study this sort of thing have told me that it's more the MIT license. Not really the same as BSD even though it came out of Berkeley. But anyway, the way I interpret it as a nonlawyer is you can do what you want with this code as long as you don't sue us.

Elizabeth: I think the fact that large cloud providers like Amazon and Google and Microsoft could build entire businesses with Postgres in them, and sell Postgres as a service … I think that has also helped its popularity, especially in the cloud world that we live in now. There's no rules there. And so the fact that so many people are willing to host it and help people run it for a fee has also just helped its popularity.

Elizabeth: What is your favorite Postgres release?

Tom: That's a hard one. Usually my favorite release is the newest one. Looking back on the conversation we just had, I would say as a high point probably 8.0 which is where we put in WAL log recovery for the reasons we mentioned. That was just a huge step forward to being a serious database instead of a plaything. I could turn it around though, and tell you which was definitely my least favorite for sure: version 13, which was absolutely full of bugs. We were fixing bugs in that for years later. It was just by far our most problematic release.

Elizabeth: Is there any particular feature in 13 that was problematic?

Tom: Mercifully blanked out most of the details.

Elizabeth: Do you have a particular feature that you authored that you're very proud of?

Tom: I see my work with Postgres as more of a huge accumulation of not-very-large individual parts. I could pick out this or that, but probably none of them would sound all that big by themselves.

Elizabeth: If you could delete one feature from Postgres tomorrow with no backwards-compatibility concerns?

Tom: I don't love the way that we've done partitioning. I don't have an idea as to how to do it better, but it's messy.

Elizabeth: I know a little bit of your history prior to joining the Postgres project. I think you did work on JPEG — the image specification. The internet thinks that you did some work on libjpeg, which was part of the Mars Perseverance camera work. Tell me a little bit about that and how that stuff affects your work in Postgres.

Tom: So I had nothing to do with the writing of the JPEG specification. But it came out and there were maybe about a dozen of us who were interested in this and said, let's sit down and write an open source implementation of it, which we did, and that became libjpeg. And I was — there was this flurry of activity at the very beginning with maybe about a dozen people involved. Then after that, it kind of went into maintenance mode. I was principal maintainer of it for five years or so, which is why my name is on it more than other people's.

When I got involved in Postgres, that became something that just sucked up all my time. And so I stopped working on libjpeg. I'm happy that some other people picked it up and ran with it, which they did eventually after I ignored it for long enough.

I know for a fact that the engineering cameras on Perseverance use libjpeg, because Joe Conway found an academic paper that said so. They've never been in any direct contact with me.

Elizabeth: Is there crossover between open image specifications and databases?

Tom: Not directly, but it definitely informs my thinking about things like software licenses. I think the fact that JPEG is absolutely everywhere today is 25% the fact that it was a really great standard that lets you make image files about 10 times smaller for the same quality as you could have before and 75% the fact that there was a free implementation that anybody could use. Without that, it would not have been put into the early web browsers and you would not be seeing it all over the net.

We made the right decision on that. And then when I came to Postgres again, the fact that it had a very liberal license was a big part of what attracted me to it.

Elizabeth: How did you get interested in this open source idea? Obviously you have a passion for collaborative open source software projects. Did you come across this in college? When did you get interested in that as an idea?

Tom: I was exposed to it when I was a grad student in the '80s. If you're in the academic environment, obviously there is software floating around all over the place that nobody particularly wants to assert copyright claims on — they'd rather share and improve and use. So I was exposed to that kind of thinking at that point.

The reason that I got involved specifically was that I had just finished up my PhD, which was largely paid for by American taxpayers, and I felt that I needed to do something to give back to the world at large. So I was looking for an open source project that I could give to the world and repay my debt to society a little bit. And libjpeg came along and I said, well, this looks like fun. I'll get involved in this.

Elizabeth: What was your PhD on?

Tom: Software architecture. The title of the thesis actually had a couple more words in that, but that was basically what it was about.

Elizabeth: I want to talk a little bit about the way the project works and the governance model. It's a little bit unique. Most open source projects at this size and scale are operated a little bit differently. They probably have a formal foundation, probably have a board, may have some kind of corporate sponsors. I'm thinking of groups like Apache Foundation, Linux Foundation, Python Software Foundation, the CNCF for Kubernetes. Postgres doesn't really work like that. The PostgreSQL Development Group isn't, as far as I know, even a formal legal entity. It's a little bit more informal than some of the other larger projects out there. It's more self-selecting. You've got a core team, self-selecting committers, really primarily all email-based. You can't even submit a pull request. Postgres code is in Git, but that's just a mirror. Everything happens by email. What do you think about the governance model?

Tom: To be honest, I don't quite know how we've managed to make it work. It was kind of forced on us by the project's initial conditions. We had this chunk of code that Berkeley had just thrown over the wall. We didn't own it. We couldn't make changes to the license. We talked about that and eventually decided that we simply didn't have the right to make changes to the license because we were not the original or primary authors of the code at that point.

So you're stuck with the license. We had a bunch of contributors, none of whom were answerable to each other; they all work for different people. So any sort of really strong governance model just would not have worked. People would have walked away from it.

So we've never had anything formal. In the beginning, the core team was the people who owned and ran our CVS server, and thereby the rights to give out commit bits or not. But that didn't last all that long. It was not too long before we had a separate infrastructure team. And so now the core committee has no formal authority whatsoever. People kind of listen to us because they always have. But I'm not really sure what would happen if we ever got into a really serious knock-down, drag-out fight about how the project should be run. But somehow it's held together for 30 years and I don't really understand how it came about.

Elizabeth: I think most folks involved in the project are pretty committed to Postgres itself. You wouldn't be writing patches if you weren't trying to make the project better. It doesn't move fast. Big patches take years to get in and even small patches can take years. But I think in some ways that creates stability in the project, because it isn't going to do the next newest thing that comes out or rewrite parts of it based on a single person's interest or a single company's interest. There's a lot of stability that you get from this "we all agree" kind of model.

Tom: I think it comes with the territory to some extent. People want their databases to be boring. When they put data in, they want to know they'll be able to get that data out tomorrow. And the requirements aren't moving that fast. Every five or 10 years, the SQL committee comes out with a new version of the standard that has some new stuff in it, and we look at it and maybe we feel like implementing it and maybe we don't. But there's not really anything driving us to make changes in a hurry. And for what we're doing, that's good.

Elizabeth: Where do you see Postgres heading next? We were talking about the governance model and some projects of this size might have a roadmap or a steering committee. Postgres doesn't really have that. We've talked a little bit about threading. Are there other things that may be coming into core Postgres that are bigger shifts? A lot of people ask about things like backups or high availability. Are you thinking about doing more of those in Postgres?

Tom: As I say, there's no project roadmap. And again, that arises out of the fact that certainly in the early years of the project, nobody could tell anybody else what to do. If you wanted to get anything done, you had to convince people that what you had was a good idea, and then maybe they'd work on it and help you. We still operate on that basis. Obviously, some people now are employed by companies that can tell them what to work on, but it's still the case that they have to convince everybody else if they want to see it land in the community code.

We've already talked about threading and the basically performance-based reasons for wanting that. The other thing that I think has been an ongoing theme for a while now is people would like to have the ability to have column-organized tables instead of the traditional row-organized tables we support right now. We already have a notion of a table access method API, but it's not general enough yet to permit somebody to put in a column-organized table. But I think people are still very interested in that concept. They're going to keep pushing in that direction. Maybe we'll get there.

Beyond that, I'm not personally in favor of rolling things into core if they work well as extensions. We have a very finite amount of engineering manpower working on the core code. Making it bigger and bigger is just going to make us stretched thinner and thinner. I'm generally in the camp that says if it can work reasonably as an extension, let's keep it as an extension.

Elizabeth: I did pretty well through most of this interview without talking about AI. I feel so proud of myself if I can get through a dinner party without talking about AI. But obviously I'm curious about what's happening with AI in your world. Are you adopting any AI coding tools or review tools?

Tom: Personally, I've done a couple of small things with Claude Code, but I can't say I've gotten into it in a major way yet.

The community at large is still having discussions about whether we want to accept submissions that are largely AI-generated. We actually had a workshop just a couple of weeks ago where a couple of dozen of us got together and talked about this stuff in person. I think the consensus among that group was, we still want a human to stand behind everything that goes into the codebase and be able to explain every decision that's in it. Even if it had been authored in part by an AI. That's not formal project policy yet. But we're going to have those discussions among the wider community soon, I think, and try to standardize on some kind of policy.

The one thing that's definitely already affecting our workflow is we're getting a huge number of AI-generated bug reports and security reports too. A lot of them are things happening behind the scenes right now. That's sort of a problem because certainly a lot of them are valid bugs we need to fix. But there's also a fair amount of slop in that from the tools not understanding basic architectural concepts. Hopefully that improves. Or at least we can figure out how to filter that stuff more effectively.

Elizabeth: I know the mailing list currently has a little bit of an embargo. So as far as I know, there aren't bots posting to the mailing list, but maybe they are and I haven't heard of it. Have you heard of nonhumans posting to the hackers list where the code is worked on and folks communicate with each other?

Tom: I haven't seen anything identifiable in that way. There's certainly been an increase in people just taking an AI-generated report and posting it directly as a bug report, that sort of thing. But I haven't seen anything that looked to me like a bot actually with a subscription to the mailing list. Maybe I've just missed it.

Elizabeth: Tell me about your local dev setup?

Tom: I do most of my work on a Linux server using Emacs and command-line tools. What I actually sit and type on is a Mac laptop, connected to the server via SSH and X11. That's basically a setup I got into years ago because I was having tremendous ergonomic problems. I needed something where I could keep my hands on my lap. So laptop sits in my lap — that's where my hands are — and I'm looking at a separate screen in front of me large enough to work with.

Elizabeth: What Linux distribution?

Tom: I worked for Red Hat years ago, so I still use mostly RHEL. The server's on RHEL 10 right now, and I've got a couple of Fedora machines lying about.

Elizabeth: Have you picked one of the new ones, the Alma Linux, the new Linux flavors after CentOS?

Tom: No, no. I actually pay Red Hat for a subscription. I still believe in what they're doing, and I think they need to be supported.

Elizabeth: What's next for you? Do you have plans beyond a life in Postgres?

Tom: No. Not really. I think I'll keep working on Postgres until either I decide I'm bored with the project or I realize I'm obsolete. And I don't know when or if those things will happen, but right now I'm quite content to keep doing what I'm doing.

Elizabeth: Thank you so much for being here today with me, Tom. I've really appreciated it. I've worked with you at a couple of different companies, and I have a special place in my heart for this project. Thank you so much for joining me today and for all the work you've done on the project.

Tom: Well, thank you so much for having me. It's been fun.

Presented by Snowflake.