Joshen Lim

thank you for an excellent question! the answer is a bit long. postgres is extremely powerful. you can write SQL to transform data any-which-way you want, and it'll find a way to produce results the fastest. this power is very useful, yet can have some negative consequences you usually don't consider when you've got a few users. it can be catastrophic when you've got a lot. say you want to search by email address + sort by last signed in time. postgres can only execute this by: 1. loading every row in memory 2. filtering out rows that don't match the search text 3. sort the remaining rows in memory on large tables, this query may take a few seconds to complete. and you might think that's acceptable, since you don't visit this page often. yet the problem isn't in the speed of the query, nor how often it runs. it's with the amount of limited resources used by it. here's why: 1. to filter out the rows, you need to scan all the rows on disk 2. a drive can only go so fast 3. and has a max IO budget after which it almost fully stops so this one query is seriously eating away at the IO budget. this causes all other IO work to slow down, so all queries naturally become slower. now once it's loaded into memory, a second issue appears. memory is limited, and the most-recent query wins. fitting a large table into memory requires removing other data from it. so queries that are usually fast, now also become slow because postgres can't find the data in memory and hast to go to disk again. this puts more pressure on the IO budget. if you ran this at the worst possible time it could seriously impact your viral app. we had to make a difficult trade-off decision: build in a weekend, scale to millions? does the users page follow it? no, it didn't. it penalized your app going viral. this isn't the @supabase way. conclusion: we must redo the page, asap. priority: minimal impact on database resources and application performance. outcome: limiting choice on the page[^1] are we content with this? not at all. we're currently identifying what's the most powerful users page that will play nicely with the rest of your application. expect a much better UX in the coming months! if you enjoyed this, I can also talk about some of the specific challenges with searching through large tables with minimal impact, and how we overcame them. [1]: old UX is still there, a few clicks away.

thank you for an excellent question! the answer is a bit long. postgres is extremely powerful. you can write SQL to transform data any-which-way you want, and it'll find a way to produce results the fastest. this power is very useful, yet can have some negative consequences you usually don't consider when you've got a few users. it can be catastrophic when you've got a lot. say you want to search by email address + sort by last signed in time. postgres can only execute this by: 1. loading every row in memory 2. filtering out rows that don't match the search text 3. sort the remaining rows in memory on large tables, this query may take a few seconds to complete. and you might think that's acceptable, since you don't visit this page often. yet the problem isn't in the speed of the query, nor how often it runs. it's with the amount of limited resources used by it. here's why: 1. to filter out the rows, you need to scan all the rows on disk 2. a drive can only go so fast 3. and has a max IO budget after which it almost fully stops so this one query is seriously eating away at the IO budget. this causes all other IO work to slow down, so all queries naturally become slower. now once it's loaded into memory, a second issue appears. memory is limited, and the most-recent query wins. fitting a large table into memory requires removing other data from it. so queries that are usually fast, now also become slow because postgres can't find the data in memory and hast to go to disk again. this puts more pressure on the IO budget. if you ran this at the worst possible time it could seriously impact your viral app. we had to make a difficult trade-off decision: build in a weekend, scale to millions? does the users page follow it? no, it didn't. it penalized your app going viral. this isn't the @supabase way. conclusion: we must redo the page, asap. priority: minimal impact on database resources and application performance. outcome: limiting choice on the page[^1] are we content with this? not at all. we're currently identifying what's the most powerful users page that will play nicely with the rest of your application. expect a much better UX in the coming months! if you enjoyed this, I can also talk about some of the specific challenges with searching through large tables with minimal impact, and how we overcame them. [1]: old UX is still there, a few clicks away.

Supabase Select Our first user conference. October 3 in SF. See you there! select.supabase.com