Jeremy Schneider
4.9K posts
Jeremy Schneider
@jer_s
Building and running reliable data platforms that scale and perform. An organizer of Seattle Postgres User Group. My opinions are my own.
Seattle, WA Katılım Mayıs 2011
415 Takip Edilen2.4K Takipçiler
Sabitlenmiş Tweet
@FranckPachot @tudor_g Your main point is correct - no FPWs on Aurora
My tweet quoted the aurora paper, just to be safe 🙂
English
@FranckPachot @tudor_g The storage nodes manipulate local SSDs and interact with database engine instances, other peer storage nodes, and the backup/restore services that continuously backup changed data to S3 and restore data from S3 as needed.
English

@AWS knows I'm a Postgres guy, so they assign me proper account numbers. I appreciate the gesture!

English

@tudor_g 🤣 yes. It is faster because it disables full-page writes in the WAL. Because torn-page prevention is offloaded to disaggregated storage. This has nothing to do with S3. Amazon Aurora does the same, with data stored on EBS.
English
@nikitabase i was really expecting some kind of pun after your first sentence
English

Do not mock your database. You can just branch it
Ben Vinegar@bentlegen
Probably a good time to remind everyone: If you’re relying on agent-generated tests where the DB is mocked out everywhere (which they absolutely love to do) - you’re gonna be in trouble
English
@kellabyte @TanelPoder lets name it "hadoop" 😅 this is a better approach with less risk of bottleneck on a single server cpu
oracle RAC is never local to the storage anyway, shared storage = always a network hop
my hesitations w db PLs as religion [1] single server cpu [2] version control of code
English

@TanelPoder Would be interesting to have a grid of nodes and the scheduler decides which workloads/apps should be tightly running on the storage node or scheduled apart. Dynamically moving on demand.
English
@sborden76 @planetpostgres here's the commit - i think it's pretty lightweight github.com/postgres/postg…
mainly another RegisterTimeout() call in postinit.c InitPostgres()
interesting discussion postgresql.org/message-id/fla…
English

@planetpostgres @jer_s @jer_s I've been hesitant to use this so far because I haven't been able to quantify the the increased overhead (if any) by the postmaster. Were you able to detect any increased overhead?
English

Jeremy Schneider (@jer_s): Postgres client_connection_check_interval postgr.es/p/7t9
English

Poisoned conn pools in Go can escalate from row-level brownouts to complete db outages. This test suite demonstrates the failure cascade and shows how PgBouncer's peering feature (v1.19+) prevents the escalation; although fixing leaks at the app level remains the true solution.
Planet PostgreSQL@planetpostgres
Jeremy Schneider (@jer_s): How Blocking-Lock Brownouts Can Escalate from Row-Level to Complete System Outages postgr.es/p/7qE
English
@xeraa @jpscaletti @cole1809 @dhh yes, my impression is that both elastic and mongo are now using terminology in a more responsible way
English

@jer_s @jpscaletti @cole1809 @dhh the elastic terminology is "free and open" but not open source
that OSI is trying to police the term "open" (while even "open source" is just established but not backed in a legal sense) is then a different story
English

Fizzy's open source is not just seeing contributions on the backend, but UI improvements as well. Really nice level up turning the confirmation popups into proper overlays by Justin Starner 👌 (before v after) github.com/basecamp/fizzy…


English
@xeraa @jpscaletti @cole1809 @dhh IIRC both mongo and elastic initially claimed SSPL was open source, before eventually backing down and changing their positions? (the history is why this DHH thing puzzles me so much) opensource.org/blog/the-sspl-…
English

@jer_s @jpscaletti @cole1809 @dhh though elasticsearch (and probably the others too) didn't call that open source (only now that it's also available under the AGPL)
English
@jpscaletti @cole1809 @dhh "source available" is the correct term. after mongodb, elasticsearch, confluent, hashicorp, etc ... i did not expect dhh (of all people!) to go the mongo/elastic route of trying to use the term "open source" like this
English
Jeremy Schneider retweetledi
@BenjDicken I have said many times to many people - the thing I love most about working with relational databases is that you can touch on almost every aspect of computer science here, from theory and high-level algorithms down to low-level hardware design details
English

Think databases are boring?
Indexing → data structures + algorithms
AI → RAG + learned indexes
MVCC → concurrent programming
Sharding → distributed systems
Query parsing → formal languages
Query planning → stats + optimization
Replication → distributed systems
Authentication → security
WAL → fault tolerance
Storage → file systems + caching
Compression → information theory
Deadlock detection → graph theory
Databases encompass tons of interesting software engineering problems.
English
Jeremy Schneider retweetledi
@dave_cramer @samokhvalov @kellabyte the new async IO code will need to be instrumented differently
but a sync OS call to read() data from a file always waits, regardless of short wait for pagecache or long wait for physical IO
ability to profile DB and see state of active conns is very valuable
English

Discovered something surprising during today's postgres.tv hacking session:
COPY has no wait events at all.
Reading from file? NULL wait_event.
Writing to file? NULL wait_event.
Tools show green "CPU" when it's actually I/O. And COPY is what pg_dump and pg_restore use by default.
We went in to add wait events for COPY TO/FROM PROGRAM, then realized the gap is much bigger than expected. Verified with pg_wait_sampling.
Good news: many of these are easy fixes — great "first patch" items if you want to contribute to Postgres. Feel free to join us
Online session: youtube.com/watch?v=wLPDt7…
Telegram group for those who want to start Postgres hacking: t.me/postgres_hacki…
-hackers thread: postgresql.org/message-id/fla…

YouTube
English
@dave_cramer @samokhvalov @kellabyte the wait_event column of pg_stat_activity is supposed to indicate if a process of "waiting" for something (like I/O)
English

@samokhvalov @kellabyte I'm not sure that answers the question? Why would there be a wait event ?
English






