Mark Imbriaco
29.7K posts

Mark Imbriaco
@markimbriaco
Head of Engineering @ Railway. Prev. Epic Games, GitHub, Heroku, 37signals. Father of 4.
Apex, NC Katılım Kasım 2007
1.8K Takip Edilen9.7K Takipçiler
Sabitlenmiş Tweet

@mcclusk @BizNasty2point0 @spittinchiclets @RearAdBsBlog Lenovo was awesome for this in game 2. The traveling Habs fans sang along to O Canada and the rest of us sang along to The Star Spangled Banner. It was great.
English

@BizNasty2point0 @spittinchiclets @RearAdBsBlog How about... the anthem singer only sings the first line and lobs it to the crowd in the playoffs.
Nothing makes my goosebumps erect like an entire stadium of fans singing the anthem together.
Go ahead and put a little heat on your team word, but sing the whole thing or stfu.
English

@TheGentleGoat You have the agency to not run their game. It’s quite simple.
English

@markimbriaco When "crushing it" means intruding on user's environments and contributing to the normalization of iterative attacks on user agency, forgive me if I'm not very supportive.
English


@ChogoSprow The next time that happens will be the first time, so I’m profoundly unconcerned.
English

@markimbriaco Don't complain when your PC gets wrecked due to bad code from RIOT corrupting the Bios
English

The single biggest thing in the Railway post mortem that other companies should pick up on is this:
"As the route caches expired, these other workloads became unreachable"
I've seen various examples of this in the past. You have some sort of workload that maintains a cached config or set of properties, and it relies on some other subsystem to be available to repopulate it (like a db somewhere). that subsystem has a downtime, and slowly you fall into degradation and eventually failure as caches expire to empty. If that subsystem takes a long time to recover, everything based on that cache is down.
A solution (note not "the" solution) is that you dont really use expiring caches like this. Instead you have a populating mechanism that defaults to the last stable state, unless fresh data is available. For a lot of systems, that allows you to maintain some stability while impacting hopefully a much smaller % of users/usage. its likely from reading this, that the other 2/3 of Railway's provider based customers would have remained online for this (again hypothetically).
I don't encourage companies to take on multi-cloud architectures (at least not cross-cloud single blast zone things) and I generally respect the Railway team a lot in what they've built. I worry a lot about the way that the major CSPs have built their giant "turn em off" buttons based on some automation. I've seen these issues first hand at AWS and through examples like yesterday at GCP. Both have gotten respectively lazy when it comes to these types of things and I believe we'll keep seeing It happen regardless who is impacted (like a company with a 7 or 8 figure a year spend...)
Railway@Railway
We have seen recovery and marked this incident as resolved. A full post mortem is now available here: blog.railway.com/p/incident-rep…
English

tar files -> rcs -> tar files -> svn -> sourcesafe -> svn -> git -> perforce -> git
ThePrimeagen@ThePrimeagen
fun fact: my career was svn (tortoise) -> git -> perforce -> git My first year at netflix was a downgrade to perforce... I hated it.
Dansk

@holman But at least we got an epic video of the new data center build after we left them.
English

This is why SQLite on the same server (which it is by default, it's a file based db) is so fast
1000x faster than making a server connection
Ben Dicken@BenjDicken
FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this
English

@chrismunns A database is really just two S3 buckets in a trenchcoat.
English

@buildwithparas @tqbf And nobody cared or noticed. Which is, in fact, the point.
English

If the community wants to save contemporary GitHub we should rally around @kdaigle becoming independent CEO. He has been there since the old days. Knows the soul of the product and knows how to do it.
English

In a world where everyone can be a creator and AI does the heavy lifting, safety nets are more important than ever.
Our UI has had undo for destructive actions for a long time, now our APIs do too.
We're building more guardrails so you can operate at agentic speed, safely.
Railway@Railway
English

@TrippTracy @ryanwhitney6 I don't think we want him here for the siren, bet or no bet. The man is cursed.
English

Frank 🐶 and I along with Huge Caniac Sting are quietly watching “Every Move You Make” from afar @ryanwhitney6
Salzy91 / Isaac Sowells Jr Stan@Salzy0991
Tripp, please keep @ryanwhitney6 as far away from the Lenovo Center as possible. Or at least until @BizNasty2point0 performs some kind of exorcism or voodoo ritual on Whit’s demons
English
Mark Imbriaco retweetledi

@mikejulian Either (a) markdown files in your repo or (b) hook up the Notion MCP and let the agent grab them from there. Or a mix of both.
English

@ryanwhitney6 The refs won the series. Terrible officiating.
English











