DataEngDude

4.2K posts

DataEngDude banner
DataEngDude

DataEngDude

@dataenggdude

Programmer, father, reader, avid outdoors lover.

เข้าร่วม Haziran 2022
417 กำลังติดตาม559 ผู้ติดตาม
DataEngDude
DataEngDude@dataenggdude·
I got the Agate hound out.
DataEngDude tweet media
English
0
0
0
16
Alexander Wagner 🇺🇦🦋
@dataenggdude I don't understand these kind of posts. The user interface is just different and it's not a replacement, it's a developer interface change. Just a different thing imo.
English
1
0
0
11
DataEngDude
DataEngDude@dataenggdude·
Lunch. Suckers.
DataEngDude tweet media
English
0
0
0
42
Zach Wilson
Zach Wilson@EcZachly·
When my five closest friends were addicts in 2011, I was an addict When my five closest friends were big tech engineers in 2016, I was a big tech engineer When my five closest friends were content creators in 2023, I was a content creator The fastest way to change your destiny is to change your friend group. You cannot change your friends to be better. But you can always find better friends! Stop thinking you need to learn yet another AI tool. If your friend group isn’t inspiring, you’ll stay stuck!
English
9
7
121
7.3K
DataEngDude
DataEngDude@dataenggdude·
On observability If you can’t see lag, throughput, and error rate for your pipelines, you’re not operating a platform; you’re hoping, and hope is not a runbook.
English
0
0
1
38
DataEngDude
DataEngDude@dataenggdude·
On schema change Schema drift isn’t proof you planned badly, it’s what happens when the business, the apps, and the vendors all keep moving; the goal isn’t to freeze schemas forever, it’s to handle change without breaking everything downstream.
English
1
0
0
57
DataEngDude
DataEngDude@dataenggdude·
On CDC CDC is powerful, but it isn’t automatically the right choice; don’t implement it just because it’s modern. Implement it when full refreshes are too slow, deletes matter, or the business actually pays for near-real-time sync.
English
0
0
0
39
DataEngDude
DataEngDude@dataenggdude·
We don't care if it's Monday night or not.
DataEngDude tweet media
English
0
0
2
37
DataEngDude
DataEngDude@dataenggdude·
On vendor noise When you pick batch, micro-batch, or streaming, this SaaS ... that Saas, you’re choosing cost, complexity, latency, and ops burden; and the vendors’ marketing is not going to spell those trade-offs out for you. Use common sense, question everything.
English
0
0
0
53
DataEngDude
DataEngDude@dataenggdude·
On batch Batch ingestion is the workhorse of most data platforms, and it’s probably what you should use most of the time, when the business question is “how fresh does this really need to be,” the honest answer is often batch, not streaming dressed up as innovation.
English
0
0
3
50
Ryan B
Ryan B@Network_Guy8·
Artemis II crew just sent this back… round earthers you good?
English
1.1K
1.3K
13.6K
1.3M
DataEngDude
DataEngDude@dataenggdude·
tools vs architecture Choosing Spark, Kafka, or DuckDB isn’t a magic pill that fixes every problem or delivers a platform immune to failure, and when someone answers every platform question by blaming a single pipeline tool, it usually means the real issues run deeper than the framework.
English
0
0
0
58
DataEngDude
DataEngDude@dataenggdude·
Bury me here.
English
0
0
1
41