Ergest Xheblati

A brief re-introduction for all my new followers. I’m Ergest (rhymes with “her guest”) I’ve spent the last decade and a half in the field of data working on everything from data engineering to data science. In 2022 I self published a book called Minimum Viable SQL Patterns (link below) In 2023 I started a newsletter called Data Patterns that combines technical aspects of data with business impact (link below) My current focus is on upgrading the methodologies used in the field of data.

@stephsmithio Not code per se, but design and build systems with code. That's the heart of engineering, not the code

I think it's more valuable than ever to learn to code. Agree or disagree? Why?

I thought a really heavy truck passed outside!

There’s only two paths for dashboards: either they never get used or they end up becoming complex applications that people use to run SQL queries and export the results to Excel.

@mistercrunch Heh, I eventually changed it to a FULL OUTER JOIN because I needed all 3 buckets (left, right and intersection)

@ergestx My most exotic join in my repertoire is the FULL OUTER JOIN. One year I submitted a talk to the Coalesce conference and it didn't get accepted so in my upsetness I thought I could start a competing conference called FULL OUTER JOIN. Still considering it.

Today I (legitimately) used a RIGHT JOIN for the first time in nearly 20 years of writing SQL. AMA

@Subhash_Peshwa I was comparing the contents of two tables and was too lazy to switch their names around to check what’s in B that’s not in A

@ergestx What was the query for?

The effort required to write a good LLM prompt is nearly equal to the effort required to design a good system. Because GIGO. Similarly the amount of effort and knowledge required to judge a good output is nearly the same so a great understanding of the fundamentals of software engineering is still essential. Therefore the only advantage of an LLM is the ability to generate code that matches your design faster than you can type it. Still I’ve been quite amazed by what even minimal prompting can do assuming I know exactly what I’m doing

@matsonj @digitallogic You finally got a Mac!

duckdb-gsheets demo narrated by yours truly for @digitallogic i had never used the extension before so the video isn't super tight, but its kind of funny :P

The only way to succeed as an AI company long term is to collect and hoard proprietary, hard to get data.

The true purpose (and promise) of analytics has always been uncovering asymmetric opportunities hiding in plain sight. Things like underpriced assets, leverage points, etc. Let's look at some examples: In the movie Moneyball (based on the book by the same title) analytics is shown to uncover asymmetric returns from a simple baseball statistic called on-base percentage (OBP). Since what you want is to score more runs than your opponent and the best way to score runs is to get on base, it makes sense that players that get on base a lot increase your chance of scoring more runs. The reason why players with high OBP were undervalued was because the prevailing notion was that the only stats that mattered were batting average (AVG) and home runs (HR) Players with high OBP but low AVG and HR were frowned upon. Nowadays this is common sense and many teams already do it therefore the leverage point has to be found elsewhere and has become far more obscure. Don't like sports? Take quantitative trading. The whole point of factor analysis is to find a way to price assets such that you pay as little as possible for as high a return as possible. Factor investing for instance is the study of financial metrics that indicate good companies to invest in. In business, uncovering a single leverage point could be worth hundreds of millions of dollars. For example finding that 90% of revenue comes from 10% of clients allows you to focus your efforts on those clients, maximize their impact then use ML/DS to profile them and find more of them. THIS is the real power of analytics and the BEST value it can offer a business.

@jlowin You’re forgetting Postgres 😀

Tech is better when it’s boring. DuckDB is the most exciting boring tech I can think of right now. What else?

It’s giving RedSox circa 1986

@lawrebx Business models (e.g. eCommerce websites) are incredibly similar in the objects (entities) that can model them and the metrics needed to model them so you start there

@ergestx 2 requires high context and strong priors How do products hurdle those issues to achieve commodification? I’ve seen the infrastructure become commodified, but like in the real world, warehouse design is far more than the superstructure…

There are two trends in the field of data I’m following very closely: 1. The rise of the “AI analyst” including agentic AI, text to SQL, etc. 2. The rise of commodified data warehouses. The first one is doomed to fail without a solid foundation of a well designed warehouse, clean data, well defined metrics. Imagine selling luxury sofas to home owners who haven’t even built the foundation yet. The second one I’m very excited about because it’s a huge opportunity to instantly level up a large number of companies and can also fuel the growth of the first trend.

@machsci IMO this is simply a feature engineering problem, not that it’s easy or anything like that

Now that we all know what metrics trees are, is it possible to discover the input/output relationships between metrics statistically? E.g., if given a list of metrics A…F, could you discover: A ↘ D ↗ ↘ B F ↗ C → E

@japborst Think about templates that automatically model a business without needing custom design

@ergestx Not sure I understand #2. Can you elaborate?

@matsonj @andrewglynch I was gonna say the same thing. Move up to the domain (finance, marketing, sales, etc) a domain expert who also deeply understands data is at least twice as valuable

@andrewglynch @ergestx Imo they return to the domain. Centralized platform team with decentralized users, like IT supported products.

@oldjacket I’ve seen Equals create a data warehouse for Stripe data and a company called Pliable offer the full thing

@ergestx What’s an example of 2. ?

Watching the comeback on Netflix about the 2004 Boston Redsox, which I saw live 20 years ago and I’m shocked at how many of the player names I remember!

@GraminMaksim Ensiferum have some good stuff. I’ve tried Opeth not quite there yet

@ergestx I can suggest Bal Sagoth, Finntroll, Ensiferum, Malokarpatan, Windir, Opeth

I thought I’d never get here but my musical tastes have evolved to favor complex compositions. So for example: 1. I no longer listen to pop 2. Hard rock I used to favor in high school now sounds like pop 3. Power metal I got into later now sounds like hard rock 4. I’ve started to listen to more symphonic metal (specifically Epica, Kamelot, Nightwish) and some darkish metal stuff (Wintersun, Insomnium, Sleep Token) Any recommendations for similar bands?