Mohammed Faisal Khan

@LukeBarousse Great post. 👏 Never heard of Analytics Engineer. Well, now i know.

Data Nerds! I ranked every data engineering tool by how often it shows up in 4M+ job postings. 📊 But here's the catch 😳. Some critical skills show up way less than they should because they're often assumed to be foundational skills for jobs. (e.g., Skills like Bash/Terminal for running pipelines) Anyway, here's the breakdown of the tiers 👇 (Note: % = how often each tool appears in DE job postings) 🔴 S TIER — Non-Negotiable The core skills needed for any DE job. Don't apply without these: 📊 SQL (~68%) — every warehouse runs on it. Query, transform, and model data. 🐍 Python (~67%) — the pipeline language. Ingestion, automation, APIs, glue between systems. ⌨️ Terminal/Bash (~11%) — every tool you'll use runs from here. This is highly undervalued in postings. 📁 Git (~11%) — version control. Every team uses it. Same posting-% caveat as Bash. ☁️ One cloud platform + warehouse (~26-46%) — AWS + Redshift, GCP + BigQuery, or Azure + Synapse. Combined cloud presence is in nearly every posting. Start with SQL, then Python. Everything else you absorb alongside them. 🟠 A TIER — Job-Ready Foundation The tool that closes the gap from "learning DE" to "hireable for modern stacks": 🪛 dbt (~10%) — only 10% of all DE postings, but 36% in Analytics Engineer (AE) roles. That's not a niche, it's a leading indicator. AE is the new hybrid role modern data teams are hiring for: part analyst, part engineer. ✅ Land the job with S + A. Pass the interview with conceptual knowledge of B Tier 👇 🟡 B TIER — Interview-Aware Know what they solve. Don't expect to code from scratch: ⚙️ Airflow (~17%) — orchestration. Built on DAGs (directed acyclic graphs). ⚡ Spark (~38%) — distributed computing for processing large datasets. 🌊 Kafka (~19%) — real-time event streaming between systems. All these depend on a foundational knowledge of Python & SQL; don't jump the gun learning these. 🟢 C TIER — Data Platform Awareness Pick the one your company uses. Understand both conceptually: ❄️ Snowflake (~26%) — pure SQL warehouse. Optimized for analytics. Modern-stack favorite. 🧱 Databricks (~24%) — lakehouse on Spark. Handles structured + unstructured. ML/AI heavy teams. 🔵 D TIER — Versatility Multipliers Lower headline demand, but high value per hour: 📊 Power BI (~15%) / Tableau (~10%) — but the kicker: in AE roles these jump to 28% / 33%. Modern data teams want pipeline builders who can also visualize. For analysts pivoting to DE, lead with this in interviews. 🟣 E TIER — Path-Dependent High demand on paper, but concentrated in legacy enterprise stacks. Skip until your job requires it: ☕ Java (~25%) — legacy enterprise data infrastructure ⚖️ Scala (~22%) — Spark's native language. Spark-heavy shops. 🎥 How did I derive this ranking? In my latest video, I walk through the concepts first (the DE lifecycle, what each tool actually solves) and then derive the tiers. (Link in comments 👇)

@officialladi_T SQLLLLLLLLLLLLLLLLLLLLLLL

Which of these tools is most sought after in today’s job market? Excel SQL Power BI Python

@initlayers Where is HNSW used?

Couldn't sleep last night. Got off the bed and somehow ended up reading about HNSW instead. What's interesting is that HNSW builds on ideas from probabilistic skip lists and NSW graphs. Instead of searching every node, it creates hierarchical graph layers where the top layers help you jump quickly across the space, and the lower layers refine the search locally. Feels surprisingly intuitive once you visualize it properly.

Working with Databricks was genuinely fun. I got to learn a lot while exploring it. I'm really thankful to @Alex_TheAnalyst for his databricks tutorial. It helped me get started and made the overall learning process much smoother 🚀 universityalliance.databricks.com/61fadc09-5d78-…

@DanKornas Great explanation. Never knew we could recommend through a prediction system.

X open-sourced its recommendation algorithm. The simple ML read: the feed is not a popularity list. It is a prediction system. It tries to predict what each viewer will do with each post. Here’s how the X ML algorithm works 🧵👇

Good afternoon ☀️ The weather today is a 10/10. Doing my 10k steps 🦶

@RaeAlisa_ @lucent_ai Codex please🙏🏻

to celebrate 3 months since lauching @lucent_ai, we're giving away 5 Codex Pro / Claude Max plans 🎁 to enter, like this post + comment which one you'd pick (codex vs claude) winners will be selected from comments in 5 days 🫶

@yabsssai Understanding the different terminologies. Also different guidelines used different terms for the same shadow, so we had to pick a single source to avoid confusion

@faisalkhan4k refining 48 structured snippets is no small task, what specific challenges did you face while ensuring those snippets are clinically grounded with your medical friend's input?

Day 2 of the Chest X-ray RAG project Spent the day auditing my Knowledge Base with a medical friend. We refined 48 structured snippets to ensure every generated report is clinically grounded. Also looked into Hybrid Search. FAISS, BM25, RRF #BuildInPublic #MedicalAI #RAG

@initlayers Quantization is such a blessing.

Quantization is everywhere these days, especially with edge AI and LLM deployment becoming more common. Probably a good time to properly understand what's actually happening under the hood instead of just calling load_in_4bit=True and moving on.

10 projects you can build to get ahead as an AI engineer: 0. Eval harness 1. RAG + reranking 2. Prompt registry 3. LLM gateway 4. Tool-calling agent 5. Synthetic data pipeline 6. LoRA fine-tune 7. Batch inference worker 8. Hallucination monitor 9. Cost/latency dashboard

@Alex_TheAnalyst Good work Alex, i will definitely follow this. A lot of JD’s are expecting us to know AI tools. Would appreciate if you could just go through them. Not much just a gist of it.

(Ignore my face) because I'm releasing a new 2026 FREE Data Analyst Bootcamp on YouTube next Week!! I created the first ever Data Analyst Bootcamp about 3 years ago and millions of people have taken it. Since then, I've created a lot more content and I wanted to update it. Here's what it already included: 1. MySQL 2. Excel 3. Tableau 4. Power BI 5. Python 6. Pandas 7. Building a Portfolio Website 8. Creating a Resume 9. Practicing for Technical Interviews 10. Azure 11. AWS 12. How to use LinkedIn to Land a Job New Content Being Added: 13. R for Data Analysis Series 14. Git and GitHub Series 15. Data Fundamentals Series 16. Databricks for Beginners Series 17. ETL in Databricks Series This will be the longest free Data Analyst Bootcamp in the world (clocking in at around 29 hours). It took 3 hours just to download it! I'm super excited and hope it'll continue to be extremely helpful to everyone out there trying to learn data skills!

@DanKornas Bookmarking this.

@yashvarma_in @foodie_coder @ZeptoNow Sooo true. All love put this post seems very robotic

@foodie_coder @ZeptoNow Bro X started feeling more like linkedin 😭

Hey All I decided to be active on Twitter again. About me :- Working with @ZeptoNow as a SDE . 2025 grad in MnC from IIT Dhanbad . ICPC’24 prelims AIR-34, also a Kanpur and Amritapuri regionalist. Will be sharing tech, CP, system design & dev experiences. What do you want to see more of?

One major issue with medical report generation models without RAG is hallucination. By grounding the model with relevant knowledge base references, the diagnosis and report quality can improve significantly.

Few days ago I started learning about RAG. Now I’m applying those skills to actual projects, and I’ll be sharing the entire project here on X. Day 1: Finding a problem to solve. I decided to work on a Chest X-ray Multimodal RAG Report Generator 🫁.

@alexkaybuilds Lovely, ill start this myself. Would like to know more about Infrayn.

Day 2 of building Infrayn. It’s live, unfinished, held together by caffeine, optimism, and several “I’ll clean that up later” decisions. But documenting the messy middle is literally the point of this series. 3 more days to go. Surely nothing catastrophic happens between now and then. 😂 medium.com/p/infrayn-is-l…

@AdalricP Nice friend. Forced you into up skilling.

A friend locked me in a room and bet me to read 60 papers on robotics in 48 hours! :) I won the bet, I always win. Everything from VLAs, world models, dexterity! The aim was to get a good technical taste for where robotics research is right now.

Interviewing is broken because the technical questions are still focused on questions that require pattern solutions not independent thought.

@ravikiran_dev7 30 apps sort of means more chances of getting successful

But seriously, What’s the point of building 30 apps in 30 days if you can’t pull a single paying user?