Santosh Shinde

The Golden Schema: Optimising Your Lakehouse for AI/BI Genie As we conclude 2025, the data landscape has undergone significant transformation compared to just a year ago. In July 2024, Gartner made a bold prediction: 30% of Generative AI projects would be abandoned by the end of 2025. Now, in December, it is evident that this prediction was accurate. Adoption has surged, with nearly 89% of organizations piloting Generative AI. However, the reality of achieving enterprise-scale success remains challenging, with success rates around 15%. This situation has led us into what experts refer to as the “Trough of Disillusionment.” The initial excitement surrounding Generative AI has diminished, revealing a crucial truth: AI's effectiveness is directly tied to the quality of the data it processes. For a deeper dive into optimising your lakehouse for AI and BI, check out the full article here: medium.com/data-science-c…

The 2026 AI Data Scientist Career Roadmap: • Month 1: Python, Pandas, Scikit-Learn • Month 2: LangChain + OpenAI • Month 3: LangGraph • Month 4: AWS Cloud  Want help? 👉 Register here: learn.business-science.io/registration-a…

GenAI + computer vision is redefining what’s possible with visual data. From multimodal CV to agentic orchestration, organizations can now interpret scenes, generate insights and act — instantly. 👉 Learn where visual AI is heading next: gtnr.it/4bPH1SQ #GartnerIT #EmergingTech #GenAI

Today we’re announcing Lakewatch — a new open, agentic SIEM. Security has changed. Attackers now use agents, operating 24/7 at machine scale, while legacy security tools were built for human-speed threats. We need tools where agents work alongside humans to keep up. Lakewatch brings a new architecture for this agentic era: • Ingest and store all enterprise data, including multimodal sources • Analyze it alongside business data with full governance • Use AI agents to automate detection, investigation, and response Security requires a fundamental platform shift. This is how teams can fight agents with agents. databricks.com/blog/databrick…

2026 is the year of agents. Here's the evidence from GitHub's fastest-growing AI repos this week (outside of OpenClaw): Agency Agents — 35K★ One command turns Claude Code into 51 AI specialists across 9 departments — product, engineering, design, marketing, QA. A full company in a prompt. → github.com/msitarzewski/a… Auto Research — 25K★ (Karpathy) Give it a goal. It plans experiments, writes code, trains, evaluates, re-optimizes. Loops until it converges. You wake up and the best version is ready. → github.com/karpathy/autor… Lightpanda — 14K★ First headless browser built from scratch for agents — not adapted from Chromium. 11× faster, 9× less memory. Infrastructure that finally matches the use case. → github.com/lightpanda-io/… llmfit — 14K★ One command benchmarks every model against your hardware — quality, speed, context, compatibility. Stops you downloading 30GB models that won't run. → github.com/10ran/llmfit CLI-Anything — 3.5K★ (only 3 days old) One line wraps any desktop app — GIMP, Blender, LibreOffice, OBS — into an agent-controllable tool. From HKU. Hit Trending in 72 hours. → github.com/HKUDS/CLI-Anyt… The pattern: every layer of the stack — orchestration, research loops, browsers, local models, desktop control, memory — is being rebuilt for agents simultaneously.

NVIDIA and Unsloth just dropped one of the best practical guides on building RL environments from scratch, and it fills the gaps that most tutorials skip entirely. Covers: - Why RL environments matter + how to build them - When RL is better than SFT - GRPO and RL best practices - How verifiable rewards and RLVR work

Jupyter notebooks for AI agent architectures github.com/Nicolepcx/ai-a…

claude code cheat sheet

Finally! A Text-to-SQL solution that actually works! (and it's open-source) When Text-to-SQL doesn’t work, we often blame the LLM or poor prompt engineering. But the real issue is usually the schema. Let me explain: You ask: “Which publishers have received royalty payments above $5,000?” Vector search pulls "publisher" and "royalty_ledger" It misses "vendor_agreement", the bridge table that connects them. So the LLM writes valid SQL that returns nothing. This is why vectors fail on real enterprise schemas. They match names. They don’t discover join paths. Foreign keys already define the path. That’s the gap QueryWeaver is built for. It’s an open-source tool by FalkorDB. QueryWeaver turns your schema into a graph. - Tables are nodes. - Foreign keys are edges. Then it walks the path and includes the bridges automatically. It seamlessly handles multi-hop chains. Tested on the BIRD Benchmark using a superhero database expanded to 60 tables, it resolved a 5-hop query by chaining through: superpower → capability_matrix → stakeholder_registry → resource_requisition → budget_allocation You can also run it locally: 𝗯𝗮𝘀𝗵 𝗶𝗱="𝟲𝗯𝟲𝗽𝗿𝗻" 𝗱𝗼𝗰𝗸𝗲𝗿 𝗿𝘂𝗻 -𝗽 𝟱𝟬𝟬𝟬:𝟱𝟬𝟬𝟬 -𝗶𝘁 𝗳𝗮𝗹𝗸𝗼𝗿𝗱𝗯/𝗾𝘂𝗲𝗿𝘆𝘄𝗲𝗮𝘃𝗲𝗿 I've shared the link to the repo in the replies.

Building Production-Ready and Scalable Agent Systems: Design Patterns for AI Agent Security @santosh-shinde/building-production-ready-and-scalable-agent-systems-design-patterns-for-ai-agent-security-90c64813bf04" target="_blank" rel="nofollow noopener">medium.com/@santosh-shind…

NEW POST Powerful context engineering is becoming a huge part of the developer experience of modern LLM tools. Birgitta Böckeler explains the current state of context configuration features, using Claude Code as an example. martinfowler.com/articles/explo…

We are cooked. This study from @AnthropicAI shows devs using AI finish tasks faster but retain less knowledge How AI impacts the formation of coding skills anthropic.com/research/AI-as…

Learn Databricks in under two hours as @Alex_TheAnalyst walks through the platform end to end, including: – Importing files and data sources – Exploring SQL Editor and notebooks – Building dashboards – And trying out AI tools like Genie and the Databricks Assistant Follow along with Databricks Free Edition: youtube.com/watch?v=CoqZTt…

Marc Andreessen: "The job is not actually the atomic unit of what happens in the workplace. The atomic unit of what happens in the workplace is the task." "A job is a bundle of tasks." “Everybody wants to talk about job loss, but really what you want to look at is task loss.” "As the tasks change enough, then that’s when the jobs change." "Ten years from now, is your job title coder, or coder-designer-product manager, or is it just, ‘I build products,’ or is it just, ‘I tell the AI how to build products.’ Whatever that job is called, it’s going to be incredibly important, because the people doing that job are going to be orchestrating the AI." @pmarca on Lenny's Podcast with @lennysan

Explore the benefits and implementation guide for real-time data for AI at: airbyte.com/agentic-data/r…

Do we need Change Data Capture (CDC) to feed fresh data to AI and analytics pipelines? Airbyte simplifies this process with its open-source and free solution: 1. Log-based CDC for PostgreSQL, MySQL, SQL Server, Oracle, and MongoDB. 2. No-code setup: connect your database, enable CDC, and sync to Snowflake, BigQuery, S3, or Vector DB. 3. Automatically handles schema evolution, normalization, and retries. 4. Ideal for feeding machine learning training, real-time business intelligence, and more, without lock-in or high costs. For more information, visit: github.com/airbytehq/airb… #dataengineering #datascience #DataOps

The more I use AI tools, and the more I observe standout devs use AI tools for building software - the more it's clear to me that standout devs before AI are standout devs when using AI tools when they lean into them Just nuts what Pete does with the project: look at it at github.com/clawdbot/clawd…

Agent Skills are now available in Cursor. Skills let agents discover and run specialized prompts and code.

We've made text-to-diagram better. Chat interface. Streaming. Smarter. Faster. Stronger.

NEW POST I had a conversation with @unmeshjoshi , and Rebecca Parsons where we talked about the feedback loop between the "what" and "how" in software development, and the role LLMs play in it. martinfowler.com/articles/convo…