Daily Databricks

There's a really rich #R package for databricks from databricks-labs. You should give it ago. If you've previously used Sparklyr or SparkR this is a worth while package to pick up for your platform. I've jumped into a quick overview below dailydatabricks.tips/Tutorials/Bric…

Did you know about With Columns Renamed? You can pass in Key Vaule Pairs of {old_name: new_name} dailydatabricks.tips/tips/Spark/wit…

Anyone still here?

NEW YEAR NEW YOU I get it but there's still time to think about this super awesome @databricks tip from last year

4/4 Read the full guide here dailydatabricks.tips/tips/Notebook/… #Databricks #DataEngineering #Python #Ipython #Spark

3/4 This is part of the #Decembricks2024 series, where we're sharing a new Databricks tip every day of December!

1/4 🧵 Just published a guide on creating Custom IPython Magics in Databricks! First, we'll cover the basics: Line magics (%) vs Cell magics (%%) Creating custom commands Registering your magics

💡 Quick fix: Try 'UNDROP TABLE WITH ID' instead of table name. And if you've got naming conflicts, use 'ALTER TABLE RENAME' first to clear the path for restoration. #TechTips #DatabaseAdmin 3/3 dailydatabricks.tips/tips/Errors/IN…

🔍 Pro tip: Use 'SHOW TABLES DROPPED' to get the exact table ID and drop time. Sometimes the issue isn't about timing - it could be naming conflicts or incorrect table references. #DatabricksSQL #DataOps 2/3

🔄 Getting the INVALID_STATE.RESTORATION_PERIOD_EXPIRED error in #Databricks when trying to undrop tables? Even within the 7-day window? Here's what you need to know...

Ever fancied using a managed identity for your #Azure #Databricks workspace that already exists and to start using it only requires a few lines of code? Find out how to use this from your Managed Resource Group dailydatabricks.tips/tips/Identity/…

4/ Ready to transform your Databricks testing workflow? Check out the full post here: 🔗dailydatabricks.tips/tips/Notebook/… #Databricks #PythonTesting #Unittest #DynamicTesting

3/ We break down the process step by step, from setting up the TestSuite to filtering and loading the discovered tests. 📝 Code samples included! 💻

🧵 Dynamically Loading Unit Tests in Databricks Notebooks 🚀 1/ Tired of the hassle of manually maintaining test suites? Want to keep your tests closer to your code? Our latest tip dives into dynamically loading unit tests using Python's unittest library. 🐍🧪

2/2 By defining a window spec once & referencing it by name, you can: 🔹Reuse complex definitions 🔹Improve query readability 🔹Avoid errors Check out the blog for a quick guide with examples! 🔗 dailydatabricks.tips/tips/Data%20En… #DataWithDatabricks #SparkTips

1/2 Are verbose window functions bogging down your @databricks SQL & PySpark code? 😰 Named window specs to the rescue! 🚀

We are over on 🟦☁️ as @Dailydatabricks.tips

💡#Decembricks Day 2 Use #DeltaLake's custom metadata to: - Track data quality metrics - Maintain audit trails - Document pipeline versions - Monitor data transformations Full guide with code examples below 👇 @Databricks #DataEngineering #Databricks