Angee Navarro

Introducing Databricks Assistant Autocomplete. Assistant Autocomplete real time code suggestions as you type in SQL and Python. It uses context from code cells, Unity Catalog metadata, DataFrame data, and more to relevant suggestions as you type. - The model powering Assistant Autocomplete was tuned and developed on Databricks with Mosaic AI. By leveraging Mosaic AI Training and Managed MLflow, we customized a model to achieve both speed and accuracy, specifically optimized for data science workloads. - Low latency is crucial for AI code completion as it directly impacts the user experience. Assistant Autocomplete utilizes Databricks Model Serving to serve the model close to users, ensuring a responsive and reliable experience.

4

🧱 At Databricks too? Don't miss this! 🧱 Date: Tuesday, June 11, 2024 Time: 11:30 AM - 11:50 AM PDT Discover how non-technical teams can leverage data observability to ensure reliable analytics! Join Kyle Kirwan, CEO of Bigeye, for an impactful session where he will: - Provide an overview of data observability. - Demonstrate how data analysts, data stewards, and data product owners can map and monitor every field. - Share strategies to eliminate broken dashboards for good.

6

So excited to have Brett Speight join the VisionWrights team because he's done this before!... - moving up the value chain into more data strategy engagements - gradually diversifying deep tech competencies (data & analytics) - still focused on leading clients to business success as long-term trusted advisors - breaking chicken & egg cycle of sales/marketing/awareness vs. consulting skills and capacity Let's get it! #DataIntoDiamonds

16

1 Comment

Last week, Max Cameron told me I look too much like a sales person when I wear a blazer and t-shirt. Gearing up for Snowflake's Data Cloud Summit this morning, I donned the blazer again, even though I know it'll scare people away since they often don't want to talk to sales folks. Why? If people are still willing to talk to me even though I look like a scary sales person, I can be more confident that ATLAS is solving a real problem and that we've nailed our messaging. (This is the same thinking underlying Jessica Livingston's advice for early startups to focus on sales, not marketing.) If the conversations fall flat, I can drop the blazer, which makes discovery-style conversations easier since folks won't be too scared to chat. Finding PMF is about operating in these two different modes at conferences and everywhere else.

8

1 Comment

There are rumors coming out of Snowflake Summit this week that Snowflake’s launching a semantic layer. Standardizing key metrics like revenue is crucial for streamlining analytics and reporting. DBT and Snowflake's semantic layer products will make this concept mainstream. But metrics aren't the only thing companies lack a standard definition for. Companies care about customers (hopefully) but often don't have a consistent customer definition. Consider a B2C company with customer interactions across multiple channels: 1. Anonymous (guest) visits from multiple devices 2. Logged-in users 3. Physical store interactions Are these interactions considered as one customer or multiple? Is there alignment on this definition between different teams? Is product measuring the funnels for the same set of customers that marketing is targeting with email campaigns? Is data science building life-time-value or churn models for the same customers who are receiving marketing emails? Without a consistent customer definition, teams can't have a shared understanding of the customer's journey. This results in a disconnected experience across product and marketing channels. There's a pressing need for a customer 360 semantic layer. That's what we're building with RudderStack Profiles.

132

1 Comment

Weren't able to attend the recent Snowflake and Databricks conferences? Tired of just reading about them on LI and want some more detail? This session is for you... Catch the replay of our 2024 Conference Season Recap ⬇️ Kyle, Zach and Eric from Bigeye cover: - The biggest conference season announcements - Top trends data leaders should be watching - How to evaluate what new tech you should invest in and more!

9

1 Comment

📅 Don't Miss Our Snowflake Session: Making Analytics Trustworthy by Default Date: Wednesday, June 5 at 3:30pm PDT Location: Theater BI113 Despite the ever-growing need for data-driven decision-making, business leaders are still foiled by unseen upstream data issues breaking their dashboards. Data observability emerged as a key trend in 2023 reports, offering a new approach to help technical teams keep data pipelines reliable. But what about non-technical teams? Join Kyle Kirwan, CEO of Bigeye, for an insightful session on how non-technical teams — including data analysts, data stewards, and data product owners — can leverage data observability. Learn how to map and monitor every field, power critical analytics, and eliminate broken dashboards for good.

25

1 Comment

10 releases Snowflake made ahead of next week's Snowflake Summit that stuck with me. Some of these have been lingering around for a while in Public Preview (they're available, but have no formal SLA backing). Others are brand new and shook the market with their timely execution. These items stuck with me, in no particular order: ➡️ Cortex (GA): managed machine learning models suited at solving structured and unstructured problems, without managing extra infrastructure ➡️ Arctic: Snowflake's open-source Large Language Model, optimized for text retrieval and RAGging ➡️ Vector datatype: Optimized storage to handle unstructured data in the context of Generative AI use cases ➡️ Iceberg: support for the popular open storage format. On top of support, it comes with considerable compression improvements compared to the standard implementation, as well as an open REST API accessible by third party products without a penalty ➡️ DocumentAI: an acquisition to extract text from PDF's in a structured process. They acquired the company quite a while back, and recently made the technology available ➡️ Cost Insight: a native magnifying glass to spot opportunities for technical improvement, from a cost-savings point of view ➡️ Aggregation policies: role-based access controls for who gets to see what level of detail and aggregation. ➡️ Considerable improvements for engineers: through native GIT support, a release framework for Native Apps ➡️ Microsoft's interoperability announcements between Fabric and Snowflake ➡️ Snowflake Container Services: Managed Kubernetes, but within the contract and control perimeters of your Snowflake instances. Over the past weeks, we have seen almost daily feature releases of considerable magnitude. As if the company is cleaning up their backlog ahead of expected announcements at the summit. Lost their product direction in the wake of GenAI? I don't think so. Snowflake is still building upon the foundations they laid out over 10 years ago, and still reaps the benefits from a visionary, cloud-first technology and GTM design. #snowflake #snowflakesummit #dataengineering

44

I put a twist on Ian Whitestone's usage of #Snowflake's useful new pruning statistics view table_pruning_history. Wish I had this one in a prior life since they are just what the doctor ordered for mature Snowflake accounts that might need some fine tuning under the hood. Michael Ybarra, Robert Breazeale, Niketh G. maybe you recognize this style of UDTF 😁. See https://lnkd.in/gaSTpkch for the docs. Code: CREATE OR REPLACE FUNCTION ROW_WEIGHTED_PRUNING_SCORE(LOOKBACK_IN_DAYS integer) RETURNS TABLE ( QUALIFIED_NAME STRING,  AVERAGE_PARTITION_PRUNING_SCORE NUMBER(26, 12), AVERAGE_ROW_PRUNING_SCORE NUMBER(26, 12), TOTAL_UNPRUNED_PARTITIONS NUMBER, TOTAL_UNPRUNED_ROWS NUMBER ) AS  $$   WITH pruning_stats as ( select   start_time::date as start_time,    database_name || '.' || schema_name || '.' || table_name as qualified_name,   sum(partitions_scanned) as sum_partitions_scanned,   sum(partitions_pruned) as sum_partitions_pruned,   sum(rows_scanned) as sum_rows_scanned,   sum(rows_pruned) as sum_rows_pruned,   DIV0(sum_partitions_pruned, (sum_partitions_scanned+sum_partitions_pruned)) as partition_pruning_score,   DIV0(sum_rows_pruned, (sum_rows_scanned+sum_rows_pruned)) as row_pruning_score from snowflake.account_usage.table_pruning_history  where start_time > current_date - LOOKBACK_IN_DAYS group by all ) SELECT QUALIFIED_NAME,  AVG(PARTITION_PRUNING_SCORE) as AVERAGE_PARTITION_PRUNING_SCORE, AVG(ROW_PRUNING_SCORE) as AVERAGE_ROW_PRUNING_SCORE, SUM(SUM_PARTITIONS_SCANNED) as TOTAL_UNPRUNED_PARTITIONS, SUM(SUM_ROWS_SCANNED) as TOTAL_UNPRUNED_ROWS FROM pruning_stats GROUP BY QUALIFIED_NAME ORDER BY TOTAL_UNPRUNED_ROWS DESC $$; SELECT * FROM TABLE(ROW_WEIGHTED_PRUNING_SCORE(90));

12

Introducing Databricks Assistant Autocomplete. Assistant Autocomplete real time code suggestions as you type in SQL and Python. It uses context from code cells, Unity Catalog metadata, DataFrame data, and more to relevant suggestions as you type.

1

📣 Segment's VP of Product, Kevin Niparko, shares how to to take advantage of AI breakthroughs, accelerating time to value, and solving problems through data strategy alignment. Full podcast episode here - https://shorturl.at/epvGK #gooddata

13

1 Comment

Episode 3 is out! Have you listened yet?? Sandy Estrada and I chatted with with James Anderson from Snowflake about all things Data Products and latest trends at Snowflake. Did you know that Snowflake does not have a Customer Success Team?? So how on earth are they making sure their customers needs are being met or exceeded? Tune in to find out, and learn how not to be a crumudgeon! All in under 40 minutes!! 🎧 Available now on Spotify and Apple Podcasts! Tune in to stay ahead in the data game. #DataScience #Technology #Podcast #Snowflake #AI #Analytics #HowIMetYourData

11

We are very excited to hear about Snowflake Artic. Of many possible applications, one key data pipeline step it can revolutionize is data cleanup. Here are a few data cleanup steps I’ve built that could have been replaced by LLMs: ▪️ Standardizing a person’s name across different data sources. Ex: Mapping Robert, Rob, Bob, R. => Robert ▪️ Standardizing a job title across sources. Ex: Mapping Software Engineer, Sr. Software Developer, Software Hacker => “Software Engineer” ▪️ Extracting the City, Zip, Country from the address. Ex: “123 W 83rd Street, NYC, 10024, US” => City:New York, Zip: 10024, Country: USA ▪️ Standardizing dirty event data: Ex: product_clicked, product_clicked_ios, product_clicked_top_of_page, product_clk => product_clicked Often data teams waste time building and maintaining these data cleanup pipelines. RudderStack Profiles makes it easy to integrate LLMs into your c360 data pipeline to handle cleanup. What data cleanup tasks do you want to eliminate with LLMs?

90

3 Comments

For the architecture in question, why choose the expensive Snowflake when Dremio offers the superior Nessie catalog, more affordable computing costs, and no vendor lock-in? Sure Snowflake has the best Share option. 1. Build Dremio open Lakehouse. 2. Use the Snowflake Share feature. comments? ambariCloud Dremio Snowflake

54

20 Comments

Introducing Databricks Assistant Autocomplete. Assistant Autocomplete real-time code suggestions as you type in SQL and Python. It uses context from code cells, Unity Catalog metadata, DataFrame data, and more to relevant suggestions as you type. This is a super cool time saver for databricks practitioners leveraging our AI Assistant!!!

21

📣Hey data pros! If you haven’t already, please fill out my quick survey that covers cloud data tool adoption, team dynamics, and optimization efforts. It only takes ~2 minutes and your feedback is invaluable! As a thank you, I’m raffling off 20 $50 Amazon gift cards. Check out the survey link in the comments.

9

2 Comments

Introducing Databricks Assistant Autocomplete. Assistant Autocomplete real time code suggestions as you type in SQL and Python. It uses context from code cells, Unity Catalog metadata, DataFrame data, and more to relevant suggestions as you type.

4