Free Databricks Databricks-Certified-Data-Engineer-Associate Practice Exam with Questions & Answers | Set: 5

https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html

Auto Loader is a Structured Streaming source that incrementally and efficiently processes new data files as they arrive in cloud storage. It supports both Python and SQL in Delta Live Tables, which are ideal for building streaming data pipelines. Auto Loader can handle near real-time ingestion of millions of files per hour and provide exactly-once guarantees when writing data into Delta Lake. Auto Loader is not designed for dashboard, machine learning, serverless, or batch workloads, which have different requirements and characteristics. References: What is Auto Loader?, Delta Live Tables

Databricks documentation for Unity Catalog lineage states that lineage is captured across queries run on Databricks , is supported for all languages , and is tracked down to the column level . It also explicitly says that lineage data includes related notebooks, jobs, and dashboards , and that the lineage can be visualized in Catalog Explorer in near real time as an interactive graph. This directly matches option D. The value of this feature is that it helps engineers understand where a table’s data came from, which upstream notebooks or jobs produced it, and which downstream assets, including dashboards, depend on it. Column-level lineage adds another layer of traceability by showing how specific fields are derived through transformations, which is especially useful for impact analysis, governance, and troubleshooting. Options A, B, and C each remove documented capabilities that Unity Catalog lineage actually supports. Therefore, the Databricks-documented answer is the full interactive graph across Delta tables and related assets, with column-level tracking included.

=========

Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as cloning, committing, pushing, pulling, and branch management. Databricks Notebooks versioning is a legacy feature that allows users to link notebooks to GitHub repositories and perform basic Git operations. However, Databricks Notebooks versioning does not support the use of multiple branches for development work, which is an advantage of using Databricks Repos. With Databricks Repos, users can create and manage branches for different features, experiments, or bug fixes, and merge, rebase, or resolve conflicts between them. Databricks recommends using a separate branch for each notebook and following data science and engineering code development best practices using Git for version control, collaboration, and CI/CD. References: Git integration with Databricks Repos - Azure Databricks | Microsoft Learn, Git version control for notebooks (legacy) | Databricks on AWS, Databricks Repos Is Now Generally Available - New ‘Files’ Feature in …, Databricks Repos - What it is and how we can use it | Adatis.

The MERGE command can be used to upsert data from a source table, view, or DataFrame into a target Delta table. It allows you to specify conditions for matching and updating existing records, and inserting new records when no match is found. This way, you can avoid writing duplicate records into a Delta table1. The other commands (DROP, IGNORE, APPEND, INSERT) do not have this functionality and may result in duplicate records or data loss234. References: 1: Upsert into a Delta Lake table using merge | Databricks on AWS 2: SQL DELETE | Databricks on AWS 3: SQL INSERT INTO | Databricks on AWS 4: SQL UPDATE | Databricks on AWS