Free Microsoft AI-300 Practice Exam with Questions & Answers | Set: 3

MLflow is the industry-standard open-source platform for experiment tracking, and Azure Machine Learning has first-class native integration with it. When you use MLflow within an Azure ML job, parameters, metrics, and artifacts are automatically logged to the run history of the AML workspace, making every run reproducible and comparable. Option A (AML job output logs) only captures console output and lacks structured parameter and metric logging. Option C (Application Insights logs) is designed for application-level telemetry, not ML experiment metadata. Option D (Azure Monitor alerts) is a reactive notification tool, not a tracking system. MLflow ' s autologging capability means that for common frameworks such as scikit-learn, XGBoost, and PyTorch, parameters and metrics are captured without a single line of custom code, directly answering Fabrikam ' s requirement for consistent experiment tracking.

Microsoft Learn Reference Topic: Track ML experiments with MLflow – Azure Machine Learning MLflow integration

The first action should be to evaluate the model output . Fabrikam already has evaluation data containing manually defined expected responses, and the technical requirements explicitly state that the organization must use its existing evaluation datasets based on real data with input-output pairs . Microsoft Foundry supports evaluating a model against an existing curated dataset to establish measurable baseline performance before optimization. Evaluation results can identify weaknesses in accuracy, relevance, groundedness, safety, or other quality dimensions and provide objective evidence for subsequent changes.

This sequencing is critical because Fabrikam also requires that advanced fine-tuning be applied only when prompt engineering is insufficient . Therefore, immediately fine-tuning the model would bypass the required baseline evaluation and optimization process. Fine-tuning should be driven by demonstrated performance gaps, not applied automatically.

Deploying directly to production is also inappropriate because leadership requires low operational risk and gradual rollout. Production traffic should not be used as the first evaluation mechanism when curated evaluation data already exists. Synthetic interaction data is useful when evaluation coverage is insufficient, but Microsoft specifically positions synthetic generation as an option when an adequate test dataset is unavailable. Fabrikam already has suitable evaluation data.

Study Guide Reference: Optimize generative AI systems and model performance — evaluate model performance, benchmark against curated datasets, iterate on prompts, and apply advanced fine-tuning when justified.

Input feature distributions differ from training data: Analyze dataset drift metrics

Model accuracy drops without code changes: Review prediction and ground truth trends

Endpoint latency increases under load: Investigate scaling and infrastructure metrics

When input feature distributions differ from training data , the correct action is to analyze dataset drift metrics . Azure Machine Learning model monitoring detects data drift by comparing the statistical distributions of production model inputs against reference data, commonly the original training dataset. Supported measures include Population Stability Index, Jensen-Shannon distance, normalized Wasserstein distance, and statistical tests such as Kolmogorov-Smirnov.

When model accuracy drops without code changes , the next investigation should focus on prediction and ground-truth trends . Azure Machine Learning model-performance monitoring compares production predictions with collected actual outcomes and can calculate classification metrics such as accuracy, precision, and recall. A declining score without deployment changes may indicate concept drift, prediction drift, or changing relationships between input features and target outcomes.

When endpoint latency increases under load , the issue is operational rather than primarily statistical. The team should investigate scaling and infrastructure metrics , including request latency, requests per minute, CPU/memory utilization, throttling, and replica capacity. Microsoft recommends using endpoint metrics to determine whether compute must scale up or out.

Rebuild the inference container image is not indicated by any of the observed signals.

Study Guide Reference: Implement machine learning model lifecycle and operations — production monitoring, data drift, model-performance monitoring, endpoint observability, and scaling.

In Azure ML command jobs, MLflow automatically creates a run context, so explicit start_run is optional. The training script must call mlflow.log_param to record each hyperparameter for the run. If the script calls this for each hyperparameter, the requirement for capturing parameters per run is met. The script must also call mlflow.log_metric to record numeric metrics such as accuracy or loss, optionally indexed by training step. Metrics and parameters logged inside an Azure ML command job are automatically associated with the correct experiment and run — no manual specification of experiment or run ID is required. If all three elements (run context, parameter logging, metric logging) are present in the script, all experiment tracking statements are True. If any element is absent, the corresponding statement is False and the tracking requirement is not met.

Microsoft Learn Reference Topic: Log metrics, parameters, and artifacts during Azure Machine Learning training runs with MLflow

In Azure Machine Learning, a workspace is the top-level resource that acts as a centralized hub for every ML activity. It stores experiments, pipelines, datasets, models, environments, and compute targets all in one place. Before you can register assets or deploy endpoints, the workspace itself must exist and be shared across the team. Fabrikam ' s core challenge is inconsistent experiment tracking, ad-hoc versioning, and no standardized deployment path — all symptoms of the absence of a single, governed workspace. Option A (registering assets) is a subsequent action only possible after the workspace exists. Option C (online endpoint) is a deployment concern, not an asset-management foundation. Option D (Foundry project) is appropriate for generative AI workloads, not for the traditional ML pipeline standardization described here. The workspace is the prerequisite that makes every other governance action possible.

Microsoft Learn Reference Topic: Azure Machine Learning workspaces – Microsoft Learn: Manage Azure Machine Learning workspaces