Free Amazon Web Services AIF-C01 Practice Exam with Questions & Answers | Set: 2

The verified answer is D. Bilingual Evaluation Understudy (BLEU) score. The company wants to evaluate whether the chatbot’s generated language matches a target communication style that includes creative spelling and shortened words. AWS AI Practitioner guidance lists BLEU, ROUGE, and BERTScore as relevant metrics for assessing foundation model performance. AWS Prescriptive Guidance describes BLEU as a metric-based evaluation method that measures n-gram overlap and focuses on precision. Because the requirement is about matching a particular style of generated language against reference examples, BLEU is the best fit among the given options.

F1 score is incorrect because it is primarily used for classification-style problems where precision and recall are combined into one score. It is not the correct metric for evaluating generated text style.

BERTScore is incorrect in this specific question because BERTScore focuses on semantic similarity using contextual embeddings. That can be useful when paraphrases should be accepted, but the question emphasizes surface language style: creative spelling, shortened words, and target-audience wording. Those are closer to token and phrase overlap against reference-style responses.

ROUGE is incorrect because ROUGE is commonly used for summarization and measures overlap with reference text, especially recall-oriented overlap. The scenario is not primarily asking whether the chatbot preserved all important content from a source document. It is asking whether the generated response matches a language style.

BLEU is therefore the best answer because it can compare the generated chatbot output to reference examples written in the desired teenage language style and measure how closely the model’s wording overlaps with those references.

Transitioning to a new large language model (LLM) on Amazon Bedrock typically involves minimal changes when the new model is pre-trained and available as a foundation model. Since the company is moving from one pre-trained LLM to another, the primary task is to ensure compatibility between the new model ' s input requirements and the existing application. Adjusting the prompt template is often necessary because different LLMs may have varying prompt formats, tokenization methods, or response behaviors, even for similar tasks like document extraction.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" When switching between foundation models in Amazon Bedrock, you may need to adjust the prompt template to align with the new model’s expected input format and optimize its performance for your use case. Prompt engineering is critical to ensure the model understands the task and generates accurate outputs. "

(Source: AWS Bedrock User Guide, Prompt Engineering for Foundation Models)

Detailed Explanation:

Option A: Create a new labeled dataset.Creating a new labeled dataset is unnecessary when transitioning to a new pre-trained LLM, as pre-trained models are already trained on large datasets. This option would only be relevant if the company were training a custom model from scratch, which is not the case here.

Option B: Perform feature engineering.Feature engineering is typically associated with traditional machine learning models, not pre-trained LLMs. LLMs process raw text inputs, and transitioning to a new LLM does not require restructuring input features. This option is incorrect.

Option C: Adjust the prompt template.This is the correct approach. Different LLMs may interpret prompts differently due to variations in training data, tokenization, or model architecture. Adjusting the prompt template ensures the new LLM understands the task (e.g., document extraction) and produces the desired output format. AWS documentation emphasizes prompt engineering as a key step when adopting a new foundation model.

Option D: Fine-tune the LLM.Fine-tuning is not required for transitioning to a new pre-trained LLM unless the company needs to customize the model for a highly specific task. Since the question does not indicate a need for customization beyond document extraction (a common LLM capability), fine-tuning is unnecessary.

Amazon Bedrock provides an option to enable invocation logging to capture and store the input and output data of the models used. This is essential for monitoring and auditing purposes, particularly when handling customer data.

Option B (Correct): " Enable invocation logging in Amazon Bedrock " : This is the correct answer as it directly enables the logging of all model invocations, ensuring transparency and traceability.

Option A: " Configure AWS CloudTrail " is incorrect because CloudTrail logs API calls but does not provide specific logging for model inputs and outputs.

Option C: " Configure AWS Audit Manager " is incorrect as Audit Manager is used for compliance reporting, not specific invocation logging for AI models.

Option D: " Configure model invocation logging in Amazon EventBridge " is incorrect as EventBridge is for event-driven architectures, not specifically designed for logging AI model inputs and outputs.

AWS AI Practitioner References:

Amazon Bedrock Logging Capabilities: AWS emphasizes using built-in logging features in Bedrock to maintain data integrity and transparency in model operations.

The verified answer is C. Controllability. AWS defines responsible AI through several dimensions, including fairness, explainability, privacy and security, safety, controllability, veracity and robustness, governance, and transparency. The exact wording in the question is the clue: the company wants mechanisms to monitor and direct AI system behavior. AWS describes controllability as having mechanisms to monitor and steer AI system behavior. That maps directly to the requirement.

Fairness is incorrect because fairness focuses on considering impacts on different groups of stakeholders. It is about avoiding unjust outcomes, biased treatment, or disproportionate harm across populations. The question does not describe demographic representation, biased outputs, or unequal model behavior.

Explainability is incorrect because explainability focuses on understanding and evaluating system outputs. It helps stakeholders understand why the model generated a result or how model behavior can be interpreted. The question is not asking about understanding a decision; it is asking about controlling and directing system behavior.

Safety is incorrect because safety focuses on reducing harmful system output and misuse. Safety is related, but the wording “monitor and direct AI system behavior” is specifically the AWS definition of controllability, not safety. Safety might include preventing toxic, harmful, or dangerous outputs, but controllability is the broader dimension about steering the system according to intended behavior.

Therefore, the responsible AI dimension being applied is controllability, because the company wants active mechanisms to guide, monitor, and steer how the generative AI application behaves in operation.

The verified answer is B. AWS Artifact. AWS Artifact provides on-demand downloads of AWS security and compliance documents, including compliance reports and certifications. AWS documentation also states that AWS Artifact can provide security and compliance documents for independent software vendors who sell products on AWS Marketplace. This directly matches the requirement involving ISV compliance reports.

AWS Artifact documentation further explains that users can access AWS reports on the AWS reports tab and access independent software vendor reports on the Third-party reports tab. This confirms that AWS Artifact is the correct service for obtaining ISV compliance documentation.

The notification requirement also points to AWS Artifact. AWS documentation states that the AWS Artifact console can be used to configure email notifications for updates on agreements and reports in AWS Artifact. AWS Artifact sends those notifications through AWS User Notifications, and users can specify notification recipients and which notifications they receive.

AWS Audit Manager is incorrect because it helps automate evidence collection and compliance assessments for the customer’s AWS usage. It does not primarily provide ISV compliance report notifications. AWS Trusted Advisor is incorrect because it provides best-practice recommendations for AWS accounts, not ISV report availability. AWS Data Exchange is incorrect because it is used to find, subscribe to, and use third-party data products. It is not the compliance report and notification portal.

Therefore, the correct service is AWS Artifact.

The correct answer is B because explainable AI (XAI) provides transparency into how models reach specific decisions. According to AWS documentation, techniques such as SHAP values (SHapley Additive exPlanations) or LIME can identify which input features (e.g., income, debt ratio, or credit history) most influenced a model’s prediction. This helps financial institutions comply with fairness and transparency requirements under regulatory frameworks like the Equal Credit Opportunity Act. AWS SageMaker Clarify is a built-in service that offers explainability reports and bias detection to enhance trust. Rule-based systems and UIs alone do not satisfy transparency standards, and accuracy improvements do not replace explainability. By implementing explainable AI, customers can understand and trust credit limit decisions, reducing bias concerns and ensuring compliance.

Referenced AWS AI/ML Documents and Study Guides:

Amazon SageMaker Clarify Documentation – Explainability and Feature Attribution

AWS Responsible AI Practices – Transparency and Accountability

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Amazon Bedrock is the managed AWS service designed specifically to develop, customize, and deploy foundation models with minimal effort.

Amazon Bedrock:

Provides direct access to multiple foundation models

Eliminates infrastructure management

Supports customization, evaluation, and deployment

Why the other options are incorrect:

SageMaker AI (B) requires more setup and ML expertise.

PartyRock (C) is for experimentation, not deployment.

Amazon Q Developer (D) is a productivity assistant.

AWS AI document references:

Amazon Bedrock Overview

Foundation Model Deployment on AWS

Reducing Development Effort with Managed AI Services

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Amazon Macie uses machine learning to:

Discover sensitive data such as PII

Classify data stored in Amazon S3

Help prevent unintended data exposure

AWS security guidance recommends Macie before data sharing to ensure compliance and privacy protection.

Why the other options are incorrect:

Kendra (A) is a search service.

Textract (C) extracts text from documents.

Data Exchange (D) shares datasets, not analyzes sensitivity.

AWS AI document references:

Amazon Macie Overview

Protecting Sensitive Data in S3

Data Privacy and Governance on AWS

The correct answer is D because Amazon Bedrock provides access to foundation models (FMs) from various providers for generative AI use cases, including text generation. It supports generating content in a consistent tone, voice, or writing style using prompts or few-shot examples.

From AWS documentation:

" Amazon Bedrock allows you to build and scale generative AI applications using foundation models from AI21 Labs, Anthropic, Cohere, Meta, Mistral, Stability AI, and Amazon. These models can generate text with controlled tone and style for applications like branding, content creation, and copywriting. "

Explanation of other options:

A. Amazon Comprehend is for natural language understanding, such as sentiment analysis and entity recognition, not generation.

B. Amazon Personalize is for building recommendation systems, not content generation.

C. Amazon Rekognition is for image and video analysis, not text generation.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Developer Guide – Generative AI Use Cases

AWS Certified Machine Learning Specialty Guide – Content Generation with FMs