Free Amazon Web Services AIF-C01 Practice Exam with Questions & Answers | Set: 3

Few-shot learning involves providing a model with a few examples (shots) to learn from. For improving intent detection accuracy in a chatbot using a large language model (LLM), the data should consist of pairs of user messages and their corresponding correct intents.

Few-shot Learning for Intent Detection:

Few-shot learning aims to enable the model to learn from a small number of examples. For intent detection, the model needs to understand the relationship between user messages and the intended action or meaning.

Providing examples of user messages and the correct user intents allows the model to learn patterns in the phrasing or language that corresponds to each intent.

Why Option C is Correct:

User Messages and Intents: These examples directly teach the model how to map a user’s input to the appropriate intent, which is the goal of intent detection in chatbots.

Improves Accuracy: By using few-shot learning with these examples, the model can generalize better from limited data, improving intent detection.

Why Other Options are Incorrect:

A. Pairs of chatbot responses and correct user intents: Incorrect because it does not focus on user input but rather on outputs.

B. Pairs of user messages and correct chatbot responses: This would be useful for response generation, not intent detection.

D. Pairs of user intents and correct chatbot responses: Again, this is not aligned with detecting intents but with generating responses.

The correct answer is C, as conversion rate (how many users upgrade to premium) and retention rate (how many stay subscribed) are primary indicators of a generative AI system’s business impact and ROI. According to AWS documentation, evaluating AI performance should go beyond technical accuracy to include business metrics that show real-world value. For a premium service, higher conversion implies successful personalization that attracts free users, while strong retention reflects sustained engagement and user satisfaction. Metrics like ARPU or query reduction are secondary indicators. AWS emphasizes outcome-based measurement to ensure AI initiatives contribute measurable ROI tied to customer adoption, loyalty, and profitability. Therefore, tracking conversion and retention gives the clearest long-term measure of financial success from generative AI.

Referenced AWS AI/ML Documents and Study Guides:

AWS Machine Learning Specialty Guide – ML Business Metrics

AWS AI Adoption Framework – Value Realization and ROI Tracking

The correct answer is Amazon Bedrock PartyRock, a playground for building and experimenting with generative AI apps in a low-cost, no-code environment. PartyRock is designed for innovation and learning. It enables users to try out prompts, LLM apps, and templates using Amazon Bedrock under a free-tier friendly setup. According to AWS, PartyRock abstracts infrastructure and allows rapid prototyping using models from Bedrock providers. This makes it ideal for early experimentation, especially for non-developers or those not ready to invest in full production pipelines. In contrast, Amazon Q Developer is for software engineering tasks, SageMaker JumpStart focuses on deploying ML models, and Q Business targets enterprise knowledge workers. None of those are as cost-effective and experimental-focused as PartyRock.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Documentation – PartyRock Overview

AWS Generative AI Learning Path – Getting Started Tools

The ML model shows 90% recall on training data but only 40% recall on unseen testing data, indicating a significant performance drop. This discrepancy suggests the model has learned the training data too well, including noise and specific patterns that do not generalize to new data, which is a classic sign of overfitting.

Exact Extract from AWS AI Documents:

From the Amazon SageMaker Developer Guide:

"Overfitting occurs when a model performs well on training data but poorly on unseen test data, as it has learned patterns specific to the training set, including noise, that do not generalize. A large gap between training and testing performance metrics, such as recall, is a common indicator of overfitting."

(Source: Amazon SageMaker Developer Guide, Model Evaluation and Overfitting)

Detailed Explanation:

Option A: The model is overfitting on the training data.This is the correct answer. The significant drop in recall from 90% (training) to 40% (testing) indicates the model is overfitting, as it performs well on training data but fails to generalize to unseen data.

Option B: The model is underfitting on the training data.Underfitting occurs when the model performs poorly on both training and testing data due to insufficient learning. With 90% recall on training data, the model is not underfitting.

Option C: The model has insufficient training data.Insufficient training data could lead to poor performance, but the high recall on trainingdata (90%) suggests the model has learned the training data well, pointing to overfitting rather than a lack of data.

Option D: The model has insufficient testing data.Insufficient testing data might lead to unreliable test metrics, but it does not explain the large performance gap between training and testing, which is more indicative of overfitting.

 Block input prompts or model responses that contain harmful content such as hate, insults, violence, or misconduct:Content filters

 Avoid subjects related to illegal investment advice or legal advice:Denied topics

 Detect and block specific offensive terms:Word filters

 Detect and filter out information in the model’s responses that is not grounded in the provided source information:Contextual grounding check

The company is using a generative AI model on Amazon Bedrock and needs to mitigate undesirable and potentially harmful content in the model’s responses. Amazon Bedrock provides several guardrail mechanisms, including content filters, denied topics, word filters, and contextual grounding checks, to ensure safe and accurate outputs. Each mitigation action in the hotspot aligns with a specific Bedrock filter policy, and each policy must be used exactly once.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

*"Amazon Bedrock guardrails provide mechanisms to control model outputs, including:

Content filters: Block harmful content such as hate speech, violence, or misconduct.

Denied topics: Prevent the model from generating responses on specific subjects, such as illegal activities or advice.

Word filters: Detect and block specific offensive or inappropriate terms.

Contextual grounding check: Ensure responses are grounded in the provided source information, filtering out ungrounded or hallucinated content."*(Source: AWS Bedrock User Guide, Guardrails for Responsible AI)

Detailed Explanation:

Block input prompts or model responses that contain harmful content such as hate, insults, violence, or misconduct: Content filtersContent filters in Amazon Bedrock are designed to detect and block harmful content, such as hate speech, insults, violence, or misconduct, ensuring the model’s outputs are safe and appropriate. This matches the first mitigation action.

Avoid subjects related to illegal investment advice or legal advice: Denied topicsDenied topics allow users to specify subjects the model should avoid, such as illegal investment advice or legal advice, which could have regulatory implications. This policy aligns with the second mitigation action.

Detect and block specific offensive terms: Word filtersWord filters enable the detection and blocking of specific offensive or inappropriate terms defined by the user, making them ideal for this mitigation action focused on specific terms.

Detect and filter out information in the model’s responses that is not grounded in the provided source information: Contextual grounding checkThe contextual grounding check ensures that the model’s responses are based on the provided source information, filtering out ungrounded or hallucinated content. This matches the fourth mitigation action.

Hotspot Selection Analysis:

The hotspot lists four mitigation actions, each with the same dropdown options: "Select...," "Content filters," "Contextual grounding check," "Denied topics," and "Word filters." The correct selections are:

First action: Content filters

Second action: Denied topics

Third action: Word filters

Fourth action: Contextual grounding check

Each filter policy is used exactly once, as required, and aligns with Amazon Bedrock’s guardrail capabilities.

Building and training a generative AI model from scratch provides the company with the most ownership and control over security responsibilities. In this scenario, the company is responsible for all aspects of the security of the data, the model, and the infrastructure.

Option D (Correct): "Building and training a generative AI model from scratch by using specific data that a customer owns": This is the correct answer because it involves complete ownership of the model, data, and infrastructure, giving the company the highest level of responsibility for security.

Option A: "Using a third-party enterprise application that has embedded generative AI features" is incorrect as the company has minimal control over the security of the AI features embedded within a third-party application.

Option B: "Building an application using an existing third-party generative AI foundation model (FM)" is incorrect because security responsibilities are shared with the third-party model provider.

Option C: "Refining an existing third-party generative AI FM by fine-tuning the model with business-specific data" is incorrect as the foundation model and part of the security responsibilities are still managed by the third party.

AWS AI Practitioner References:

Generative AI Security Scoping Matrix on AWS: AWS provides a security responsibility matrix that outlines varying levels of control and responsibility depending on the approach to developing and using AI models.

Amazon Bedrock Agents provide the capability to automate repetitive tasks and orchestrate complex workflows using generative AI models. This is particularly beneficial for customer support inquiries, where quick and efficient processing is crucial.

Option B (Correct): "Automation of repetitive tasks and orchestration of complex workflows": This is the correct answer because Bedrock Agents can automate common customer service tasks and streamline complex processes, improving response times and efficiency.

Option A: "Generation of custom foundation models (FMs) to predict customer needs" is incorrect as Bedrock agents do not create custom models.

Option C: "Automatically calling multiple foundation models (FMs) and consolidating the results" is incorrect because Bedrock agents focus on task automation rather than combining model outputs.

Option D: "Selecting the foundation model (FM) based on predefined criteria and metrics" is incorrect as Bedrock agents are not designed for selecting models.

AWS AI Practitioner References:

Amazon Bedrock Documentation: AWS explains that Bedrock Agents automate tasks and manage complex workflows, making them ideal for customer support automation.

Moderation APIs, such as Amazon Rekognition’s Content Moderation API, can help filter and block inappropriate or unwanted images from being returned by a chatbot. These APIs are specifically designed to detect and manage undesirable content in images.

Option A (Correct): "Implement moderation APIs": This is the correct answer because moderation APIs are designed to identify and filter inappropriate content, ensuring the chatbot does not return unwanted images.

Option B: "Retrain the model with a general public dataset" is incorrect because retraining does not directly prevent inappropriate content from being returned.

Option C: "Perform model validation" is incorrect as it ensures model correctness, not content moderation.

Option D: "Automate user feedback integration" is incorrect because user feedback does not prevent inappropriate images in real-time.

AWS AI Practitioner References:

AWS Content Moderation Services: AWS provides moderation APIs for filtering unwanted content from applications.

AWS CloudTrail: Logs all API calls to Amazon Bedrock.

Amazon S3 Intelligent-Tiering: Optimizes storage costs for long-term retention with automatic tiering.

According to Amazon Bedrock Logging Documentation:

“CloudTrail records API activity and events, and logs can be stored in S3. For cost optimization, use S3 Intelligent-Tiering to retain logs long-term.”