Free Amazon Web Services MLA-C01 Practice Exam with Questions & Answers | Set: 8

The correct answer is A. Create Amazon Managed Streaming for Apache Kafka (Amazon MSK) Serverless clusters to process the data.

Amazon MSK Serverless allows organizations to run Apache Kafka workloads without managing or provisioning the underlying infrastructure. Unlike MSK Provisioned clusters (Option B), serverless clusters automatically scale to accommodate variable workloads and handle operational tasks such as patching, scaling, and monitoring. This meets the company’s requirement to avoid managing or scaling infrastructure while providing near real-time streaming analytics.

Option C, using Amazon Kinesis Data Streams with Application Auto Scaling, provides a managed streaming solution but is not based on open-source technology. While Kinesis can handle near real-time data, the question specifically emphasizes the company’s desire for open-source technology, which points toward Kafka.

Option D, self-hosting Apache Flink on EC2, requires manual management of servers, scaling, patching, and container orchestration. This approach contradicts the requirement of not managing or scaling infrastructure and introduces operational complexity.

MSK Serverless provides seamless integration with open-source Kafka APIs, enabling applications to produce and consume streaming data in real time while AWS handles scaling and availability. It is ideal for analytics pipelines where data ingestion is continuous and variable, and infrastructure overhead must be minimized.

By choosing MSK Serverless, the company can implement near real-time updates for its analytics platform using open-source Kafka without the operational burden of cluster management, fully aligning with AWS best practices for deployment and orchestration of ML workflows in a managed, scalable, and serverless manner.

To implement a manual approval-based workflow ensuring that only approved models are deployed to production endpoints, Amazon SageMaker provides integrated tools such as SageMaker Pipelines and the SageMaker Model Registry.

SageMaker Pipelines is a robust service for building, automating, and managing end-to-end machine learning workflows. It facilitates the orchestration of various steps in the ML lifecycle, including data preprocessing, model training, evaluation, and deployment. By integrating with the SageMaker Model Registry, it enables seamless tracking and management of model versions and their approval statuses.

Implementation Steps:

Define the Pipeline:

Create a SageMaker Pipeline encompassing steps for data preprocessing, model training, evaluation, and registration of the model in the Model Registry.

Incorporate a Condition Step to assess model performance metrics. If the model meets predefined criteria, proceed to the next step; otherwise, halt the process.

Register the Model:

Utilize the RegisterModel step to add the trained model to the Model Registry.

Set the ModelApprovalStatus parameter to PendingManualApproval during registration. This status indicates that the model awaits manual review before deployment.

Manual Approval Process:

Notify the designated approver upon model registration. This can be achieved by integrating Amazon EventBridge to monitor registration events and trigger notifications via AWS Lambda functions.

The approver reviews the model ' s performance and, if satisfactory, updates the model ' s status to Approved using the AWS SDK or through the SageMaker Studio interface.

Deploy the Approved Model:

Configure the pipeline to automatically deploy models with an Approved status to the production endpoint. This can be managed by adding deployment steps conditioned on the model ' s approval status.

Advantages of This Approach:

Automated Workflow: SageMaker Pipelines streamline the ML workflow, reducing manual interventions and potential errors.

Governance and Compliance: The manual approval step ensures that only thoroughly evaluated models are deployed, aligning with organizational standards.

Scalability: The solution supports complex ML workflows, making it adaptable to various project requirements.

By implementing this solution, the company can establish a controlled and efficient process for deploying models, ensuring that only approved versions reach production environments.