Title

Deploy an AI chat-bot app on an Ampere A1 instance using Minikube

About Or Title

About This Workshop

Desc Long

Generative AI inference using ARM-based CPUs has proven to be very effective, however we need more proof points to support this claim. Thus, we conducted extensive research to test popular open-source LLM models such as Llama 2, Mistral, and Orcas with Ampere Altra ARM-based CPUs on Oracle Cloud Infrastructure(OCI).

Ampere A1 compute shapes provide flexible VM shapes and bare metal options across numerous regions with competitive pricing while providing flexibility in choosing CPU and memory. This allowed us to run various sized open-source LLM models and derive a conclusion from our hypothesis.

This guide will provide a thorough, step-by-step process for creating, provisioning, and deploying the necessary resources to access the lama.cpp, or an application of your choosing.

Workshop Length

1 hour, 30 minutes

Outline

Lab 1 - Setting up VCN and Networing
Lab 2 - Creating Compute Instance
Lab 3 - Setting compute and installing dependencies
Lab 4 - Pulling the chat-bot image
Lab 5 - Deployment of the application
Lab 6 - Interacting with the application

Prerequisites

Familiarity with Kubernetes and cloud native concepts of deployment and containerization is required.
Some understanding of linux shell commands.
Familiarity with Oracle Cloud Infrastructure (OCI) components like OCI Compute, networking, OCIR
Basic familiarity with open-source tools like GIT and GitHub.

Other Workshops you might like

NVIDIA Morpheus on OCI: AI-Driven Cybersecurity Powered by NVIDIA GPUs

NVIDIA Morpheus on OCI: AI-Driven Cybersecurity Powered by NVIDIA GPUs

Automated NVIDIA Morpheus with Oracle Cloud Infrastructure (OCI) by leveraging NVIDIA GPUs such as (..)

2 hrs
OCI Email Delivery

OCI Email Delivery

A walkthrough of configuring Oracle's Email Delivery Service including a creation of an email (..)

1 hr
Monitor GPU Metrics in Oracle Cloud Infrastructure (OCI) with DCGM, Grafana and Prometheus

Monitor GPU Metrics in Oracle Cloud Infrastructure (OCI) with DCGM, Grafana and Prometheus

Learn how to set up and automate GPU metric monitoring in Oracle Cloud Infrastructure using NVIDIA (..)

2 hrs
Try Machine Learning and Predict Optimal Pit Strategy with Oracle Red Bull Racing

Try Machine Learning and Predict Optimal Pit Strategy with Oracle Red Bull Racing

In this workshop, you’ll have the opportunity to experiment with predictive models and choose which (..)

2 hrs 2730 Views