Skip to Main Content
Run a simple RAG enabled chatbot in OKE using NVIDIA NIM, Qdrant and Gradio

About This Workshop

Youtube Video

About This Workshop
Running AI workloads in Oracle Kubernetes Engine (OKE) is easier than you think. Oracle Cloud Infrastructure (OCI) provides a wide variety of GPU-enabled nodes, both virtual and bare metal, tailored to run as worker nodes in your Kubernetes cluster. These come pre-installed with NVIDIA drivers and device plugin daemon-set, simplifying setup.
With OCI, you can harness the power of Large Language Models (LLMs) securely within your own tenancy, enabling them to provide answers based on your enterprise's data. By implementing Retrieval Augmented Generation (RAG) pipelines, you can enhance LLM capabilities, allowing them to provide accurate answers on data they weren't originally trained on. This approach not only improves accuracy but also reduces costs associated with model training and fine-tuning.

Workshop Info

3 hours
  • Lab 1 - Provision of resources to run JupyterHub Notebook
  • Lab 2 - Run JupyterHub Notebook to chat with LLM
  • Lab 3 - Retrieval Augmented Generation (RAG) Application

- Administrative access to an OCI tenancy.
- Ability to spin-up A10 instances in OCI.
- Ability to create resources with Public IP addresses (Load Balancer, Instances, OKE API Endpoint).
- Access to HuggingFace.
- Accept selected HuggingFace model license agreement.

Other Workshops you might like