{"id":16002,"date":"2024-07-04T09:49:03","date_gmt":"2024-07-04T16:49:03","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=16002"},"modified":"2025-06-13T16:36:50","modified_gmt":"2025-06-13T23:36:50","slug":"accelerate-rag-ai-couchbase-nvidia","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/accelerate-rag-ai-couchbase-nvidia\/","title":{"rendered":"Accelerate Couchbase-Powered RAG AI Application With NVIDIA NIM\/NeMo and LangChain"},"content":{"rendered":"

Today, we’re excited to announce our new integration with NVIDIA NIM\/NeMo. In this blog post, we present a solution concept of an interactive chatbot based on a Retrieval Augmented Generation<\/em> (RAG)<\/span>\u00a0<\/span>architecture with Couchbase Capella as a Vector database. The retrieval and generation phases of the RAG pipeline are accelerated by NVIDIA NIM\/NeMo with <\/span>just a few lines of code.\u00a0<\/span><\/p>\n

Enterprises across various verticals strive to offer the best customer service to their customers. To achieve this, they are arming their frontline workers such as ER nurses, store sales associates, and help desk representatives, with AI-powered interactive question-and-answer (QA) chatbots to retrieve relevant and up-to-date information quickly. <\/span><\/p>\n

Chatbots are usually based on <\/span>RAG<\/span><\/a>, an AI framework used for retrieving facts from the enterprise\u2019s knowledge base to ground LLM responses in the most accurate and recent information. It involves three distinct phases, which starts with the retrieval of the most relevant context using <\/span>vector search<\/span><\/a>, augmentation of the user\u2019s query with the context, and, finally, generating relevant responses using an LLM.<\/span><\/p>\n

The problem with existing RAG pipelines is that calls to the embedding service in the retrieval phase for converting user prompts into vectors can add significant latency, slowing down applications that require interactivity. Vectorizing a document corpus consisting of millions of PDFs, docs, and other knowledge bases can take a long time to vectorize, increasing the likelihood of using stale data for RAG. Further, users find it challenging to accelerate inference (tokens\/sec) cost-efficiently to reduce the response time of their chatbot applications.<\/span><\/p>\n

Figure 1 depicts a performant stack that will enable you to easily develop an <\/span>interactive customer service chatbot. It consists of the StreamLit application framework, LangChain for orchestration, Couchbase Capella for indexing and searching vectors, and NVIDIA NIM\/NeMo for accelerating the retrieval and generation stages.<\/span><\/p>\n

\"NVIDIA<\/a>

Figure 1: Conceptual Architecture of a QA Chatbot built using Capella and NVIDIA NIM\/NeMo<\/p><\/div>\n

Couchbase Capella, a high-performance database-as-a-service (DBaaS), allows you to get started quickly with storing, indexing, and querying operational, vector, text, time series, and geospatial data while leveraging the flexibility of JSON. You can easily integrate Capella for <\/span>vector search<\/span><\/a> or semantic search without the need for a separate vector database by integrating an orchestration framework such as <\/span>LangChain<\/span><\/a> or <\/span>LlamaIndex<\/span><\/a> into your production RAG pipeline. It offers the <\/span>hybrid search<\/span><\/a> capability, which blends vector search with traditional search to improve search performance significantly. Further, you can extend vector search to the edge using Couchbase mobile for edge AI use cases.<\/span><\/p>\n

Once you have configured Capella Vector Search, you can proceed to choose a performant model from the <\/span>NVIDIA API Catalog<\/span><\/a>, which offers a broad spectrum of foundation models that span open-source, NVIDIA AI foundation, and custom models, optimized to deliver the best performance on NVIDIA accelerated infrastructure. These models are deployed as <\/span>NVIDIA NIM<\/span><\/a> either on-prem or in the cloud using easy-to-use prebuilt containers via a single command. NeMo Retriever, <\/span>a part of NVIDIA NeMo,<\/span> offers information retrieval with the lowest latency, highest throughput, and maximum data privacy.\u00a0<\/span><\/p>\n

The chatbot that we have developed using the aforementioned stack will allow you to <\/span>upload your PDF documents and ask questions interactively. It uses NV-QA-Embed<\/em>, a GPU-accelerated text embedding model used for question-answer retrieval, and <\/span>Llama 3 – 70B<\/span><\/a>, which is packaged as a NIM and accelerated on NVIDIA infrastructure. The <\/span>langchain-nvidia-ai-endpoints<\/span><\/a> package contains LangChain integrations for building applications with models on NVIDIA NIM. <\/span>Although we have used NVIDIA-hosted endpoints for prototyping purposes, we recommend that you consider using self-hosted NIM by referring to the <\/span>NIM documentation<\/span><\/a> for production deployments.\u00a0<\/span><\/p>\n

You can use this solution to support use cases that require quick information retrieval such as:<\/span><\/p>\n