GemmaGroq: DocQnA with RAG

Building a Document Q&A Chatbot with Gamma and Grok API: A Step-by-Step Guide

Introduction

In today's digital age, natural language processing (NLP) has become an integral part of various applications, from virtual assistants to customer service chatbots. One of the most exciting developments in NLP is the emergence of advanced language models like Gamma. These models enable the creation of sophisticated document Q&A chatbots that can understand and respond to user queries with remarkable accuracy.

In this tutorial, we'll explore how to create an end-to-end document Q&A chatbot using Gamma and the Grok API. We'll cover everything from setting up the environment to deploying the chatbot for real-world use. By the end of this tutorial, you'll have a working chatbot capable of answering questions based on the content of any provided document.

What is Gamma?

Gamma is a family of lightweight, state-of-the-art open-source language models developed by Google. These models are designed for open-source projects and are built using cutting-edge research and technology similar to the Gemini models. Gamma models are specifically tailored for various applications, including code assistance, open vision language models, and recurrent models.

What is Grok API?

Grok API is an inferencing engine known for its high-speed performance. By leveraging the Grok API, our chatbot can process queries and generate responses swiftly, enhancing the overall user experience.

Setting Up the Environment

Before we dive into the implementation, let's set up our environment. We'll need to install the necessary libraries and configure our environment variables.

Step 1: Install Required Libraries

First, we need to install the Python libraries that will allow us to interact with Gamma and Grok API. We use pip, the package installer for Python, to install these libraries.

pip install gamma-models
pip install grok-api

gamma-models: This library provides the necessary functions to interact with Gamma models.
grok-api: This library allows us to make requests to the Grok API endpoints.

Step 2: Configure Environment Variables

Next, we need to set up environment variables to store our API keys securely. Environment variables are a common way to keep sensitive information like API keys out of your codebase.

import os
os.environ['GAMMA_API_KEY'] = 'YOUR_GAMMA_API_KEY'
os.environ['GROK_API_KEY'] = 'YOUR_GROK_API_KEY'

By storing API keys in environment variables, we ensure they are not hardcoded into our script, enhancing security and flexibility.

Implementing the Chatbot

Now that our environment is set up, let's implement the chatbot functionality. We'll break this down into several steps to make the process clear and manageable..

Step 1: Load Documents

First, we need to load the document we want our chatbot to process. The Document class from the gamma library helps us to handle this task.

import gamma
from gamma import Document

document = Document('path/to/document.pdf')

Document('path/to/document.pdf'): This line initializes a Document object by loading a PDF document from the specified path.

Step 2: Split Text into Chunks

For efficient processing, we split the document into smaller, manageable chunks. This helps in handling large documents and improves retrieval accuracy.

chunks = document.split_into_chunks(512)

split_into_chunks(512): This method divides the document into chunks of 512 tokens each. Smaller chunks make it easier to find relevant sections when answering queries.

Step 3: Utilize the Retriever Function

We use the Retriever class to handle the retrieval of relevant chunks based on user queries. This class uses various techniques to find and rank chunks according to their relevance to the query.

from gamma.retriever import Retriever

retriever = Retriever(chunks)

Retriever(chunks): This initializes a retriever object with the previously created chunks, allowing it to search through these chunks efficiently.

Step 4: Implement the Chatbot Logic

Now we define the main function that will process user queries and return responses. The retrieve method of the Retriever class is used to find the most relevant chunk for a given query.

def chatbot(query):
    response = retriever.retrieve(query)
    return response

retrieve(query): This method takes a user query and returns the most relevant chunk from the document.

Testing the Chatbot

Let's test our chatbot with a sample query to see how it performs. This step helps us verify that our chatbot is functioning correctly.

query = 'What is the main topic of this document?'
response = chatbot(query)
print(response)

chatbot(query): This function call sends the query to our chatbot and retrieves the response, which is then printed to the console.

Performance Evaluation

To evaluate the performance of our chatbot, we measure metrics such as response time and accuracy. Response time indicates how quickly the chatbot can process and answer a query, while accuracy measures how correctly it answers the questions.

import time

start_time = time.time()
response = chatbot(query)
end_time = time.time()
response_time = end_time - start_time
print(f'Response Time: {response_time:.2f} seconds')

Below is the full code implementation for creating a document Q&A chatbot using Gamma and the Grok API. This example leverages the Streamlit library for building a user-friendly interface and demonstrates how to set up and run the chatbot.

import streamlit as st
import os
from langchain_groq import ChatGroq
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains import create_retrieval_chain
from langchain_community.vectorstores import FAISS
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from dotenv import load_dotenv
import os
load_dotenv()

## load the GROQ And OpenAI API KEY 
groq_api_key=os.getenv('GROQ_API_KEY')
os.environ["GOOGLE_API_KEY"]=os.getenv("GOOGLE_API_KEY")

st.title("Gemma Model Document Q&A")

llm=ChatGroq(groq_api_key=groq_api_key,
             model_name="Llama3-8b-8192")

prompt=ChatPromptTemplate.from_template(
"""
Answer the questions based on the provided context only.
Please provide the most accurate response based on the question
<context>
{context}
<context>
Questions:{input}

"""
)

def vector_embedding():

    if "vectors" not in st.session_state:

        st.session_state.embeddings=GoogleGenerativeAIEmbeddings(model = "models/embedding-001")
        st.session_state.loader=PyPDFDirectoryLoader("./us_census") ## Data Ingestion
        st.session_state.docs=st.session_state.loader.load() ## Document Loading
        st.session_state.text_splitter=RecursiveCharacterTextSplitter(chunk_size=1000,chunk_overlap=200) ## Chunk Creation
        st.session_state.final_documents=st.session_state.text_splitter.split_documents(st.session_state.docs[:20]) #splitting
        st.session_state.vectors=FAISS.from_documents(st.session_state.final_documents,st.session_state.embeddings) #vector OpenAI embeddings





prompt1=st.text_input("Enter Your Question From Doduments")


if st.button("Documents Embedding"):
    vector_embedding()
    st.write("Vector Store DB Is Ready")

import time



if prompt1:
    document_chain=create_stuff_documents_chain(llm,prompt)
    retriever=st.session_state.vectors.as_retriever()
    retrieval_chain=create_retrieval_chain(retriever,document_chain)
    start=time.process_time()
    response=retrieval_chain.invoke({'input':prompt1})
    print("Response time :",time.process_time()-start)
    st.write(response['answer'])

    # With a streamlit expander
    with st.expander("Document Similarity Search"):
        # Find the relevant chunks
        for i, doc in enumerate(response["context"]):
            st.write(doc.page_content)
            st.write("--------------------------------")

Conclusion

In this tutorial, we've successfully built a document Q&A chatbot using Gamma and the Grok API. By leveraging the capabilities of these advanced language models and inferencing engines, we can create intelligent chatbots capable of extracting valuable insights from textual data with speed and precision. This foundational guide provides the essential steps to get your chatbot up and running, allowing you to further refine and expand its capabilities based on your specific needs.

Code Repository

You can find the complete code repository for this tutorial on GitHub: https://github.com/Hairul343/GemmaGroq-DocQnA-RAG

Stay Tuned

Stay tuned for more tutorials and updates on the latest developments in NLP and AI. Happy coding! Happy discovering and Always curious everyone

Step-by-Step Guide to Building a Q&A Chatbot with Gamma and Grok API