[] 2 - QA and Chat over Documents - Langchian

 

QA and Chat over Documents


Refer to an overview of Langchain here: 

Introduction

QA is one of Langchain's main applications. Anything that has to do with finding an answer in a sea of words can be referred to as question and answer(QA). Although this simple function seems weak, it can actually be used to build many things. QA is related to extraction, analyzing structured data, and summarization, which all deal with processing data with LLM. Use cases: Use cases | 🦜️🔗 Langchain.

The question would be what makes QA special and different from the other use cases? I believe QA allows for fast questions to answer. This makes sense when you take a look at the workflow of QA: 

load > split > embed > store > retrieve > output

QA process

load

loading "Document". "Document" is a class of Langchain, which contains the main content and metadata. You can load the document from different sources using Langchain document_loader tools, supported by many useful integrations: Integrations | 🦜️🔗 Langchain. 

For example, I am using a website loader tool to load text from a website


Split

Splitting "Document" into chunks, or a list of "Document". You can define the size of the chunks as well as how much the chunks overlap with the last one.

This step is important because it sets the foundation for fast retrieval of relevant documents for answer generation.

Embed

Embed the documents into vectors. To embed some text into a vector means to convert a string into a vector that is a numerical representation of that string. This step essentially puts a tag on each document, which can be used to retrieve them based on the characteristics of the document content(text). 

I am not familiar with how it works on the inside, but embedding algorithms/models are able to generate features for a text automatically.

Store

Store the vector in a vectorstore. Chroma and FAISS are good local vectorstores, while Pinecone is the most popular cloud vectorstore.

Retrieve

Documents from the vectorstore are retrieved based on their corresponding embedding vector. To retrieve the documents, a query/question needs to be passed. The query will be embedded into a vector and used to search for similar document vectors. By default, the four most relevant documents will be retrieved.

Output

The documents with relevant information to the question and the question itself will be passed to an LLM model for an answer. The important point is that only a few relevant documents are used to answer a question. If we skipped steps 2-5, the LLM will have to process the whole loaded document to generate an answer. This is inefficient, and the LLM may not take so many texts at once.

Langchain functions

Langchain offers a few ways to do the above process, with different levels of abstraction. Here are the functions that can get an answer to a question:

  1. load > VectorstoreIndexCreator(langchain.indexes) > answer
  2. load > split > store > RetrievalQA(langchain.chains) > answer
  3. load> split > store > retrieve              > load_QA_chain(langchain.chains.question_answering）> answer
process without abstraction: load > split > embed > store > retrieve > output >                                                 answer

Find the code here: code_gallery/Langchain/QA over documents at main · lambo131/code_gallery (github.com). I won't go over the code here, but there are some comments I have to say.

For me, RetrievalQA is the best level of abstraction. It allows me to define how the original document is split. 

In the code, there is an example of customizing the LLM prompt in the QA chain(the prompt template where LLM takes in the retrieved docs).

I also tried GPT4ALL as my LLM. GPT4ALL is a free personal LLM run locally. I was able to run gpt4 on my laptop! However, GPT4ALL runs very slowly in Langchain.


Simple web app project

I use QA with docs to make a web tool that you can input a website link and ask questions about the contents of the website.

Video

Explore questions

- What are some ways to customize the format of the output answer

what I want to make next?
- PDF QA web app
- Question based web scrapper QA web app