This repository contains a Python-based solution for processing and responding to natural language queries using Langchain and OpenAI 3.5 LLM. The system integrates various services from the Langchain library, OpenAI, and Pinecone to execute a multi-step process that involves text loading, embedding generation, and information retrieval.
- Document Loading and Processing: Uses TextLoader to load textual data, which is then split into manageable chunks.
- Embedding Generation: Utilizes OpenAIEmbeddings to convert text data into embeddings that facilitate semantic search.
- Vector Storage and Retrieval: Leverages PineconeVectorStore for storing and retrieving text embeddings efficiently, ensuring that relevant information is accessible for query processing.
- Advanced Query Processing: Combines document retrieval with AI-driven chat capabilities to provide contextually relevant answers to user queries. This is orchestrated through a sophisticated chain setup that includes retrieval and document processing strategies.