A character filter that replaces characters in the input string. It uses a regular expression - Bug in langchain with Azure Search retriever?

[levalencia]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

async def policy_gpt_chain_lcel(prompt, username, inputs, outputs):
    async def llm_thread_gpt_lcel(prompt, username, inputs, outputs):
        llm = AzureChatOpenAI(
            azure_endpoint=SHD_AZURE_OPENAI_ENDPOINT,
            openai_api_version="2023-03-15-preview",
            deployment_name=SHD_OPENAI_DEPLOYMENT_NAME,
            openai_api_key=SHD_OPENAI_KEY,
            openai_api_type="Azure",
            model_name=SHD_OPENAI_GPT_MODEL_NAME,
            streaming=True,
            temperature=0
        )        

        embeddings = setup_embeddings()
        vector_store = setup_vector_store(embeddings)
        retriever = UserRetrieverPolicy2(vectorstore=vector_store, username=username)
        
        demo_ephemeral_chat_history = ChatMessageHistory()
        for i in range(len(inputs)):
            demo_ephemeral_chat_history.add_user_message(inputs[i])
            demo_ephemeral_chat_history.add_ai_message(outputs[i])
            
        contextualize_q_system_prompt = (
            "Given a chat history and the latest user question "
            "which might reference context in the chat history, "
            "formulate a standalone question which can be understood "
            "without the chat history. Do NOT answer the question, "
            "just reformulate it if needed and otherwise return it as is."
        )
        contextualize_q_prompt = ChatPromptTemplate.from_messages(
            [
                ("system", contextualize_q_system_prompt),
                MessagesPlaceholder("chat_history"),
                ("human", "{input}"),
            ]
        )
        history_aware_retriever = create_history_aware_retriever(llm, retriever, contextualize_q_prompt) 
        qa_prompt = ChatPromptTemplate.from_messages(
            [
                ("system", ANSWER_PROMPT_POLICY),
                MessagesPlaceholder("chat_history"),
                ("human", "{input}"),
            ]
        )
        question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)  | StrOutputParser()
             
        rag_chain_from_docs = create_retrieval_chain(history_aware_retriever, question_answer_chain) 
        conversational_rag_chain = RunnableWithMessageHistory(
            rag_chain_from_docs,
            lambda session_id: demo_ephemeral_chat_history, 
            input_messages_key="input",
            history_messages_key="chat_history",
            output_messages_key="answer",
        )


        async for chunk in conversational_rag_chain.astream( {"input": prompt},config={"configurable": {"session_id": "unbound"}}):
            print(str(chunk).encode())
            yield str(chunk).encode()
    
    async for chunk in llm_thread_gpt_lcel(prompt, username, inputs, outputs):
        yield chunk

import os
from dotenv import load_dotenv
from langchain.vectorstores import VectorStore
from langchain.schema import Document
from langchain.schema import BaseRetriever
from typing import List

from langchain_core.callbacks import CallbackManagerForRetrieverRun
from langchain_core.documents import Document
from langchain_core.retrievers import BaseRetriever


from config.config import NUMBER_OF_CHUNKS_TO_RETURN

# Code from: https://redis.com/blog/build-ecommerce-chatbot-with-redis/
class UserRetrieverPolicy2(BaseRetriever):

    """
    UserRetriever class extends BaseRetriever and is designed for retrieving relevant documents
    based on a user query using hybrid similarity search with a VectorStore.

    Attributes:
    - vectorstore (VectorStore): The VectorStore instance used for similarity search.
    - username (str): The username associated with the documents, used for personalized retrieval.

    Methods:
    - clean_metadata(self, doc): Cleans the metadata of a document, extracting relevant information for display.
    - get_relevant_documents(self, query): Retrieves relevant documents based on a user query using hybrid similarity search.

    Example:
    retriever = Retriever(vectorstore=vector_store, username="john_doe")
    relevant_docs = retriever.get_relevant_documents("How does photosynthesis work?")
    for doc in relevant_docs:
        print(doc.metadata["Title"], doc.page_content)
    """

    vectorstore: VectorStore
    username: str

    def clean_metadata(self, doc):
        """
        Cleans the metadata of a document.

        Parameters:
            doc (object): The document object.

        Returns:
            dict: A dictionary containing the cleaned metadata.

        """
        metadata = doc.metadata

        return {
            "file_id": metadata["title"], 
            "source": metadata["title"] + "_page=" + str(int(metadata["chunk_id"].split("_")[-1])+1), 
            "page_number": str(int(metadata["chunk_id"].split("_")[-1])+1), 
            "document_title": metadata["document_title_result"] ,
            "document_owning_department": metadata["document_owning_department_result"],
            "manual" : metadata["manual_result"]
        }

               
    def _get_relevant_documents(        self, query: str, *, run_manager: CallbackManagerForRetrieverRun
        ) -> List[Document]:
        """
        Retrieves relevant documents based on a given query.

        Args:
            query (str): The query to search for relevant documents.

        Returns:
            list: A list of relevant documents.

        """
        docs = []
        is_match_filter = ""
        load_dotenv()
        admins = os.getenv('ADMINS', '')
        admins_list = admins.split(',')
        is_admin = self.username.split('@')[0] in admins_list

        if not is_admin:
            is_match_filter = f"search.ismatch('{self.username.split('@')[0]}', 'usernames_result')"

        
        for doc in self.vectorstore.similarity_search(query, search_type="hybrid", k=NUMBER_OF_CHUNKS_TO_RETURN, filters=is_match_filter):
            cleaned_metadata = self.clean_metadata(doc)
            docs.append(Document(
                page_content=doc.page_content,
                metadata=cleaned_metadata))
            
        
        return docs
    
    
    

@app.route('/policy_lcel_quart', methods=['POST'])
async def policy_lcel_quart():
    if authenticate_request():
        data = await request.get_json()
        prompt = data.get("prompt", "What are the company values?")
        chat_history = data.get("chat_history", [])
        username = data.get("username", "")
        inputs = [item.get('input') for item in chat_history]
        outputs = [item.get('output') for item in chat_history]        
        async def generate():
            start_time = time.time()
            try:
                async for chunk in policy_gpt_chain_lcel(prompt, username, inputs, outputs):
                    yield chunk
            finally:
                end_time = time.time()
                execution_time = (end_time - start_time) * 1000  # Convert to milliseconds
                log_custom_metric(tc, 'policy_lcel_quarts', execution_time)
        return Response(generate(), content_type='text/event-stream')
    else:
        return jsonify({"error": "Unauthorized. Invalid or missing API key."}), 401

Description

The following code is throwing this error in Azure Console Logs

2024-07-02T05:45:49.958282665Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:701: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.958351266Z """Parameter group.
2024-07-02T05:45:49.960335495Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:860: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.960359196Z """
2024-07-02T05:45:49.963946348Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:701: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.963965648Z """Parameter group.
2024-07-02T05:45:49.965008563Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:860: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.965039064Z """
2024-07-02T05:45:49.967698302Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:701: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.967717703Z """Parameter group.
2024-07-02T05:45:49.968734017Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_models_py3.py:860: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.968750518Z """
2024-07-02T05:45:49.999791569Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/_generated/models/_search_index_client_enums.py:67: SyntaxWarning: invalid escape sequence '\ '
2024-07-02T05:45:49.999859070Z """This parameter is only valid if the query type is semantic. If set, the query returns
2024-07-02T05:45:50.074343753Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5100: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.074422654Z pattern: str = "\W+",
2024-07-02T05:45:50.076212480Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5181: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.076272081Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.077820704Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5233: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.077839604Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.078564615Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5100: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.078589815Z pattern: str = "\W+",
2024-07-02T05:45:50.079150123Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5181: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.079169123Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.079602130Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5233: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.079619030Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.080145738Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5325: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.080163438Z pattern: str = "\W+",
2024-07-02T05:45:50.085307113Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5325: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.085343213Z pattern: str = "\W+",
2024-07-02T05:45:50.088388558Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5100: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.088406458Z pattern: str = "\W+",
2024-07-02T05:45:50.088411858Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5181: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.088417758Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.089087968Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5233: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.089103768Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.089109368Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5325: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.089114068Z pattern: str = "\W+",
2024-07-02T05:45:50.109335162Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5100: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.109366863Z pattern: str = "\W+",
2024-07-02T05:45:50.109372763Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5181: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.109377463Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.109383263Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5233: SyntaxWarning: invalid escape sequence '\s'
2024-07-02T05:45:50.109387963Z """A character filter that replaces characters in the input string. It uses a regular expression
2024-07-02T05:45:50.109392063Z /tmp/8dc99dd633b9950/antenv/lib/python3.12/site-packages/azure/search/documents/indexes/_generated/models/_models_py3.py:5325: SyntaxWarning: invalid escape sequence '\W'
2024-07-02T05:45:50.109396663Z pattern: str = "\W+",

System Info

System Information

OS: Windows
OS Version: 10.0.22621
Python Version: 3.12.3 (tags/v3.12.3:f6650f9, Apr 9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)]

Package Information

langchain_core: 0.2.9
langchain: 0.2.5
langchain_community: 0.2.5
langsmith: 0.1.82
langchain_openai: 0.1.9
langchain_text_splitters: 0.2.1
langchainhub: 0.1.20

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph

I am also using pip package: azure-search-documents 11.4.0

[dosubot[bot]]

To resolve the syntax warnings related to invalid escape sequences in the Azure Search documents package, you should use raw string literals by prefixing the string with an r. This is particularly useful for strings that contain backslashes (\), which are common in file paths and regular expressions.

For example, if you encounter such warnings, you can convert the problematic strings to raw strings. Here is a general example:

pattern: str = r"\W+"

In your specific case, you should ensure that any string literals that might contain backslashes are prefixed with r to avoid invalid escape sequence warnings. This should be applied to all instances where the warnings are being raised.

Here is an example of how you might modify the code:

pattern: str = r"\W+"

By making these changes, you should be able to resolve the syntax warnings related to invalid escape sequences in the Azure Search documents package ^[1].

To continue talking to Dosu, mention @dosu.

[levalencia]

@dosu how do I fix this in langchain? I cant reproduce the errror locally on my environment, but when deployed to Azure App Service I get this error

[dosubot[bot]]

To fix the syntax warnings related to invalid escape sequences in the Azure Search documents package when deploying to Azure App Service, you should ensure that any string literals containing backslashes are prefixed with r to denote raw strings. This is particularly important for strings used in regular expressions or file paths.

Here is an example of how you might modify the code:

pattern: str = r"\W+"

In your specific case, you should ensure that all instances where the warnings are being raised are updated to use raw string literals. This should be applied to all the problematic strings in the generated models of the azure-search-documents package.

Since you cannot reproduce the error locally, it might be due to differences in the environment or Python version. Ensure that your local environment matches the deployment environment as closely as possible.

If the issue persists, you might need to manually edit the generated models in the azure-search-documents package or use a script to automate this process. Here is a general approach to automate the fix:

1. Create a script to update the generated models:

import re
import os

def fix_invalid_escape_sequences(file_path):
    with open(file_path, 'r') as file:
        content = file.read()

    # Replace invalid escape sequences with raw strings
    content = re.sub(r'(?<!r)"(.*?\\.*?)(?<!r)"', r'r"\1"', content)

    with open(file_path, 'w') as file:
        file.write(content)

# Path to the directory containing the generated models
models_dir = '/path/to/azure/search/documents/_generated/models/'

for root, _, files in os.walk(models_dir):
    for file in files:
        if file.endswith('.py'):
            fix_invalid_escape_sequences(os.path.join(root, file))

2. Run the script before deploying your application:

Ensure that this script is executed as part of your deployment process to fix the invalid escape sequences in the generated models.

By making these changes, you should be able to resolve the syntax warnings related to invalid escape sequences in the Azure Search documents package when deploying to Azure App Service.

To continue talking to Dosu, mention @dosu.

[levalencia]

@hwchase17 @dosu

I found the solution, but not the root cause.
Downgrading Python from 3.12 to 3.11 in Azure App Service fixes this issue. I wonder where this issue comes from? Azure Search Retriever from langchain? Azure Search SDK from MIcrosoft? or Python itself.

[dosubot[bot]]

The issue you're encountering is due to changes in Python 3.12 related to how escape sequences are handled in regular expressions. Python 3.12 has stricter handling of escape sequences, which can lead to warnings if they are not properly defined.

In your case, the warnings are coming from the azure-search-documents package, which likely contains string literals with backslashes that are not prefixed with r to denote raw strings. This stricter handling in Python 3.12 is causing the syntax warnings.

Downgrading to Python 3.11 avoids this issue because the stricter handling of escape sequences was introduced in Python 3.12. To permanently resolve the issue, you would need to ensure that all string literals with backslashes in the azure-search-documents package are correctly defined as raw strings.

There is no specific code in the Azure Search Retriever from LangChain that uses regular expressions and could be affected by these changes ^[1]. The root cause is the azure-search-documents package and the changes in Python 3.12 itself.

To continue talking to Dosu, mention @dosu.