Failed with deepseek-r1 #18

meefen · 2025-01-30T01:57:58Z

Thanks for the nice repo!

I was able to run it with llama3.1 but encountered an error with generate_query when running on deepseek-r1. Any reason why?

Traceback (most recent call last):
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph_api/queue.py", line 244, in worker
    await asyncio.wait_for(consume(stream, run_id), timeout)
  File "/Users/macuser/.pyenv/versions/3.11.8/lib/python3.11/asyncio/tasks.py", line 489, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph_api/stream.py", line 292, in consume
    raise e from None
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph_api/stream.py", line 282, in consume
    async for mode, payload in stream:
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph_api/stream.py", line 233, in astream_state
    event = await wait_if_not_done(anext(stream, sentinel), done)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph_api/asyncio.py", line 72, in wait_if_not_done
    raise e.exceptions[0] from None
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/pregel/__init__.py", line 1899, in astream
    async for _ in runner.atick(
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/pregel/runner.py", line 527, in atick
    _panic_or_proceed(
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/pregel/runner.py", line 619, in _panic_or_proceed
    raise exc
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/pregel/retry.py", line 128, in arun_with_retry
    return await task.proc.ainvoke(task.input, config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/utils/runnable.py", line 499, in ainvoke
    input = await step.ainvoke(input, config, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langgraph/utils/runnable.py", line 287, in ainvoke
    ret = await asyncio.create_task(coro, context=context)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 588, in run_in_executor
    return await asyncio.get_running_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.pyenv/versions/3.11.8/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/.cache/uv/archive-v0/uyc5n4k0VupSgTHxi-1TD/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 579, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/macuser/src/ollama-deep-researcher/./src/assistant/graph.py", line 31, in generate_query
    return {"search_query": query['query']}
                            ~~~~~^^^^^^^^^
KeyError: 'query'
During task with name 'generate_query' and id 'ed895b40-783d-d5de-dbd4-d6b127e31fbc'

The text was updated successfully, but these errors were encountered:

brucebai · 2025-01-30T13:57:53Z

have the same issue ~

rlancemartin · 2025-01-30T21:47:54Z

@brucebai / @meefen which specific distilled r1 model are you using?

Almost certainly an issue with producing structured output.

If you tell me the model, I can try to repro.

meefen · 2025-01-30T21:51:59Z

For me, I was using deepseek-r1, the same one shown in the demo.

It is 8b, and I did try to add deepseek-r1:8b or deepseek-r1:latest, but had the same outcome.

rlancemartin · 2025-01-30T22:43:47Z

For me, I was using deepseek-r1, the same one shown in the demo.
It is 8b, and I did try to add deepseek-r1:8b or deepseek-r1:latest, but had the same outcome.

Strange, I assume this is an issue w/ the local model failing to produce JSON object w/ the correct query key.

This can indeed happen.

I just tried w/ 1.5b and it was OK.

But, I could absolutely see cases where it fails to a JSON object w/ query key.

I will try to improve the prompt now.

rlancemartin · 2025-01-30T22:45:23Z

@meefen if easy, can you add a print statement in assistant/graph.py" --

print(result.content) # add
query = json.loads(result.content)
print(print) # add

I want to see what you are getting.

rlancemartin · 2025-01-30T23:04:47Z

FWIW, I also just updated the prompt. Pls pull and see if you still see this problem:
73a0fc4

andyxudong86 · 2025-01-31T06:19:06Z

same issue for me. it''s good when using llama3.1:8b, but failed when using deepseek r1:7b or 1.5b. same error.

2025-01-31T06:17:13.432797Z [error ] Background run failed [langgraph_api.queue] api_variant=local_dev run_attempt=1 run_created_at=2025-01-31T06:16:56.643864+00:00 run_ended_at=2025-01-31T06:17:13.431770+00:00 run_exec_ms=16337
run_id=1efdf9af-8f35-6672-acad-ea80d74be41e run_started_at=2025-01-31T06:16:57.094363+00:00
Traceback (most recent call last):
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\queue.py", line 244, in worker
await asyncio.wait_for(consume(stream, run_id), timeout)
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\asyncio\tasks.py", line 489, in wait_for
return fut.result()
^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 292, in consume
raise e from None
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 282, in consume
async for mode, payload in stream:
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 233, in astream_state
event = await wait_if_not_done(anext(stream, sentinel), done)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\asyncio.py", line 72, in wait_if_not_done
raise e.exceptions[0] from None
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel_init_.py", line 2007, in astream
async for _ in runner.atick(
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\runner.py", line 527, in atick
_panic_or_proceed(
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\runner.py", line 619, in _panic_or_proceed
raise exc
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\retry.py", line 128, in arun_with_retry
return await task.proc.ainvoke(task.input, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\utils\runnable.py", line 532, in ainvoke
input = await step.ainvoke(input, config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\utils\runnable.py", line 320, in ainvoke
ret = await asyncio.create_task(coro, context=context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 588, in run_in_executor
return await asyncio.get_running_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 579, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\ollama-deep-researcher./src/assistant/graph.py", line 31, in generate_query
return {"search_query": query['query']}
~~~~~^^^^^^^^^
KeyError: 'query'

rlancemartin · 2025-01-31T06:27:32Z

same issue for me. it''s good when using llama3.1:8b, but failed when using deepseek r1:7b or 1.5b. same error.

2025-01-31T06:17:13.432797Z [error ] Background run failed [langgraph_api.queue] api_variant=local_dev run_attempt=1 run_created_at=2025-01-31T06:16:56.643864+00:00 run_ended_at=2025-01-31T06:17:13.431770+00:00 run_exec_ms=16337 run_id=1efdf9af-8f35-6672-acad-ea80d74be41e run_started_at=2025-01-31T06:16:57.094363+00:00 Traceback (most recent call last): File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\queue.py", line 244, in worker await asyncio.wait_for(consume(stream, run_id), timeout) File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\asyncio\tasks.py", line 489, in wait_for return fut.result() ^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 292, in consume raise e from None File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 282, in consume async for mode, payload in stream: File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\stream.py", line 233, in astream_state event = await wait_if_not_done(anext(stream, sentinel), done) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\asyncio.py", line 72, in wait_if_not_done raise e.exceptions[0] from None File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel__init__.py", line 2007, in astream async for _ in runner.atick( File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\runner.py", line 527, in atick _panic_or_proceed( File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\runner.py", line 619, in _panic_or_proceed raise exc File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\pregel\retry.py", line 128, in arun_with_retry return await task.proc.ainvoke(task.input, config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\utils\runnable.py", line 532, in ainvoke input = await step.ainvoke(input, config, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph\utils\runnable.py", line 320, in ainvoke ret = await asyncio.create_task(coro, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 588, in run_in_executor return await asyncio.get_running_loop().run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 579, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Andy\ollama-deep-researcher./src/assistant/graph.py", line 31, in generate_query return {"search_query": query['query']} ~~~~~^^^^^^^^^ KeyError: 'query'

did you pull latest?

if so, can log:

print(result.content) # add
query = json.loads(result.content)
print(print) # add

and send here?

andyxudong86 · 2025-01-31T06:37:24Z

pls check the follwoing.

2025-01-31T06:35:06.092941Z [error ] Traceback (most recent call last):
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\starlette\routing.py", line 693, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\lifespan.py", line 30, in lifespan
await collect_graphs_from_env(True)
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\graph.py", line 257, in collect_graphs_from_env
graph = await run_in_executor(None, _graph_from_spec, spec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 588, in run_in_executor
return await asyncio.get_running_loop().run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core\runnables\config.py", line 579, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Andy\AppData\Local\Programs\Python\Python311\Lib\site-packages\langgraph_api\graph.py", line 295, in _graph_from_spec
modspec.loader.exec_module(module)
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "C:\Users\Andy\ollama-deep-researcher./src/assistant/graph.py", line 159, in
print(result.content) # add
^^^^^^
NameError: name 'result' is not defined
[uvicorn.error] api_variant=local_dev
2025-01-31T06:35:06.092941Z [error ] Application startup failed. Exiting. [uvicorn.error] api_variant=local_dev

rlancemartin · 2025-01-31T15:51:12Z

print

This means that there is no output from your local LLM call.

    llm_json_mode = ChatOllama(model=configurable.local_llm, temperature=0, format="json")
    result = llm_json_mode.invoke(
        [SystemMessage(content=query_writer_instructions_formatted),
        HumanMessage(content=f"Generate a query for web search:")]
    )

1/ you are sure that Ollama app is running?
2/ you tested a simple call w/ model of interest in isolation (outside of the app)?
E.g., ChatOllama(model=your_model, format="json").
3/ what local llm are you using?

andyxudong86 · 2025-02-01T06:51:25Z

yes, ollama is running
simple call from outside is good
local llm is deepseek-r1:7b

DoLife · 2025-02-02T18:55:44Z

I have the same issue. All other ollama models work except for deepseek r1. I am running deepseek-r1:latest the 7billion model.

As for the log output, the following:
{

}

As you can see, it is all empty. Exactly as pasted here.

rlancemartin · 2025-02-02T19:21:22Z

I have the same issue. All other ollama models work except for deepseek r1. I am running deepseek-r1:latest the 7billion model.

As for the log output, the following:
{

}

As you can see, it is all empty. Exactly as pasted here.

Very strange. Small point: you mean Deepseek 8b right? What's the exact model name you are specifying?

DoLife · 2025-02-03T05:13:04Z

I have the same issue. All other ollama models work except for deepseek r1. I am running deepseek-r1:latest the 7billion model.
As for the log output, the following:
{
}

As you can see, it is all empty. Exactly as pasted here.

Very strange. Small point: you mean Deepseek 8b right? What's the exact model name you are specifying?

It is the 7b model, 4.7gb, there is a 8b model, but it is not the one I am using.

DoLife · 2025-02-03T05:43:13Z

Just as a side note. I don`t see anywhere where you provide the api endpoint for ollama. I had to include it manually to get the code working. base_url="http://127.0.0.1:11434"

rlancemartin · 2025-02-03T05:57:07Z

Just as a side note. I don`t see anywhere where you provide the api endpoint for ollama. I had to include it manually to get the code working. base_url="http://127.0.0.1:11434"

I'm running the app on Mac and have not needed to supply. I will check with the Ollama folks.

DoLife · 2025-02-03T08:07:08Z

Just as a side note. I don`t see anywhere where you provide the api endpoint for ollama. I had to include it manually to get the code working. base_url="http://127.0.0.1:11434"

I'm running the app on Mac and have not needed to supply. I will check with the Ollama folks.

I am running on a windows system, maybe that is the difference. I kept getting a ConnectError: WinError 10049

Here is the output of chatgpt regarding the issue:
Addressing errors

OK, let me see: "format_exc_info" should be removed for pretty exceptions. The crucial issue is httpx ConnectError: WinError 10049, indicating an incorrect or missing IP address configuration.

Pinpointing the issue

I’m digging into the httpx ConnectError: WinError 10049 error. It’s likely due to an incorrect IP address in the httpx request or a misconfigured host.

Fixing issues

To address the structlog warning, remove format_exc_info from the processors. For the httpx connection error, validate the host and port parameters.

Fixing the address

To resolve the httpx connection error, ensure the host address is correctly specified in the langchain_ollama config, possibly in ollama client instantiation or relevant methods in langchain_ollama/chat_models.py.

DoLife · 2025-02-03T08:08:54Z

Just as a side note. I don`t see anywhere where you provide the api endpoint for ollama. I had to include it manually to get the code working. base_url="http://127.0.0.1:11434"

I'm running the app on Mac and have not needed to supply. I will check with the Ollama folks.

Thank you for this repository. It is great!

DoLife · 2025-02-03T08:12:07Z

Just as a side note. I don`t see anywhere where you provide the api endpoint for ollama. I had to include it manually to get the code working. base_url="http://127.0.0.1:11434"

I'm running the app on Mac and have not needed to supply. I will check with the Ollama folks.

Ollama was written for Mac, so everything will work correctly for it out of the box.

junglegao · 2025-02-06T09:46:54Z

How to run ChatOllama out of the app? I got the same issue in mac, and ollama was running.

from langchain_ollama import ChatOllama
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'langchain_ollama'

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed with deepseek-r1 #18

Failed with deepseek-r1 #18

meefen commented Jan 30, 2025

brucebai commented Jan 30, 2025

rlancemartin commented Jan 30, 2025

meefen commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 30, 2025

andyxudong86 commented Jan 31, 2025

rlancemartin commented Jan 31, 2025

andyxudong86 commented Jan 31, 2025

rlancemartin commented Jan 31, 2025 •

edited

Loading

andyxudong86 commented Feb 1, 2025

DoLife commented Feb 2, 2025 •

edited

Loading

rlancemartin commented Feb 2, 2025

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

rlancemartin commented Feb 3, 2025 •

edited

Loading

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

junglegao commented Feb 6, 2025 •

edited

Loading

Failed with deepseek-r1 #18

Failed with deepseek-r1 #18

Comments

meefen commented Jan 30, 2025

brucebai commented Jan 30, 2025

rlancemartin commented Jan 30, 2025

meefen commented Jan 30, 2025 • edited Loading

rlancemartin commented Jan 30, 2025 • edited Loading

rlancemartin commented Jan 30, 2025 • edited Loading

rlancemartin commented Jan 30, 2025

andyxudong86 commented Jan 31, 2025

rlancemartin commented Jan 31, 2025

andyxudong86 commented Jan 31, 2025

rlancemartin commented Jan 31, 2025 • edited Loading

andyxudong86 commented Feb 1, 2025

DoLife commented Feb 2, 2025 • edited Loading

rlancemartin commented Feb 2, 2025

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

rlancemartin commented Feb 3, 2025 • edited Loading

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

DoLife commented Feb 3, 2025

junglegao commented Feb 6, 2025 • edited Loading

meefen commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 30, 2025 •

edited

Loading

rlancemartin commented Jan 31, 2025 •

edited

Loading

DoLife commented Feb 2, 2025 •

edited

Loading

rlancemartin commented Feb 3, 2025 •

edited

Loading

junglegao commented Feb 6, 2025 •

edited

Loading