Error parsing agent response: TypeError: Cannot read properties of undefined (reading 'length')

[MX-Goliath]

When using ollama with the qwen2.5 or qwen2.5:14b model, I get this error:
Error parsing agent response: TypeError: Cannot read properties of undefined (reading 'length')

[kevinpiac]

It seems qwen2.5 either does not return any response or respond with an invalid JSON format.

I won't investigate further since it's not the main recommended model, but feel free to investigate.

[j-corral]

I got the same error with other ollama model (llama3.2 - llama3.3)
It seems to come from the ChatOllama.invoke part which does not works like the OpenaI one.
After having a quick look at LangChain, the ChatOllama may require to configure a structured Output : https://js.langchain.com/docs/integrations/chat/ollama/#withstructuredoutput.

Ollama result

[Openator] 💡 Starting manager agent
[Openator] 💡 Defining next task...
serialized {
  "endGoal": "Search for \"google pixel\" then click on the cheapest product.",
  "taskHistorySummary": []
}
response:  AIMessage {
  "content": "{ \n  \"actions\": [\n    { \n      \"type\": \"scrollUp\", \n      \"goal\": \"0 Pixel Above - See more elements\" \n    }, \n    {\n      \"type\": \"extractContent\",\n      \"goal\": \"Extract DOM Elements\"\n    }, \n    {\n      \"type\": \"fillInput\", \n      \"index\": 11, \n      \"text\": \"google pixel\" \n    }\n  ],\n  \"userTaskAndTaskHistory\": {\n    \"endGoal\": \"Search for \\\"google pixel\\\" then click on the cheapest product.\",\n    \"taskHistorySummary\": []\n  },\n  \"currentUrl\": \"https://www.dealabs.com/\",\n  \"memoryLearnings\": [\n    { \n      \"location\": \"0 Pixel Above\", \n      \"description\": \"See more elements\"\n    }, \n    {\n      \"location\": \"1-5 Pixel Below\", \n      \"description\": \"Multiple div elements\"\n    }, \n    {\n      \"location\": \"6-10 Pixel Below\", \n      \"description\": \"Specific div element with h2 tag\"\n    }\n  ],\n  \"nextGoal\": {\n    \"goal\": \"Click on the cheapest product\",\n    \"description\": \"Find and click on the cheapest product for \\\"google pixel\\\"\"\n  },\n  \"feedback\": null\n}",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3.2",
    "created_at": "2025-03-01T11:07:36.791369Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 8850737916,
    "load_duration": 1103750291,
    "prompt_eval_count": 2048,
    "prompt_eval_duration": 2716000000,
    "eval_count": 275,
    "eval_duration": 4439000000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 2048,
    "output_tokens": 275,
    "total_tokens": 2323
  }
}
parsed: {
  actions: [
    { type: 'scrollUp', goal: '0 Pixel Above - See more elements' },
    { type: 'extractContent', goal: 'Extract DOM Elements' },
    { type: 'fillInput', index: 11, text: 'google pixel' }
  ],
  userTaskAndTaskHistory: {
    endGoal: 'Search for "google pixel" then click on the cheapest product.',
    taskHistorySummary: []
  },
  currentUrl: 'https://www.dealabs.com/',
  memoryLearnings: [
    { location: '0 Pixel Above', description: 'See more elements' },
    {
      location: '1-5 Pixel Below',
      description: 'Multiple div elements'
    },
    {
      location: '6-10 Pixel Below',
      description: 'Specific div element with h2 tag'
    }
  ],
  nextGoal: {
    goal: 'Click on the cheapest product',
    description: 'Find and click on the cheapest product for "google pixel"'
  },
  feedback: null
}
safeActions: [
  { type: 'scrollUp', goal: '0 Pixel Above - See more elements' },
  { type: 'extractContent', goal: 'Extract DOM Elements' },
  { type: 'fillInput', index: 11, text: 'google pixel' }
]
Error parsing agent response: TypeError: Cannot read properties of undefined (reading 'nextGoal')
    at Openator.defineNextTask (WORKSPACE/AI/agents/openator/dist/index.js:867:71)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async WORKSPACE/AI/agents/openator/dist/index.js:807:30
[Openator] 💡 Executing task: Keep trying
[Openator] ✅ Keep trying
[Openator] 💡 Defining next task...

OPenAI result

[Openator] 💡 Starting manager agent
[Openator] 💡 Defining next task...
serialized {
  "endGoal": "Search for \"google pixel\" then click on the cheapest product.",
  "taskHistorySummary": []
}
parsed: {
  currentState: {
    evaluationPreviousGoal: 'We need to accept the cookies to proceed with the search.',
    memory: "Cookies acceptance is required to interact with the site. End goal is to search for 'google pixel' and click on the cheapest product.",
    nextGoal: 'Accept the cookies to proceed.'
  },
  actions: [
    {
      name: 'clickElement',
      params: [Object],
      description: "Click the 'Tout accepter' button to accept cookies."
    }
  ]
}
safeActions: [
  {
    name: 'clickElement',
    params: { index: 24 },
    description: "Click the 'Tout accepter' button to accept cookies."
  }
]
task: Task {
  id: 'b81cf29e-9c89-4296-9caa-9a15f72b6a32',
  goal: 'Accept the cookies to proceed.',
  actions: [
    TaskAction {
      data: [Object],
      id: 'da4a9c80-d256-4348-8f71-468f4a069c50',
      _status: 'pending',
      result: ''
    }
  ],
  _status: 'pending',
  _reason: undefined
}

At the moment, I don't have time to investigate further.

[kevinpiac]

Thanks for investigating this @j-corral

From by understanding configuring structured output is necessary for ToolCalling only, which does not seem related (mentioned here):

For models that support tool calling, you can also call .withStructuredOutput() to get a structured output from the tool.

In your example, we can see that the ollama3.3 model responds with a valid JSON that has been parsed correctly.

However when we pretty-print it, it looks like:

{
  "actions": [
    {
      "type": "scrollUp",
      "goal": "0 Pixel Above - See more elements"
    },
    {
      "type": "extractContent",
      "goal": "Extract DOM Elements"
    },
    {
      "type": "fillInput",
      "index": 11,
      "text": "google pixel"
    }
  ],
  "userTaskAndTaskHistory": {
    "endGoal": "Search for \"google pixel\" then click on the cheapest product.",
    "taskHistorySummary": []
  },
  "currentUrl": "https://www.dealabs.com/",
  "memoryLearnings": [
    {
      "location": "0 Pixel Above",
      "description": "See more elements"
    },
    {
      "location": "1-5 Pixel Below",
      "description": "Multiple div elements"
    },
    {
      "location": "6-10 Pixel Below",
      "description": "Specific div element with h2 tag"
    }
  ],
  "nextGoal": {
    "goal": "Click on the cheapest product",
    "description": "Find and click on the cheapest product for \"google pixel\""
  },
  "feedback": null
}

The problem is that it does not match the expected response format as defined here

Feel free to share more details if you investigate on your end. I would be happy we can solve this but I don't really know how we can influence the model to give better answers.

[MX-Goliath]

I was able to resolve this issue by increasing the contextual window size for Qwen. It also worked with Mistral Nemo. Ollama has a default window size of 2048, so I created a new model based on existing ones and added PARAMETER num_ctx 16384 to Modelfile.

[kevinpiac]

@MX-Goliath Thanks for coming up with this solution!

Would you mind sharing your code so (or issuing a PR) so I can integrate to the code base for everyone?