Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substantial Agent Search Updates #3994

Merged
merged 1 commit into from
Feb 14, 2025

Conversation

joachim-danswer
Copy link
Contributor

@joachim-danswer joachim-danswer commented Feb 13, 2025

Description

Updates:

  • refined subquestion generation gets access to the answers to the initial ones (code + prompts)
  • major general prompt work
  • added time logging to verification steps
  • added standard timings to all LangGraph nodes with LLM executions
  • updated agent_test framework
  • added validation step after the refined answer generation
  • fixed doc string construction for sub-questions
  • revised doc context strategy
  • streaming out more docs than cited/verified
  • also includes timing changes

Here is the linear ticket: https://linear.app/danswer/issue/DAN-1452/agent-search-update

How Has This Been Tested?

Local tests with plenty of questions

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

  • This PR should be backported (make sure to check that the backport attempt succeeds)
  • [Optional] Override Linear Check

Copy link

vercel bot commented Feb 13, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
internal-search ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 14, 2025 6:59pm

Copy link
Contributor

@evan-danswer evan-danswer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did a huddle to do review together, LGTM

Copy link
Contributor

@evan-danswer evan-danswer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as long as you tested and it works!


# Sort by score in descending order, handling None scores
sorted_sections = sorted(
unique_sections.values(), key=lambda x: x.center_chunk.score or 0, reverse=True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If multiple docs have the same score (say 0) this does not give a unique order

relevant_doc_ids = [doc.center_chunk.document_id for doc in relevant_docs]

# Start with relevant docs or fallback to original question docs
streaming_documents = relevant_docs.copy()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no more fallback to original question docs? If so, remove "fallback to" comment

@joachim-danswer joachim-danswer force-pushed the agent-search-prompt-change-v2_a branch from deb0305 to 89be53c Compare February 14, 2025 18:54
@joachim-danswer joachim-danswer added this pull request to the merge queue Feb 14, 2025
Merged via the queue into main with commit 6687d5d Feb 14, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants