Merge from main #3
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- **Description:** Adapt JinaEmbeddings to run with the new Jina AI Rerank API - **Twitter handle:** https://twitter.com/JinaAI_ - [ ] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ --------- Co-authored-by: Bagatur <[email protected]> Co-authored-by: Bagatur <[email protected]>
Thank you for contributing to LangChain! - [x] **PR title**: Fix misplaced zep cloud example links - [x] **PR message**: - **Description:** Fixes misplaced links for vector store and memory zep cloud examples - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.
Thank you for contributing to LangChain! - [x] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" **Description:** In VoyageAI text-embedding examples use voyage-law-2 model - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.
…of RecursiveUrlLoader document (#20875) **Description:** The RecursiveUrlLoader loader offers a link_regex parameter that can filter out URLs. However, this filtering capability is limited, and if the internal links of the website change, unexpected resources may be loaded. These resources, such as font files, can cause problems in subsequent embedding processing. > https://blog.langchain.dev/assets/fonts/source-sans-pro-v21-latin-ext_latin-regular.woff2?v=0312715cbf We can add the Content-Type in the HTTP response headers to the document metadata so developers can choose which resources to use. This allows developers to make their own choices. For example, the following may be a good choice for text knowledge. - text/plain - simple text file - text/html - HTML web page - text/xml - XML format file - text/json - JSON format data - application/pdf - PDF file - application/msword - Word document and ignore the following - text/css - CSS stylesheet - text/javascript - JavaScript script - application/octet-stream - binary data - image/jpeg - JPEG image - image/png - PNG image - image/gif - GIF image - image/svg+xml - SVG image - audio/mpeg - MPEG audio files - video/mp4 - MP4 video file - application/font-woff - WOFF font file - application/font-ttf - TTF font file - application/zip - ZIP compressed file - application/octet-stream - binary data **Twitter handle:** @coolbeevip --------- Co-authored-by: Bagatur <[email protected]>
…20876) For driver connection reusage, we introduce passing the graph object to neo4j integrations
**Description:** This PR removes an unnecessary code snippet from the documentation. The snippet in question is not relevant to the content and does not contribute to the overall understanding of the topic. It contained redundant imports and unused code, potentially causing confusion for readers. **Issue:** There is no specific issue number associated with this change. **Dependencies:** No additional dependencies are required for this change. --------- Co-authored-by: Bagatur <[email protected]> Co-authored-by: Bagatur <[email protected]>
Does not update docs.
Thank you for contributing to LangChain!
- [x] **PR title**
- [x] **PR message**:
- **Description:** Deprecate persist method in Chroma no longer exists
in Chroma 0.4.x
- **Issue:** #20851
- **Dependencies:** None
- **Twitter handle:** AndresAlgaba1
- [x] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.
- [x] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/
---------
Co-authored-by: Bagatur <[email protected]>
Description: Fix tqdm import in QuantizedBiEncoderEmbeddings
Co-authored-by: kaka <[email protected]> Co-authored-by: Bagatur <[email protected]> Co-authored-by: jingsi <[email protected]> Co-authored-by: Bagatur <[email protected]>
Add __version__ to cli
…-server. (#20468) Description: Add support for Semantic topics and entities. Classification done by pebblo-server is not used to enhance metadata of Documents loaded by document loaders. Dependencies: None Documentation: Updated. Signed-off-by: Rahul Tripathi <[email protected]> Co-authored-by: Rahul Tripathi <[email protected]>
…tion (#20833) - **Description**: - **add support for more data types**: by default `IpexLLM` will load the model in int4 format. This PR adds more data types support such as `sym_in5`, `sym_int8`, etc. Data formats like NF3, NF4, FP4 and FP8 are only supported on GPU and will be added in future PR. - Fix a small issue in saving/loading, update api docs - **Dependencies**: `ipex-llm` library - **Document**: In `docs/docs/integrations/llms/ipex_llm.ipynb`, added instructions for saving/loading low-bit model. - **Tests**: added new test cases to `libs/community/tests/integration_tests/llms/test_ipex_llm.py`, added config params. - **Contribution maintainer**: @shane-huang
…-space separated list (#20434) - **Description:** Changes `lanchain_core.output_parsers.CommaSeparatedListOutputParser` to handle `,` as a delimiter alongside the previous implementation which used `, ` as delimiter. - **Issue:** Started noticing that some results returned by LLMs were not getting parsed correctly when the output contained `,` instead of `, `. - **Dependencies:** No - **Twitter handle:** not active on twitter. <!--- If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. -->
…exception (#20219) **Description:** Fixes a bug in the HuggingGPT task execution logic here: except Exception as e: self.status = "failed" self.message = str(e) self.status = "completed" self.save_product() where a caught exception effectively just sets `self.message` and can then throw an exception if, e.g., `self.product` is not defined. **Issue:** None that I'm aware of. **Dependencies:** None **Twitter handle:** https://twitter.com/michaeljschock Co-authored-by: Bagatur <[email protected]>
…books (#20002) - [ ] **Kinetica Document Loader**: "community: a class to load Documents from Kinetica" - [ ] **Kinetica Document Loader**: - **Description:** implemented KineticaLoader in `kinetica_loader.py` - **Dependencies:** install the Kinetica API using `pip install gpudb==7.2.0.1 `
…20269) **Description**: _PebbloSafeLoader_: Add support for pebblo server and client version **Documentation:** NA **Unit test:** NA **Issue:** NA **Dependencies:** None --------- Co-authored-by: Bagatur <[email protected]>
) **Description:** AzureSearch vector store has no tests. This PR adds initial tests to validate the code can be imported and used. **Issue:** N/A **Dependencies:** azure-search-documents and azure-identity are added as optional dependencies for testing --------- Co-authored-by: Matt Gotteiner <[email protected]> Co-authored-by: Bagatur <[email protected]>
`langchain_pinecone.Pinecone` is deprecated in favor of `PineconeVectorStore`, and is currently a subclass of `PineconeVectorStore`. ```python @deprecated(since="0.0.3", removal="0.2.0", alternative="PineconeVectorStore") class Pinecone(PineconeVectorStore): """Deprecated. Use PineconeVectorStore instead.""" pass ```
…py assertions (issue-20919) (#20920) **Description:** Fix integrated test case test_recursive_url_loader.py Local testing successful ```shell (venv) lei@LeideMacBook-Pro community % poetry run pytest tests/integration_tests/document_loaders/test_recursive_url_loader.py ================================================================================ test session starts ================================================================================ platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.4.0 -- /Users/zhanglei/Work/github/langchain/venv/bin/python cachedir: .pytest_cache rootdir: /Users/zhanglei/Work/github/langchain/libs/community configfile: pyproject.toml plugins: syrupy-4.6.1, asyncio-0.20.3, cov-4.1.0, vcr-1.0.2, mock-3.12.0, anyio-3.7.1, dotenv-0.5.2, requests-mock-1.11.0, socket-0.6.0 asyncio: mode=Mode.AUTO collected 6 items tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader PASSED [ 16%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic PASSED [ 33%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader FAILED [ 50%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent PASSED [ 66%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_loading_invalid_url PASSED [ 83%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties PASSED [100%] ===================================================================================== FAILURES ====================================================================================== __________________________________________________________________________ test_sync_recursive_url_loader ___________________________________________________________________________ def test_sync_recursive_url_loader() -> None: url = "https://docs.python.org/3.9/" loader = RecursiveUrlLoader( url, extractor=lambda _: "placeholder", use_async=False, max_depth=2 ) docs = loader.load() > assert len(docs) == 23 E AssertionError: assert 24 == 23 E + where 24 = len([Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html', 'title': '3.9.18 Documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/py-modindex.html', 'content_type': 'text/html', 'title': 'Python Module Index — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/download.html', 'content_type': 'text/html', 'title': 'Download — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/howto/index.html', 'content_type': 'text/html', 'title': 'Python HOWTOs — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/whatsnew/index.html', 'content_type': 'text/html', 'title': 'Whatâ\x80\x99s New in Python — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/c-api/index.html', 'content_type': 'text/html', 'title': 'Python/C API Reference Manual — Python 3.9.18 documentation', 'language': None}), ...]) tests/integration_tests/document_loaders/test_recursive_url_loader.py:38: AssertionError ================================================================================= warnings summary ================================================================================== tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties /Users/zhanglei/.pyenv/versions/3.11.4/lib/python3.11/html/parser.py:170: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. k = self.parse_starttag(i) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================ slowest 5 durations ================================================================================ 56.75s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic 38.99s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader 31.20s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties 30.37s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent 15.44s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader ============================================================================== short test summary info ============================================================================== FAILED tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader - AssertionError: assert 24 == 23 ================================================================ 1 failed, 5 passed, 5 warnings in 172.97s (0:02:52) ================================================================ (venv) zhanglei@LeideMacBook-Pro community % poetry run pytest tests/integration_tests/document_loaders/test_recursive_url_loader.py ================================================================================ test session starts ================================================================================ platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.4.0 -- /Users/zhanglei/Work/github/langchain/venv/bin/python cachedir: .pytest_cache rootdir: /Users/zhanglei/Work/github/langchain/libs/community configfile: pyproject.toml plugins: syrupy-4.6.1, asyncio-0.20.3, cov-4.1.0, vcr-1.0.2, mock-3.12.0, anyio-3.7.1, dotenv-0.5.2, requests-mock-1.11.0, socket-0.6.0 asyncio: mode=Mode.AUTO collected 6 items tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader PASSED [ 16%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic PASSED [ 33%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader PASSED [ 50%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent PASSED [ 66%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_loading_invalid_url PASSED [ 83%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties PASSED [100%] ================================================================================= warnings summary ================================================================================== tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties /Users/zhanglei/.pyenv/versions/3.11.4/lib/python3.11/html/parser.py:170: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. k = self.parse_starttag(i) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================ slowest 5 durations ================================================================================ 46.99s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic 32.43s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader 31.23s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent 30.75s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties 15.89s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader ===================================================================== 6 passed, 5 warnings in 157.42s (0:02:37) ===================================================================== (venv) lei@LeideMacBook-Pro community % ``` **Issue:** #20919 **Twitter handle:** @coolbeevip
Adds a first version of the migrate script.
Test a helper function that was added earlier.
**Description:** Update LarkSuite loader doc to give an example for loading data from LarkSuite wiki. **Issue:** None **Dependencies:** None **Twitter handle:** None
Issue: the `langkit` package is not presented in the `pyproject.toml` but it is a requirement for the `WhyLabsCallbackHandler` Change: added `langkit` --------- Co-authored-by: Chester Curme <[email protected]>
More doc updates por favor @baskaryan!
…tors (#21209) If Session and/or keyspace are not provided, they are resolved from cassio's context. So they are not required. This change is fully backward compatible.
Thank you for contributing to LangChain! community:baichuan[patch]: standardize init args updated `baichuan_api_key` so that aliased to `api_key`. Added test that it continues to set the same underlying attribute. Test checks for `SecretStr` updated `temperature` with Pydantic Field, added unit test. Related to #20085
Reverts #21174 Hey team - going to revert this because it doesn't seem necessary for testing. We should only be adding optional + extended_testing dependencies for deps that have extended tests. otherwise it just increases probability of dependency conflicts in the community lockfile.
Issue: `load_qa_chain` is placed in the __init__.py file. As a result, it is not listed in the API Reference docs. BTW `load_qa_chain` is heavily presented in the doc examples, but is missed in API Ref. Change: moved code from init.py into a new file. Related: #21266
The NemoEmbeddings is deprecated, instead use langchain-nvidia-ai-endpoints NVIDIAEmbeddings interface. cc: @mattf --------- Co-authored-by: Daniel Glogowski <[email protected]> Co-authored-by: andyjessen <[email protected]> Co-authored-by: Chris Germann <[email protected]> Co-authored-by: gere <[email protected]> Co-authored-by: Chester Curme <[email protected]>
…Yellowbrick vector store (#20856) - **Description:** Add LSH-based indexing to the Yellowbrick vector store module - **Twitter handle:** @markcusack --------- Co-authored-by: markcusack <[email protected]> Co-authored-by: markcusack <[email protected]> Co-authored-by: Eugene Yurtsev <[email protected]>
Description: this change adds args_schema (pydantic BaseModel) to
YahooFinanceNewsTool for correct schema formatting on LLM function calls
Issue: currently using YahooFinanceNewsTool with OpenAI function calling
returns the following error "TypeError("YahooFinanceNewsTool._run() got
an unexpected keyword argument '__arg1'")". This happens because the
schema sent to the LLM is "input: "{'__arg1': 'MSFT'}"" while the method
should be called with the "query" parameter.
Co-authored-by: Bagatur <[email protected]>
valid for from_documents() as well fixes #21251
- **Description:** fix: variable names in root validator not allowing pass credentials as named parameters in llm instancing, also added sambanova's sambaverse and sambastudio llms to __init__.py for module import
- **Description:** SambaNova hosted embeddings integration
Thank you for contributing to LangChain!
- [ ] **PR title**: "package: description"
- Where "package" is whichever of langchain, community, core,
experimental, etc. is being modified. Use "docs: ..." for purely docs
changes, "templates: ..." for template changes, "infra: ..." for CI
changes.
- Example: "community: add foobar LLM"
- [ ] **PR message**: ***Delete this entire checklist*** and replace
with
- **Description:** a description of the change
- **Issue:** the issue # it fixes, if applicable
- **Dependencies:** any dependencies required for this change
- **Twitter handle:** if your PR gets announced, and you'd like a
mention, we'll gladly shout you out!
- [ ] **Add tests and docs**: If you're adding a new integration, please
include
1. a test for the integration, preferably unit tests that do not rely on
network access,
2. an example notebook showing its use. It lives in
`docs/docs/integrations` directory.
- [ ] **Lint and test**: Run `make format`, `make lint` and `make test`
from the root of the package(s) you've modified. See contribution
guidelines for more: https://python.langchain.com/docs/contributing/
Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to pyproject.toml files (even optional
ones) unless they are required for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
- If you are adding something to community, do not re-import it in
langchain.
If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, hwchase17.
… parameter (#20695) - **Description:** add **DocumentRelevanceOverrideConfigurations** request parameter to Kendra retriever Co-authored-by: Simone Caserini <[email protected]> Co-authored-by: Bagatur <[email protected]>
[Standardized model init args #20085](#20085) - Enable premai chat model to be initialized with `model_name` as an alias for `model`, `api_key` as an alias for `premai_api_key`. - Add initialization test `test_premai_initialization` --------- Co-authored-by: Chester Curme <[email protected]>
Relates [#17048] Description : Applied fix to redis and neo4j file. Error was : `Cannot override writeable attribute with read-only property` fix with the same solution of [[langchain/libs/community/langchain_community/chat_message_histories/elasticsearch.py](https://github.com/langchain-ai/langchain/blob/d5c412b0a9a8af02754de60f93d93c3efa33ccdb/libs/community/langchain_community/chat_message_histories/elasticsearch.py#L170-L175)] --------- Co-authored-by: Bagatur <[email protected]> Co-authored-by: Chester Curme <[email protected]>
@efriis @hwchase17 --------- Co-authored-by: Erick Friis <[email protected]>
**Description:** Adding chat completions to the Together AI package, which is our most popular API. Also staying backwards compatible with the old API so folks can continue to use the completions API as well. Also moved the embedding API to use the OpenAI library to standardize it further. **Twitter handle:** @Nutlope - [x] **Add tests and docs**: If you're adding a new integration, please include - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Erick Friis <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.