How to set max tokens for OllamaEmbeddings from langchain_ollama.embeddings

[Marsman1996]

For OllamaEmbeddings from langchain_community.embeddings, I can use the following code to set max tokens:

embedding_client = OllamaEmbeddings(
    base_url=f"http://localhost:11434",
    model="nomic-embed-text",
    num_ctx=6144,
)

But how to set max tokens for OllamaEmbeddings from langchain_ollama.embeddings?

[dosubot[bot]]

To set the maximum number of tokens for OllamaEmbeddings from langchain_ollama.embeddings, you can use the num_ctx parameter, similar to how you do it in langchain_community.embeddings. Here's an example:

from langchain_ollama import OllamaEmbeddings

embedding_client = OllamaEmbeddings(
    base_url="http://localhost:11434",
    model="nomic-embed-text",
    num_ctx=6144,
)

The num_ctx parameter sets the size of the context window used to generate the next token ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[Marsman1996]

@dosu The code you provided not work, here is the error message:

Traceback (most recent call last):
  File "/home/marsman1996/afgen/afgenllm/./test.py", line 3, in <module>
    embedding_client = OllamaEmbeddings(
        base_url="http://localhost:11434",
        model="nomic-embed-text",
        num_ctx=6144,
    )
  File "/home/marsman1996/afgen/afgenllm/.venv/lib/python3.13/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for OllamaEmbeddings
num_ctx
  Extra inputs are not permitted [type=extra_forbidden, input_value=6144, input_type=int]
    For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

And here is the definition of OllamaEmbeddings:

class OllamaEmbeddings(BaseModel, Embeddings):
    """Ollama embedding model integration.

    Set up a local Ollama instance:
        Install the Ollama package and set up a local Ollama instance
        using the instructions here: https://github.com/ollama/ollama .

        You will need to choose a model to serve.

        You can view a list of available models via the model library (https://ollama.com/library).

        To fetch a model from the Ollama model library use ``ollama pull <name-of-model>``.

        For example, to pull the llama3 model:

        .. code-block:: bash

            ollama pull llama3

        This will download the default tagged version of the model.
        Typically, the default points to the latest, smallest sized-parameter model.

        * On Mac, the models will be downloaded to ~/.ollama/models
        * On Linux (or WSL), the models will be stored at /usr/share/ollama/.ollama/models

        You can specify the exact version of the model of interest
        as such ``ollama pull vicuna:13b-v1.5-16k-q4_0``.

        To view pulled models:

        .. code-block:: bash

            ollama list

        To start serving:

        .. code-block:: bash

            ollama serve

        View the Ollama documentation for more commands.

        .. code-block:: bash

            ollama help

    Install the langchain-ollama integration package:
        .. code-block:: bash

            pip install -U langchain_ollama

    Key init args — completion params:
        model: str
            Name of Ollama model to use.
        base_url: Optional[str]
            Base url the model is hosted under.

    See full list of supported init args and their descriptions in the params section.

    Instantiate:
        .. code-block:: python

            from langchain_ollama import OllamaEmbeddings

            embed = OllamaEmbeddings(
                model="llama3"
            )

    Embed single text:
        .. code-block:: python

            input_text = "The meaning of life is 42"
            vector = embed.embed_query(input_text)
            print(vector[:3])

        .. code-block:: python

            [-0.024603435769677162, -0.007543657906353474, 0.0039630369283258915]

    Embed multiple texts:
        .. code-block:: python

             input_texts = ["Document 1...", "Document 2..."]
            vectors = embed.embed_documents(input_texts)
            print(len(vectors))
            # The first 3 coordinates for the first vector
            print(vectors[0][:3])

        .. code-block:: python

            2
            [-0.024603435769677162, -0.007543657906353474, 0.0039630369283258915]

    Async:
        .. code-block:: python

            vector = await embed.aembed_query(input_text)
           print(vector[:3])

            # multiple:
            # await embed.aembed_documents(input_texts)

        .. code-block:: python

            [-0.009100092574954033, 0.005071679595857859, -0.0029193938244134188]
    """  # noqa: E501

    model: str
    """Model name to use."""

    base_url: Optional[str] = None
    """Base url the model is hosted under."""

    client_kwargs: Optional[dict] = {}
    """Additional kwargs to pass to the httpx Client. 
    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
    """

    _client: Client = PrivateAttr(default=None)  # type: ignore
    """
    The client to use for making requests.
    """

    _async_client: AsyncClient = PrivateAttr(default=None)  # type: ignore
    """
    The async client to use for making requests.
    """

    model_config = ConfigDict(
        extra="forbid",
    )

    @model_validator(mode="after")
    def _set_clients(self) -> Self:
        """Set clients to use for ollama."""
        client_kwargs = self.client_kwargs or {}
        self._client = Client(host=self.base_url, **client_kwargs)
        self._async_client = AsyncClient(host=self.base_url, **client_kwargs)
        return self

[dosubot[bot]]

The error you're encountering is due to the OllamaEmbeddings class from langchain_ollama.embeddings not accepting the num_ctx parameter. The class is configured to forbid extra parameters that are not explicitly defined in its initialization.

To resolve this, you should remove the num_ctx parameter from your instantiation of OllamaEmbeddings. The current implementation of OllamaEmbeddings in langchain_ollama.embeddings does not support setting the context window size directly through a parameter like num_ctx. You can only use the parameters that are defined in the class, such as model, base_url, and client_kwargs ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[pvz122]

I have the same question too. Can anyone offer a help?

[SyedBaqarAbbas]

Hello @Marsman1996 and @pvz122,
I noticed that this functionality was missed in the package langchain_ollama. I have created a PR to #29296 solve your issue.

[Marsman1996]

Now I'm wondering why they mark OllamaEmbeddings from langchain_community.embeddings as deprecated when langchain_ollama.embeddings not ready...