together: add chat models, use openai base

docs/docs/integrations/llms/together.ipynb

@@ -7,15 +7,21 @@
    "source": [
     "# Together AI\n",
     "\n",
-    "> The Together API makes it easy to fine-tune or run leading open-source models with a couple lines of code. We have integrated the world’s leading open-source models, including Llama-2, RedPajama, Falcon, Alpaca, Stable Diffusion XL, and more. Read more: https://together.ai\n",
+    "> The Together API makes it easy to query and fine-tune leading open-source models with a couple lines of code. We have integrated the world’s leading open-source models, including Llama-3, Mixtral, DBRX, Stable Diffusion XL, and more. Read more: https://together.ai\n",
     "\n",
     "To use, you'll need an API key which you can find here:\n",
-    "https://api.together.xyz/settings/api-keys. This can be passed in as init param\n",
+    "https://api.together.ai/settings/api-keys. This can be passed in as init param\n",
     "``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.\n",
     "\n",
-    "Together API reference: https://docs.together.ai/reference"
+    "Together API reference: https://docs.together.ai"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "1c47fc36",
+   "metadata": {},
+   "source": []
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -28,40 +34,43 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 3,
+   "execution_count": null,
+   "id": "637bb53f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Running chat completions with Together AI\n",
+    "\n",
+    "from langchain_core.prompts import ChatPromptTemplate\n",
+    "from langchain_together import ChatTogether\n",
+    "\n",
+    "chat = ChatTogether()\n",
+    "\n",
+    "# using chat invoke\n",
+    "chat.invoke(\"Tell me fun things to do in NYC\")\n",
+    "\n",
+    "# using chat stream\n",
+    "for m in chat.stream(\"Tell me fun things to do in NYC\"):\n",
+    "    print(m)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
    "id": "e7b7170d-d7c5-4890-9714-a37238343805",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on a large amount of text data. It is able to generate text that is similar to the training data, and can be used for tasks such as language translation, question answering, and text summarization.\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on a large amount of text data. It is able to generate text that is similar to the training data, and can be used for tasks such as language translation, question answering, and text summarization.\n",
-      "\n",
-      "A: A large language model is a neural network that is trained on\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
+    "# Running completions with Together AI\n",
+    "\n",
     "from langchain_together import Together\n",
     "\n",
     "llm = Together(\n",
-    "    model=\"togethercomputer/RedPajama-INCITE-7B-Base\",\n",
-    "    temperature=0.7,\n",
-    "    max_tokens=128,\n",
-    "    top_k=1,\n",
+    "    model=\"codellama/CodeLlama-70b-Python-hf\",\n",
     "    # together_api_key=\"...\"\n",
     ")\n",
     "\n",
-    "input_ = \"\"\"You are a teacher with a deep knowledge of machine learning and AI. \\\n",
-    "You provide succinct and accurate answers. Answer the following question: \n",
-    "\n",
-    "What is a large language model?\"\"\"\n",
-    "print(llm.invoke(input_))"
+    "print(llm.invoke(\"def bubble_sort(): \"))"
    ]
   }
  ],

libs/partners/together/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2023 LangChain, Inc.
+Copyright (c) 2024 LangChain, Inc.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

libs/partners/together/Makefile

@@ -11,7 +11,6 @@ integration_test integration_tests: TEST_FILE=tests/integration_tests/
 test tests integration_test integration_tests:
 	poetry run pytest $(TEST_FILE)
 
-
 ######################
 # LINTING AND FORMATTING
 ######################

libs/partners/together/README.md

@@ -1,48 +1,28 @@
 # langchain-together
 
-This package contains the LangChain integration for Together's generative models.
+This package contains the LangChain integrations for [Together AI](https://www.together.ai/) through their [APIs](https://docs.together.ai/).
 
-## Installation
+## Installation and Setup
 
-```sh
+- Install the LangChain partner package
+
+```bash
 pip install -U langchain-together
 ```
 
-## Embeddings
-
-You can use Together's embedding models through `TogetherEmbeddings` class.
-
-```py
-from langchain_together import TogetherEmbeddings
+- Get your Together AI api key from the [Together Dashboard](https://api.together.ai/settings/api-keys) and set it as an environment variable (`TOGETHER_API_KEY`)
 
-embeddings = TogetherEmbeddings(
-    model='togethercomputer/m2-bert-80M-8k-retrieval'
-)
-embeddings.embed_query("What is a large language model?")
-```
-
-## LLMs
+## Chat Completions
 
-You can use Together's generative AI models as Langchain LLMs:
+This package contains the `ChatTogether` class, which is the recommended way to interface with Together AI chat models.
 
-```py
-from langchain_together import Together
-from langchain_core.prompts import PromptTemplate
+ADD USAGE EXAMPLE HERE.
+Can we add this in the langchain docs?
 
-llm = Together(
-    model="togethercomputer/RedPajama-INCITE-7B-Base",
-    temperature=0.7,
-    max_tokens=64,
-    top_k=1,
-    # together_api_key="..."
-)
+NEED to add image endpoint + completions endpoint as well
 
-template = """Question: {question}
-Answer: """
-prompt = PromptTemplate.from_template(template)
+## Embeddings
 
-chain = prompt | llm
+See a [usage example](https://python.langchain.com/docs/integrations/text_embedding/together/)
 
-question = "Who was the president in the year Justin Beiber was born?"
-print(chain.invoke({"question": question}))
-```
+Use `togethercomputer/m2-bert-80M-8k-retrieval` as the default model for embeddings.

libs/partners/together/langchain_together/__init__.py

@@ -1,9 +1,5 @@
+from langchain_together.chat_models import ChatTogether
+from langchain_together.completions import Together
 from langchain_together.embeddings import TogetherEmbeddings
-from langchain_together.llms import Together
-from langchain_together.version import __version__
 
-__all__ = [
-    "__version__",
-    "Together",
-    "TogetherEmbeddings",
-]
+__all__ = ["ChatTogether", "Together", "TogetherEmbeddings"]

libs/partners/together/langchain_together/chat_models.py

@@ -0,0 +1,103 @@
+"""Wrapper around Together AI's Chat Completions API."""
+
+import os
+from typing import (
+    Any,
+    Dict,
+    List,
+    Optional,
+)
+
+import openai
+from langchain_core.pydantic_v1 import Field, SecretStr, root_validator
+from langchain_core.utils import (
+    convert_to_secret_str,
+    get_from_dict_or_env,
+)
+from langchain_openai import ChatOpenAI
+
+
+class ChatTogether(ChatOpenAI):
+    """ChatTogether chat model.
+
+    To use, you should have the environment variable `TOGETHER_API_KEY`
+    set with your API key or pass it as a named parameter to the constructor.
+
+    Example:
+        .. code-block:: python
+
+            from langchain_together import ChatTogether
+
+
+            model = ChatTogether()
+    """
+
+    @property
+    def lc_secrets(self) -> Dict[str, str]:
+        return {"together_api_key": "TOGETHER_API_KEY"}
+
+    @classmethod
+    def get_lc_namespace(cls) -> List[str]:
+        return ["langchain", "chat_models", "together"]
+
+    @property
+    def lc_attributes(self) -> Dict[str, Any]:
+        attributes: Dict[str, Any] = {}
+
+        if self.together_api_base:
+            attributes["together_api_base"] = self.together_api_base
+
+        return attributes
+
+    @property
+    def _llm_type(self) -> str:
+        """Return type of chat model."""
+        return "together-chat"
+
+    model_name: str = Field(default="meta-llama/Llama-3-8b-chat-hf", alias="model")
+    """Model name to use."""
+    together_api_key: Optional[SecretStr] = Field(default=None, alias="api_key")
+    """Automatically inferred from env are `TOGETHER_API_KEY` if not provided."""
+    together_api_base: Optional[str] = Field(
+        default="https://api.together.ai/v1/chat/completions", alias="base_url"
+    )
+
+    @root_validator()
+    def validate_environment(cls, values: Dict) -> Dict:
+        """Validate that api key and python package exists in environment."""
+        if values["n"] < 1:
+            raise ValueError("n must be at least 1.")
+        if values["n"] > 1 and values["streaming"]:
+            raise ValueError("n must be 1 when streaming.")
+
+        values["together_api_key"] = convert_to_secret_str(
+            get_from_dict_or_env(values, "together_api_key", "TOGETHER_API_KEY")
+        )
+        values["together_api_base"] = values["together_api_base"] or os.getenv(
+            "TOGETHER_API_BASE"
+        )
+
+        client_params = {
+            "api_key": (
+                values["together_api_key"].get_secret_value()
+                if values["together_api_key"]
+                else None
+            ),
+            "base_url": values["together_api_base"],
+            "timeout": values["request_timeout"],
+            "max_retries": values["max_retries"],
+            "default_headers": values["default_headers"],
+            "default_query": values["default_query"],
+        }
+
+        if not values.get("client"):
+            sync_specific = {"http_client": values["http_client"]}
+            values["client"] = openai.OpenAI(
+                **client_params, **sync_specific
+            ).chat.completions
+        if not values.get("async_client"):
+            async_specific = {"http_client": values["http_async_client"]}
+            values["async_client"] = openai.AsyncOpenAI(
+                **client_params, **async_specific
+            ).chat.completions
+        return values

libs/partners/together/langchain_together/llms.py → libs/partners/together/langchain_together/completions.py

@@ -1,4 +1,5 @@
 """Wrapper around Together AI's Completion API."""
+
 import logging
 import warnings
 from typing import Any, Dict, List, Optional
@@ -13,16 +14,14 @@
 from langchain_core.pydantic_v1 import Extra, SecretStr, root_validator
 from langchain_core.utils import convert_to_secret_str, get_from_dict_or_env
 
-from langchain_together.version import __version__
-
 logger = logging.getLogger(__name__)
 
 
 class Together(LLM):
     """LLM models from `Together`.
 
     To use, you'll need an API key which you can find here:
-    https://api.together.xyz/settings/api-keys. This can be passed in as init param
+    https://api.together.ai/settings/api-keys. This can be passed in as init param
     ``together_api_key`` or set as environment variable ``TOGETHER_API_KEY``.
 
     Together AI API reference: https://docs.together.ai/reference/completions
@@ -35,39 +34,39 @@ class Together(LLM):
             model = Together(model_name="mistralai/Mixtral-8x7B-Instruct-v0.1")
     """
 
-    base_url: str = "https://api.together.xyz/v1/completions"
+    base_url: str = "https://api.together.ai/v1/completions"
     """Base completions API URL."""
     together_api_key: SecretStr
-    """Together AI API key. Get it here: https://api.together.xyz/settings/api-keys"""
+    """Together AI API key. Get it here: https://api.together.ai/settings/api-keys"""
     model: str
-    """Model name. Available models listed here: 
+    """Model name. Available models listed here:
         Base Models: https://docs.together.ai/docs/inference-models#language-models
         Chat Models: https://docs.together.ai/docs/inference-models#chat-models
     """
     temperature: Optional[float] = None
     """Model temperature."""
     top_p: Optional[float] = None
-    """Used to dynamically adjust the number of choices for each predicted token based 
-        on the cumulative probabilities. A value of 1 will always yield the same 
-        output. A temperature less than 1 favors more correctness and is appropriate 
-        for question answering or summarization. A value greater than 1 introduces more 
+    """Used to dynamically adjust the number of choices for each predicted token based
+        on the cumulative probabilities. A value of 1 will always yield the same
+        output. A temperature less than 1 favors more correctness and is appropriate
+        for question answering or summarization. A value greater than 1 introduces more
         randomness in the output.
     """
     top_k: Optional[int] = None
-    """Used to limit the number of choices for the next predicted word or token. It 
-        specifies the maximum number of tokens to consider at each step, based on their 
-        probability of occurrence. This technique helps to speed up the generation 
-        process and can improve the quality of the generated text by focusing on the 
+    """Used to limit the number of choices for the next predicted word or token. It
+        specifies the maximum number of tokens to consider at each step, based on their
+        probability of occurrence. This technique helps to speed up the generation
+        process and can improve the quality of the generated text by focusing on the
         most likely options.
     """
     max_tokens: Optional[int] = None
     """The maximum number of tokens to generate."""
     repetition_penalty: Optional[float] = None
-    """A number that controls the diversity of generated text by reducing the 
+    """A number that controls the diversity of generated text by reducing the
         likelihood of repeated sequences. Higher values decrease repetition.
     """
     logprobs: Optional[int] = None
-    """An integer that specifies how many top token log probabilities are included in 
+    """An integer that specifies how many top token log probabilities are included in
         the response for each token generation step.
     """
 
@@ -109,7 +108,7 @@ def _format_output(self, output: dict) -> str:
 
     @staticmethod
     def get_user_agent() -> str:
-        return f"langchain-together/{__version__}"
+        return "langchain-together/0.3"
 
     @property
     def default_params(self) -> Dict[str, Any]: