Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Standardize ChatGroq #22751

Merged
merged 7 commits into from
Jun 14, 2024
203 changes: 201 additions & 2 deletions libs/partners/groq/langchain_groq/chat_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,15 +85,214 @@
To use, you should have the
environment variable ``GROQ_API_KEY`` set with your API key.

Any parameters that are valid to be passed to the groq.create call can be passed
in, even if not explicitly saved on this class.
Any parameters that are valid to be passed to the groq.create call
can be passed in, even if not explicitly saved on this class.

Example:
.. code-block:: python

from langchain_groq import ChatGroq

model = ChatGroq(model_name="mixtral-8x7b-32768")

Setup:
Install ``langchain-groq`` and set environment variable
``GROQ_API_KEY``.

.. code-block:: bash

pip install -U langchain-groq
export GROQ_API_KEY="your-api-key"

Key init args — completion params:
model_name: str
Name of Groq model to use. E.g. "mixtral-8x7b-32768".
temperature: float
Sampling temperature. Ranges from 0.0 to 1.0.
max_tokens: Optional[int]
Max number of tokens to generate.
model_kwargs: Dict[str, Any]
Holds any model parameters valid for create call not
explicitly specified.

Key init args — client params:
custom_get_token_ids: Optional[Callable[[str], List[int]]]
Optional encoder to use for counting tokens.
groq_api_base: Optional[str]
Base URL path for API requests, leave blank if not using a proxy
or service emulator.
metadata: Optional[Dict[str, Any]]
Metadata to add to the run trace.

See full list of supported init args and their descriptions in the params
section.

Instantiate:
.. code-block:: python

from langchain_groq import ChatGroq

model = ChatGroq(
model_name="mixtral-8x7b-32768",
temperature=0.0,
max_retries=2,
# streaming=...,
# other params...
)

Invoke:
.. code-block:: python

messages = [
("system", "You are a helpful translator. Translate the user
sentence to French."),
("human", "I love programming."),
]
model.invoke(messages)

.. code-block:: python

AIMessage(content='The English sentence "I love programming" can
be translated to French as "J\'aime programmer". The word
"programming" is translated as "programmer" in French.',
response_metadata={'token_usage': {'completion_tokens': 38,
'prompt_tokens': 28, 'total_tokens': 66, 'completion_time':
0.057975474, 'prompt_time': 0.005366091, 'queue_time': None,
'total_time': 0.063341565}, 'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop',
'logprobs': None}, id='run-ecc71d70-e10c-4b69-8b8c-b8027d95d4b8-0')

Check failure on line 164 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W293)

langchain_groq/chat_models.py:164:1: W293 Blank line contains whitespace
Stream:
.. code-block:: python

for chunk in model.stream(messages):
print(chunk)

.. code-block:: python

content='' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='The' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content=' English' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content=' sentence' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
...
content=' program' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='".' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='' response_metadata={'finish_reason': 'stop'}
id='run-4e9f926b-73f5-483b-8ef5-09533d925853

.. code-block:: python

stream = model.stream(messages)
full = next(stream)
for chunk in stream:
full += chunk
full

.. code-block:: python

AIMessageChunk(content='The English sentence "I love programming"
can be translated to French as "J\'aime programmer".
Here\'s the breakdown of the sentence:\n\n* "J\'aime" is the
French equivalent of "I love"\n* "programmer" is the French
infinitive for "to program"\n\nSo, the literal translation
is "I love to program". However, in English we often omit the
"to" when talking about activities we love, and the same applies
to French. Therefore, "J\'aime programmer" is the correct and
natural way to express "I love programming" in French.',
response_metadata={'finish_reason': 'stop'},
id='run-a3c35ac4-0750-4d08-ac55-bfc63805de76')

Async:
.. code-block:: python

await model.ainvoke(messages)

.. code-block:: python

AIMessage(content='The English sentence "I love programming" can
be translated to French as "J\'aime programmer". The word
"programming" is translated as "programmer" in French. I hope
this helps! Let me know if you have any other questions.',
response_metadata={'token_usage': {'completion_tokens': 53,
'prompt_tokens': 28, 'total_tokens': 81, 'completion_time':
0.083623752, 'prompt_time': 0.007365126, 'queue_time': None,
'total_time': 0.090988878}, 'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop',
'logprobs': None}, id='run-897f3391-1bea-42e2-82e0-686e2367bcf8-0')

Tool calling:
.. code-block:: python

from langchain_core.pydantic_v1 import BaseModel, Field

class GetWeather(BaseModel):
'''Get the current weather in a given location'''

location: str = Field(..., description="The city and state,
e.g. San Francisco, CA")

class GetPopulation(BaseModel):
'''Get the current population in a given location'''

location: str = Field(..., description="The city and state,
e.g. San Francisco, CA")

model_with_tools = model.bind_tools([GetWeather, GetPopulation])
ai_msg = model_with_tools.invoke("What is the population of NY?")
ai_msg.tool_calls

.. code-block:: python

[{'name': 'GetPopulation',
'args': {'location': 'NY'},
'id': 'call_bb8d'}]

See ``ChatGroq.bind_tools()`` method for more.

Structured output:
.. code-block:: python

from typing import Optional

from langchain_core.pydantic_v1 import BaseModel, Field

class Joke(BaseModel):
'''Joke to tell user.'''

setup: str = Field(description="The setup of the joke")
punchline: str = Field(description="The punchline to the joke")
rating: Optional[int] = Field(description="How funny the joke
is, from 1 to 10")

structured_model = model.with_structured_output(Joke)
structured_model.invoke("Tell me a joke about cats")

.. code-block:: python

Joke(setup="Why don't cats play poker in the jungle?",
punchline='Too many cheetahs!', rating=None)

See ``ChatGroq.with_structured_output()`` for more.

Response metadata
.. code-block:: python

ai_msg = model.invoke(messages)
ai_msg.response_metadata

.. code-block:: python

{'token_usage': {'completion_tokens': 70,
'prompt_tokens': 28,
'total_tokens': 98,
'completion_time': 0.111956391,
'prompt_time': 0.007518279,
'queue_time': None,
'total_time': 0.11947467},
'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1',
'finish_reason': 'stop',
'logprobs': None}
"""

client: Any = Field(default=None, exclude=True) #: :meta private:
Expand Down
Loading