ChatFireworks

Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the Fireworks docs.

This guide will help you getting started with ChatFireworks chat models. For detailed documentation of all ChatFireworks features and configurations head to the API reference.

Overview​

Integration details​

Class	Package	Local	Serializable	PY support	Package downloads	Package latest
ChatFireworks	@langchain/community	❌	✅	✅

Model features​

See the links in the table headers below for guides on how to use specific features.

Setup​

To access ChatFireworks models you’ll need to create a Fireworks account, get an API key, and install the @langchain/community integration package.

Credentials​

Head to the Fireworks website to sign up to Fireworks and generate an API key. Once you’ve done this set the FIREWORKS_API_KEY environment variable:

export FIREWORKS_API_KEY="your-api-key"

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

Installation​

The LangChain ChatFireworks integration lives in the @langchain/community package:

npm i @langchain/community @langchain/core

yarn add @langchain/community @langchain/core

pnpm add @langchain/community @langchain/core

Instantiation​

Now we can instantiate our model object and generate chat completions:

import { ChatFireworks } from "@langchain/community/chat_models/fireworks";

const llm = new ChatFireworks({
  model: "accounts/fireworks/models/llama-v3p1-70b-instruct",
  temperature: 0,
  maxTokens: undefined,
  timeout: undefined,
  maxRetries: 2,
  // other params...
});

Invocation​

const aiMsg = await llm.invoke([
  [
    "system",
    "You are a helpful assistant that translates English to French. Translate the user sentence.",
  ],
  ["human", "I love programming."],
]);
aiMsg;

AIMessage {
  "id": "chatcmpl-9rBYHbb6QYRrKyr2tMhO9pH4AYXR4",
  "content": "J'adore la programmation.",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "completionTokens": 8,
      "promptTokens": 31,
      "totalTokens": 39
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 31,
    "output_tokens": 8,
    "total_tokens": 39
  }
}

console.log(aiMsg.content);

J'adore la programmation.

Chaining​

We can chain our model with a prompt template like so:

import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant that translates {input_language} to {output_language}.",
  ],
  ["human", "{input}"],
]);

const chain = prompt.pipe(llm);
await chain.invoke({
  input_language: "English",
  output_language: "German",
  input: "I love programming.",
});

AIMessage {
  "id": "chatcmpl-9rBYM3KSIhHOuTXpBvA5oFyk8RSaN",
  "content": "Ich liebe das Programmieren.",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "completionTokens": 6,
      "promptTokens": 26,
      "totalTokens": 32
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 26,
    "output_tokens": 6,
    "total_tokens": 32
  }
}

Behind the scenes, Fireworks AI uses the OpenAI SDK and OpenAI compatible API, with some caveats:

• Certain properties are not supported by the Fireworks API, see here.
• Generation using multiple prompts is not supported.

API reference​

For detailed documentation of all ChatFireworks features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_community_chat_models_fireworks.ChatFireworks.html

Related​

• Chat model conceptual guide
• Chat model how-to guides