StreamChat

Bot API allows you to try out different Large Language Models (LLMs) from providers like Mistral, OpenAI, and Claude.

For more details on the supported LLM models and their capabilities, please refer to the ChatBot.

Stream chat with LLM

POST /stream_chat

Headers

Name	Value
Content-Type	application/json
Authorization	Bearer <token>

Body

Name	Type	Description
llm	string	-> The llm parameter specifies the type of Large Language Model (LLM) to be used. The supported values are: ClaudeAI OpenAI GeminiAI OpenRouterAI
llm_model	string	-> The llm_model parameter specifies the name of the Large Language Model (LLM) to be used. The supported values for this parameter depend on the llm you have selected. For example, if the llm is OpenAI, the supported llm_model values include: "gpt-4.1-mini-2025-04-14" "gpt-4.1-2025-04-14" Similarly, if the llm is ClaudeAI, the supported llm_model values include: "claude-3-7-sonnet-latest" "claude-opus-4-20250514" Ensure that the llm_model value you provide is compatible with the selected model_type. -> Supported LLMs and their Models are : GeminiAI gemini-2.5-pro gemini-2.5-flash ClaudeAI claude-3-7-sonnet-latest claude-opus-4-20250514 claude-sonnet-4-20250514 OpenAI gpt-4.1-mini-2025-04-14 gpt-4o-mini gpt-4.1-2025-04-14 o3-mini o1 o3 o4-mini-2025-04-16 (Note: Must be configured with temperature=1 ,while runninng this model) OpenRouterAI x-ai/grok-3-beta x-ai/grok-3-mini-beta perplexity/sonar-pro perplexity/sonar-reasoning-pro perplexity/sonar-reasoning perplexity/sonar-deep-research deepseek/deepseek-chat deepseek/deepseek-r1
history	List of Dictionary (Python) or map (C++)	-> The history parameter allows you to provide the previous chat history and the last user message. The history should follow a specific pattern: The history should start with a user message. After each user message, there should be a response from the assistant. The last message in the history should be a user message or query. -> The history should be provided as a list of dictionaries, where each dictionary represents a message. The dictionary should have the following structure: { "role": "user", "content": { "text": "please give 50 word story in English", "image_data": [ { "url": "string", "details": "low" } ] } } The image_data parameter is a list that can contain up to 5 image URLs. The details parameter is optional and can be used to specify the level of detail required for the image analysis (only for OpenAI models). Here's an example of the history parameter: "history": [ { "role": "user", "content": { "text": "hey", "image_data": [] } }, { "role": "assistant", "content": { "text": "Hello! How can I assist you today?", "image_data": [] } }, { "role": "user", "content": { "text": "Please analyze this image", "image_data": [ { "url": "https://res.cloudinary.com/qolaba/image/upload/v1695690455/kxug1tmiolt1dtsvv5br.jpg", "details": "low" } ] } } ]
temperature	float	-> The temperature parameter accepts a float value between 0 and 1. This parameter helps control the level of determinism in the output from the Large Language Model (LLM). -> Higher temperature values (closer to 1) will result in more diverse and creative output, while lower values (closer to 0) will lead to more deterministic and conservative responses.
image_analyze	bool	-> If you are passing image URLs and want the model to analyze the images, set the image_analyze parameter to true.
enable_tool	bool	-> To use the tools supported by the Chat API, enable the enable_tool parameter. The Chat API currently supports two tools: Vector Search Internet Search -> After enabling the enable_tool parameter, you can provide the details of the tool you want to use in the tools parameter.
system_msg	string	-> The system_msg parameter allows you to set a system message for the Large Language Model (LLM). This message can be used to provide context or instructions to the model, which can influence the tone and behavior of the generated responses.
tools	dictionary or map	The tools parameter allows you to configure the capabilities available to the assistant during a conversation. It defines which tools are enabled, how much past context to consider, and specifies settings for embeddings, PDFs, and image generation. This is useful for tailoring the assistant’s functionality to your use case. 🔹 Structure jsonCopyEdit"tools": { "tool_list": { "image_generation": false, "image_generation1": false, "image_editing": false, "search_doc": false, "internet_search": false, "python_code_execution_tool": false, "csv_analysis": false }, "number_of_context": 3, "pdf_references": [], "embedding_model": [ "text-embedding-3-large" ], "image_generation_parameters": {} } 🔹 Field Descriptions tool_list: An object with boolean flags to enable or disable specific tools. Setting a value to true activates that feature. image_generation: Enables image creation from text prompts. image_generation1: Optional alternate switch for image generation (if supported). image_editing: Enables uploading and editing images. search_doc: Allows document-based searching. internet_search: Enables real-time web searches. python_code_execution_tool: Allows execution of Python code within a safe sandbox. csv_analysis: Enables processing and analysis of uploaded CSV files. number_of_context: An integer defining how many previous user-assistant interactions to retain in memory. Higher values help preserve conversation flow. pdf_references: A list of PDFs that the assistant can reference for contextual information should be stored in vector store database. Each entry includes an unique id which comes from the output of vector store API. If you're adding more than one document using pdf_references, make sure you add the same number of entries in the embedding_model array. embedding_model: we support "text-embedding-3-large" as the embedding model. image_generation_parameters: An object to define settings like size or style for image generation. Leave it empty for default behavior.
token	string	Authentication token needed for the request.
orgID	string	Identifies the organization associated with the request.
function_call_list	string	Lists the functions called during the process.
systemId	string	Specifies the system identifier linked to the request.
last_user_query	string	Provides the last user query made in the chat.

Example of Input body parameters :

data '{
  "llm": "GeminiAI",
  "llm_model": "gemini-2.5-flash",
  "history": [
    {
      "role": "user",
      "content": {
        "text": "please give 50 word story in English",
        "image_data": [
          {
            "url": "string",
            "details": "low"
          }
        ]
      }
    }
  ],
  "temperature": 0.7,
  "image_analyze": false,
  "enable_tool": false,
  "system_msg": "You are a helpful assistant.",
  "tools": {
    "tool_list": {
      "image_generation": false,
      "image_generation1": false,
      "image_editing": false,
      "search_doc": false,
      "internet_search": false,
      "python_code_execution_tool": false,
      "csv_analysis": false
    },
    "number_of_context": 3,
      "pdf_references": [
      "output vectorstore1",
      "output vectorstore2"
    ],
    "embedding_model": [
      "text-embedding-3-large",
      "text-embedding-3-large"
    ],
    "image_generation_parameters": {}
  },
  "token": "123",
  "orgID": "string",
  "function_call_list": [],
  "systemId": "string",
  "last_user_query": "string"
}'

After passing the necessary parameters and executing the Chat API, you will receive a stream response. A successful response will look similar to the following:

Response

200

{
  "output": null, 
  "error": null, 
  "error_data": null
}

500

{
  "output": null, 
  "error": string, 
  "error_data": string
}

The Chat API response is a streaming response, which means you will receive the output in chunks as the model generates the response. The response will continue to stream until the generation is complete.

The response will contain the following elements:

• output: This object contains the generated text output from the model.

The text output from the LLM can be obtained from the output parameter. When the response is complete, the final chunk will contain a null value in the output parameter, indicating the end of the stream.

Run the API

To test this API, please use the following link:

https://app.theneo.io/api-runner/qolaba/ml-apis/api-reference/streamchatapp.theneo.io