跳到正文
GitHub·· 1 天前AI 评分62

Stream Agents 0.7 改用 Go 后端构建文本、语音和视频智能体

GetStream/Agents

AI 导读

Stream Agents 0.7 将后端从 Python 迁移到 Go,用于构建文本、语音和视频智能体,并提供 Python、Node、Ruby、PHP、.NET、Rust 等后端 SDK 及 Swift、Kotlin、React、Flutter、React Native 客户端 SDK。

正文

StreamAgents

Note

Using Python for Agents no longer makes sense. For 0.7 we've moved to Go, with SDKs for other languages.

Stream Agents is a Go backend for building text, voice and video agents. You can build AI/agentic apps like Muse, Decagon and Harvey in a day.

It's faster to build agents:

  1. Agents are defined in a folder structure + YAML
  2. SDKs handle chat/audio client and state
  3. All the components you need are integrated with sensible defaults. (Guardrails, Knowledge, Routing, Connectors, Sandboxes, skills, telephony)

Performance is excellent:

  1. Go runs with high concurrency and great performance
  2. The edge acceleration offloads your backend and lowers latency
  3. Audio forwarding is optimized with WARP similar to OpenAI

Sovereign AI

The combination of routing, persistence and memory makes it easy to switch the underlying models. Stay in control of how costs scale. Start with the frontier models and run your own when it makes sense.

Skill

The skill enables you to build complex AI apps in hours:

  • Ask Claude/Cursor/Lovable to setup Stream, with getstream.io/SKILL.md

Quickstart

The heavy lifting is done in Go. But you can use the agents in any environment:

Pick your backend language:

Then add chat, voice and video to your app with a client SDK:

Agent Folders and CLI

The fastest way to define an agent is using the folder structure

You can quickly create a folder using the CLI

curl -fsSL https://getstream.io/cli.sh | bash
getstream agents init my-agent --template text

The structure looks something like this:

agents/my-agent/
  knowledge/       ← indexed markdown and URLs
    faq.md
    urls.yaml
  skills/          ← skills loaded as needed
    research.md
  simulations/
  agent.yaml       ← config for the agent
  routing.yaml     ← routing
  instructions.md  ← system prompt
  guardrail.md     ← prevent abuse

After you create an agent you can quickly test it

getstream agents test my-agent

Note that you can also configure agents through the SDKs or dashboard.

Simulations

Simulations play scripted callers against your agent so you can catch regressions before you ship.

agents-simulations-readme.mp4

Client Side/ Browser & App

You can create a session for persistence. After the session is created, you can uses responses.create across all supported LLMs.

import { Client } from "@stream-io/stream-agents";

const api = new Client({ apiKey: "your_api_key" });
await api.setUser("jlahey", userToken); // a token your backend minted for this user

const session = await api.agent("support").sessions.create();
await session.responses.create("Do I have any open support cases?");

for await (const event of session.events()) {
  if (event.kind === "response_delta") console.log(event.text);
  if (event.kind === "responded") break;
}

Also note how the agent is referenced by use case. IE "support" and not the name of the model. The design here is that it should be trivial to switch from a frontier model to open or your own.

Full apps with chat, voice and tools, one per client SDK:

Tool Calling

Any backend SDK can provide tools to the agent. Tools are registered once for the app, and each agent's agent.yaml names the ones it uses in a tool group. Here's a Python based example:

import asyncio

from vision_agents.core.llm import ToolCallContext
from vision_agents.plugins import stream

client = stream.Client()


@client.tools.register()
async def customer_case_lookup(tool_call_context: ToolCallContext) -> list:
    """Look up the user's open support cases."""
    user = tool_call_context.user
    if not user.id:
        raise ValueError("unknown user")
    # pretend this calls your CRM
    return await crm.open_cases(user.id)


asyncio.run(client.run_dispatch())

Examples / What can you build

  • Simple chat: text, knowledge, skills, function calling, a connector and an MCP server
  • Simple voice
  • Inbound calling: answers a restaurant's number and takes an order
  • Outbound calling: rings a candidate and screens them
  • Video: security camera, avatar
  • Artemis and Awesome No Moat: coming soon

Features included

Simulations & Monitoring

Agent simulations and monitoring are built-in.

Cost Transparency

It's very easy for AI costs to spiral out of control. We aim to give you the tools you need to keep costs under control

  • It's extremely easy to switch model
  • Session monitoring shows you stats about token usage. So you can optimize tools/skills etc to reduce token usage
  • Aggregated stats show token usage for skills, tools across all your tools
  • Tags allow you to track specific customers, products etc
  • Spending limits

Set daily, weekly and monthly spend limits in USD on the organization, API keys, users, apps and teams. See spend limits.

agents-cost-reporting-readme.mp4

Connectors

Connectors give an agent a provider's tools, or let people talk to it on a messaging channel. Add one in agent.yaml or the dashboard, with no code. Any other MCP server can be added as a custom connector. See the connectors docs.

  • Messaging channels: Slack bot, WhatsApp, SMS (Telnyx), iMessage (Linq)
  • Communication: Slack, Gmail
  • Documents: Notion, Google Docs, Google Drive
  • Scheduling: Google Calendar, Calendly, Cal.com
  • Project management and engineering: Linear, GitHub, Sentry
  • Customers and revenue: Salesforce, HubSpot, Gong
  • Commerce: Shopify, Instacart
  • Analytics: BigQuery, Metabase

Benchmarks (coming soon)

  • Agent development speed
  • Edge performance
  • Agent concurrency
  • Voice benchmark

Alternatives

How Stream Agents compares to the alternatives, see Stream vs alternatives:

  • OpenAI or Anthropic directly
  • Agent frameworks: Eve, AI SDK, LangChain
  • Routers: OpenRouter, Bifrost, LiteLLM
  • Voice AI frameworks: LiveKit, Pipecat, Retell, Agora, Vapi
  • Support SaaS: Sierra, Decagon, Serval

Free tier

You can run it yourself, or use the hosted version. Its free tier has generous limits:

  • $25 of inference and other costs included
  • 50k agent hosting minutes / month
  • 150k AI infra text messages
  • 500 concurrent agents
  • 10k monthly active agent users
  • 1M API calls
  • 1M records storage
  • Custom voices
  • Team collaboration

Past those limits, usage costs:

What Price
AI voice infra $0.002 / min
AI infra text $0.0001 / message
Audio RTC $0.30 / 1k participant min
SD video RTC $0.75 / 1k participant min
Stored records $1.50 / 1M after first 1M
Bandwidth $0.10 / GB
Tokens, telephony, knowledge, search & other forwarded costs cost + 5% + payment fees

See agent pricing.

来源:GitHub · github.com