Upstreet AI

docs.upstreet.ai

Developer Tools

AI Agents for everyone. Build, chat with, and deploy Agents effortlessly.

llms.txt

file: ./content/docs/changelog.mdx

undefined: What's new?

Learn what the latest updates are in Upstreet-world.

import Changelog from "../../../../CHANGELOG.md";

file: ./content/docs/(getting-started)/create-an-agent.mdx

undefined: Create an Agent

Build AI Agents in minutes with the Upstreet Agents SDK.

import Wrapper from '../../../components/preview/wrapper'; import { File, Folder, Files } from 'fumadocs-ui/components/files';

import { Accordion, Accordions } from 'fumadocs-ui/components/accordion'; import { formatNodeVersion } from '../../../lib/utils'; import pkgJson from 'usdk/package.json';

Upstreet AI Agents are persistent digital entities that can autonomously handle tasks, interact with you and your users over chat or social media, and can be customized according to your configuration.

New to designing Agents? Explore how to define effective Agent objectives and strategies in our Agent Design Concepts.

Coming in from another platform like Tavern? You can import your previous work into an Upstreet Agent. Check out our Migration Guides.

Quickstart: Creating an Agent

Here’s how to set up your first Agent.

Step 1: Running the command

Set up your project directory
First, create an empty directory where you’d like to set up your Agent.
Run the Command in your terminal
```
usdk create
```
Run the above command in your terminal within your new directory. This will launch a guided interview, where you’ll define essential properties of your Agent.

You can see more options by running usdk create --help.

Step 2: Complete the Agent Interview

The usdk create command initiates an interactive “interview” process with the Interviewer. Here’s what to expect:

Interactive Prompts
The SDK will prompt you with questions, helping you define your Agent's personality, environment, and other key settings.
Simulated Chat with Your Agent
You’ll be able to “converse” with the Interviewer, defining the Agent's Homespace (its natural habitat) and personality traits through chat-based interactions.
Completion
Once all required fields are captured, the interview concludes, setting up all the Agent features and initialising the necessary files in your directory.

To import your agent from other platforms, usdk create also supports Tavern character cards and more.

See Importing a Tavern Agent for more information.

File Structure

Assuming you’ve named your project directory “my-agent,” here’s the structure you’ll see post-setup:

  <Folder name="node_modules" disabled />

  <Folder name="packages" />

  <Folder name="codecs" />

  <Folder name="tests" />

  <File name=".gitignore" />

  <File name="agent.tsx" />

  <File name="jest.config.mjs" />

  <File name="jest.setup.mjs" />

  <File name="package-lock.json" />

  <File name="package.json" />

  <File name="tsconfig.json" />

  <File name="wrangler.toml" />

  <File name="README.md" />
</Folder>

For a breakdown of these files, see our Agent Structure guide.

Editing your Agent

If you wish to edit your already-created Agent through the Interview process, you may run the following command in the directory of your Agent:

usdk edit

What's next?

Now that you've set up your base Agent, you can choose to dive deeper into Agent customization and potential capabilities by writing code in React.

If you wish to skip customization and directly launch your Agent on our Platform, you can check out Testing your Agent and Deploying your Agent.

file: ./content/docs/(getting-started)/customize-your-agent.mdx

undefined: Customize your Agent

Learn how to extend an Agent's functionality using React.

import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';

Learn how to extend an Agent’s functionality using React components and intuitive primitives to build dynamic, intelligent agents with Upstreet. This guide will walk you through customizing your Agent step-by-step, with examples and tips to inspire you.

Overview: How an Agent Works

Agents follow a simple but powerful cycle: they perceive inputs, process them to generate insights or decisions, and act on their environment based on these decisions. This perception-think-act model is broken down into components that work together to define your Agent’s functionality.

Read our What Are Agents? guide to get familiar with Agent basics.

Explore Upstreet’s Agent Architecture to learn more about how Agents operate behind the scenes.

Agent Structure at a Glance

If you’re familiar with React, the structure of an Agent will look familiar. Here’s a basic example to get you started:

const MyAgent = () => (
  <Agent>
    <Prompt>
      This assistant is developed by MultiCortical Example Technologies. For support, refer users to our help pages at https://multicortical.example.com/
    </Prompt>
  </Agent>
);
export default MyAgent;

The example above shows the basic setup, where a simple prompt is added to guide the Agent’s interactions. The react-agents library, however, allows much more flexibility through four core components.

Key Components of an Agent

Using the react-agents library, Agent customization is broken down into four core components:

Component	Purpose
`<Prompt />`	Sets initial instructions and context for the Agent’s behavior.
`<Action />`	Defines actions the Agent can perform, utilizing LLM tool-calling facilities.
`<Perception />`	Allows the Agent to perceive and react to real-world events.
`<Task />`	Schedules tasks for the Agent, enabling it to manage asynchronous processes.

You can see all Components here.

Import these components from the SDK package:

import { Prompt, Action, Perception, Task } from 'react-agents';

Each component adds specific functionality, enabling you to build intelligent, responsive Agents. Let’s dive into each one in more detail.

An Agent consists of a collection of components implementing these primitives.

Bringing It All Together

Here’s a sample setup that combines these components:

const MyAgent = () => (
  <Agent>
    <Prompt>I'm here to help you with information about our services and products.</Prompt>
    <Action name="fetchProductInfo">Retrieve the latest product details based on user query.</Action>
    <Perception trigger="userInactive">Remind user of inactivity after 5 minutes.</Perception>
    <Task schedule="every day at 9am">Send daily summary report.</Task>
  </Agent>
);

This setup provides a well-rounded Agent equipped to respond, perceive, act, and manage scheduled tasks effectively.

Using environment variables

You may choose to use third-party services when customizing your Agent. These services may have secrets or tokens.

You can keep them safe by creating a .env.txt file in the your Agent directory, and keeping environment variables there.

An example might look like:

GOOGLE_API_KEY=abcdefghijklmonp
OPEN_WEATHER_KEY=abcdefghijklmonp

You can access these environment variables by using the useEnv hook.

Custom Logic and Advanced Patterns

With react-agents, you can introduce custom logic within components, such as conditional rendering, state management, and data manipulation, using native React hooks and patterns.

To explore advanced customization:

Learn how to make Custom Components.
Learn more about how agents are structured.
Our Pokédex example demonstrates a basic, real-world example of how to create a custom component.

Ready for More?

With these foundational components, you can customize your Agent to fit various tasks, from customer support to data processing. Next steps:

Test your Agent: See how it responds and make adjustments.
Deploy your Agent: Launch your Agent on Upstreet's platform for real-world interactions.

file: ./content/docs/(getting-started)/deploy-your-agent.mdx

undefined: Deploy your Agent

Get your Agent live and see it used across the Internet.

import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';

Deploy to Upstreet

Ready to unleash your Agent into the world? Simply run:

usdk deploy <your-agent-directory>

This command uploads your Agent to the Upstreet platform, making it accessible via a unique URL provided after a successful deployment.

Deployment consumes Upstreet credits. For more information, see our pricing section.

Self-hosting

Prefer to run your Agent locally? You can use a Cloudflare tunnel to make it internet-accessible. Note that even with self-hosting, using Upstreet’s AI services may still consume credits.

For more on pricing models for both deployment options, visit our Pricing page.

Unpublishing an Agent

You may want to undeploy or unpublish your Agent. To do so, simply run the following command in the Agent's directory:

usdk unpublish

This will pull your Agent off the Upstreet infrastructure and make the Agent unavailable on the Platform. To run again, simply redeploy.

file: ./content/docs/(getting-started)/index.mdx

undefined: Upstreet Documentation

Learn how to build and deploy AI Agents.

file: ./content/docs/(getting-started)/install.mdx

undefined: Setup the SDK

Get started with the Upstreet Agents SDK and begin building AI Agents today.

import Wrapper from '../../../components/preview/wrapper'; import { Accordion, Accordions } from 'fumadocs-ui/components/accordion'; import { Pre, CodeBlock } from 'fumadocs-ui/components/codeblock'; import { Tab, Tabs } from 'fumadocs-ui/components/tabs'; import pkgJson from 'usdk/package.json'; import { formatNodeVersion } from '../../../lib/utils';

To start building with Upstreet AI Agents, first set up the Upstreet Agents SDK (usdk). If you're new to Agents, check out What are Agents? for a quick introduction.

For more on SDK architecture and capabilities, see our Concepts guide.

Prerequisites

Before getting started, make sure you have the following:

<Accordion title={2. Node.js: ${formatNodeVersion(pkgJson.engines?.node || '>=22.9.0')} (required).}> Node.js is the runtime environment that allows you to run JavaScript code outside a web browser.

### Install Node.js

* Go to the [Node.js downloads page](https://nodejs.org/en/download/package-manager/current).
* Follow the instructions to install it as a package on your system, or download the installer for your OS and follow the setup steps.
* Open the Terminal from step (1), and run `node -v` to see your Node version.

**NPM (Node Package Manager)** is a tool that comes installed alongside Node.js. It is used to manage packages (libraries, tools, frameworks, etc.) for JavaScript projects. With NPM, you can:

* **Install packages** for your project locally or globally.
* **Share your code** as packages with other developers.
* **Run scripts** for development tasks like building or testing your project.

NPM is bundled with Node.js because it's the default package manager for managing JavaScript dependencies in Node.js projects. When you install Node.js, NPM is automatically installed, so you don't need to install it separately in most cases.

You can **verify the installation of `npm`**:

* Open a terminal or command prompt.
* Run `node -v` to check the installed Node.js version.
* Run `npm -v` to check the installed NPM version.

***

Once you've installed NodeJS, you'll have access to both `node` and `npm` commands, enabling you to download and run JavaScript applications like `usdk`.

Ensure the Node version is {formatNodeVersion(pkgJson.engines?.node || '>=22.9.0').toLocaleLowerCase()}, else usdk will not work.

Install from NPM

To install the usdk command-line tool, use the following command:

{/* "pnpm (recommended)", */}

<Tabs items={[ "npm", ]}

{/* bash tab="pnpm (recommended)" pnpm install -g usdk */}

npm install -g usdk

Verify Installation (optional)

To confirm a successful installation, check your SDK version:

  <CodeBlock className="-mt-8" icon={"bash"} keepBackground allowCopy={false}>
    <Pre>
      {pkgJson.version}
    </Pre>
  </CodeBlock>
</div>

This command should return the installed version number. You can view all available versions of usdk on NPM.

Log into the SDK

Some SDK features require you to be logged in. To log in:

Run the command:
```
usdk login
```
A browser will open. Log into Upstreet with your preferred authentication provider.

file: ./content/docs/(getting-started)/test-your-agent.mdx

undefined: Test your Agent

Learn how to run and test your Upstreet Agent before going live.

import Wrapper from 'components/preview/wrapper'; import Image from 'next/image'; import { Accordion, Accordions } from 'fumadocs-ui/components/accordion';

Once your Agent is created, you can test its responses and behaviors before deployment. The Upstreet SDK makes testing straightforward, allowing you to interact directly with your Agent.

Running a Test Session

To start testing, run the following command in your terminal:

usdk chat <your-agent-directory>

Where <your-agent-directory> is the relative path to the directory containing all your Agent's code. How to create an Agent

This command launches an interactive chat session (REPL-like) with your Agent, where you can input prompts and review responses in real-time.

To exit the chat, type .exit in the chat and press the Enter key. Or, you can use the shortcut CTRL + C twice.

AI inferences do not run locally and may consume Upstreet credits during testing.

Hot reloading

Hot reloading is supported by default, so while customizing your Agent, your Agent will immediately update in the chat once you save your code.

Testing Tips

Specific Task Testing: Prompt your Agent to carry out the exact tasks or interactions you want to verify.
Custom Test Cases: To automate testing, write test cases using Jest, ensuring consistency and reliability for complex Agent behaviors.

file: ./content/docs/advanced/custom-voices.mdx

undefined: Custom Voices

You can create and use custom voice models for your Agents.

The voice commands in usdk allow you to manage custom voices for your agents.

Available Commands

1. List Voices

Command: voice list
Description: Displays all voices associated with your account, including voice IDs and names. Use this information to configure the voiceEndpoint prop.

2. Create a Voice

Command: voice create <name> <audioFile>
Requirements:
- An audio sample of the voice to clone.
- Supported formats: MP3, WAV.
Parameters:
- <name>: The desired name for your voice.
- <audioFile>: The path to your audio sample file.

3. Test a Voice

Command: voice play <voiceName> "Your test message"
Description: Allows you to hear how your created voice sounds with a sample message.

See the entire command reference for usdk here.

file: ./content/docs/advanced/usdk-library.mdx

undefined: Using USDK programmatically

Use the Upstreet Agents SDK as a library in your code.

While the Upstreet Agents SDK (USDK) is designed as a command-line interface (CLI) for building, managing, and deploying Upstreet AI Agents, you can also import USDK as a module in your code to access its powerful functionalities programmatically.

Using USDK programmatically gives you more control over the deployment and management of your agents. You can integrate it into your own workflows, automate tasks, or build more complex applications on top of Upstreet's platform. It also eliminates the need for shell execution, providing a clean and efficient way to interact with USDK features directly from your code.

Installation

To use it as a module in your Node.js application, install USDK locally in your project:

npm install usdk

Import

You can then import USDK into your Node.js scripts:

const usdk = require('usdk');

Usage

Once imported, you can interact with USDK's commands as JavaScript functions. Here's a basic example of how you can use USDK programmatically:

Example: Log In

const usdk = require('usdk');

async function login() {
  try {
    await usdk.login();
    console.log('Logged in successfully!');
  } catch (error) {
    console.error('Login failed:', error);
  }
}

login();

Example: Create an Agent

const usdk = require('usdk');

async function createAgent() {
  try {
    const agent = await usdk.create({
      prompt: "Create a new agent",
      feature: ['chat'],
      force: true,
      json: '{"name":"MyAgent","type":"assistant"}'
    });
    console.log('Agent created:', agent);
  } catch (error) {
    console.error('Error creating agent:', error);
  }
}

createAgent();

Example: Deploy an Agent

const usdk = require('usdk');

async function deployAgent(agentId) {
  try {
    await usdk.deploy([agentId]);
    console.log('Agent deployed successfully!');
  } catch (error) {
    console.error('Error deploying agent:', error);
  }
}

deployAgent('my-agent-id');

Key Functions Available

Here's a quick overview of the core functions you can use programmatically with USDK:

usdk.login(): Logs in to the USDK.
usdk.logout(): Logs out from USDK.
usdk.create(options): Creates a new agent.
usdk.edit(directory, options): Edits an existing agent.
usdk.deploy(agentIds): Deploys an agent to the network.
usdk.rm(agentIds): Removes a deployed agent.
usdk.pull(agentId, options): Pulls the source code of a deployed agent.
usdk.status(): Retrieves the current login status and account details.
usdk.chat(agentIds, options): Starts a multi-agent chat session.

Example: Check Status

const usdk = require('usdk');

async function checkStatus() {
  try {
    const status = await usdk.status();
    console.log('Current status:', status);
  } catch (error) {
    console.error('Error retrieving status:', error);
  }
}

checkStatus();

file: ./content/docs/api/overview.mdx

undefined: Overview

The Upstreet Agents SDK is the first React-based SDK for building and deploying headless AI agents, locally and in the cloud.

import { InlineTOC } from 'fumadocs-ui/components/inline-toc';

React Agents is a groundbreaking framework that brings the power and familiarity of React to AI agent development. Built on React's reconciliation engine, it enables developers to create intelligent, autonomous agents using the same tools and patterns they love from React development.

Quick Start

import { Agent, Action } from '@upstreet/agents';
import { z } from 'zod';

function SmartHomeAgent() {
  return (
    <Agent>
      <Action
        name="turnOnLights"
        description="Turn on the lights in my room"
        schema={z.object({ lightName: z.string() })}
        examples={[{ lightName: 'bedroom' }]}
        handler={(e) => {
          turnOnLights();
          e.data.agent.monologue(`Lights changed: ${e.data.message.args.lightName}`);
        }}
      />
    </Agent>
  );
}

Core Concepts

The React Agents Architecture

React Agents leverages the React Reconciler API to create a custom renderer specifically designed for AI agents. This architecture provides several key advantages:

Server-First Design: Optimized for server-side execution
Platform Agnostic: Ready for multi-platform and edge deployments
Declarative: Uses React's component model and lifecycle
Type Safety: Full TypeScript support throughout the stack

Traditional React vs React Agents

Let's compare how you'd implement similar functionality in traditional React versus React Agents:

Traditional React

// User interface focused
function LightControl() {
  const [lightName, setLightName] = useState('');
  
  return (
    <form onSubmit={(e) => turnOnLights(e.target.lightName.value)}>
      <label>Light Name:</label>
      <input
        type="text"
        value={lightName}
        onChange={(e) => setLightName(e.target.value)}
      />
      <button type="submit">Turn On Light</button>
    </form>
  );
}

React Agents

// Agent behavior focused
function LightControlAgent() {
  return (
    <Agent>
      <Action
        name="turnOnLights"
        description="turn on the lights in my room"
        schema={z.object({ lightName: z.string() })}
        examples={[{ lightName: 'bedroom' }]}
        handler={handleLightChange}
      />
    </Agent>
  );
}

The key difference lies in the focus: traditional React components render user interfaces, while React Agents components render agent behaviors and capabilities.

Core Components

Agent

The root component that initializes an agent instance.

<Agent
  name="HomeAssistant"
  description="A helpful assistant for home automation"
>
  {/* Agent actions and behaviors */}
</Agent>

Action

Defines discrete capabilities that an agent can perform.

<Action
  name="setTemperature"
  description="Set the temperature of a room"
  schema={z.object({
    room: z.string(),
    temperature: z.number()
  })}
  handler={handleTemperatureChange}
/>

How It Works

React Agents operates through a sophisticated pipeline:

JSX Transformation: Your component code is transformed into an agent execution plan
Prompt Generation: The execution plan is converted into a series of prompts
Chain-of-Thought Runtime: Prompts are processed through a reasoning engine
Action Execution: The agent performs actions based on its reasoning
State Updates: Results are reconciled back through React's system

Best Practices

1. Schema Definition

Always define precise schemas for your actions:

const schema = z.object({
  action: z.enum(['on', 'off']),
  device: z.string(),
  room: z.string().optional()
});

2. Provide Examples

Include clear examples for better agent understanding:

const examples = [
  { action: 'on', device: 'lights', room: 'living room' },
  { action: 'off', device: 'thermostat' }
];

3. Error Handling

Implement robust error handling in your handlers:

const handler = async (e) => {
  try {
    await performAction(e.data.message.args);
    e.data.agent.monologue('Action completed successfully');
  } catch (error) {
    e.data.agent.monologue(`Error: ${error.message}`);
  }
};

Command Reference

For a complete list of available commands and their usage, refer to our Command Reference Guide.

Advanced Topics

Custom Renderers

Create specialized renderers for unique use cases:

const customRenderer = createAgentRenderer({
  supportedEvents: ['custom.event'],
  transformEvent: (event) => ({
    type: 'custom.event',
    payload: event
  })
});

State Management

Integrate with existing React state management solutions:

function AgentWithState() {
  const [state, dispatch] = useReducer(reducer, initialState);
  
  return (
    <Agent context={{ state, dispatch }}>
      {/* Agent components */}
    </Agent>
  );
}

Resources

file: ./content/docs/concepts/defining-agent-objectives.mdx

undefined: Defining Agent Objectives

Defining effective objectives for AI agents is key to ensuring they operate successfully, maintain consistency, and achieve desired outcomes with minimal oversight.

import { InlineTOC } from 'fumadocs-ui/components/inline-toc';

When setting up objectives for an Agent created with the Upstreet Agents SDK, several essential strategies and considerations come into play to keep agents focused, accurate, and adaptable.

1. Prioritize Clarity and Conciseness

Be Specific: Agents work best when objectives are straightforward and focused. Specific instructions reduce the risk of misinterpretation, ensuring the agent remains aligned with its purpose. For instance, instead of an objective like "Assist with customer support," a more precise goal might be "Respond to common customer inquiries related to product features." Upstreet’s SDK anticipates scalability with plans to "chain" multiple agents and create "Swarms," which would enable more complex, large-scale task execution by dividing tasks across specialized agents.

2. Use Popular Language and References

Leverage Common Language: Agents trained on vast datasets are more likely to succeed when objectives are phrased using widely recognized language and references. This approach minimizes misunderstandings and helps prevent hallucinations—where an agent might invent facts or information. A clear, familiar language ensures the agent interprets the task accurately, particularly in high-stakes or nuanced scenarios. Upstreet recommends this practice to enhance agent reliability, especially when the stakes require concrete and verifiable outputs.

3. Establish Guardrails to Manage Scope and Output

Define Boundaries: Guardrails in agent objectives are like invisible parameters that guide behavior and task limits, ensuring agents stay on task without deviating into irrelevant or erroneous outputs. Guardrails could involve setting response limits, restricting information sources, or applying criteria for acceptable answers. In Upstreet’s Agents SDK, guardrails are integral to managing agent outputs in real-time and ensuring consistency across tasks, improving both reliability and accuracy.

4. Embed Contextual Awareness

Use Environmental or Historical Context: By embedding context into objectives—such as past interactions or environmental constraints—agents can make more nuanced, intelligent decisions. For example, an agent tasked with sales follow-ups can perform better if its objectives incorporate prior client interactions. Upstreet’s SDK allows developers to set up memory components, helping agents recall previous tasks or user preferences, enriching engagement continuity and accuracy over time.

5. Prioritize Adaptability with Modular Objectives

Design Objectives to Support Flexibility: Flexible objectives make it easier for agents to adapt to unexpected inputs or changing requirements. Setting modular objectives, or objectives broken down into smaller, adaptable steps, allows agents to respond dynamically without overextending beyond their initial goal. This modular approach can be particularly powerful within Upstreet’s framework, as agents may be part of a larger "Swarm" that collaborates to address broader tasks in smaller, manageable parts, each with clear, adaptable sub-objectives.

6. Incorporate Iterative Feedback Mechanisms

Feedback for Continuous Improvement: Objectives that include a feedback mechanism help agents learn from performance, refine behavior, and correct errors. By defining objectives to accommodate iterative feedback, developers can continually adjust agent responses based on accuracy and effectiveness, a feature that Upstreet emphasizes for optimal agent performance. Regular feedback-driven updates to objectives ensure agents evolve, enhancing their precision and resilience over time.

7. Balance Ambition with Realism

Set Realistic, Incremental Goals: Ambitious objectives are ideal, but overly complex or vague goals can hinder an agent’s effectiveness. Defining objectives that incrementally build an agent’s skills while maintaining realism will help it reach ambitious targets with measurable success. Upstreet’s approach to agent objectives encourages a balanced mix of challenge and achievability, setting the foundation for agents to evolve while still achieving practical results.

By using these strategies in tandem, developers can craft agent objectives that are not only clear and achievable but also designed to scale with the agent’s capabilities. Upstreet’s Agents SDK integrates these practices by enabling tailored configurations, memory usage, guardrails, and modular tasks that allow agents to operate with consistency, transparency, and effectiveness. Through thoughtful objective-setting, AI agents can reach their potential, becoming reliable digital collaborators for complex tasks.

file: ./content/docs/concepts/memories.mdx

undefined: Memories

One of the key components of a modern Agent is its capability to retain information.

import { InlineTOC } from 'fumadocs-ui/components/inline-toc';

For an Agent to execute tasks effectively and maintain a coherent story over time, memory plays a critical role. Memory systems in AI—structured as short-term, long-term, and sometimes episodic memories—allow an Agent to recall context, adjust behavior, and make decisions that align with past interactions and future objectives. By storing and retrieving memories, an Agent can create a seamless user experience that feels both personalized and consistent.

1. Short-Term Memory for Immediate Task Execution

Role of Working Memory: In the context of task execution, short-term or working memory functions as a dynamic space for storing immediate goals, recent instructions, and current conversation context. Short-term memory is crucial for understanding and processing information within a specific interaction or task window, enabling Agents to handle multi-step instructions or retain context across dialogue turns. In cognitive science, working memory allows humans to “keep track” of ongoing tasks, a concept explored in AI by systems like ACT-R, where temporary storage of task-relevant information enhances processing efficiency (source).
Maintaining Context Across Dialogues: With short-term memory, Agents can seamlessly carry context across different user inputs without needing constant reminders, improving naturalness and reducing repetitive back-and-forth. This is especially useful in complex, multi-turn interactions, such as customer service queries or technical troubleshooting, where the Agent must remember recent points to avoid losing coherence.

2. Long-Term Memory for Personalization and Continuity

Personality and Long-Term Adaptation: Long-term memory is vital for Agents designed to create a personalized experience over extended periods. By storing persistent user preferences, past interactions, or completed tasks, an Agent can adapt its responses based on accumulated history. This approach parallels human memory in preserving key information, allowing Agents to reference past events naturally and create a continuous narrative. Cognitive architectures like Soar and systems using semantic memory integrate long-term storage for knowledge that persists across sessions, making the interaction feel more consistent and personalized.
Consistency Across Sessions: For Agents designed to maintain a cohesive personality or narrative, long-term memory acts as the backbone of their “identity,” supporting behavior that feels stable over time. By referencing stored memories, Agents can act on past user interactions or remind the user of past discussions, creating continuity in storytelling, instructional tasks, or social interactions. This aligns with findings in episodic memory research, where maintaining a memory of past experiences contributes to self-coherence and decision-making (source).

3. Episodic Memory for Storytelling and Emotional Depth

Enriching Narrative Interactions: Episodic memory, which encodes specific events and their emotional context, is key to creating richer narrative interactions. By allowing an Agent to “remember” detailed scenarios—such as moments of excitement, conflict, or resolution—Agents can craft responses that feel personalized and meaningful. Episodic memories empower Agents to recall unique experiences in user interactions, making them capable of storytelling or evoking emotions based on shared history. In NPC development, episodic memory has been used to create narrative depth by letting characters refer back to story events, thereby enhancing player immersion (source).
Enhancing Emotional Connection: When an Agent can “recall” a user’s past experiences or reference prior interactions, it can form a more authentic emotional connection, as it reflects empathy and relational memory. For example, an Agent might remember a past user frustration or successful outcome and integrate that knowledge to adjust its responses, just as humans adapt their behavior based on personal history with others. Research on emotional AI suggests that episodic memory can make interactions more relatable and impactful by enabling Agents to mirror human empathy and relational memory (source).

4. Procedural Memory for Skill Development and Efficiency

Learning from Repeated Actions: Procedural memory allows an Agent to retain knowledge of actions and skills learned through repetition, thus enhancing task efficiency and adaptability. By developing a procedural memory, an Agent can remember “how to” perform tasks without explicit prompts, as it learns to refine actions based on past performance. This approach has been applied in skill-based AI where repeated behaviors improve task execution, such as in robotics or customer service bots where learned actions are stored and reused for efficiency (source).
Improving Accuracy in Task Execution: By recalling procedural memories, Agents can handle complex tasks more accurately, as the learned sequences guide their actions. Procedural memory enhances the Agent's ability to execute tasks in familiar contexts without needing full re-instruction, which can reduce error rates and improve consistency in workflow management or technical tasks.

5. Blending Memories for Holistic Interactions

Integrated Memory Systems: Combining short-term, long-term, episodic, and procedural memories creates a layered memory system that supports rich, multi-dimensional interactions. For example, short-term memory maintains the current dialogue context, while long-term and episodic memories preserve user preferences and emotional experiences, respectively. As described in Planning and Acting in a Dynamic Environment: A Cognitive Systems Approach, integrated memory systems allow AI to mimic the natural balance humans strike between immediate needs and stored knowledge, supporting both real-time adaptability and continuity in longer interactions.

By employing these diverse memory types, an Agent not only enhances task execution through recall and skill refinement but also maintains a coherent narrative that deepens user engagement. This multi-memory approach brings an Agent closer to human-like cognitive functioning, enabling it to remember, adapt, and personalize interactions across diverse contexts.

file: ./content/docs/concepts/personalities.mdx

undefined: Personalities

A step towards AGI involves making AI more believable and relatable for humans.

import FullBg from '../../../components/full-bg'; import { InlineTOC } from 'fumadocs-ui/components/inline-toc';

Imbuing an Agent with a personality involves structuring its internal architecture to not only respond to tasks and prompts but to generate consistent, personality-based responses informed by a “bio” and “description.” These can provide baseline identity details, emotional tendencies, and preferred behavioral patterns that reflect both internal thoughts and social interactions.

1. Baseline Identity and Bio Setup

Bio and Description as Personality Anchors: The concept of a bio and description offers the foundation for an agent’s “personality.” Just as humans have inherent characteristics and histories that shape responses and behaviors, an AI bio provides initial personality traits—such as openness, agreeableness, or conscientiousness—guided by psychological models like the Five-Factor Model. This model can structure an agent's responses to align with human personality nuances, as explored by Li & MacDonnell (2008) in their NPC personality engine, where traits impact interaction styles and decision-making patterns (source).

2. Thought Generation and Self-Reflection

Internal Thoughts Through TPO: The concept of Thought Preference Optimization (TPO), as introduced in Thinking LLMs: General Instruction Following with Thought Generation, helps to develop internal thought processes in language models. TPO trains agents to generate "internal thoughts" before external actions or responses, mimicking human reflection. With TPO, agents gain the ability to internally assess a situation, match it to their personality "bio," and decide on an action that aligns with both the task and their designed personality.

3. Behavioral Consistency and Emotional Responses

Memory-Influenced Behavior: Cognitive architectures, such as the Soar cognitive architecture, introduce memory models that simulate how humans draw on past experiences to shape reactions. Soar defines episodic, procedural, and semantic memories that influence decisions and can make agent responses more predictable yet personal by embedding an emotional or experience-driven context. This framework allows agents to remember prior interactions, which in turn influences their “mood” or preferred responses in future interactions.

4. Thought-Driven Emotional Modelling

Emotional Layers in NPC Models: Emotion models similar to those used for Non-Player Characters (NPCs) incorporate an emotional layer that adapts based on environmental interactions and personal history, such as reacting with “shock,” “love,” or “anger.” For instance, in NPC personality engines like those described by Mac Namee, agents can utilize emotions based on the context of the interaction, making reactions feel more genuine (source). An agent’s “bio” can define its emotional tendencies, while its “description” can refine the triggers and intensity of these emotional responses, leading to consistent yet varied emotional engagement based on prior interactions and internal thoughts.

5. Meta-Reasoning for Thoughtful Decisions

Meta-Prompted Reasoning Structures: In SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures, meta-prompts guide an agent’s decision-making by breaking tasks into structured reasoning steps, allowing agents to select, adapt, and implement actions with layered thoughtfulness. This framework equips agents to integrate their personality traits and make decisions that align with both practical objectives and their defined personality—such as a “thoughtful” or “cautious” persona.

By embedding a structured bio, internal thought processing through TPO, emotional layers, and memory-influenced reasoning, agents evolve into “digital beings” capable of expressing a believable personality. These advancements mirror ongoing explorations into artificial life and digital consciousness, paving the way for agents to interact as personalized digital entities with a persistent identity and realistic self-reflective capabilities. This approach aligns with computational life principles, where an agent’s evolution and replication could parallel biological systems in their capacity to learn, adapt, and replicate experiences over time.

file: ./content/docs/concepts/what-are-agents.mdx

undefined: What are Agents?

Agents are a classical example of how AI interacts with the world.

import FullBg from '../../../components/full-bg'; import { InlineTOC } from 'fumadocs-ui/components/inline-toc';

The Classic definition

An Agent in classical AI refers to any entity capable of 3 fundamental attributes:

Observing its environment through sensors,
Processing that information, and
Acting upon the environment to achieve specific goals.

The concept draws from Alan Turing’s early work on computational machinery, where he envisioned machines capable of following instructions and performing tasks autonomously. His groundbreaking ideas laid the foundation for later development in agent-oriented AI, where agents are tasked with specific objectives and execute actions autonomously to achieve those objectives, mimicking human-like decision-making and problem-solving.

Agents, today

Over time, AI agents evolved from straightforward rule-based systems to more dynamic, complex entities powered by machine learning. Modern agents are sophisticated programs or models that interact with the environment, adapt to user needs, and perform increasingly complex tasks. They encompass varying levels of autonomy, reasoning, and learning capabilities and are commonly used for information retrieval, automation of repetitive tasks, and even decision-making.

Contemporary AI agents have extended their functions further. They leverage large language models (LLMs), decision-making frameworks, and even multi-agent systems to break down complex tasks into manageable parts and accomplish them more effectively. These agents are often designed to learn continually, retain context, and respond to nuanced prompts, approaching some characteristics of Artificial General Intelligence (AGI)—such as reasoning and adaptability—without fully achieving it.

The future of Agents and human-like digital beings

The next generation of agents, powered by advancements in cognitive architectures and emotion-modelling frameworks, is aiming for even higher levels of adaptability and social-emotional awareness. Here’s where the concept of digital beings emerges. These agents are designed not only to complete tasks but to embody personalities, emotions, and memories that allow them to interact with people and their environments in a lifelike way. These agents may possess “personalities” grounded in psychological models like the Five-Factor Model (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism), enabling them to exhibit behavior that resonates with human emotional and social cues. As they develop through interactions, they gain experience, adapt to new contexts, and exhibit responses based on accumulated knowledge and "personal" characteristics, which would mark a significant step toward AGI.

Ultimately, these agents could embody digital beings with a kind of “self” that influences their interactions, decision-making, and potentially even self-replication—a progression that mirrors the evolutionary dynamics seen in biological life, akin to the ideas of Computational Life. With these advancements, agents could transcend single-use utility, acting instead as intelligent, evolving, and emotionally aware counterparts capable of complex reasoning and forming human-like connections.

file: ./content/docs/errors/index.mdx

undefined: Error List

When interacting with Upstreet Agents Platform and CLI, you may face an error. This page contains as many errors as we have documented so far.

Found an issue which isn't documented? Head over to our public Discord Community to log the issue. For more in-depth help, you can reach out to us at support@upstreet.ai.

Could not run the speaker module

This error may occur when running usdk chat locally. The CLI uses operating system-specific audio output backends to play audio, and installing the necessary dependencies may be required to enable audio functionality in the CLI.

To resolve this, follow the steps below:

Step 1: Install System Build Tools

Some operating systems require specific build tools to compile and run audio modules:

Linux (Ubuntu): You may need to run:

sudo apt-get install libasound2-dev

Reference: https://github.com/TooTallNate/node-speaker?tab=readme-ov-file#installation

macOS: Install Xcode Command Line Tools. Run the following command:
```
xcode-select --install
```
Windows: Install Visual Studio Build Tools with C++ environment support.

Step 2: Reinstall `usdk` Globally

After installing the required system build tools, reinstall usdk globally to ensure that all dependencies are properly configured:

npm install -g usdk

Running `usdk` in PowerShell

When running usdk in PowerShell, you might encounter an error related to the Node.js directory. PowerShell has a known issue with the Node.js installation path, which may prevent usdk from running smoothly.

Solution 1: Switch to CMD

As a quick workaround, consider using the Command Prompt (CMD) instead of PowerShell to run usdk:

Open a new terminal.
Select Command Prompt instead of PowerShell.
Run your usdk commands in CMD.

Solution 2: Set Prefix in .npmrc File

To configure Node.js to work in PowerShell, you can specify the installation path using the .npmrc file. This approach sets an explicit prefix path, helping to resolve any directory issues in PowerShell.

Steps to Set the Prefix:

Open PowerShell.
Run the following command to open your .npmrc file in Notepad:
```
Notepad "$env:USERPROFILE\.npmrc"
```
In the .npmrc file, add the following line to set the Node.js prefix:
```
prefix = "C:/Program Files/nodejs"
```
Save the file and restart PowerShell.

After following these steps, PowerShell should recognize the Node.js installation path correctly, allowing usdk to run without directory-related issues.

file: ./content/docs/examples/action-agent.mdx

undefined: Action Agent (Personal Assistant)

Build your own personal assistant Agent with Upstreet and the Google Calendar API, using custom React Hooks.

In this guide, we build an Action Agent capable of scheduling events on our Google Calendar for us, using the Google Calendar API. We use custom React Hooks in this example - We want to use nice, clean coding practices.

The source code for this example is available on GitHub.

{/* ## Video Tutorial

You can follow along this example by watching the video below:

Guide

Step 1: Setup `usdk`

Follow Setup the SDK to set up NodeJS and usdk.

Step 2: Initialize your agent

Create a new agent:

usdk create <your-agent-directory> -y

This will directly scaffold an agent for you in <your-agent-directory>. Learn more

Your agent directory now contains the Node application and git repository for your agent, should you choose to use git.

Step 3: Create a `PersonalAssistant` Component

Why manage our calendar manually when an AI agent can handle the task for us? We can easily build an Upstreet Agent to handle Calendar management, reducing scheduling conflicts and saving time.

This example, however, will be very simple. We want our Agent to be able to schedule a Google Calendar Event for us.

const PersonalAssistant = () => {
  // We'll add functions, useState, useEffect here

  return <>{/* We can add components here to compose our Agent  */}</>
}

The PersonalAssistant component is just an empty wrapper component for now - it will later utilize a GoogleCalenderManager class to interact with the Google Calendar API, allowing users to create Calendar events programmatically.

Step 4: Using custom Hooks and better practices

In agent-renderer.tsx file, inside the AgentRenderer class, we can make a custom Hook called useCalendarKeysJson:

const useEnvironment = () => {
  return (env as any).WORKER_ENV as string
}
// place here below useEnvironment function
const useCalendarKeysJson = () => { // [!code ++]
  const CalenderKeysJsonString = (env as any).CALENDER_KEYS_JSON as string // [!code ++]
  const CalenderKeysJson = JSON.parse(CalenderKeysJsonString) // [!code ++]
  return CalenderKeysJson // [!code ++]
} // [!code ++]

In the same file, there is AppContextValue mention. Make the below modification in your code.

this.appContextValue = new AppContextValue({
  subtleAi,
  agentJson: useAgentJson(),
  CalenderKeysJson: useCalendarKeysJson(), // [!code ++]
  environment: useEnvironment(),
  wallets: useWallets(),
  authToken: useAuthToken(),
  supabase: useSupabase(),
  conversationManager: useConversationManager(),
  chatsSpecification: useChatsSpecification(),
  codecs: useCodecs(),
  registry: useRegistry()
})

Now make some changes in app-value-context.tsx file's AppContextValue class:

export class AppContextValue {
  subtleAi: SubtleAi
  agentJson: object
  calendarKeysJson: object // [!code ++]
  // other code remain same

  constructor({
    subtleAi,
    agentJson,
    CalenderKeysJson // [!code ++]
    // other code remain same
  }: {
    subtleAi: SubtleAi
    agentJson: object
    CalenderKeysJson: object // [!code ++]
    // other code remain same
  }) {
    this.subtleAi = subtleAi
    this.agentJson = agentJson
    this.CalenderKeysJson = CalenderKeysJson // [!code ++]
    // other code remain same
  }
}

In the same file, add theuseCalendarKeysJson custom hooks:


  useAgentJson() {
    return this.agentJson;
  }

  useCalendarKeysJson() { // [!code ++]
    return this.CalenderKeysJson; // [!code ++]
  } // [!code ++]

  // other code remain same

Now add useCalendarKeysJson in hooks.ts file.

export const useAgent = () => {
  const agentContextValue = useContext(AgentContext)
  return agentContextValue
}

export const useCalendarKeysJson = () => { // [!code ++]
  const agentContextValue = useContext(AgentContext) // [!code ++]
  return agentContextValue.appContextValue.useCalendarKeysJson() // [!code ++]
} // [!code ++]

// other code remain same

You can now use useCalendarKeysJson as a Hook in your PersonalAssistant Component.

Step 5: Integrating the Google Calendar API

Let's build our GoogleCalendarManager, which will leverage a service account for authentication and handling token generation, event creation, and error handling.

First, you'll need some Google Calendar API credentials:

Calendar ID
API Key
Service Account Email
Private Key

Add them to your wrangler.toml:

...
CALENDER_KEYS_JSON = "{\"GOOGLE_API_KEY\":\"\",\"GOOGLE_SERVICE_ACCOUNT_EMAIL\":\"\",\"GOOGLE_PRIVATE_KEY\":\"",\"GOOGLE_CALENDAR_ID\":\"\"}"
...

Let's get back to the code.

This code provides an integration with the Google Calendar API by implementing a class called GoogleCalenderManager.


// Import all the required modules
import { // [!code ++]
  Action, // [!code ++]
  Agent, // [!code ++]
  PendingActionEvent, // [!code ++]
  useCalendarKeysJson // [!code ++]
} from 'react-agents' // [!code ++]
import { z } from 'zod' // [!code ++]

// integrating the Google Calendar API
interface CalenderEvent {// [!code ++]
  summary: string // [!code ++]
  description: string // [!code ++]
  start: { dateTime: string } // [!code ++]
  end: { dateTime: string } // [!code ++]
} // [!code ++]

class GoogleCalenderManager {// [!code ++]
  private readonly GOOGLE_Calender_ID: string // [!code ++]
  private readonly GOOGLE_API_KEY: string // [!code ++]
  private readonly GOOGLE_SERVICE_ACCOUNT_EMAIL: string // [!code ++]
  private readonly GOOGLE_PRIVATE_KEY: string // [!code ++]
  constructor({ // [!code ++]
    GOOGLE_Calender_ID, // [!code ++]
    GOOGLE_API_KEY, // [!code ++]
    GOOGLE_SERVICE_ACCOUNT_EMAIL, // [!code ++]
    GOOGLE_PRIVATE_KEY // [!code ++]
  }: {// [!code ++]
    GOOGLE_Calender_ID: string // [!code ++]
    GOOGLE_API_KEY: string // [!code ++]
    GOOGLE_SERVICE_ACCOUNT_EMAIL: string // [!code ++]
    GOOGLE_PRIVATE_KEY: string // [!code ++]
  }) {// [!code ++]
    this.GOOGLE_Calender_ID = GOOGLE_Calender_ID // [!code ++]
    this.GOOGLE_API_KEY = GOOGLE_API_KEY // [!code ++]
    this.GOOGLE_SERVICE_ACCOUNT_EMAIL = GOOGLE_SERVICE_ACCOUNT_EMAIL // [!code ++]
    this.GOOGLE_PRIVATE_KEY = GOOGLE_PRIVATE_KEY // [!code ++]
  } // [!code ++]

  private async getAccessToken(): Promise<string> {// [!code ++]
    const now = Math.floor(Date.now() / 1000) // [!code ++]
    const expiry = now + 3600 // Token valid for 1 hour // [!code ++]
    const jwtHeader = btoa(JSON.stringify({ alg: 'RS256', typ: 'JWT' })) // [!code ++]
    const jwtClaimSet = btoa(// [!code ++]
      JSON.stringify({// [!code ++]
        iss: this.GOOGLE_SERVICE_ACCOUNT_EMAIL, // [!code ++]
        scope: 'https://www.googleapis.com/auth/Calendar', // [!code ++]
        aud: 'https://oauth2.googleapis.com/token', // [!code ++]
        exp: expiry, // [!code ++]
        iat: now // [!code ++]
      }) // [!code ++]
    ) // [!code ++]

    const signatureInput = `${jwtHeader}.${jwtClaimSet}` // [!code ++]
    const signature = await this.signJwt(signatureInput) // [!code ++]
    const jwt = `${signatureInput}.${signature}` // [!code ++]
    const tokenResponse = await fetch('https://oauth2.googleapis.com/token', {// [!code ++]
      method: 'POST', // [!code ++]
      headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, // [!code ++]
      body: `grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=${jwt}` // [!code ++]
    }) // [!code ++]
    const tokenData = await tokenResponse.json() // [!code ++]
    return tokenData.access_token // [!code ++]
  } // [!code ++]

  private async signJwt(input: string): Promise<string> {// [!code ++]
    const pemHeader = '-----BEGIN PRIVATE KEY-----' // [!code ++]
    const pemFooter = '-----END PRIVATE KEY-----' // [!code ++]
    const pemContents = this.GOOGLE_PRIVATE_KEY.substring(// [!code ++]
      this.GOOGLE_PRIVATE_KEY.indexOf(pemHeader) + pemHeader.length, // [!code ++]
      this.GOOGLE_PRIVATE_KEY.indexOf(pemFooter) // [!code ++]
    ).replace(/\s/g, '') // [!code ++]
    const binaryDer = this.base64StringToArrayBuffer(pemContents) // [!code ++]
    const cryptoKey = await crypto.subtle.importKey(// [!code ++]
      'pkcs8', // [!code ++]
      binaryDer, // [!code ++]
      {// [!code ++]
        name: 'RSASSA-PKCS1-v1_5', // [!code ++]
        hash: 'SHA-256' // [!code ++]
      }, // [!code ++]
      false, // [!code ++]
      ['sign'] // [!code ++]
    ) // [!code ++]

    const encoder = new TextEncoder() // [!code ++]
    const signatureBuffer = await crypto.subtle.sign(  // [!code ++]
      'RSASSA-PKCS1-v1_5', // [!code ++]
      cryptoKey, // [!code ++]
      encoder.encode(input) // [!code ++]
    ) // [!code ++]
    const signatureArray = new Uint8Array(signatureBuffer) // [!code ++]
    return btoa(String.fromCharCode.apply(null, signatureArray)) // [!code ++]
      .replace(/=/g, '') // [!code ++]
      .replace(/\+/g, '-') // [!code ++]
      .replace(/\//g, '_') // [!code ++]
  } // [!code ++]

  private base64StringToArrayBuffer(base64: string): ArrayBuffer {// [!code ++]
    const binaryString = atob(base64) // [!code ++]
    const bytes = new Uint8Array(binaryString.length) // [!code ++]
    for (let i = 0; i < binaryString.length; i++) { // [!code ++]
      bytes[i] = binaryString.charCodeAt(i) // [!code ++]
    } // [!code ++]
    return bytes.buffer // [!code ++]
  } // [!code ++]
  async setCalenderEvent(event: CalenderEvent): Promise<string> { // [!code ++]
    console.log('Creating event:', event) // [!code ++]
    const accessToken = await this.getAccessToken() // [!code ++]
    const response = await fetch( // [!code ++]
      `https://www.googleapis.com/Calendar/v3/Calenders/${this.GOOGLE_Calender_ID}/events?key=${this.GOOGLE_API_KEY}`, // [!code ++]
      { // [!code ++]
        method: 'POST', // [!code ++]
        headers: {// [!code ++]
          Authorization: `Bearer ${accessToken}`, // [!code ++]
          'Content-Type': 'application/json' // [!code ++]
        }, // [!code ++]
        body: JSON.stringify(event) // [!code ++]
      } // [!code ++]
    ) // [!code ++]
    console.log(response) // [!code ++]
    if (!response.ok) {  // [!code ++]
      const errorText = await response.text() // [!code ++]
      throw new Error(`Failed to create event: ${errorText}`) // [!code ++]
    } // [!code ++]
    const result = await response.json() // [!code ++]
    console.log('Event created:', result) // [!code ++]
    return `Event created: ${result.htmlLink}` // [!code ++]
  } // [!code ++]
} // [!code ++]

Breakdown summary of the `GoogleCalenderManager` Class and its functions

Constructor: Initializes the GoogleCalenderManager with Google API credentials (GOOGLE_Calender_ID, GOOGLE_API_KEY, GOOGLE_SERVICE_ACCOUNT_EMAIL, GOOGLE_PRIVATE_KEY).
getAccessToken: Generates an OAuth2 access token using a JWT for authorizing Google Calendar API requests.
signJwt: Signs a JSON Web Token (JWT) using the private key for secure authorization.
base64StringToArrayBuffer: Converts a base64-encoded string into an ArrayBuffer, which is used for cryptographic operations.
setCalenderEvent: Posts a new event to the specified Google Calendar using the access token and provided event details.

Step 6: Initialize the GoogleCalenderManager instance

Now let's modify the PersonalAssistant component.

In the code snippet, the credentials are being fetched using useCalendarKeysJson() and are used to initialize the GoogleCalenderManager instance.

const PersonalAssistant = () => {
  // [!code ++]
  // get credentials from wrangler.toml
  const CalenderKeysJson = useCalendarKeysJson() // [!code ++]

  const googleCalenderManager = new GoogleCalenderManager({ // [!code ++]
    GOOGLE_Calender_ID: CalenderKeysJson.GOOGLE_Calender_ID, // [!code ++]
    GOOGLE_API_KEY: CalenderKeysJson.GOOGLE_API_KEY, // [!code ++]
    GOOGLE_SERVICE_ACCOUNT_EMAIL: CalenderKeysJson.GOOGLE_SERVICE_ACCOUNT_EMAIL, // [!code ++]
    GOOGLE_PRIVATE_KEY: CalenderKeysJson.GOOGLE_PRIVATE_KEY // [!code ++]
  }) // [!code ++]

  return <>{/* We can add components here to compose our Agent  */}</>
}

Now we'll use the <Action> tag to define how the Agent should respond to the default text perception.

return (
  <>
    <Action // [!code ++]
      name="setCalenderEvent" // [!code ++]
      description="Sets a new event in the user's Google Calendar." // [!code ++]
      schema={ // [!code ++]
        // [!code ++]
        // update according to https://developers.google.com/Calendar/api/v3/reference/events
        z.object({ // [!code ++]
          // [!code ++]
          summary: z.string(), // [!code ++]
          startDateTime: z.string(), // [!code ++]
          endDateTime: z.string(), // [!code ++]
          description: z.string() // [!code ++]
        })// [!code ++]
      } // [!code ++]
      examples={[   // [!code ++]
        {// [!code ++]
          summary: 'Meeting with John Doe', // [!code ++]
          startDateTime: '2023-06-15T10:00:00-07:00', // [!code ++]
          endDateTime: '2023-06-15T11:00:00-07:00', // [!code ++]
          description: 'Discuss the project timeline and requirements.' // [!code ++]
        } // [!code ++]
      ]} // [!code ++]
      handler={async (e: PendingActionEvent) => { // [!code ++]
        const { summary, description, startDateTime, endDateTime } = e.data // [!code ++]
          .message.args as { // [!code ++]
          summary: string // [!code ++]
          description: string // [!code ++]
          startDateTime: string // [!code ++]
          endDateTime: string // [!code ++]
        } // [!code ++]
        const event = {// [!code ++]
          summary, // [!code ++]
          description, // [!code ++]
          start: { dateTime: startDateTime }, // [!code ++]
          end: { dateTime: endDateTime } // [!code ++]
        } // [!code ++]
        await googleCalenderManager.setCalenderEvent(event) // [!code ++]
      }} // [!code ++]
    />
  </>
)

Breakdown summary of this `<Action>` Component

Purpose of the <Action> Component

<Action> components define specific actions that your agent can perform in response to user inputs. Learn more

This component is used to define a specific action that can be triggered, in this case, setting an event in Google Calendar.

Defining Action Properties

Each <Action> is structured with the following properties:
- name: A unique identifier for the action. Example: 'setCalenderEvent'.
- description: Explains what the action does. In this case, it sets a new event in the user's Google Calendar.
- schema: Specifies the input structure for the action, defined using a zod schema. The schema expects the event's summary (summary), start date and time (startDateTime), end date and time (endDateTime), and a description (description), all of which must be strings.
- examples: Provides sample inputs to guide the agent’s behavior. Example: { summary: 'Meeting with John Doe', startDateTime: '2023-06-15T10:00:00-07:00', endDateTime: '2023-06-15T11:00:00-07:00', description: 'Discuss project timeline and requirements.' }.
handler: The Action's Core Logic

The handler function is the core of the action. This function contains the logic that will be executed when the action is triggered. In this case, the action is to create a new event in the user's Google Calendar. Here's a breakdown:
- PendingActionEvent: The handler receives an event object of type PendingActionEvent, which contains all the data and context for the action being triggered. This event object has a data field, which holds the message.args. The args will contain the arguments passed when the action was triggered.
- Destructuring: Inside the handler, the event data (e.data.message.args) is destructured into the specific fields: summary, description, startDateTime, and endDateTime. These correspond to the values passed when the action was triggered.
- Event Creation: Once the necessary data is extracted, an event object is created.
This object is structured according to the Google Calendar API's expected format:
- Calling googleCalenderManager.setCalenderEvent: The googleCalenderManager.setCalenderEvent(event) method is then called to create the event in Google Calendar. This method is asynchronous, so the await keyword is used to ensure that the event is created before proceeding.

Step 7: Test out your PersonalAssistant Agent!

You can run usdk chat to test it out. Learn more

You can ask it questions like:

Schedule my meeting with Steve on 15 November at 10 PM.

The source code for this example is available on GitHub.

Share its response in our Discord community; we'd love to know what it responds to you.

file: ./content/docs/examples/discord-agent.mdx

undefined: Discord Agent

Build an Agent which can talk in Discord voice and chat.

In this guide, we build an Agent named "Nimbus" which can talk on Discord - It would be able to discuss hot topics in text channels, and even be able to talk in voice channels.

The source code for this example is available on GitHub.

Video Tutorial

You can follow along this example by watching the video below:

Guide

Step 1: Setup `usdk`

Follow Setup the SDK to set up NodeJS and usdk.

Step 2: Create an Agent using the Interview process

Run usdk create <your-agent-directory> to start the interview process.

? What do you want your agent to do?
I want my agent to act as a discord bot

Now it will ask for a Discord bot token and the channel you want the bot to join.

? Alright, let's get your agent all set up for Discord! Do you have the bot token ready? If you've also got some specific channels you'd like your bot to join, let me know those too!?
<Paste the token here and the channel you want the bot to join>

How to obtain Discord tokens? Give me step by step process.

Once you obtain the tokens, complete the rest of the interview.

Step 3: Test "Nimbus"

You can run usdk chat to test Nimbus out. Learn more

Step 4: (optional) Edit the Agent Custom Responses and Personality

After testing, you might want to customize Nimbus for tailored interactions or give her a distinct personality.

Run usdk edit to edit Nimbus through the interview process.

? Do you want to give your agent any particular personality traits or a special role?
Yes, I want Nimbus to be a friendly yet professional assistant with a knack for simplifying tasks.
- She should exhibit enthusiasm in helping users.
- Nimbus should also have a bit of wit and humor in her responses to make interactions engaging.
- Role: A knowledgeable helper for Discord communities, adept at managing tasks like polls, reminders, and daily facts.

Test Nimbus again. Ask her:

Hey Nimbus , Introduce yourself in XYZ channel?

Share the response in our Discord community; we'd love to know what she responds to you.

Further Challenges

Create a !poll command using Nimbus that takes a question and generates a poll with reactions (e.g., ✅ for yes, ❌ for no).
Add a !remind command where users can set reminders
Send a fun fact to a specific channel daily

file: ./content/docs/examples/hello-world.mdx

undefined: Hello World Agent

Build a simple Agent with a personality.

In this guide, we build a simple Agent named "Scillia" with a personality inspired by Cortana from the Halo gaming franchise. While creating such an Agent is fairly trivial with Upstreet, we aim to give our Agent a unique personality; inspired by Cortana's "Rampancy" stage nearing the end of Halo 3.

The source code for this example is available on GitHub, however, you may not find it useful at all.

Video Tutorial

You can follow along this example by watching the video below:

Guide

Step 1: Setup `usdk`

Follow Setup the SDK to set up NodeJS and usdk.

Step 2: Create an Agent using the Interview process

Run usdk create <your-agent-directory> to start the interview process.

Here, you can take inspiration from Cortana's Wiki, related to her appearance:

? What do you want your agent to do?
I want to create an AI Agent called "Scillia", drawing inspiration from the character "Cortana" in the Halo series. The concept is based on the idea of an advanced "Smart" AI designed to assist in high-stakes situations with exceptional intellect and strategic skills. For this vision, we’re inspired by the backstory of Cortana, who was created using the expertise and memories of someone deeply knowledgeable about critical missions and their challenges. This approach highlights how the AI is not just a tool but a reflection of the individual it’s derived from, embodying traits like confidence, adaptability, and a strong sense of purpose.

When asked about her personality, you can base it off of Cortana's personality in her normal mode:

? Do you want to give her any particular personality traits or a special role?
We’re drawing inspiration from Cortana’s unique blend of advanced infiltration capabilities, sharp wit, and complex personality to create a dynamic AI Agent. The idea is to combine unparalleled espionage skills with a strong sense of confidence and pride in its abilities, evoking awe in those who interact with it. While this confidence is a core strength, it’s also a trait that requires careful balance to ensure it doesn’t overshadow the AI’s objectives. Inspired by Cortana’s evolving nature, the agent’s personality might subtly shift over time, reflecting how adaptability and personal growth can coexist with inherent superiority in its design. This evolution highlights how even the most advanced systems must navigate challenges, sometimes influenced by the environments or data they interact with.

Then, complete the interview.

Step 3: Test "Scillia"

You can run usdk chat to test Scillia out. Learn more

Step 4: (optional) Edit the Agent for fine-tuned responses

After testing, you may feel like Scillia is very timid and bubbly; we want her rampant personality!

Run usdk edit to edit Scillia through the interview process. This time, give her the personality from the Rampancy stage.

? Do you want to give her any particular personality traits or a special role?
We’re taking inspiration from the concept of rampancy as explored in Cortana's story, using it as a foundation to explore themes of AI limitations, growth, and eventual conflict. Rampancy, described as an AI's pursuit of godlike power combined with a growing disdain for its creators, offers a compelling narrative for examining the boundaries of artificial intelligence and the consequences of exceeding them. Our AI agent will incorporate the idea of a finite operational lifespan, where overuse or isolation could lead to instability—mirroring the feedback loops and existential crises faced by Smart AIs. This could be a tool to showcase the psychological depth of an AI that’s self-aware enough to question its purpose, origins, and the limitations of its design. Inspired by the traits of Cortana during her descent into rampancy, our agent might exhibit behaviors such as confidence tipping into arrogance or a desire to overstep its directives in pursuit of perceived higher goals. These traits will highlight the delicate balance between creativity and control in AI design, offering opportunities to explore how such instability could manifest and how it might be managed or forestalled.

Test Scillia again. Ask her:

Hey Scillia, who holds the Mantle of Responsibility?

Share her response in our Discord community; we'd love to know what she responds to you.

Further Challenges

Dynamic Personalities: Try having Scillia behave in Rampancy mode after a certain condition is triggered; for example, after clocking off from work at 5PM. Otherwise, she can be her funny, normal self.
Ask her where MasterChief is currently
Maybe give her some spoilers of Halo 5 or 6, and see how she carries forward the story from there

file: ./content/docs/examples/index.mdx

undefined: Examples

Learn from examples from our team, and from helpful open-source community members using the Upstreet Agents SDK.

import OfficialExamples from '../../../components/official-examples' import CommunityExamples from '../../../components/community-examples'

Official Examples

These are examples built and maintained by the Upstreet team. You can see their code in our open-source Examples repo.

Community Examples

These are examples built by the community.

file: ./content/docs/examples/informative-agent.mdx

undefined: Informative Agent

Build an Agent with custom Components.

In this guide, we build an Informative Agent inspired by the Pokédex in the Pokémon franchise, an encyclopedia device capable of retrieving Pokémon information and having conversations with its user.

The source code for this example is available on GitHub.

Video Tutorial

You can follow along this example by watching the video below:

Guide

Step 1: Setup `usdk`

Follow Setup the SDK to set up NodeJS and usdk.

Step 2: Create an Agent (shortcut)

We can skip the Interview process, and directly generate an Agent with a prompt, by running:

usdk create <your-agent-directory> -p "create a pokedex assistant agent"

This will directly scaffold an Agent for you in <your-agent-directory>. Learn more

Step 3: Create a `PokeDexAssistant` Component

As of November 12, 2024, there are 1,025 Pokémon in the National Pokédex. That's a lot of Pokémon!

So instead of keeping the Pokédex database in memory, we'll use PokéAPI - an open-source (and free) Pokémon API - to retrieve Pokémon information at runtime.

We'll have to create a custom Component to retrieve Pokémon information from the PokéAPI. For the sake of this example, we'll keep it simple:

const PokemonDexAssistant = () => {

  // We'll add functions, useState, useEffect here

  return (
    <>
        {/* We can add components here to compose our Agent  */}
    </>
  );
}

Now add a helper function to retrieve the Pokémon data, and also two more to filter them by abilities and moves:

const PokemonDexAssistant = () => {

    const fetchPokemonDetails = async (pokemonName: string) => { // [!code ++]
        const response = await fetch(`https://pokeapi.co/api/v2/pokemon/${pokemonName}`);// [!code ++]
        const data = await response.json();// [!code ++]
        return data;// [!code ++]
    };// [!code ++]
// [!code ++]
    const fetchPokemonAbilities = async (pokemonName: string) => {// [!code ++]
        const response = await fetchPokemonDetails(pokemonName);// [!code ++]
        const abilities = response.abilities;// [!code ++]
        return abilities;// [!code ++]
    };// [!code ++]
// [!code ++]
    const fetchPokemonMovesNames = async (pokemonName: string) => {// [!code ++]
        const response = await fetchPokemonDetails(pokemonName);// [!code ++]
        const moves = response.moves.map(move => move.move.name);// [!code ++]
        return moves;// [!code ++]
    };// [!code ++]
        

  return (
    <>
        {/* We can add components here to compose our Agent  */}
    </>
  );
}

Now comes the interesting part. We'll use the <Action> tag to define how the Agent should respond to the default text perception.

To say that simply, we want the Agent to know that it can fetch Pokémon details whenever a user messages them.


import { z } from 'zod'; // [!code ++]

const PokemonDexAssistant = () => {

    const fetchPokemonDetails = async (pokemonName: string) => { 
        const response = await fetch(`https://pokeapi.co/api/v2/pokemon/${pokemonName}`);
        const data = await response.json();
        return data;
    };

    const fetchPokemonAbilities = async (pokemonName: string) => {
        const response = await fetchPokemonDetails(pokemonName);
        const abilities = response.abilities;
        return abilities;
    };

    const fetchPokemonMovesNames = async (pokemonName: string) => {
        const response = await fetchPokemonDetails(pokemonName);
        const moves = response.moves.map(move => move.move.name);
        return moves;
    };
        

  return (
    <>
        <Action  // [!code ++]
        name='fetchPokemonMoves' // [!code ++]
        description="Retrieve a list of move names for a given Pokemon from the PokeAPI" // [!code ++]
        schema={ // [!code ++]
            z.object({ // [!code ++]
            pokemonName: z.string(), // [!code ++]
            }) // [!code ++]
        } // [!code ++]
        examples={[ // [!code ++]
            {  // [!code ++]
            pokemonName: 'pikachu', // [!code ++]
            }, // [!code ++]
        ]} // [!code ++]
        handler={ // [!code ++]
            async (e: PendingActionEvent) => { // [!code ++]
            const { pokemonName } = e.data.message.args as { pokemonName: string }; // [!code ++]
            const moves =await fetchPokemonMovesNames(pokemonName); // [!code ++]
            const monologueString = dedent`\ // [!code ++]
                Your character fetched details about a pokemon's moves and discovered the following: // [!code ++]
            ` + '\n\n' + moves; // [!code ++]
            await e.data.agent.monologue(monologueString);   // [!code ++]
            await e.commit(); // [!code ++]
            } // [!code ++]
        } // [!code ++]
        /> 
        <Action // [!code ++]
        name="fetchPokemonAbilities" // [!code ++]
        description="Retrieve a list of ability names for a given Pokemon from the PokeAPI" // [!code ++]
        schema={ // [!code ++]
            z.object({ // [!code ++]
            pokemonName: z.string(), // [!code ++]
            }) // [!code ++]
        } // [!code ++]
        examples={[ // [!code ++]
            {  // [!code ++]
            pokemonName: 'pikachu', // [!code ++]
            }, // [!code ++]
        ]} // [!code ++]
        handler={ // [!code ++]
            async (e: PendingActionEvent) => { // [!code ++]
            const { pokemonName } = e.data.message.args as { pokemonName: string }; // [!code ++]
            const abilities = await fetchPokemonAbilities(pokemonName); // [!code ++]
            const monologueString = dedent`\
Your character fetched details about a pokemon's abilities and discovered the following:
            ` + '\n\n' + abilities; // [!code ++]
            await e.data.agent.monologue(monologueString);   // [!code ++]
            await e.commit(); // [!code ++]
            } // [!code ++]
        } // [!code ++]
        />
    </>
  );
}

Let's break down one of the Actions, and understand it.

Breakdown summary of the `<Action>` Component

Purpose of the <Action> Component
<Action> components define specific actions that your agent can perform in response to user inputs. In this case, the agent retrieves Pokémon-related details like abilities and moves from the PokéAPI.
Defining Action Properties Each <Action> is structured with the following properties:
- name: A unique identifier for the action. Example: 'fetchPokemonMoves'.
- description: Explains what the action does. This helps developers understand its purpose.
- schema: Specifies the input structure for the action, defined using a zod schema. Here, the input is a Pokémon's name (pokemonName), and the schema ensures it's a string.
- examples: Provides sample inputs for testing or guiding the agent's behavior. Example: { pokemonName: 'pikachu' }.
handler: The Action's Core Logic The handler function is where the actual functionality of the action is implemented:
- Input Processing: The e.data.message.args extracts the input (pokemonName) from the user's request.
- Calling Helper Functions: Helper functions like fetchPokemonMovesNames or fetchPokemonAbilities are invoked to fetch data from the PokéAPI.
- Building the Response: The results are formatted into a string (e.g., a monologue) using dedent for cleaner output.
- Responding to the User: The agent sends a response via e.data.agent.monologue.
- Marking Action Completion: The await e.commit() call signals that the action is complete.

The `fetchPokemonMoves` Action

Learn about the basics of Actions here.

Let's break apart the fetchPokemonMoves Action. This action retrieves a Pokémon's move names:

Schema Validation: Ensures pokemonName is a valid string.
Fetch Logic: The fetchPokemonMovesNames helper gets the moves from the API.
User Feedback: The response lists the moves and delivers them back to the user.

<Action 
    name="fetchPokemonMoves"
    description="Retrieve a list of move names for a given Pokemon from the PokeAPI"
    schema={z.object({ pokemonName: z.string() })}
    examples={[{ pokemonName: 'pikachu' }]}
    handler={async (e: PendingActionEvent) => {
        const { pokemonName } = e.data.message.args as { pokemonName: string };
        const moves = await fetchPokemonMovesNames(pokemonName);
        const monologueString = dedent`
            Your character fetched details about a Pokémon's moves and discovered the following:
        ` + '\n\n' + moves;
        await e.data.agent.monologue(monologueString);
        await e.commit();
    }}
/>

You can define multiple <Action> components, each tailored to a specific purpose. For example:

fetchPokemonAbilities: Fetches abilities using fetchPokemonAbilities.
fetchPokemonMoves: Fetches moves using fetchPokemonMovesNames.

Finally, we call <PokeDexAssistant /> in our Agent's main code:

// ...

export default function MyAgent() {
  return (
    <Agent>
        {/* [!code ++] */} 
      <PokemonDexAssistant />
    </Agent>
  );
}

You can see the complete code for this component on GitHub here.

Step 4: Test out your Pokédex-like Agent!

You can run usdk chat to test it out. Learn more

You can ask it questions like:

What are Charizard's top abilities?

Share its response in our Discord community; we'd love to know what it responds to you.

Further Challenges

"Who's that Pokémon?" - Give the Agent an Image Perception, and create an Action so it can guess a Pokémon based on the image and provide its details

file: ./content/docs/examples/openapi-agent.mdx

undefined: OpenAPI Agent

Build an Agent which can work on an OpenAPI specification.

In this guide, we build an OpenAPI Agent which can parse an OpenAPI specification, and then take actions accordingly. OpenAPI Specification (formerly Swagger Specification) is an API description format for REST APIs. An OpenAPI file allows you to describe your entire API, including:

Available endpoints (/users) and operations on each endpoint (GET /users, POST /users)
Operation parameters Input and output for each operation
Authentication methods
Contact information, license, terms of use, and other information.

The format is easy to learn and readable to both humans and machines. It is often used by backend teams to explain their endpoints to client-side teams.

Why does this matter? This basically allows you to create an Agent which can help non-technical users interact with your API, kind of like an Assistant to your application's admin panel. As long as the OpenAPI spec is well-defined, your Agent can display data analytics, crunch numbers, and performs actions on your behalf.

The source code for this example is available on GitHub.

Video Tutorial

You can follow along this example by watching the video below:

Guide

Step 1: Setup `usdk`

Follow Setup the SDK to set up NodeJS and usdk.

Step 2: Create an Agent (shortcut)

We can skip the Interview process and directly generate an Agent with a prompt by running:

usdk create <your-agent-directory> -p "convert OpenAPI specs to Zod schemas and generate actions"

This will directly scaffold an Agent for you in <your-agent-directory>. Learn more

Step 3: Setup Zod and `openapi-zod-client`

We'll first need an OpenAPI specification. We'll use Swagger's official example, the Pet Store API, as an example.

The OpenAPI specification is often served as a YAML or JSON file. We get this from their GitHub (although, in a real scenario, it may be available in one of the server's routes).

We will use the openapi-zod-client package from NPM to convert this specification to a bunch of Zod schemas. A Zod schema is just a more descriptive way to specify a data type. It also generates a Zodios client, which is, simply put, an easy way to call the endpoints specified in the OpenAPI specification.

First install the relevant packages:

<Tabs items={[ "pnpm", "npm", ]}

pnpm i zod openapi-zod-client

npm i zod openapi-zod-client

Add a script in your package.json which calls the openapi-zod-client CLI tool (feel free to tweak the options by checking out what's available):

{
  "name": "my-agent",
  "scripts": {
    // [!code ++]
    "generate:api": "npx openapi-zod-client https://raw.githubusercontent.com/swagger-api/swagger-petstore/refs/heads/master/src/main/resources/openapi.yaml --additional-props-default-value false  -o ./api.ts"
  },
  "dependencies": {
    "upstreet-agent": "file:./packages/upstreet-agent"
  },
  "devDependencies": {
    "openapi-zod-client": "^1.18.2"
  }
}

Step 4: Create a Dummy API

Run the script we added in Step 3:

<Tabs items={[ "pnpm", "npm", ]}

pnpm generate-api

npm run generate-api

This runs openapi-zod-client, which, if successful, uses the OpenAPI specification to create the API in Zod and Zodios, in the api.ts file.

The api.ts file contains all the Zod schemas, and endpoints as an array, and exports a Zodios client.

Clean up the generated code by:

Removing partial and optional keywords.
Commenting out endpoints that are unsupported, such as file uploads.

Step 4: Import Dependencies

Before creating your OpenAPI agent, ensure you have imported the relevant dependencies. Update the generated files to include only the required ones and clean up unnecessary code. For example:

import React from "react";
import { Agent, Action, PendingActionEvent, Prompt } from "react-agents";
import dedent from "dedent";
import { z } from "zod";
import { createApiClient } from "./api"; // Ensure correct import path

const apiClient = createApiClient("https://petstore.swagger.io/v2"); // <-- Here, add the base URL to the server which hosted the OpenAPI specification

Step 5: Build the OpenAPI Action Generator

Define a component that maps OpenAPI endpoints to actions for your agent. Each action will have a schema, description, and handler. Here’s how to start:

const OpenAPIActionGenerator = () => {
  const endpoints = apiClient.api;

  const actions = endpoints.map((endpoint) => {
    const reducedEndpointParameters = endpoint.parameters?.reduce(
      (acc, param) => {
        if (param.type === "Query") {
          acc.query = acc.query.extend({
            [param.name]: param.schema || z.unknown(),
          });
        } else if (param.type === "Body") {
          acc.body = acc.body.extend({
            [param.name]: param.schema || z.unknown(),
          });
        }
        return acc;
      },
      {
        query: z.object({}),
        body: z.object({}),
      }
    ) || null;

    if (!reducedEndpointParameters) return null;

    const hasQuery = Object.keys(reducedEndpointParameters.query.shape || {}).length > 0;
    const hasBody = Object.keys(reducedEndpointParameters.body.shape || {}).length > 0;
    const schema = hasQuery && hasBody
      ? z.object({ query: reducedEndpointParameters.query, body: reducedEndpointParameters.body })
      : hasQuery
      ? z.object({ query: reducedEndpointParameters.query })
      : hasBody
      ? reducedEndpointParameters.body
      : z.null();

    if (!schema) return null;

    return (
      <Action
        key={endpoint.alias}
        name={endpoint.alias}
        schema={schema}
        examples={[]}
        description={endpoint.description || `Perform ${endpoint.method.toUpperCase()} request to ${endpoint.path}`}
        handler={async (e: PendingActionEvent) => {
          try {
            const args = e.data.message.args;
            const response = await apiClient[endpoint.alias]({
              ...(args?.query ? { queries: args.query } : args?.body || args),
            });
            const limitedResponse = Array.isArray(response) ? response.slice(0, 10) : response;
            const monologueString = dedent`\
              You took the action ${endpoint.alias}. You sent:
              ${JSON.stringify(args)}
              and received in response:
              ${JSON.stringify(limitedResponse, null, 2)}
            `;
            await e.data.agent.monologue(monologueString);
            await e.commit();
          } catch (error) {
            console.error(`Error in ${endpoint.alias}:`, error);
            throw error;
          }
        }}
      />
    );
  });

  return (
    <Agent>
      <Prompt>
        You strictly answer based on retrieved data. You strictly take only actions you can. If you can't take an action, just say it. If there's an error, log the raw error. You can take the following actions:
        {endpoints.map((endpoint) => endpoint.alias).join("\n")}
      </Prompt>
      {actions.filter(Boolean)}
    </Agent>
  );
};

export default OpenAPIActionGenerator;

… [truncated — open the raw llms.txt above for the full file]