Fireworks AI Docs

Docs

• Exporting Billing Metrics: Export billing and usage metrics for all Fireworks services
• Usage & Cost Breakdown: Break down usage and rated costs by deployment, model, API key, or custom tags — via firectl or the billingUsage API
• Service Accounts: How to manage and use service accounts in Fireworks
• Custom SSO: Set up custom Single Sign-On (SSO) authentication for Fireworks AI
• Managing users: Add, delete, and manage roles for users in your Fireworks account
• Create a Message: Anthropic-compatible endpoint.
• Cancel Reinforcement Fine-tuning Job
• Create API Key
• Create Batch Inference Job
• Create Dataset
• Load LoRA
• Create Deployment
• Create dpo job
• Create Evaluation Job
• Create Evaluator: Creates a custom evaluator for scoring model outputs. Evaluators use the Eval Protocol to define test cases, run model inference, and score responses. They are used with evaluation jobs and Reinforcement Fine-Tuning (RFT).
• Create Model
• Create Reinforcement Fine-tuning Job
• Create Reinforcement Fine-tuning Step
• Create Router
• Create secret
• Create Supervised Fine-tuning Job
• Create User
• Create embeddings
• Delete API Key
• Delete Batch Inference Job
• Delete Dataset
• Unload LoRA
• Delete Deployment
• Delete dpo job
• Delete Evaluation Job
• Delete Evaluator: Deletes an evaluator and its associated versions and build artifacts.
• Delete Model
• Delete Reinforcement Fine-tuning Job
• Delete Reinforcement Fine-tuning Step
• Delete Response: Deletes a model response by its ID. Once deleted, the response data will be gone immediately and permanently.
• Delete Router
• Delete secret
• Delete Supervised Fine-tuning Job
• Execute one training step for keep-alive Reinforcement Fine-tuning Step
• Generate an image with FLUX.1 [schnell] FP8
• Generate or edit an image with FLUX.1 Kontext
• Get Account
• Get Batch Inference Job
• Get Account Usage
• Get Dataset
• Get Dataset Download Endpoint
• Get Dataset Upload Endpoint
• Get LoRA
• Get Deployment
• Get Deployment Shape
• Get Deployment Shape Version
• Get dpo job
• Get dpo job metrics file endpoint
• Get Evaluation Job
• Get Evaluation Job execution logs (stream log endpoint + tracing IDs).
• Get Evaluator: Retrieves an evaluator by name. Use this to monitor build progress after creation (step 6 in the Create Evaluator workflow).
• Get Evaluator Build Log Endpoint: Returns a signed URL to download the evaluator's build logs. Useful for debugging BUILD_FAILED state.
• Get Evaluator Source Code Endpoint: Returns a signed URL to download the evaluator's source code archive. Useful for debugging or reviewing the uploaded code.
• Get Evaluator Upload Endpoint: Returns signed URLs for uploading evaluator source code (step 3 in the Create Evaluator workflow). After receiving the signed URL, upload your .tar.gz archive using HTTP PUT with Content-Type: application/octet-stream header.
• Get generated image from FLUX.1 Kontext
• Get Model
• Get Model Download Endpoint
• Get Model Upload Endpoint
• Get Quota: Gets a single quota by resource name.
• Get Reinforcement Fine-tuning Job
• Get Reinforcement Fine-tuning Step
• Get Response
• Get Router
• Get Secret: Retrieves a secret by name. Note that the value field is not returned in the response for security reasons. Only the name and key_name fields are included.
• Get Supervised Fine-tuning Job
• Get User
• Introduction
• List Accounts
• List API Keys
• List Batch Inference Jobs
• List Datasets
• List LoRAs
• List Deployment Shapes Versions
• List Deployments
• List dpo jobs
• List Evaluation Jobs
• List Evaluators: Lists all evaluators for an account with pagination support.
• List Models
• List Quotas: Lists all quotas for an account.
• List Reinforcement Fine-tuning Jobs
• List Reinforcement Fine-tuning Steps
• List Responses: Get a list of all responses for the authenticated account.
• List Routers
• List Secrets: Lists all secrets for an account. Note that the value field is not returned in the response for security reasons. Only the name and key_name fields are included for each secret.
• List Supervised Fine-tuning Jobs
• List Users
• Create Chat Completion: Create a completion for the provided prompt and parameters.
• Create Completion: Create a completion for the provided prompt and parameters.
• Create Response: Creates a model response, optionally interacting with custom tools via the Model Context Protocol (MCP). This endpoint supports conversational continuation and streaming.
• Prepare Model for different precisions
• Rerank documents: Rerank documents for a query using relevance scoring
• Resume Dpo Job
• Resume Reinforcement Fine-tuning Job
• Resume Rlor Trainer Job
• Resume Supervised Fine-tuning Job
• Scale Deployment to a specific number of replicas or to zero
• Undelete Deployment
• Update Dataset
• Update LoRA
• Update Deployment
• Update Evaluator: Updates evaluator metadata (display_name, description, default_dataset). Changing requirements or entry_point triggers a rebuild. To upload new source code, set prepare_code_upload: true then follow the upload flow.
• Update Model
• Update Quota: Updates a quota.
• Update Router
• Update secret
• Update User
• Upload Dataset Files: Provides a streamlined way to upload a dataset file in a single API request. This path can handle file sizes up to 150Mb. For larger file sizes use Get Dataset Upload Endpoint.
• Validate Dataset Upload
• Validate Evaluator Upload: Triggers server-side validation of the uploaded source code (step 5 in the Create Evaluator workflow). The server extracts and processes the archive, then builds the evaluator environment. Poll Get Evaluator to monitor progress.
• Validate Model Upload
• Autoscaling: Configure how your deployment scales based on traffic
• Performance benchmarking: Measure and optimize your deployment's performance with load testing
• Client-side performance optimization: Optimize your client code for maximum performance with dedicated deployments
• Exporting Metrics: Export metrics from your dedicated deployments to your observability stack
• Regions: Fireworks runs a global fleet of hardware on which you can deploy your models.
• Reserved capacity
• Routers: Distribute traffic across multiple deployments for A/B testing, traffic migration, and load distribution.
• Speculative Decoding: Speed up generation with draft models and n-gram speculation
• Cloud Integrations: Cloud Integrations
• Agent Frameworks: Build production-ready AI agents with Fireworks and leading open-source frameworks
• Microsoft Foundry: Deploy frontier open models inside your Azure subscription, billed through Azure.
• Claude Code: Use Claude Code with Fireworks AI models
• Development Setup with Fireworks Docs MCP: Configure the Fireworks AI Docs MCP server for Claude Code and Cursor
• GitHub Copilot: Use Fireworks AI models in GitHub Copilot Chat via a custom endpoint
• MLOps & Observability: Track and monitor your Fireworks AI deployments with leading MLOps and observability platforms
• Cookbooks: Interactive Jupyter notebooks demonstrating advanced use cases and best practices with Fireworks AI
• Courses: Standalone end-to-end examples showing how to use Fireworks to solve real-world use cases
• How do I close my Fireworks.ai account?
• I have multiple Fireworks accounts. When I try to login with Google on Fireworks' web UI, I'm getting signed into the wrong account. How do I fix this?
• What email does GitHub authentication use?
• What email does LinkedIn authentication use?
• What should I do if I can't access my company account after being invited when I already have a personal account?
• Are there discounts for bulk usage?
• Are there extra fees for serving fine-tuned models?
• How does billing and credit usage work?
• How many tokens per image?: Learn how to calculate token usage for images in vision models and understand pricing implications
• How much does Fireworks cost?
• Is prompt caching billed differently for serverless models?
• How do credits work?
• Why might my account be suspended even with remaining credits?
• Are there any quotas for serverless?
• Do you provide notice before removing model availability?
• Do you support Auto Scaling?
• How does autoscaling affect my costs?
• How does billing and scaling work for on-demand GPU deployments?
• How does billing work for on-demand deployments?
• How does the system scale?
• Are there SLAs for serverless?
• What are the rate limits for on-demand deployments?
• What factors affect the number of simultaneous requests that can be handled?
• What’s the supported throughput?
• Why am I experiencing request timeout errors and slow response times with serverless LLM models?
• Does Fireworks support custom base models?
• Does the API support batching and load balancing?
• FLUX image generation
• How do I control output image sizes when using SDXL ControlNet?
• How to check if a model is available on serverless?
• There’s a model I would like to use that isn’t available on Fireworks. Can I request it?
• What factors affect the number of simultaneous requests that can be handled?
• Fireworks Agent: Classification: Benchmark base models, fine-tune on labeled data, and pick the best classifier — automatically.
• Fireworks Agent: Preference Learning (DPO/ORPO): Run preference fine-tuning end-to-end with optional base-model sweep, automatic pair generation, and pairwise evaluation.
• Fireworks Agent: Evaluator Authoring: Have Fireworks Agent generate a reusable evaluator from your dataset — for scoring candidates in an SFT sweep, or for use with Managed RFT.
• Fireworks Agent Overview: Describe what you want, approve the plan and cost, get a deployed fine-tuned model.
• Fireworks Agent: Supervised Fine-Tuning: Run end-to-end SFT with Fireworks Agent — dataset inspection, hyperparameter sweep, evaluator-guided selection, and a deployed winner.
• Use Fireworks Agent with Claude Code, Cursor, Codex, and other coding agents: Install the Fireworks Agent skill file once and drive end-to-end fine-tuning from your coding agent.
• Training Overview: Launch RFT jobs using the eval-protocol CLI
• Remote Environment Setup: Implement the /init endpoint to run evaluations in your infrastructure
• Debug SFT tokenization: Download rendered token IDs and loss masks for supervised fine-tuning jobs.
• Deploying Fine Tuned Models: Deploy one or multiple LoRA models fine tuned on Fireworks using live merge or multi-LoRA
• Direct Preference Optimization
• Agent Tracing: Understand where your agent runs and how tracing enables reinforcement fine-tuning
• Evaluators: Understand the fundamentals of evaluators and reward functions in reinforcement fine-tuning
• Supervised Fine Tuning - Text
• Supervised Fine Tuning - Vision: Learn how to fine-tune vision-language models on Fireworks AI with image and text datasets
• Training Overview
• Basics: Understand the reinforcement learning fundamentals behind RFT
• Managed Fine-Tuning Overview: Fine-tune models with Fireworks-managed infrastructure — no custom code required.
• Monitor Training: Track RFT job progress and diagnose issues in real-time
• Price comparison vs Tinker: Estimate the cost of multi-turn agentic RL rollouts on Fireworks compared to Tinker's per-token pricing
• Parameter Tuning: Learn how training parameters affect model behavior and outcomes
• Single-Turn Training Quickstart: Train a model to be an expert at answering GSM8K math questions
• Remote Agent Quickstart: Train an SVG drawing agent running in a remote environment
• Overview: Train models using reinforcement learning in minutes
• Cost Estimator: Estimate and optimize the cost of your RFT training jobs
• RFT parameters reference: Checkpoint, resume, and GRPO metrics fields for reinforcement fine-tuning recipes.
• Ledger & Debugging for RL Rollouts: Inspect snapshot history, reset the ledger, and understand how in-flight requests behave during a weight swap.
• Incremental Snapshots (ARC2): Build ARC2 incremental checkpoints, use per-file hints, and signal delta hot-loads for BYOT RL rollout integrations.
• RL Rollouts with Your Own Trainer: Integrate an external RL trainer with Fireworks inference: hot-load new checkpoints from your bucket and run rollouts via the OpenAI-compatible API.
• Secure Training (BYOB): Fine-tune models while keeping sensitive data and components under your control
• Checkpoints and Resume: Save training progress, resume from failures, and promote checkpoints to deployable models — driven by the recipe.
• Cookbook: Distillation: Single-teacher OPD and routed multi-teacher policy distillation with cookbook recipes.
• Cookbook: DPO: Direct Preference Optimization with pairwise data using the cookbook recipe.
• The Cookbook: Ready-to-run training recipes for GRPO, DPO, SFT, and distillation built on top of the Training API.
• Cookbook Reference: Configuration classes, checkpoint utilities, and gradient accumulation normalization for cookbook recipes.
• Cookbook: Reinforcement Learning: Async RL on Fireworks — write a rollout function, the recipe owns the loop (gate, advantage, weight sync, KL/TIS, PPO, checkpoints). Runs async or fully synchronous.
• Cookbook: SFT: Supervised fine-tuning via the cookbook's sft_loop recipe.
• Weight sync: How a trainer's updated weights reach the serving deployment during RL training.
• Introduction: Fireworks Training API — custom training loops with full Python control over objectives, while Fireworks handles distributed GPU infrastructure.
• Loss Functions: Built-in loss functions and custom objectives via forward_backward_custom.
• Quickstart: Get a custom training loop running in minutes with the Fireworks Training API.
• Cleanup and Teardown: Delete trainer jobs and deployments after experiments to avoid leaked resources.
• DeploymentManager (Compatibility): Legacy SDK reference for direct deployment lifecycle and weight-sync management.
• DeploymentSampler: Client-side tokenized sampling from inference deployments for training and evaluation.
• FireworksClient: Account-level operations that don't require a running trainer job.
• FiretitanServiceClient & TrainingClient: Connect to a trainer endpoint and use the training client for forward/backward passes, optimizer steps, and checkpointing.
• TrainerJobManager (Compatibility): Legacy SDK reference for service-mode trainer job lifecycle management.
• WeightSyncer (Legacy): Backward-compatibility reference for the old standalone checkpoint-then-sync helper.
• Saving and Loading: SDK-level reference for checkpoint save, load, weight sync, and promotion.
• Training and Sampling: End-to-end SDK walkthrough: bootstrap resources, train, checkpoint, and sample through a serving deployment.
• Training Shapes: Pre-configured GPU and model training profiles that simplify distributed training setup.
• Vision Inputs: Fine-tune vision-language models (VLMs) with the Training API using multimodal chat data containing images and text.
• Training Prerequisites & Validation: Requirements, validation checks, and common issues when launching RFT jobs
• Using Secrets: Learn how to create secrets that can be utilized within your reward function.
• Warm Start from Fine-Tuned Models: Continue training from a previously fine-tuned model with RFT
• Training Guide: UI: Launch RFT jobs using the Fireworks dashboard
• Weighted Training: Control which samples have greater influence during RFT training
• Fire Pass Setup: Kimi K2.6 Turbo for personal agentic coding — Fire Pass (Early Access), $49 / month
• Concepts: This document outlines basic Fireworks AI concepts.
• Glossary: Definitions for key terms used across Fireworks AI documentation.
• Build with Fireworks AI: Fast inference and fine-tuning for open source models
• Deployments Quickstart: Deploy models on dedicated GPUs in minutes
• Serverless Quickstart: Make your first Serverless API call in minutes
• Batch API: Process large-scale async workloads
• Completions API: Use the completions API for raw text generation with custom prompt templates
• Tool Calling: Connect models to external tools and APIs
• Inference Error Codes: Common error codes, their meanings, and resolutions for inference requests
• Deployments: Configure and manage on-demand deployments on dedicated GPUs
• Using predicted outputs: Use Predicted Outputs to boost output generation speeds for editing / rewriting use cases
• Prompt caching
• Embeddings & Reranking: Generate embeddings and rerank results for semantic search
• Text Models: Query, track and manage inference for text models
• Vision Models: Query vision-language models to analyze images and visual content
• Account quotas: Account-wide request limits, spending tiers, budget controls, and on-demand GPU quotas
• Reasoning: How to use reasoning with Fireworks models
• Which model should I use?: Find the best open models for your use case or migrate from closed source models like Claude, GPT, and Gemini
• Reliability and Error Handling: Recommended patterns for timeouts, retries, and error handling when building production applications on the Fireworks API.
• Responses API
• Inference for RL Rollouts: Session affinity, weight-swap behavior, and MoE Router Replay for rollout traffic on Fireworks inference deployments.
• Audit & Access Logs: Monitor and track account activities with audit logging for Enterprise accounts
• Zero Data Retention: Data retention policies at Fireworks
• Data Security: How we secure and handle your data for inference and training
• Understanding LoRA performance: Understand the performance impact of LoRA fine-tuning, optimization strategies, and deployment considerations.
• Video & Audio Inputs: Query multimodal models to process video and audio content directly
• Kimi K2 family: Using Kimi K2 family models in agentic and tool-calling workflows on Fireworks.
• Quantization: Reduce model precision to improve performance and lower costs
• Custom Models: Upload, verify, and deploy your own models from Hugging Face or elsewhere
• Upload via REST API: Programmatically upload custom models using the Fireworks REST API
• Serverless Overview: How Serverless inference works on Fireworks: serving paths, billing, request/response headers, prompt caching, model lifecycle, and when to choose Serverless over On-demand
• Serverless Pricing: Per-token serverless pricing for text, vision, and embedding models, including Priority and Fast serving paths
• Serverless Rate Limits: Adaptive rate limits grow and shrink with your usage
• Serverless Serving Paths: Standard, Priority, and Fast serving paths on Fireworks Serverless
• Structured Outputs: Enforce output formats using JSON schemas or custom grammars
• Anthropic compatibility: Use Anthropic SDKs with Fireworks, and understand the supported surface for the Anthropic-compatible Messages API.
• firectl account get: Prints information about an account.
• firectl account list: Prints all accounts the current signed-in user has access to.
• firectl api-key create: Creates an API key for the signed in user or a specified service account user.
• firectl api-key delete: Deletes an API key.
• firectl api-key get: Prints information about an API key.
• firectl api-key list: Prints all API keys for the signed in user.
• firectl audit-logs list: Lists audit logs for the signed in user.
• Authentication: Authentication for access to your account
• firectl batch-inference-job create: Creates a batch inference job.
• firectl batch-inference-job delete: Deletes a batch inference job.
• firectl batch-inference-job get: Retrieves information about a batch inference job.
• firectl batch-inference-job list: Lists all batch inference jobs in an account.
• firectl billing export-metrics: Exports billing metrics
• firectl billing get-usage: Prints account usage and rated costs for a time range.
• firectl billing list-invoices: Prints information about invoices.
• firectl billing notification-settings: Manage notification settings.
• firectl billing notification-settings get: Get notification settings for an account.
• firectl billing notification-settings update: Update notification settings for an account.
• firectl credit-redemption list: Lists credit code redemptions for the current account.
• firectl credit-redemption redeem: Redeems a credit code.
• firectl dataset create: Creates and uploads a dataset.
• firectl dataset delete: Deletes a dataset.
• firectl dataset download: Downloads a dataset to a local directory.
• firectl dataset get: Prints information about a dataset.
• firectl dataset list: Prints all datasets in an account.
• firectl dataset update: Updates a dataset.
• firectl deployed-model get: Prints information about a deployed model.
• firectl deployed-model list: Prints all deployed models in the account.
• firectl deployed-model update: Update a deployed model.
• firectl deployment create: Creates a new deployment.
• firectl deployment delete: Deletes a deployment.
• firectl deployment get: Prints information about a deployment.
• firectl deployment list: Prints all deployments in the account.
• firectl deployment scale: Scales a deployment to a specified number of replicas.
• firectl deployment-shape-version get: Prints information about a deployment shape version.
• firectl deployment-shape-version list: Prints all deployment shape versions of this deployment shape.
• firectl deployment undelete: Undeletes a deployment.
• firectl deployment update: Update a deployment.
• firectl dpo-job cancel: Cancels a running dpo job.
• firectl dpo-job create: Creates a dpo job.
• firectl dpo-job delete: Deletes a dpo job.
• firectl dpo-job export-metrics: Exports metrics for a dpo job.
• firectl dpo-job get: Retrieves information about a dpo job.
• firectl dpo-job list: Lists all dpo jobs in an account.
• firectl dpo-job resume: Resumes a dpo job.
• firectl evaluator-revision alias: Alias an evaluator revision
• firectl evaluator-revision delete: Delete an evaluator revision
• firectl evaluator-revision get: Get an evaluator revision
• firectl evaluator-revision list: List evaluator revisions
• firectl identity-provider create: Creates a new identity provider.
• firectl identity-provider get: Prints information about an identity provider.
• firectl identity-provider list: List identity providers for an account
• firectl model create: Creates and uploads a model.
• firectl model delete: Deletes a model.
• firectl model download: Download a model.
• firectl model get: Prints information about a model.
• firectl model list: Prints all models in an account.
• firectl model load-lora: Loads a LoRA model.
• firectl model prepare: Prepare models for different precisions
• firectl model unload-lora: Unloads a LoRA model.
• firectl model update: Updates a model.
• firectl model upload: Resumes or completes a model upload.
• firectl quota get: Prints information about a quota.
• firectl quota list: Prints all quotas.
• firectl quota update: Updates a quota.
• firectl reinforcement-fine-tuning-job cancel: Cancels a running reinforcement fine-tuning job.
• firectl reinforcement-fine-tuning-job create: Creates a reinforcement fine-tuning job.
• firectl reinforcement-fine-tuning-job delete: Deletes a reinforcement fine-tuning job.
• firectl reinforcement-fine-tuning-job get: Retrieves information about a reinforcement fine-tuning job.
• firectl reinforcement-fine-tuning-job list: Lists all reinforcement fine-tuning jobs in an account.
• firectl reinforcement-fine-tuning-job resume: Resumes a failed reinforcement fine-tuning job.
• firectl reinforcement-fine-tuning-job update: Update fields on a reinforcement fine-tuning job.
• firectl reservation get: Prints information about a reservation.
• firectl reservation list: Prints active reservations.
• firectl rlor-trainer-job cancel: Cancels a running rlor trainer job.
• firectl rlor-trainer-job create: Creates a rlor trainer job.
• firectl rlor-trainer-job delete: Deletes a rlor trainer job.
• firectl rlor-trainer-job get: Retrieves information about a rlor trainer job.
• firectl rlor-trainer-job list: Lists all rlor trainer jobs in an account.
• firectl router create: Creates a router.
• firectl router delete: Deletes a router.
• firectl router get: Prints information about a router.
• firectl router list: Prints all routers in the account.
• firectl router update: Update a router.
• firectl secret create: Creates a secret for the signed in user.
• firectl secret delete: Deletes a secret.
• firectl secret get: Retrieves a secret by name.
• firectl secret list: Lists all secrets for the signed in user.
• firectl secret update: Updates an existing secret.
• firectl set-api-key: Sets the default API key in ~/.fireworks/auth.ini.
• firectl supervised-fine-tuning-job cancel: Cancels a running supervised fine-tuning job.
• firectl supervised-fine-tuning-job create: Creates a supervised fine-tuning job.
• firectl supervised-fine-tuning-job delete: Deletes a supervised fine-tuning job.
• firectl supervised-fine-tuning-job get: Retrieves information about a supervised fine-tuning job.
• firectl supervised-fine-tuning-job list: Lists all supervised fine-tuning jobs in an account.
• firectl training-shape clone: Clones an existing training shape to a new shape.
• firectl training-shape create: Creates a new training shape.
• firectl training-shape delete: Deletes a training shape and all its versions.
• firectl training-shape get: Prints information about a training shape.
• firectl training-shape list: Lists training shapes in the account.
• firectl training-shape update: Updates a training shape (mutable fields only). Creates a new version automatically.
• firectl training-shape-version get: Prints information about a training shape version.
• firectl training-shape-version list: Lists training shape versions.
• firectl training-shape-version update: Update a training shape version.
• firectl upgrade: Upgrades the firectl binary to the latest version.
• firectl user create: Creates a new user.
• firectl user delete: Deletes a user.
• firectl user get: Prints information about a user.
• firectl user list: Prints all users in the account.
• firectl user update: Updates a user.
• firectl version: Prints the version of firectl
• firectl whoami: Shows the currently authenticated user
• Getting started: Learn to create, deploy, and manage resources using Firectl
• OpenAI compatibility
• Python SDK
• Changelog

OpenAPI Specs

• merged.openapi
• gateway.openapi
• text-completion.openapi
• responses.openapi
• openapi
• gateway-extra.openapi
• anthropic-messages.openapi

Optional

• Demos