OpenPipe

Docs

• Delete Dataset: Delete a dataset.
• Delete Model: Delete an existing model.
• Get Model: Get a model by ID.
• List Datasets: List datasets for a project.
• List Models: List all models for a project.
• Chat Completions: OpenAI-compatible route for generating inference and optionally logging the request.
• Create Dataset: Create a new dataset.
• Add Entries to Dataset: Add new dataset entries.
• Create Model: Train a new model.
• Judge Criteria: Get a judgement of a completion against the specified criterion
• Report: Record request logs from OpenAI models
• Report Anthropic: Record request logs from Anthropic models
• Update Metadata: Update tags metadata for logged calls matching the provided filters.
• Base Models: Train and compare across a range of the most powerful base models.
• Caching: Improve performance and reduce costs by caching previously generated responses.
• Anthropic Proxy
• Proxying to External Models
• Gemini Proxy
• Chat Completions
• Criterion Alignment Sets: Use alignment sets to test and improve your criteria.
• API Endpoints: Use the Criteria API for runtime evaluation and offline testing.
• Criteria: Align LLM judgements with human ratings to evaluate and improve your models.
• Criteria Quick Start: Create and align your first criterion.
• Exporting Data: Export your past requests as a JSONL file in their raw form.
• Importing Request Logs: Search and filter your past LLM requests to inspect your responses and build a training dataset.
• Datasets: Collect, evaluate, and refine your training data.
• Datasets Quick Start: Create your first dataset and import training data.
• Relabeling Data: Use powerful models to generate new outputs for your data before training.
• Uploading Data: Upload external data to kickstart your fine-tuning process. Use the OpenAI chat fine-tuning format.
• Deployment Types: Learn about serverless, hourly, and dedicated deployments.
• Direct Preference Optimization (DPO)
• DPO Quick Start: Train your first DPO fine-tuned model with OpenPipe.
• Code Evaluations: Write custom code to evaluate your LLM outputs.
• Criterion Evaluations: Evaluate your LLM outputs using criteria.
• Head-to-Head Evaluations: Evaluate your LLM outputs against one another using head-to-head evaluations.
• Evaluations: Evaluate the quality of your LLMs against one another or independently.
• Evaluations Quick Start: Create your first head to head evaluation.
• External Models
• Fallback options: Safeguard your application against potential failures, timeouts, or instabilities that may occur when using experimental or newly released models.
• Fine Tuning via API: Fine tune your models programmatically through our API.
• Fine-Tuning Quick Start: Train your first fine-tuned model with OpenPipe.
• Reward Models (Beta): Train reward models to judge the quality of LLM responses based on preference data.
• Fine Tuning via Webapp: Fine tune your models on filtered logs or uploaded datasets. Filter by prompt id and exclude requests with an undesirable output.
• Pruning Rules: Decrease input token counts by pruning out chunks of static text.
• Exporting Logs: Export your past requests as a JSONL file in their raw form.
• Logging Requests: Record production data to train and improve your models' performance.
• Logging Anthropic Requests
• Updating Metadata Tags
• Installing the SDK
• Quick Start: Get started with OpenPipe in a few quick steps.
• OpenPipe Documentation: Software engineers and data scientists use OpenPipe's intuitive fine-tuning and monitoring services to decrease the cost and latency of their LLM operations. You can use OpenPipe to collect and analyze LLM logs, create fine-tuned models, and compare output from multiple models given the same input.
• Overview: OpenPipe is a streamlined platform designed to help product-focused teams train specialized LLM models as replacements for slow and expensive prompts.
• Pricing Overview

OpenAPI Specs

• openapi