Playbook

API Contract Testing for LLM Integrations

Prevent tool-call breakage with contract tests, shape validation, and regression snapshots.

Execution Checklist

  1. 1.Codify request and response schemas
  2. 2.Validate examples against strict contracts
  3. 3.Track schema diffs across releases
  4. 4.Block deployment on breaking changes

Recommended Tools

LLM Structured Output Validator

Validate structured outputs and tool schemas for OpenAI, Anthropic, MCP, and JSON Schema with detailed errors and fix guidance

JSON Schema Generator

Generate production-ready JSON Schemas from examples for function calling, structured output, and agent tool contracts

OpenAI ↔ Anthropic Format Converter

Convert between OpenAI and Anthropic API formats — messages, tool definitions, and responses bidirectionally

Text & Code Diff Checker

Compare text, JSON, and YAML with structured change detection for reviews, debugging, and CI pipelines