Setting up Ollama Guardrails

Ollama Guardrails

Use Portkey to enforce structured inputs, safe outputs, and usage policies across all Ollama requests. Configure verdict actions like async checks, request denial, and feedback logging, according to your enforcement needs.

Ollama Guardrails

Ollama makes it easy to run powerful language models like LLaMA 3, Mistral, and Gemma locally—but running these models safely in production still requires robust guardrails. With Portkey, you can apply input/output validation, safety checks, and usage controls to every Ollama request—without modifying your application code.

Ollama is widely used for privacy-sensitive workloads, rapid prototyping, and edge deployments. With full support for Ollama’s local APIs, Portkey helps you enforce guardrails, monitor model behavior, and route requests—bringing enterprise-grade governance to locally hosted LLMs.

With Portkey, you can:

• Protect your AI stack from security threats with built-in guardrails
• Route requests with precision and zero latency based on guardrail checks
• View guardrails verdicts, latency, and pass/fail status for every check in real time.
• Enforce org-wide AI safety policies across all your teams, workspaces and models.
• Integrate existing guardrail infrastructure through simple webhook calls
• Secure vector embedding requests

Portkey supports all Ollama models out of the box and can be deployed in hours, not weeks, making it the easiest way to bring enterprise-grade control to your AI stack.

World-Class Guardrail Partners

Integrate top guardrail platforms with Portkey to run your custom policies seamlessly — from content filtering and PII detection to moderation and compliance. Ensure every AI request is safe, auditable, and aligned with your enterprise standards.

Guardrail checks you can apply to Ollama

Portkey offers both deterministic and LLM-powered guardrails that work seamlessly with Ollama's APIs. You can apply these checks to inputs, outputs, or both.

Input guardrails

• Regex Match: Enforce patterns on input prompts
• Sentence / Word / Character Count: Control verbosity
• Lowercase Detection: Control verbosity
• Ends With: Validate specific prompt endings
• Webhook: Enforce custom business logic
• JWT Token Validator: Verify token authenticity
• Model Whitelist: Allow only approved models per route
• Moderate Content: Block unsafe or harmful prompts
• Check Language: Enforce language constraints
• Detect PII: Prevent sensitive info in prompts
• Detect Gibberish: Block incoherent or low-quality input

Output guardrails

• Regex / Sentence / Word / Character Count: Ensure required words or phrases
• JSON Schema / JSON Keys: Ensure required words or phrases
• Valid URLs: Validate links in responses
• Contains Code: Detect code in specific formats
• Webhook: Post-process or validate output
• Detect PII / Detect Gibberish: Ensure required words or phrases

How to add guardrails to Ollama with Portkey

Putting Portkey Guardrails in production is just a 4-step process:

1. Create Guardrail Checks
2. Create Guardrail Actions
3. Enable Guardrail through Configs
4. Attach the Config to a Request

Guardrail action settings

• Async (TRUE): Run guardrails in parallel to the request.
• Deny Request (TRUE): Block the request or response if any guardrail fails.
• Send Feedback on Success/Failure: Attach metadata based on guardrail results.

Frequently Asked Questions

• Do guardrails add latency to requests?
• Can I block a request if a guardrail fails?
• What happens if I don’t want to block, just observe?
• Are OpenAI Guardrails only for inputs?
• What’s the difference between basic and pro guardrails?

Safeguard your Ollama requests now.

Whether you're running production workloads, or scaling across teams, Portkey’s guardrails give you control without compromise.