Welcome to the Validaitor LLM Docs

Work in Progress

This documentation is still a work in progress and may be incomplete or contain errors.

Introduction

Validaitor is a tool that allows you to create and run tests for your large language model based application.

Using our app you can automatically test your LLM applications in multiple categories such as:

• Fairness
• Robustness
• Privacy
• Security
• Truthfulness

and more.

Additionally we provide tooling that allows you to demonstrate your compliance with regulations and standards such as the EU AI Act or ISO 42001 and collaboration features to work together with your team or even external auditors to ensure the quality of your AI application.

Overview

The Validaitor app is divided into multiple sections that allow you to configure and run tests for your LLM application in a structured way.

The main sections for testing are:

• AI Systems
• LLM Apis
• AI Models
• Data Assets
• Tests
• Test Suites
• Prompt Collections
• Metrics