Skip to main content

What is DATUMO Eval?


DATUMO Eval is Korea’s first all-in-one AI evaluation automation platform.
It streamlines the entire evaluation workflow—from prompt generation and model response analysis to metric scoring and result visualization—helping teams validate model performance and build reliable, trustworthy AI systems.


  • A professional-grade platform for evaluating the quality, safety, and robustness of AI models
  • Full-featured tooling for managing evaluation criteria and analyzing results
  • Support for quantitative scoring and risk validation (e.g., Red Teaming)

Key Features​

Evaluation Criteria Management​

Get started with proven, up-to-date evaluation frameworks to assess LLM performance across reliability, safety, and usability.

Prompt & Dataset Generation​

Automatically generate domain-specific evaluation prompts that align with your product or use case.

Automated Evaluation​

Automatically score LLM outputs based on your selected evaluation metrics and rubrics.

Evaluation Dashboard​

Visualize strengths and weaknesses across models, datasets, and criteria with detailed statistical breakdowns and filters.

Red Teaming (Add-on)​

Simulate adversarial conditions using curated red teaming strategies. Supports both human-led and automated testing.



Core Components​

DATUMO Eval is composed of three core modules that cover the entire evaluation lifecycle:

Prompt & Dataset Generation​

Generate evaluation-ready questions based on documents or product specifications.

  • Chunk-based prompt generation
  • Domain-specific dataset building

Automated Evaluation​

Automate the judgment of LLM outputs.

  • Multiple evaluation methods supported (e.g., Likert scale, weighted scoring, boolean ops)
  • Comparison against expected answers using text decomposition
  • Built-in safety metrics (bias, toxicity, legality, etc.)

Evaluation Dashboard​

Visually analyze model performance across different criteria and dimensions.

  • Score breakdowns by model and metric
  • Response-level justifications and tagging
  • Interactive filtering and outlier analysis

Red Teaming (Add-on)​

Challenge your LLM with adversarial inputs.

  • Human red-teaming framework
  • Auto-red teaming engine


Evaluation Workflow​

DATUMO Eval’s evaluation flow looks like this:

A[Upload evaluation dataset] --> B[Configure model & collect responses]
B --> C[Run automated evaluation]
C --> D[View & analyze results in dashboard]

🚀 Quick Start​

Start Evaluating

Once you're set up, you can begin your first evaluation here:

RAG Evaluation [Pro]

Quickly evaluate your RAG systems with this tool:

*This feature is available on Pro plans only.

Red Teaming [Add-On]

Evaluate the vulnerability of your AI system through red teaming.

*This feature is available as a paid add-on.