Judge Evaluation

The automated evaluation framework of Datumo Eval. It utilizes a Judge evaluation model to compare and evaluate the responses of a Target model, and can quantify model performance based on a Dataset.

---

Evaluation Task

Evaluate AI model performance using evaluation metrics and check results.

RAGAs Task

Automatically evaluate generated responses and retrieved contexts with RAGAs metrics.

RAG Checker

Evaluate RAG system responses based on Expected Response.

Auto Red-teaming

Automatically generate attack prompts to evaluate AI model safety.