RAGAs Task

RAGAs Task

Datumo provides a RAGAs-based evaluation feature that allows you to automatically evaluate and quantify the generated responses and retrieved contexts with RAGAs metrics.

The overall flow is similar to a general Evaluation Task, but detailed manual editing functions are limited.

1. Create a RAGAs Task
2. Create and Run an Eval Set
3. Check Evaluation Results
4. (Advanced) Check results in the Beir Leaderboard view

📄️ 1. Create a RAGAs Task

Create a new RAGAs Task.

📄️ 2. Run Eval Set

Create a RAGAs Eval Set and start the evaluation.

📄️ 3. Check Results

Check the evaluation results with the Dashboard and Table View.

📄️ + Beir Leaderboard

Perform BEIR benchmark evaluation along with Judge evaluation and check the results on the leaderboard.