Metric: Add Answer Conciseness #4
Description
Is your feature request related to a problem? Please describe.
The current evaluation suite can detect if an answer is incomplete, but it cannot detect if an answer is overly verbose or "rambles." For chatbot applications, concise answers are critical.
Describe the solution you'd like
Add a new AI-powered metric to the RAGEvaluator called score_conciseness. This metric would evaluate if the generated answer is more verbose than necessary to address the user's question.
Describe alternatives you've considered
A simple character count could be used, but an LLM-as-a-Judge approach would provide a more nuanced, semantic evaluation of conciseness.
Additional context
This adds another layer of sophistication to the evaluation suite, making it even more valuable for production use cases.