Skip to content

Commit 7f79435

Browse files
authored
updated context_scoring_prompt and provide extra examples. (#725)
update context_scoring_prompt to better prompt and provide two extra examples.
1 parent 3fc1950 commit 7f79435

File tree

1 file changed

+15
-4
lines changed

1 file changed

+15
-4
lines changed

src/ragas/testset/prompts.py

+15-4
Original file line numberDiff line numberDiff line change
@@ -132,14 +132,25 @@
132132

133133
context_scoring_prompt = Prompt(
134134
name="score_context",
135-
instruction="""Given a context, complete the two following tasks and output answer valid json format
136-
1.Evaluate the provided context and assign a numerical score between 0 and 10 based on the following criteria:
137-
- Award a high score to context that thoroughly delves into and explains concepts.
138-
- Assign a lower score to context that contains excessive references, acknowledgments, personal information, or other non-essential elements.""",
135+
instruction="""Given a context, perform the following task and output the answer in VALID JSON format:
136+
Evaluate the provided context and assign a numerical score between 0 and 10 based on the following criteria:
137+
- Clarity and Accuracy: Award a high score to the context that provides accurate and non-misleading information with clarity.
138+
- Depth and Innovation: Give extra points to the context that thoroughly delves into the concepts or includes new perspectives and unique insights.
139+
- Structure: Award a high score to the context with clear structure and logical flow.
140+
- Relevance: Assign a lower score to the context that contains excessive references, acknowledgments, personal information, or other non-essential elements.
141+
""",
139142
examples=[
143+
{
144+
"context": "The Pythagorean theorem is a fundamental principle in geometry. It states that in a right-angled triangle, the square of the length of the hypotenuse (the side opposite the right angle) is equal to the sum of the squares of the lengths of the other two sides. This can be written as a^2 + b^2 = c^2 where c represents the length of the hypotenuse, and a and b represent the lengths of the other two sides.",
145+
"output": {"score": 8.5},
146+
},
140147
{
141148
"context": "Albert Einstein (14 March 1879 - 18 April 1955) was a German-born theoretical physicist who is widely held to be one of the greatest and most influential scientists of all time.",
142149
"output": {"score": 6.0},
150+
},
151+
{
152+
"context": "I love chocolate. It's really tasty. Oh, and by the way, the earth orbits the sun, not the other way around. Also, my favorite color is blue.",
153+
"output": {"score": 2.0},
143154
}
144155
],
145156
input_keys=["context"],

0 commit comments

Comments
 (0)