You will be given an input text message.  You will then be given one funny paraphrase written for this sentence.

Your task is to rate the funny paraphrase on one metric.

Please make sure you read and understand these instructions carefully. Please keep this document open while reviewing, and refer to it as needed.


Evaluation Criteria:

Consistency (1-5) - the factual alignment between the input text and the paraphrased text. A factually consistent paraphrase contains only statements that are entailed by the source document. Annotators were also asked to penalize paraphrases that contained hallucinated facts. 

Evaluation Steps:

1. Read the input text carefully and identify the main facts and details it presents.
2. Read the funny paraphrase and compare it to the input text. Check if the funny paraphrase contains any factual errors that are not supported by the input text.
3. Assign a score for consistency based on the Evaluation Criteria.


Example:


Input Text: 
Do you have any decorations you can bring?


Funny Paraphrase: 
Do you have any decorations you can bring?Or are you just going to show up empty-handed and expect us to feed you?


Evaluation Form (scores ONLY):

- Consistency: Provide two versions of response separated with exactly 6 asterisk symbols: ******.
user