Delulu si Mare: LLM Hallucinations and ROUGE
A Valentine's Day look at why large language models hallucinate and how ROUGE, BLEU, and other metrics measure it.
Article
Ask a large language model to write a love letter from a list of shared memories and it may add details that were never there. In evaluation these fabricated outputs are called hallucinations.
Metrics such as ROUGE, BLEU, BERTScore, and perplexity compare generated text against a reference. A worked ROUGE example scores a paraphrased anniversary note at 82% recall and 82% precision, an F1 of 0.82. Fine-tuning and prompt engineering reduce hallucinations further.
Visual story
