# Metrics

## BLUE

 Used to compare generated translations,

Compared n-grams generated translation to the n-grams of ground truth reference translation.  n-grams essentially means chunks of n words.  

Average of precision of n-grams over different values of n

{% embed url="<https://youtu.be/M05L1DhFqcw>" %}

## ROUGE

Used for text summarization&#x20;

Compared n-grams of generated summary to the n-grams of ground truth reference summary.   &#x20;

calculate recall and precision of n-grams then calculate the F1 score.

Can do this for difference value of n.&#x20;

## Perplexity

This is piece about relation between - perplexity and bits-per-character (from information in sentence perspective), this helps understand how perplexity/cross entropy relates to information-theoretic motivation.

Metric to evaluate language models.

Perplexity is an exponent of cross entropy loss of generated output.&#x20;

<figure><img src="https://1877261540-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LFDuA0A2VRqmT31Blrq%2Fuploads%2FBUk1TwPCIIdLY6OQbpJ9%2Fimage.png?alt=media&#x26;token=bf5def1a-5219-430c-9ab9-0b6ba6fdd75b" alt=""><figcaption></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://theshank.gitbook.io/ai/transformers/metrics.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.