FAQ: show how to apply the query instruction, and what omitting it costs

#4

The FAQ says omitting the query instruction causes "a performance degradation" but doesn't say how to apply it or how large the effect is. This adds the explicit call and a measured figure.

On 2AIRTC (12,587 native Amharic documents, 240 native topics, 7,084 human relevance judgments), harrier-oss-v1-270m @ 31de22b scores nDCG@10 0.7159 via model.encode(queries) and 0.7850 via prompt_name="web_search_query" — +0.0691, paired t-test over 240 topics, p = 4.1e-14, bootstrap 95% CI [+0.0526, +0.0860]. Documents truncated to 2,000 characters with max_seq_length=2493 so nothing is truncated; document embeddings byte-identical across conditions. On BEIR SciFact (English) the same measurement gives +0.0089.

The note about default_prompt_name is there because it's the natural thing to reach for and it doesn't work — it applies to every encode() call, so it would instrument documents too. Related: #1, which addresses the encode_query path.

One collection, one language; the 27b variant wasn't tested.

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment