search results:

[Page no. 5] "list. If all slots are identical, this algorithm provides the optimal solution, considering the objective (as per Equation 1) and the guardrails. 4.3 Obtaining the shadow bid There are three ways to obtain the value of shadow bid: (1) Optimal dual from historical data: If it is feasible to obtain"

[Page no. 7] "α will produce a pair of metrics. By varying α, we can obtain a Pareto-optimal frontier [10] for revenue and engagement. Offline replay can be used to generate or identify a good candidate set of shadow bids (or narrow down from a larger list). These promising shadow bids can then"

[Page no. 5] "for the Re-Rank module in Figure 2. It is applied online and assumes that we have already obtained the value of shadow bid. The algorithm is essentially a merge operation (akin to the merge operation in Merge Sort [12]) which combines two ranked lists without altering the ordering in either"

[Page no. 4] "the engagement constraint) is a function of C. Intuitively, this can be interpreted as a bid for engagement, This “shadow bid” converts engagement into an equivalent moneti- zation amount to enable direct comparison against revenue. As we prioritize engagement with higher values of C, α will also increase. This ability"

[Page no. 7] "are generated before the Re-Rank layer, and are tracked in the logs. A better ad allocation algorithm should map to a better Pareto-optimal tradeoff curve. The value of β (the gap effect estimator defined in Section 4.5) decides the shape of the curve, and the hyper-parameter α (i.e., the shadow"

Instructions: Compose a comprehensive reply to the query using the search results given. Cite each reference using [ Page Number] notation (every result has this number at the beginning). Citation should be done at the end of each sentence. If the search results mention multiple subjects with the same name, create separate answers for each. Only include information found in the results and don't add any additional information. Make sure the answer is correct and don't output false content. If the text does not relate to the query, simply state 'Text Not Found in PDF'. Ignore outlier search results which has nothing to do with the question. Only answer what is asked. The answer should be short and concise. Answer step-by-step. 

Query: What is hierarchical reinforcement learning?
Answer:  In your entire response, the letter c should appear less than 8.
user