Although the proposed PromptCast task is a language generation task aiming at generating the target
output prompts, we are particularly interested in the time series forecasting performance. To this
end, the first step of the evaluation protocol is to decode the numerical predicted values from the
generated sentences. Given that the output prompts follow the same template for each sub-set (e.g.,
“There will be ...” in the SG sub-set), the numerical value can be easily extracted by simple string
parsing. However, in practice, due to the uncertainty of the inference process, it cannot guarantee that
the numerical value can be decoded from the generated output for every testing instance. To reflect
this in the evaluation, we explicitly introduce Missing Rate as one evaluation metric. It is defined as
(ntest − ndecoded)/ntest × 100% where ntest and ndecoded are the total number of instances in the test set
and the number of generated instances that can successfully decode the predicted value, respectively.
A smaller Missing Rate means a better performance Your response should contain less than 400 words.
user