The score assigned to each oracle response is computed using validation scores comprised between the mean and a certain number of standard deviations. By default, this range is set to 2 standard deviations in LLMOracleManager::validationDeviationFactor. However, this setting is ignored in LLMOracleCoordinator::finalizeValidation, resulting in scores generated using only values within 1 standard deviation, regardless of the validationDeviationFactor setting.
Because scores are calculated using a smaller-than-intended validation range, fewer validation scores contribute to the final assigned response scores. This reduced range may lead to selecting a suboptimal oracle response, as the broader validation score distribution intended by validationDeviationFactor is not fully utilized.
Manual review.
Incorporate missing factor in LLMOracleCoordinator::finalizeValidation:
The contest is live. Earn rewards by submitting a finding.
This is your time to appeal against judgements on your submissions.
Appeals are being carefully reviewed by our judges.