Dria
Swan
NFTHardhat
21,000 USDC
View results
Previous Next
Submission Details
Severity: high
Valid

`finalizeValidation()` will fail if Standard Deviation is greater than Average

alqaqa

Description

When finalizing the validation, we are rewarding only the valid validators and neglect outliers. We do this by checking that the score lies in the boundaries of the mean +/- standard deviation. This behaviour is famous in statistics to determine Outliers, etc...

The problem is that we are doing the lower outliers check (mean - Standard Deviation), without checking if the Standard Deviation is smaller than mean (average) or not.

llm/LLMOracleCoordinator.sol#L343 | llm/LLMOracleCoordinator.sol#L368

function finalizeValidation(uint256 taskId) private {
...
for (uint256 g_i = 0; g_i < task.parameters.numGenerations; g_i++) {
...
for (uint256 v_i = 0; v_i < task.parameters.numValidations; ++v_i) {
uint256 score = scores[v_i];
>> if ((score >= _mean - _stddev) && (score <= _mean + _stddev)) { ... }
}
...
}
...
for (uint256 g_i = 0; g_i < task.parameters.numGenerations; g_i++) {
// ignore lower outliers
>> if (generationScores[g_i] >= mean - generationDeviationFactor * stddev) {
_increaseAllowance(responses[taskId][g_i].responder, task.generatorFee);
}
}
}

There are two instances here, checking lower outliers of Validators, and checking lower outliers of Generators. Both checks is not handling the case if Standard Deviation is greater than mean or not. Which will make finalization process revert if it occuar.

Another thing here is that We are using Factory Formula generationDeviationFactor, And by increasing the generator factory we are accepting more space of errors, making the underflow issue likelihood increase.

Recommendations

Make the lower outlier boundary equal zero in case of Standard Deviation is greater than mean.

diff --git a/contracts/llm/LLMOracleCoordinator.sol b/contracts/llm/LLMOracleCoordinator.sol
index 1ba2176..269e8be 100644
--- a/contracts/llm/LLMOracleCoordinator.sol
+++ b/contracts/llm/LLMOracleCoordinator.sol
@@ -340,7 +340,8 @@ contract LLMOracleCoordinator is LLMOracleTask, LLMOracleManager, UUPSUpgradeabl
uint256 innerCount = 0;
for (uint256 v_i = 0; v_i < task.parameters.numValidations; ++v_i) {
uint256 score = scores[v_i];
- if ((score >= _mean - _stddev) && (score <= _mean + _stddev)) {
+ uint256 lowerBoundry = _stddev <= _mean ? _mean - _stddev : 0
+ if ((score >= lowerBoundry) && (score <= _mean + _stddev)) {
innerSum += score;
innerCount++;
@@ -365,7 +366,8 @@ contract LLMOracleCoordinator is LLMOracleTask, LLMOracleManager, UUPSUpgradeabl
(uint256 stddev, uint256 mean) = Statistics.stddev(generationScores);
for (uint256 g_i = 0; g_i < task.parameters.numGenerations; g_i++) {
// ignore lower outliers
- if (generationScores[g_i] >= mean - generationDeviationFactor * stddev) {
+ uint256 lowerBoundry = generationDeviationFactor * stddev <= mean ? mean - generationDeviationFactor * stddev : 0
+ if (generationScores[g_i] >= lowerBoundry) {
_increaseAllowance(responses[taskId][g_i].responder, task.generatorFee);
}
}
Updates

Lead Judging Commences

inallhonesty Lead Judge 8 months ago
Submission Judgement Published
Validated
Assigned finding tags:

Underflow in `LLMOracleCoordinator::validate`

Support

FAQs

Can't find an answer? Chat with us on Discord, Twitter or Linkedin.