Dria
Swan
NFTHardhat
21,000 USDC
View results
Previous Next
Submission Details
Severity: high
Valid

The final validation for a request will revert due to underflow in `LLMOracleCoordinator::finalizeValidation` when standard deviation is higher than the mean.

ericselvig

Vulnerability Details

The finalizeValidation function in LLMOracleCoordinator calculates the standard deviation of the scores given by all the validators for a particular genertion. This _stddev is then used to determine the range of the acceptable scores as:
score >= _mean - _stddev && score <= _mean + _stddev.

However, if the standard deviation is higher than the mean, the transaction will revert due to an underflow in the calculation of the range.

A similar issue occurs when calculating the range of acceptable scores for the generations in the same function.

The chances of this happening are not very rare either, even if one value if different than the rest, the mean of the data may be less than the standard deviation as shown in the PoC.

File: llm/LLMOracleCoordinator.sol
​
function finalizeValidation(uint256 taskId) private {
// code...
for (uint256 v_i = 0; v_i < task.parameters.numValidations; ++v_i) {
uint256 score = scores[v_i];
@> if ((score >= _mean - _stddev) && (score <= _mean + _stddev)) {
// code...
}
}
// code...
for (uint256 g_i = 0; g_i < task.parameters.numGenerations; g_i++) {
@> if (generationScores[g_i] >= mean - generationDeviationFactor * stddev) {
_increaseAllowance(responses[taskId][g_i].responder, task.generatorFee);
}
}
// code...
}

https://github.com/Cyfrin/2024-10-swan-dria/blob/c8686b199daadcef3161980022e12b66a5304f8e/contracts/llm/LLMOracleCoordinator.sol#L343

https://github.com/Cyfrin/2024-10-swan-dria/blob/c8686b199daadcef3161980022e12b66a5304f8e/contracts/llm/LLMOracleCoordinator.sol#L368

Proof of Concept

Add this to LLMOracleCoordinator.test.ts:

describe("final validation fails if standard deviation is higher than the mean", function () {
const [numGenerations, numValidations] = [1, 4];
​
// mean = 3.3125e18
// standard deviation = 3.9186e18
// standard deviation > mean
let dummyScores = [parseEther("0"), parseEther("1.5"), parseEther("1.75"), parseEther("10")];
​
this.beforeAll(async () => {
const [, , , , , , , , val3, val4] = await ethers.getSigners();
validators.push(val3, val4);
await transferTokens(token, [
[val3.address, STAKES.validatorStakeAmount],
[val4.address, STAKES.validatorStakeAmount],
]);
await registerOracles(token, registry, [], [val3, val4], STAKES);
​
taskId++;
});
​
it("should revert the final validation", async function () {
await safeRequest(coordinator, token, requester, taskId, input, models, {
difficulty,
numGenerations,
numValidations,
});
​
await safeRespond(coordinator, generators[0], output, metadata, taskId, 0n);
​
for (let i = 0; i < numValidations - 1; i++) {
await safeValidate(coordinator, validators[i], [dummyScores[i]], metadata, taskId, BigInt(i));
}
​
// the last validation fails with underflow panic error
await expect(
safeValidate(
coordinator,
validators[numValidations - 1],
[dummyScores[numValidations - 1]],
metadata,
taskId,
BigInt(numValidations - 1)
)
).to.be.revertedWithPanic(0x11);
});
});

Impact

The completion of the task will never be successful and the generators and validators will not receive their fees.

Tools Used

Manual Review

Recommendations

Use a different method to calculate the range of acceptable scores when the standard deviation is higher than the mean.

Updates

Lead Judging Commences

inallhonesty Lead Judge 8 months ago
Submission Judgement Published
Validated
Assigned finding tags:

Underflow in `LLMOracleCoordinator::validate`

Support

FAQs

Can't find an answer? Chat with us on Discord, Twitter or Linkedin.