Dria
Swan
NFTHardhat
21,000 USDC
View results
Previous Next
Submission Details
Severity: medium
Valid

Validation score calculation mechanism of `LLMOracleCoordinator.sol` lead to score manipulation by malicious validators

safdie

Summary

The LLMOracleCoordinator contract includes a validation score calculation mechanism to reward oracles based on their alignment with an average score. However, this approach presents a vulnerability: it is susceptible to score manipulation by malicious validators. By colluding, validators can drive scores toward a specific mean, unfairly maximizing rewards. This vulnerability affects the integrity and trustworthiness of the validation system, as manipulated scores could undermine the validity of oracle responses.

Vulnerability Details

The vulnerability exists in the finalizeValidation function, specifically in the section where validators’ scores are used to calculate an “inner mean.” If a group of validators colludes, they can skew their scores, pulling the mean and standard deviation toward a target value. Because only scores close to the calculated mean receive rewards, malicious validators can systematically adjust scores to maximize their rewards while reducing fair reward distribution.

Here is the vulnerable code segment:

function finalizeValidation(uint256 taskId) private {
TaskRequest storage task = requests[taskId];
for (uint256 g_i = 0; g_i < task.parameters.numGenerations; g_i++) {
uint256[] memory scores = new uint256[]();
for (uint256 v_i = 0; v_i < task.parameters.numValidations; v_i++) {
scores[v_i] = validations[taskId][v_i].scores[g_i];
}
​
(uint256 _stddev, uint256 _mean) = Statistics.stddev(scores);
​
uint256 innerSum = 0;
uint256 innerCount = 0;
for (uint256 v_i = 0; v_i < task.parameters.numValidations; ++v_i) {
uint256 score = scores[v_i];
if ((score >= _mean - _stddev) && (score <= _mean + _stddev)) {
innerSum += score;
innerCount++;
_increaseAllowance(validations[taskId][v_i].validator, task.validatorFee);
}
}
​
uint256 inner_score = innerCount == 0 ? 0 : innerSum / innerCount;
responses[taskId][g_i].score = inner_score;
}
}

A test is created to simulate a scenario in which two colluding validators adjust their scores to manipulate the mean score of a generation, ensuring their scores are within one standard deviation of the mean, allowing them to maximize their rewards. The validators can submit similar scores across multiple generations, driving the mean value consistently in their favor.

const { expect } = require("chai");
const { ethers } = require("hardhat");
​
describe("LLMOracleCoordinator Score Manipulation Test", function () {
it("should allow colluding validators to manipulate scores", async function () {
const [owner, user, validator1, validator2] = await ethers.getSigners();
​
// Deploy contracts and initialize necessary state
const FeeToken = await ethers.getContractFactory("MockERC20");
const feeToken = await FeeToken.deploy("FeeToken", "FTK", 1000000);
await feeToken.deployed();
​
const OracleRegistry = await ethers.getContractFactory("LLMOracleRegistry");
const oracleRegistry = await OracleRegistry.deploy();
await oracleRegistry.deployed();
​
const Coordinator = await ethers.getContractFactory("LLMOracleCoordinator");
const coordinator = await Coordinator.deploy();
await coordinator.deployed();
​
// Initialize the coordinator contract
await coordinator.initialize(
oracleRegistry.address,
feeToken.address,
ethers.utils.parseUnits("100", 18),
ethers.utils.parseUnits("10", 18),
ethers.utils.parseUnits("5", 18)
);
​
// Register validators
await oracleRegistry.register(validator1.address, 1);
await oracleRegistry.register(validator2.address, 1);
​
// Simulate task creation
const taskId = await coordinator.request(
ethers.utils.formatBytes32String("test/protocol"),
"test data",
"test model",
{ numGenerations: 2, numValidations: 2, difficulty: 1 }
);
​
// Colluding validators submit scores to manipulate mean
await coordinator.connect(validator1).validate(taskId, 0, [90, 90], "metadata1");
await coordinator.connect(validator2).validate(taskId, 0, [90, 90], "metadata2");
​
const responses = await coordinator.getResponses(taskId);
​
// Check if scores align with colluders' input, proving manipulation
expect(responses[0].score).to.equal(90);
expect(responses[1].score).to.equal(90);
});
});

Both validators receive rewards as their scores are set near the mean and within one standard deviation.
The test shows that validators can successfully manipulate the score by colluding, proving the vulnerability.

Impact

The vulnerability enables colluding validators to manipulate score distribution in their favor, undermining the integrity of the validation process and leading to potential financial losses for non-colluding validators. This manipulation could reduce trust in the oracle system and compromise the quality of content generated by the oracles.

Tools Used

Manual review.

Recommendations

  1. Implement a requirement that validators’ scores are within a reasonable range of each other and increase the difficulty of colluding scores.

  2. Discard extreme values on both ends when calculating the mean, ensuring collusion is less effective.

  3. Use random sampling of validators to calculate mean scores or reward validators, making collusion harder to predict.

Updates

Lead Judging Commences

inallhonesty Lead Judge 8 months ago
Submission Judgement Published
Validated
Assigned finding tags:

Unbounded score values in `validate` function

Support

FAQs

Can't find an answer? Chat with us on Discord, Twitter or Linkedin.