Dria
Swan
NFTHardhat
21,000 USDC
View results
Previous Next
Submission Details
Severity: low
Invalid

Single Fee Model Does Not Account for Varying LLM Costs

0xnbvc

Vulnerability Details

The protocol uses a single fee model for all LLM models, despite significant cost variations between different models. For example:

  • GPT-4 costs approximately $0.03/1K tokens

  • GPT-3.5 costs approximately $0.0005/1K tokens

  • Claude 3 Opus costs $0.015/1K tokens

  • Claude 3 Sonnet costs $0.003/1K tokens

This means that validators running more expensive models like GPT-4 will have significantly higher operational costs compared to those running cheaper models like GPT-3.5, yet they receive the same compensation.

In LLMOracleManager.sol, the fee structure is model-agnostic:

LLMOracleManager.sol#L105-L120

/// @notice Get the total fee for a given task setting.
/// @param parameters The task parameters.
/// @return totalFee The total fee for the task.
/// @return generatorFee The fee paid to each generator per generation.
/// @return validatorFee The fee paid to each validator per validated generation.
function getFee(LLMOracleTaskParameters calldata parameters)
public
view
returns (uint256 totalFee, uint256 generatorFee, uint256 validatorFee)
{
uint256 diff = (2 << uint256(parameters.difficulty));
generatorFee = diff * generationFee;
validatorFee = diff * validationFee;
totalFee =
platformFee + (parameters.numGenerations * (generatorFee + (parameters.numValidations * validatorFee)));
}

Impact

  • Validators are disincentivized from using more expensive (and potentially more capable) models

  • Validators using expensive models may operate at a loss

  • The protocol may end up with most validators using only the cheapest models, reducing the diversity and quality of responses

  • This could lead to a "race to the bottom" where validators optimize for cost rather than quality

Recommendations

Implement model-specific fees that reflect the actual costs of different LLMs.

Updates

Lead Judging Commences

inallhonesty Lead Judge 11 months ago
Submission Judgement Published
Invalidated
Reason: Design choice

Support

FAQs

Can't find an answer? Chat with us on Discord, Twitter or Linkedin.