Summary

The LLMOracleCoordinator's respond and validate functions are limited by numGenerations and numValidations parameters that incentivizes speed over quality in AI responses. This design encourages generators to prioritize quick responses over better-quality outputs.

Also with “temporal staking” vulnerability attacker can manipulate oracle response.

Vulnerability Details

respond stores all response values in the responses array, but finalizeValidation only considers as valid the first numGenerations of responses that came in. validate is also the same.

The response process involves two steps:

1. Calculate proof-of-work nonce

2. Generate AI response

While nonce calculation time might be similar across generators, AI response generation time varies significantly. Modern models like gpt-o1 take longer but produce better results. However, the current system only processes the first N responses, creating a "fastest wins" scenario.

Impact

This can be linked to “temporal staking” vulnerability to attack the protocol.

The current implementation allows anyone to become a generator or validator at no cost, allowing them to manipulate the oracle by obtaining nonces for multiple accounts in parallel and then performing malicious responses and validations in a single block.

Tools Used

None

Recommendations

Rather than managing with numGenerations, numValidations, it is recommended to use the request parameter to set how long you want to treat responses as valid, so that all responses that come in within a certain time are treated as valid. (However, in this case, you will need to change how you manage responses to prepare for DoS).