rate_request), the playground and the dashboard. Sending again from the same place replaces what that place sent before.
Endpoint
Send exactly one of
thumbs, rating or score. A send is the whole feedback: a label or comment you leave out the second time is cleared.
The answer is the feedback as it was saved:
404 with the code request_not_found. A call can be scored a moment after its answer has come back. Each key can send up to 600 scores a minute, and a workspace 3,000, apart from model calls; past that the answer is 429.
Score an answer with the OpenAI SDK
The completion’sid is chatcmpl- and the request id, so you can send it as it is. For the other endpoints, read the X-Request-Id header from the raw response.
With plain HTTP
From an agent
The Runcrate MCP server has arate_request tool, so an agent can score the answers it gets back (an eval loop, a reviewer agent). Its scores are kept as the agent’s, apart from your code’s.
Send one of
thumbs, rating or score.
In the dashboard
- Observability: open a request and rate it with the thumbs under Rating, with a note if you like. The panel also lists what your code and your agents sent. The Rating filter lists only answers rated good, rated bad, or rated at all, and the summary shows the share rated good once anything in the selection has a rating.
- Playground: rate the answer under the result, or any run in Recent runs.
Reading feedback back
GET https://www.runcrate.ai/api/v1/requests/<request id>/feedback with your API key lists every place’s feedback on a request, newest first.
How scores read
- Above
0is a good answer, below0a bad one, and0(a rating of 3) neither. - A request rated in several places counts as rated good if any place rated it good, and as rated bad if any rated it bad.
- Feedback is kept as long as the request: 30 days.