Opik integration for manual annotation of LLM judge

[jacobthebanana]

Log LLM Judge perspective to Comet Opik for manual annotation.

New environment variables:

export OPIK_BASE_URL="..."
export OPIK_PROJECT_NAME="..."

# optional for self-hosted installation
export OPIK_API_KEY="..."

TODO:

• Include dataset name in data sent to opik server.

[Jacob-Chmura]

Presumably we'll need to uv add opik

I have not used this framework before, but the integration seems reasonable. Will read more about it

[jacobthebanana]

Tests are fixed. Ready for review

[Jacob-Chmura]

Great work!

[Jacob-Chmura]

Nit, but this check can go outside the loop

[Jacob-Chmura]

Are there any other scripts missing here? I.e. looping through and setting the env variables? Given the results of our manual annotation trail, I think this merge may be postponed in preference of automated eval via mechanical turks etc (for rebuttals). Will keep this open for the short term.

[Jacob-Chmura]

Thanks for updating everything. I'm not sure if we'll merge this as is, but we can keep this branch for the time being.