WIP: copilot hook POC

[spencatro-pub]

This change gets basic hooks working in copilot cli. There are still issues:

• No model attribution
• Only works in copilot-cli (chat doesn't call hooks?)

Anecdotally, this might be a little overly aggressive at calling lines AI when they might have been human. I am not sure if this is specific to copilot, or would also be an issue in other harnesses as well.

This definitely needs more baking, but I think this is a decent starting point.

(to see attribution data, check out `https://github.com/spencatro-pub/pull/2` , until #34 is resolved)

[spencatro-pub]

Quick notes on what I tried re: copilot model detection:

• Parsing logs in ~/.copilot/ - at first glance, these logs do not appear complete (only shows "default" model selection, doesn't log when model is changed etc). too brittle, and maybe a little too anti-privacy anyways
• Checking the env calling the hook - again nothing super useful in here

To clarify what I mean be "upstream changes" being needed: the cleanest path here is probably just copilot-cli adding the model information to the payload passed to the hook.

[muraalee]

Thanks to @spencatro-pub for the PR, I’ll review it soon.

Quick heads-up: I’m in the middle of a major update to Agentblame v3 to support improved diffing (via a snapshot-based approach), prompt capture, and attribution of lines back to prompts (and related metadata). This started as a small change but has grown significantly; I expect to wrap it up in the next couple of days.

Once v3 lands, you’ll likely need to rebase and test against it, then submit an updated PR.

[spencatro-pub]

Sure thing! I am excited to see what v3 brings. And yeah, I wouldn't worry too much about reviewing this yet (hence WIP). I imagine this feature will live in a branch for a while; again I do think agentblame will need changes in copilot upstream to fully claim support. (I was foolishly hoping copilot-cli might be open source so I could go sign a CLA, but appears not to be the case... darn. I'll try to get some better info to file a clear issue.) It looks like they are going for a near copy of claude's hooks, so hopefully the following design/impl there will be pretty straightforward on their end.

But yes, agreed, this branch is not worth actual scrutiny yet (esp if no one else on the team is even using copilot-cli yet, I am happy to drive my own monster for a bit). I def need to bake on this branch, find the edge cases, and learn more about the potentially quite vast schism between MS's docs and current hook implementation 😅

[muraalee]

Please update the PR for the latest agentblame version.

[muraalee]

Agent Blame 3.0 has been shipped, please pull from main and send me the PR

[spencatro-pub]

Backed out the v2 implementation, going to study v3 and copilot a little. Will either update or close this PR this week, but for now this is just for reference in case anyone else out there also wants to hack on copilot.

[spencatro-pub]

Added MVP for v3 copilot-cli snapshot tracking.

Open issues:
No session IDs: there is basically no way to correlate individual hook payloads with e.g. a larger session. Every prompt, tool call, etc creates a new session. (Could consider just using the last created session instead, but will probably lead to false associations. Any solution that isn't adding the needed metadata to hook outputs on copilot's side feels like it will be too brittle imo)
Unit testing: there isn't any precedent here so I haven't added any yet, but I think adding unit tests that test known payloads could be helpful, especially since copilot hook docs are a bit..... bare...

Quick demo video:
https://github.com/user-attachments/assets/4700c86b-f627-4671-ac05-c1a504a4345c