Secrets not updated - operator runs into rate limit issues (receiving generic error message)

[dorthrithil]

Your environment

Operator Version: 1.10.1
Connect Server Version: Using service account
Kubernetes Version: 1.24.0

What happened?

I created 6 OnePasswordItems recently:

apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
  name: xxxxx
  namespace: xxxxx
spec:
  itemPath: "vaults/xxxxx/items/xxxxxx"

Now, I changed a value in 1Pwd and expected to see it reflected in the k8s Secret after some time. But that didn't happen. Instead I see rate limit errors in the pod logs of the operator. I wonder how I can hit the limits so frequently with just 6 items? I have a normal 1Pwd account type, it should have 1000 read and 100 write operations / h.

UPDATE: As pointed out in comment below vault/item lookup should be consistent. The vault lookup should mimic how we handle item lookup for consistency and to reduce unnecessary requests.

We should also look into the generic reconciliation error that may be appearing if the error is actually actually masking rate limit exceeded error messages.

What did you expect to happen?

Not hitting rate limits, see my Secret updated.

Notes & Logs

2026-02-16T21:16:27Z	ERROR	update_op_kubernetes_secrets_task	failed to retrieve 1Password item at path vaults/xxxxx/items/xxxxx for secret xxxxxx	{"error": "failed to 'getVaultID' for vaultNameOrID='xxxxx': failed to get vault by title \"xxxxx\": failed to GetVaultsByTitle using 1Password SDK: performing a vault operation: rate limit exceeded"}
github.com/1Password/onepassword-operator/pkg/onepassword.(*SecretUpdateHandler).updateKubernetesSecrets
	/workspace/pkg/onepassword/secret_update_handler.go:178
github.com/1Password/onepassword-operator/pkg/onepassword.(*SecretUpdateHandler).UpdateKubernetesSecretsTask
	/workspace/pkg/onepassword/secret_update_handler.go:47
main.main.func6
	/workspace/cmd/main.go:349
[same log for 5 other OnePasswordItems - repeats every 10 minutes]

2026-02-16T21:23:47Z	INFO	controller_onepassworditem	1Password rate limit hit. Requeuing after 15 minutes.	{"Request.Namespace": "xxxxxx", "Request.Name": "xxxxxxx"}
[same log for 5 other OnePasswordItems, repeats every 15 minutes]

[JillRegan]

Hi @dorthrithil ! 👋

You're right, it looks like you are hitting a rate limit using service account.

With your 6 secrets, we poll on a schedule (whatever you set for the polling interval but default is 10mins) and we query 1Password for each item on every run. So every polling interval the operator would be making 6 new queries. Also, depending if you are using the vault name instead of the UUID in the item path we have to double the queries. First we need to make a request for the vault UUID using the name and then use the UUID in the request for the item. The name also applies for item name. If you are using name for both vault and item, this would make it 18 queries every polling interval.

Also worth noting rate limits apply per service account token for hourly usage, and daily limits are account-wide for all service accounts in the 1Password account. So other automation or CLI usage with the same token or other tokens in the account can contribute to hitting those limits.

[dorthrithil]

Hi Jill!
Thanks, that makes sense. I wasn't using UUIDs for name and vault, so I should have been hitting the daily limit. Changed them to UUIDs now, so tomorrow I should hopefully see it recover. I'll let you know here.

But now another strange thing is happening, I get:

2026-02-17T20:37:07Z	ERROR	Reconciler error	{"controller": "onepassworditem", "controllerGroup": "onepassword.com", "controllerKind": "OnePasswordItem", "OnePasswordItem": {"name":"xxxx","namespace":"xxxx"}, "namespace": "xxxx", "name": "xxxxx", "reconcileID": "xxxx", "error": "failed to retrieve item: failed to get item for vaultID='VAULT-ABC' and itemNameOrID='ITEM-DEF': no items found with identifier \"ITEM-DEF\" in vault \"VAULT-ABC\""}

But the pasted IDs are definitely correct, the same as when e.g. using "Copy Private Link" from the context menu in 1pwd.

I've had a look in the code and discovered some issues:

1. In getVaultID I noticed, that it will "First try to get vault by title", so using the vault UUID directly will not make any difference in terms of rate limit currently. I guess this can be flipped around? For items its implemented the other way round "If it looks like a UUID, try fetching by ID first".
2. The actual rate limit error message may be masked by incorrect error handling leading to the above message which makes no sense while rate limits apply and the passed value and item IDs are correct. Couldn't confirm this without proper testing though.

[JillRegan]

@dorthrithil Good catch there! You’re right about getVaultID. We currently always try get vault by title first and only treat the value as a UUID if that fails. Like you mentioned the logic for item id does the opposite. We should flip the vault handling to match this, as it helps reduce the requests!

As for the error messaging are you still seeing that no items found error now that the rate limit has refreshed? 🤔

[dorthrithil]

Nope, the no items found errors are gone, yesterday 22:41 GMT+1, so a little more then 24h after I swapped the names for UUIDs. I think this is another indicator for no items found actually masking rate limit exceeded error messages.

[JillRegan]

Thanks for confirming @dorthrithil 👍

I am going to leave this issue open and have just updated the description and title slightly so we can look into update the vault/item query as well as fix the generic messaging you received.