OpenTelemetry Metrics layer

[liamdiprose]

Hey again,

We've using OpenTelemetry to track our task metrics. We couldn't adapt this metrics adapter to work: https://github.com/DoumanAsh/metrics-opentelemetry so we created this layer.

We'd love for it to be merged upstream. Let me know how I can help it happen.

Cheers,
Liam

[geofmureithi]

Thanks for this contribution!

There is only some small changes requested mainly to avoid confusion between job, message, and request. To do so, we prefer to use task as a general term, job for mutable requests and message for immutable requests.

[geofmureithi]

Also:

1. You need to rebase as there are some conflicts
2. Add an entry to the CHANGELOG.md file.

[liamdiprose]

Thank you for clearing up the terminology. I've made the changes you requested and rebased on main.

On the topic of metric names, OpenTelemetry have a catalogue of semantic conventions for choosing metric names. For example, axum-otel-metrics uses the HTTP Semantic Conventions with metric names like http.server.request.duration (the periods are converted to _ by opentelemetry-prometheus for the /metrics endpoint in the example).

There aren't any semantic conventions for task scheduler metrics but the General guidelines recommend we consider prior art when choosing metric names. The Messaging conventions are designed with queues in mind and might fit apalis.

Based on my understanding, we would get these metric names

Prometheus Metric	OpenTelemetry Messaging Metric
tasks	messaging.client.consumed.messages
task_duration	messaging.process.duration

These semantic conventions also come with required attributes. Let me know if this changes your mind on the metric names for the otel layer.

[liamdiprose]

Let me know how I can help with the security audit.

[geofmureithi]

LGTM! Thanks for this contribution