LOG-8122: Add proposal for namespaced log forwarder#1912
LOG-8122: Add proposal for namespaced log forwarder#1912jcantrill wants to merge 1 commit intoopenshift:masterfrom
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@jcantrill: This pull request references LOG-8122 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@jcantrill: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
|
||
| ### Non-Goals | ||
| * Event collection and forwarding | ||
| * Guaranteed log collection with no logs lost |
There was a problem hiding this comment.
Is there any level of assurance of log forwarding given? ie guaranteed unless ....
There was a problem hiding this comment.
Log collection is always best effort. There currently is no way for any log collection agent (not just Red Hat's agent) to induce back pressure and signal a source or kublet that logs have not been collected. This means availability of logs is beholden to the log size and rotation settings configured for all containers. Accomplishing anything closer to a "collect all logs" guarantee would require an alternate log driver and high availability of a collector.
I am open to suggestions of adding more details or a statement that maybe we could strive to achieve.
| resources: # optional: corev1.ResourceRequirements | ||
| networkPolicy: # optional: obsv1.NetworkPolicy | ||
| tolerations: | ||
| nodeSelector: |
There was a problem hiding this comment.
Does this create a misconfiguration risk where a user's application pod is (re)scheduled to a node which does not overlap the CLD nodeSelector? ie any update to application scheduling constraints could require a corresponding change to CLD node selection?
There was a problem hiding this comment.
Does this create a misconfiguration risk where a user's application pod is (re)scheduled to a node which does not overlap the CLD nodeSelector? ie any update to application scheduling constraints could require a corresponding change to CLD node selection?
Possibly. My thoughts are administrators can deploy one or more distributors that cover scenarios or use cases that meet their requirements: CLD for that very important workload, CLD for general work loads, a catch all fall back CLD. CLDs are routinely matched to LFs meaning that maybe the responsibility for LF shifts from one CLD to another. We provide status and alerts to support the LogForwarder. We do need to be concerned with scheduling and the potential for log collection disruption. There probably is also a case where we may get duplicate log collection for cases where responsibilities are shifted from one CLD to another.
| priority: # the priority of this distributor in relation to others. larger is higher priority | ||
| collectionPolicy: | ||
| container: # only collects container logs | ||
| namespaceMatchLabels: # map[string]string namespace labels to evaluate when collecting logs |
There was a problem hiding this comment.
How does this relate to rbac controls on namespace/pod access? Can a user create a LogForwarder that will have access to logs for pods in namepaces they don't have access to?
There was a problem hiding this comment.
How does this relate to rbac controls on namespace/pod access? Can a user create a LogForwarder that will have access to logs for pods in namepaces they don't have access to?
CLDs beget a CLF in the openshift-logging namespace which is a reserved namespace that requires elevated permissions to access. These CLFs are no different then what we currently provide. Additionally these CLFs will be granted permissions to collect application logs across the cluster. Essentially, these are cluster level services.
LogForwarders are namespaced objects that result in a deployment which receives logs from the CLF, but only logs for workloads in the same namespace. The CLFs will be configured to isolate source to sink for a namespace to ensure one collection and forwarding effort does not impact another. From the admin of the LF's perspective, he will require no additional RBAC
|
@jcantrill From my perspective, this feature (namespaced log forwarder) is a high priority. It addresses a need raised by several customers, making it a critical capability for them to manage log forwarding at a namespace level. Whether it lands in the next immediate release or further down the road, this feature request remains highly relevant from my perspective. The core of the proposal was to introduce namespaced log forwarding based on a central distributor (Cluster Log Forwarder, or CLF). Why it is Still Relevant:
|
|
I also think this is relevant. Needs planning/prioritization relative to other stuff, but definitely deserves to be on the backlog. |
5 participants
This PR:
https://issues.redhat.com/browse/LOG-8122