diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/guide.en-gb.md b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/guide.en-gb.md new file mode 100644 index 00000000000..a1450dcc183 --- /dev/null +++ b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/guide.en-gb.md @@ -0,0 +1,207 @@ +--- +title: Introduction to Public Cloud Logs +excerpt: Understand how logging works in OVHcloud Public Cloud +updated: 2026-02-15 +--- + +## Objective + +This guide helps you understand the concepts and usage of logging in OVHcloud Public Cloud. It explains how logs are generated, collected, and consumed, and provides links to detailed guides for subscribing to logs from each Public Cloud service (Compute, Block Storage, Load Balancer, etc.). + +### What are Public Cloud Logs? + +Public Cloud services generate logs that can help you monitor, troubleshoot, and analyze your cloud infrastructure. + +Each service may offer different types of logs, known as "kinds" (e.g., access, audit, error), which can be consumed and stored according to your needs. + +## Requirements + +- Access to the [OVHcloud Control Panel](/links/manager). +- A [Public Cloud project](/links/public-cloud/public-cloud) in your OVHcloud account +- A [Logs Data Platform](/links/manage-operate/ldp) account +- [At least one Stream](/pages/manage_and_operate/observability/logs_data_platform/getting_started_quick_start) created inside this account +- At least one instance of a product compatible with Public Cloud logs (see the [list below](#compatible-ovhcloud-products)). + +### Use cases + +#### Application troubleshooting + +Public Cloud logs help you troubleshoot application and infrastructure issues by giving access to logs generated by your services. You can filter and search these logs to identify anomalies and errors, which makes diagnosing and resolving problems more efficient. In distributed cloud architectures, logs from different services can be correlated to get a complete view of your environment. + +#### Security and data compliance + +Public Cloud logs help you detect security incidents, track administrative actions, and support compliance with certifications and regulatory requirements. By keeping a record of operations and access events, logs provide essential evidence for audits and security investigations. + +Depending on the service and the log destination you choose, access to logs can be restricted using fine-grained permissions. This allows you to control who can view or manage specific logs, strengthening your security posture and improving overall data governance. + +#### Infrastructure monitoring + +Public Cloud logs provide valuable insights into the health and behavior of your infrastructure. By analyzing service and system logs, you can detect abnormal patterns, errors, or performance issues and react before they impact your workloads or users. + +Depending on the tools and destinations you use to consume your logs, you can build monitoring workflows such as alerts or visualizations to better understand the activity of your Public Cloud services and share this information across your teams. + +## Compatible Public Cloud services + +The following table lists the Public Cloud services that currently provide logs and explains where to find the corresponding guides to enable or retrieve them. +Availability and capabilities may vary depending on the service. This list will be updated as new services become compatible. + +| Product name | Availability | Guide link | +| :----------- | :----------- | :--------- | +| Public Cloud - Compute | OVHcloud Control Panel & API | [Public Cloud Openstack Logs Forwarding](/pages/public_cloud/public_cloud_network_services/replace) | +| Public Cloud - Block Storage | OVHcloud Control Panel & API | [Public Cloud Openstack Logs Forwarding](/pages/public_cloud/public_cloud_network_services/replace) | +| Public Cloud - Network | OVHcloud Control Panel & API | [Public Cloud Openstack Logs Forwarding](/pages/public_cloud/public_cloud_network_services/replace) | +| Public Cloud - Object Storage | OVHcloud Control Panel & API | Link roadmap git ? | +| Public Cloud - Managed Kubernetes Service | OVHcloud Control Panel & API | [Managed Kubernetes Service Audit Logs Forwarding](/pages/public_cloud/containers_orchestration/managed_kubernetes/forwarding-audit-logs-to-logs-data-platform) | +| Public Cloud - Load Balancer | OVHcloud Control Panel & API | [Public Cloud Load Balancer TCP / HTTP / HTTPS Logs Forwarding](/pages/public_cloud/public_cloud_network_services/technical-resources-05-lb_logs_2_customers) | +| Public Cloud - Managed Databases | OVHcloud Control Panel & API | [Public Cloud Databases - How to setup logs forwarding](/pages/public_cloud/public_cloud_databases/databases_16_logs_to_customer) | +| Public Cloud - Analytics | OVHcloud Control Panel & API | [Analytics - How to setup logs forwarding](/pages/public_cloud/data_analytics/analytics/analytics_logs_to_customer) | +| Public Cloud - AI | OVHcloud Control Panel & API | Link roadmap git ? | + +## Instructions + +### On the product side + +On Public Cloud services that provide logging capabilities, you can access logs from the OVHcloud Control Panel through a dedicated `Logs`{.action} tab. This section allows you to view available logs and configure how they are collected or forwarded. Once you click this tab, you will see the two main components of Public Cloud logs: a **live-tail** panel and a **subscription** panel. + +![live-tail](images/logs_tab.png){.thumbnail} + +#### Live-tail panel + +On the left side, the live-tail panel lets you see the logs of this service in **real time**. It is useful to have a quick overview of the ongoing activity on this service. + +It lets you pause/play the live-tail, clear the current session, scroll, and also includes a basic search: + +![basic-search](images/basic_search.png){.thumbnail} + +#### Subscription panel + +On the right side, the subscription panel lets you **subscribe** to this service's logs to make them available in your Logs Data Platform account, in the data stream(s) of your choice. This panel displays all your active log subscriptions for this service and lets you create new ones. + +If you click on the `Subscribe`{.action} button (or `Subscribe to another data stream`{.action} if you already have some existing subscriptions), you will land on this page: + +![create-subscription-page](images/subscribe_page.png){.thumbnail} + +This is where you have to select the Logs Data Platform stream in which you want this service's logs to be available. Using the top left dropdown list, select your Logs Data Platform account. The datagrid in the center displays all the data streams in the selected Logs Data Platform account and shows you information about each data stream: its name/description, its retention and if indexing is enabled. It also shows you if this data stream already has some other active logs subscriptions. + +The last column contains the `Subscribe`{.action} and `Unsubscribe`{.action} buttons. Clicking on `Subscribe`{.action} will start sending your service logs in the selected data stream, while clicking on `Unsubscribe`{.action} will stop sending them. + +You can choose to create multiple subscriptions for a given service, to have its service logs available in multiple data streams. This can be useful for use cases which will be detailed further down this guide. + +> [!primary] +> +> - Only the service logs generated **after** the subscription creation will be available in your data stream. +> - If you choose to unsubscribe, the logs already sent to your data stream will remain; they will only be deleted when they reach your stream's configured retention. You just won't see newly generated logs. +> - Logs Data Platform is pay-as-you-go: you pay for the amount of logs sent in your data stream. If you subscribed to a product that generates no logs (because it has no activity), you won't pay anything. + +#### Logs kinds + +A given Public Cloud can offer different types of logs you can subscribe to. These different types of logs are named "`kinds`". When a product has more than one log kind, the live-tail & subscriptions panels become contextualized to the selected kind: + +![manager-with-kind](images/multiple_kind.png){.thumbnail} + +In this example, both the live-tail only display and the subscriptions listed on the right panel are related to the "`REST API audit logs`". Selecting another kind in the top dropdown list will display different live-tail content and subscriptions. + +This division of a product's logs into several kinds provides you more flexibility. For example if a product has many kinds (e.g. `access`, `audit`, `error` and `ssh`) and you are only interested in `audit` logs, you can choose to subscribe only to this `audit` and not to the others. Another example: you are interested in `audit` logs and `error` logs, but you want to send these logs in two different data streams: that's possible by creating two subscriptions based on two different kinds targeting two different data streams. + +We will dig into this use case later in this guide. + +### On the Logs Data Platform side + +Now that you have created logs subscription(s), you can consume these logs the way you want in your Logs Data Platform data stream. Depending on your needs, you can configure your data stream in different (non-exclusive) ways. For instance: + +- Enabling indexation to benefit from the OpenSearch/Graylog feature set. +- Enabling [long-term storage](/pages/manage_and_operate/observability/logs_data_platform/archive_cold_storage) to safely keep your logs for years. +- Enabling web-socket to consume your logs from a third-party software, such as [ldp-tail](/pages/manage_and_operate/observability/logs_data_platform/cli_ldp_tail). + +All the possible ways to consume your logs are summarized in the [Introduction to Logs Data Platform](/pages/manage_and_operate/observability/logs_data_platform/getting_started_introduction_to_LDP). + +> [!primary] +> +> In the case of Public Cloud logs, all the "Log Generation" and "Log Ingestion" are managed by OVHcloud. You only have to care about the "Storage" and "Query and visualization" parts. + +You can also manage your logs subscription from a data stream point of view. To do so, go to the Logs Data Platform section of the [OVHcloud Control Panel](/links/manager), select a service and click on the `Data stream`{.action} tab. In the table listing your streams you can see a `Subscriptions` column displaying how many logs subscriptions target a given stream. + +In this table, click the `...`{.action} button then click `Manage subscriptions`{.action}. The next page will list all the logs subscriptions targeting this data stream: + +![list-from-stream](images/list_from_stream.png){.thumbnail} + +For each subscription, you can see the type of service it comes from, the service name and the subscribed logs kind. + +You can also delete a logs subscription from this view by clicking the corresponding bin icon. + +### Working with the API + +To ease your integration jobs with Public Cloud logs, we made the related API endpoints consistent across all Public Cloud. This means all compatible products expose the **same** set of API endpoints, with the **same** suffixes, the **same** payloads and the **same** responses. The available endpoints are the following: + +| **Method** | **Path** | **Description** | +| :--------: | :----------------------------------------------------- | :---------------------------------------------------------------- | +| `GET` | `/xxx/{serviceName}/log/kind` | List product's log kinds | +| `GET` | `/xxx/{serviceName}/log/kind/{name}` | Get the detail of a log kind | +| `GET` | `/xxx/{serviceName}/log/subscription` | List existing log subscription from this product instance | +| `POST` | `/xxx/{serviceName}/log/subscription` | Create a new log subscription from this product instance | +| `GET` | `/xxx/{serviceName}/log/subscription/{subscriptionId}` | Get the detail of a log subscription | +| `DELETE` | `/xxx/{serviceName}/log/subscription/{subscriptionId}` | Delete a log subscription | +| `POST` | `/xxx/{serviceName}/log/url` | Generate a temporary URL to retrieve logs (used by the live-tail) | + +#### Public Cloud logs and the IAM cross-identity service delegation + +You can face use cases where the product you want logs from and the Logs Data Platform service are not owned by the same OVHcloud account. Example: + +- Alice is the owner of a Managed Kubernetes Service instance. +- Bob is the owner of a Logs Data Platform instance. +- Alice wants her Managed Kubernetes Services logs to be available in Bob's Logs Data Platform data stream. + +This is possible thanks to OVHcloud IAM: + +- Bob needs to create an IAM policy to allow Alice's identity to use the `ldp:apiovh:output/graylog/stream/forwardTo` IAM action on his Logs Data Platform data stream. +- Then Alice will be able to use the `POST /cloud/project/{serviceName}/kube/{kubeId}/log/subscription` API endpoint, and give Bob's `streamId` in this endpoint payload. + +You can find more information about IAM policies in the [How to use IAM policies using the OVHcloud Control Panel](/pages/account_and_service_management/account_information/iam-policy-ui) guide. + +### Interesting strategies + +As explained earlier, the model of Public Cloud logs gives you a lot of flexibility regarding how you want to gather/isolate/consume your logs. In this section we will see some common patterns. These are just examples, they aren't an exhaustive list of all the Public Cloud logs configuration possibilities. + +#### Many products to One data stream, per environment + +In this example let say you have a tech stack composed of many Public Cloud. This stack is deployed on multiple environments (development, production). One interesting strategy could be to send each environment's logs to a different data stream, and configure these 2 data streams in a different way: + +![many-to-one-per-env](images/many_to_one_per_env.png){.thumbnail} + +In the configuration above: + +- All the logs of your development environment are available in a data stream. Only the indexation feature is enabled on this data stream. +- All the logs of your production environment are available in another data stream. The indexation and long-term storage feature are both enabled. + +This means that: + +- your development environment logs won't be long-term stored, as you probably don't need it and don't want to pay for it. +- your production environment logs will be stored in a dedicated data stream, making it easier for you to configure alerts or dashboards based on this stream. + +### Many products to Many data streams + +Lets take the same example again, where you have a tech stack made of several Public Cloud. But in this case you have multiple teams, each one working on a different component. For security reasons you want the team `A` to access the Load Balancer logs but not the Managed Databases logs. +Also you have a Security team that needs to access all the audit logs of each component. We can imagine a setup such as the following one: + +![many-to-many](images/many_to_many.png){.thumbnail} + +In the configuration above: + +- All the Load Balancer & Managed Kubernetes *applicative* logs are sent to a data stream (blue arrow). +- All the Managed Databases *applicative* logs are sent to another data stream (blue arrow). +- All their *audit* logs are sent to another data stream (red arrow). + +This means that: + +- you can grant your Security Team to see only the data stream containing all the audit logs of all the components of your tech stack. +- you can grant your Developer Team to see only the data stream containing the Load Balancer / Managed Kubernetes logs. +- you can grant your Database Administrator Team to see only the data stream contaning your Managed Database logs/ +- you can configure the indexation/long-term storage/etc. as you wish on each of these streams. You can also set different retentions for each stream. + +## Go further + +- Getting Started: [Quick Start](/pages/manage_and_operate/observability/logs_data_platform/getting_started_quick_start) +- Documentation: [Guides](/products/observability-logs-data-platform) +- Create an account: [Try it!](https://www.ovh.com/en/order/express/#/express/review?products=~(~(planCode~'logs-account~productId~'logs))) + +Join our [community of users](/links/community). diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/basic_search.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/basic_search.png new file mode 100644 index 00000000000..897ca296d55 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/basic_search.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/list_from_stream.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/list_from_stream.png new file mode 100644 index 00000000000..f789de1a342 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/list_from_stream.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/logs_tab.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/logs_tab.png new file mode 100644 index 00000000000..d8ac980ee33 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/logs_tab.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_many.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_many.png new file mode 100644 index 00000000000..43e48017f76 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_many.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_one_per_env.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_one_per_env.png new file mode 100644 index 00000000000..ac85df3419b Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/many_to_one_per_env.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/multiple_kind.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/multiple_kind.png new file mode 100644 index 00000000000..9f6a692a637 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/multiple_kind.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/subscribe_page.png b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/subscribe_page.png new file mode 100644 index 00000000000..e809c419063 Binary files /dev/null and b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/images/subscribe_page.png differ diff --git a/pages/public_cloud/public_cloud_cross_functional/logs-concepts/meta.yaml b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/meta.yaml new file mode 100644 index 00000000000..d286c0f02b1 --- /dev/null +++ b/pages/public_cloud/public_cloud_cross_functional/logs-concepts/meta.yaml @@ -0,0 +1,2 @@ +id: a53a23c4-a63d-44c0-b2d4-969244727977 +full_slug: public-cloud-logs-concepts \ No newline at end of file