Mechanism to filter out unnecessary logs when publishing to DAS from API Manager analytics #161
Description
API Manager analytics are deployed in API Cloud. We have recently found an issue where log publishers are trying to publish logs to DAS but it throws an error saying connection failed etc, these logs are logged back in publisher node again and it causes a riffling effect trying to publish those errors back to DAS.
This causes API manager nodes to perform slow and causing a severe effect on API manager functionality too.
DAS node can go offline, can fail to handle incoming load etc. So we can not guarantee that DAS will always accept events from API manager publishers.
To fix this issue, API Manager log publisher should be able to configure what not to be published such as DAS connectivity issues, or logs from the particular tenant(super tenant) etc. API Manager analytics needs to be improved to configure a log filtering mechanism.
Moved from https://wso2.org/jira/projects/SHAREDANLY/issues/SHAREDANLY-4