Tekton Long-Term Log Storage
KubeRocketCI Portal provides the capability to view long-term logs for pipelines. The long-term logs are stored in the OpenSearch cluster and can be viewed in the KubeRocketCI Portal after pipeline cleanup or deletion. This guide describes how to configure and view long-term logs in the KubeRocketCI Portal.
Long-Term Log Access Workflow​
The following diagram illustrates the workflow for accessing long-term logs for pipelines in the KubeRocketCI Portal:
Configuration​
To configure long-term log storage for pipelines in the KubeRocketCI Portal, follow the steps below:
-
Install and configure OpenSearch cluster:
noteTo install OpenSearch cluster, we recommend to use the edp-cluster-add-ons repository, which contains already prepared configurations for OpenSearch installation.
To install OpenSearch cluster using the add-ons repository, follow the steps below:
-
Clone the forked edp-cluster-add-ons repository.
-
Navigate to the
clusters/core/addons/opensearch
directory and configure thevalues.yaml
file with the necessary values for OpenSearch cluster installation.noteEnsure that Fluent Bit is configured to send container logs to the
logstash-edp
index in the OpenSearch cluster, as the KubeRocketCI Portal uses this index to retrieve long-term logs. -
Install the OpenSearch cluster using Helm or Argo CD methods.
-
-
Install and configure KrakenD API Gateway:
noteTo install KrakenD API Gateway with OpenSearch connection, we recommend to use the edp-cluster-add-ons repository, which contains already prepared configurations for KrakenD installation. For more details, refer to the KrakenD Integration guide.
To install KrakenD API Gateway with OpenSearch connection, follow the steps below:
-
Clone the forked edp-cluster-add-ons repository.
-
Navigate to the
clusters/core/addons/krakend
directory and configure thevalues.yaml
file with the necessary values for KrakenD installation.noteEnsure that the KrakenD endpoint object for OpenSearch has the
/search/logs
endpoint name, as the KubeRocketCI Portal uses this endpoint to fetch long-term logs. -
Ensure that the KrakenD configuration secret contains the OpenSearch connection variables.
- Manifests
- External Secrets Operator
apiVersion: v1
kind: Secret
metadata:
name: krakend
namespace: krakend
type: Opaque
stringData:
OPENSEARCH_URL: https://opensearch-cluster-master.logging:9200
OPENSEARCH_CREDS: <base64-encoded-credentials>{
"OPENSEARCH_URL": "https://opensearch-cluster-master.logging:9200",
"OPENSEARCH_CREDS": "<base64-encoded-credentials>"
} -
Install the KrakenD API Gateway using Helm or Argo CD methods.
-
Viewing Long-Term Logs​
After configuring long-term log storage for Pipelines in the KubeRocketCI Portal, follow the steps below to view long-term logs:
-
Navigate to the KubeRocketCI Portal and sign in with appropriate credentials.
-
In the Components section, navigate to the appropriate component (if component is not created yet, refer to the Add Application guide to create a new one).
-
In the component window, in the right upper corner, click the GIT button to navigate to the component repository (e.g., GitHub repository).
-
In the component repository, create a Pull Request to trigger the review pipeline execution.
-
In the Pull Request checks section, click the Show all checks button and navigate to the review pipeline in the KubeRocketCI Portal by clicking the Details button.
-
In the pipeline window, check the pipeline status. Ensure that the pipeline is successfully executed and the logs are available.
-
Navigate to the Pipelines section and find the executed review pipeline. Delete the pipeline by clicking the Delete button.
-
Return to the Pull Request and click the Show all checks button. Navigate to the review pipeline again by clicking the Details button.
-
After the pipeline deletion, the long-term logs are displayed in the pipeline window.
This example demonstrates how to view long-term logs for review pipeline, but the same approach can be applied to other pipeline types.