Create and Use Custom Tekton Pipelines

This use case explains how to create and use personalized Tekton pipelines in KubeRocketCI. While KubeRocketCI offers pre-configured Tekton pipelines for common use cases, custom pipelines allow you to adapt workflows to meet unique project requirements. It also provides guidance on integrating and using these custom Tekton pipelines within your development processes.

Goals

• Provide a clear method for creating and configuring custom Tekton tasks and pipelines for build, review, deployment, and clean processes within KubeRocketCI.
• Offer a detailed guide on integrating custom pipelines with KubeRocketCI, ensuring seamless connection and functionality.
• Clarify the process for replacing existing build, review, deployment, and clean pipelines with custom pipelines, highlighting necessary steps and considerations.
• Demonstrate the benefits of using custom Tekton pipelines in KubeRocketCI, such as improved flexibility, scalability, and efficiency in development workflows.

Prerequisites

Before proceeding with this use case, ensure the following prerequisites are met:

• Access to a KubeRocketCI instance with permissions to create and edit Projects and Environments.
• A configured KubeRocketCI instance with at least one active Git Server (e.g., GitHub, GitLab or Bitbucket).
• Configured Argo CD instance with the Add-ons repository added.

Scenario

• Create a Tekton library in KubeRocketCI for storing and managing custom Tekton resources.
• Define custom Tekton pipelines for build, review, deployment, or clean processes based on specific project requirements.
• Integrate custom Tekton pipelines with KubeRocketCI to automate build, review, and deployment workflows using Argo CD and the add-ons repository.
• Replace existing build, review, deploy, or clean pipelines with custom pipelines.

Creating a Tekton Library for Custom Pipelines

The first step in creating and using custom Tekton pipelines is to set up a Tekton library in KubeRocketCI. This library acts as a central repository for storing and managing custom Tekton tasks, pipelines, and trigger templates.

note

The Tekton library template can be found in the helm-helm-pipeline repository.

To create a Tekton library, follow these steps:

1. Log in to the KubeRocketCI portal:

   

2. On the main menu, click the Manage Namespaces button:

   

3. Ensure the Namespace value points to the namespace with the KubeRocketCI installation:

   

   note

   Don't forget to press Enter to add the namespace to the allowed namespaces list.

4. Create a new Project with the Application type using the Create strategy. Select the Projects section and click + Create project:

   

5. Select Custom configuration -> Library -> Create and click Continue:

   

6. On the Git & project info tab, define the following values and click Continue:

   • Git server: github
   • Owner: github_account_name
   • Repository name: tekton-custom-pipelines
   • Default branch: main
   • Project name: tekton-custom-pipelines
   • Description: Repository for storing and managing custom Tekton resources

   

7. On the Build config tab, define the values and click Continue:

   • Library code language Helm
   • Language version/framework: Pipeline
   • Build tool: Helm
   • Codebase versioning type: semver
   • Start version from: 0.1.0
   • Suffix: SNAPSHOT

   

8. On the Review tab, verify the project configuration and click Create project:

   

9. On the congratulations page, click Open project:

   

Once the library is created, you can start adding custom Tekton tasks, pipelines, and trigger templates to the library.

Defining Custom Tekton Pipelines

After creating the Tekton library, the next step is to define custom Tekton pipelines for build, review or deployment processes based on your project requirements.

To define custom Tekton pipelines, follow these steps:

1. In the top-right corner of the Project details page, click the Git button to open the Git repository for the tekton-custom-pipelines library:

   

2. Clone the repository to your local machine:

   

   You can use the following command to clone the repository:

   git clone <git-repo-url>

3. Examine the repository structure.

   By default, the repository structure looks as follows:

   tekton-custom-pipelines
   ├── Chart.yaml
   ├── templates
   │   ├── pipelines
   │   │   ├── deploy
   │   │   │   ├── custom-clean.yaml
   │   │   │   └── custom-deploy.yaml
   │   │   ├── bitbucket-build-default.yaml
   │   │   ├── bitbucket-build-semver.yaml
   │   │   ├── bitbucket-build-lib-default.yaml
   │   │   ├── bitbucket-build-lib-semver.yaml
   │   │   ├── bitbucket-review.yaml
   │   │   ├── gerrit-build-default.yaml
   │   │   ├── gerrit-build-semver.yaml
   │   │   ├── gerrit-build-lib-default.yaml
   │   │   ├── gerrit-build-lib-semver.yaml
   │   │   ├── gerrit-review.yaml
   │   │   ├── github-build-default.yaml
   │   │   ├── github-build-semver.yaml
   │   │   ├── github-build-lib-default.yaml
   │   │   ├── github-build-lib-semver.yaml
   │   │   ├── github-review.yaml
   │   │   ├── gitlab-build-default.yaml
   │   │   ├── gitlab-build-semver.yaml
   │   │   ├── gitlab-build-lib-default.yaml
   │   │   ├── gitlab-build-lib-semver.yaml
   │   │   └── gitlab-review.yaml
   │   ├── resources
   │   │   └── npm-settings.yaml
   │   ├── tasks
   │   │   ├── deploy
   │   │   │   └── hello-world-deploy.yaml
   │   │   └── hello-world.yaml
   │   └── triggers
   │       ├── custom-clean.yaml
   │       └── custom-deploy.yaml
   └── values.yaml

   • The templates directory is a main directory that contains templates for custom Tekton resources. This directory is divided into subdirectories for pipelines, tasks, triggers, and resources.

   • The templates/pipelines directory contains Tekton pipeline templates for build and review processes. For each Git server (e.g., GitHub, GitLab, Bitbucket, or Gerrit), there are specific templates, organized by component and versioning type.

     • The appropriate template should be selected based on the configuration of your component. For example:

       • If the component is created in GitHub, has the Application component type, and uses semver versioning, you should use the github-build-semver.yaml template.
       • If the component is created in Bitbucket, has the Library component type, and uses the default versioning, you should use the bitbucket-build-lib-default.yaml template.

       Here is an example illustrating the naming conventions used in template filenames:

       

   • The templates/pipelines/deploy directory contains Tekton pipeline templates for deploy and clean processes. Select the appropriate template based on the deployment process requirements.

   • The templates/tasks directory contains Tekton task templates for specific tasks that are part of the build, review, or deployment processes.

   • The templates/triggers directory contains Tekton trigger templates for triggering the deployment or clean processes.

   • The templates/resources directory is intended for various templates, such as config maps and secrets, which can be mounted into custom Tekton tasks as needed.

4. Modify the existing pipeline and task templates or create new ones based on your project requirements.

   Result: This step will demonstrate an example of creating custom Tekton resources. After completing this step, you will have a custom Tekton task, pipeline, and trigger template.

   Usage: The created custom Tekton resources can be used later for components on the KubeRocketCI platform. This flow serves as an example that can be used when creating your own, specific custom Tekton resources for your project's needs

   Consider the scenario where a custom task hello-world needs to be created and used in a custom Tekton pipeline. Follow the steps below to achieve this:

   1. Create a custom Tekton task.

      Create a custom Tekton task called hello-world to be used in a custom build pipeline. This task should be created in the templates/tasks directory with the name hello-world.yaml. The task will accept two parameters: BASE_IMAGE and username. The BASE_IMAGE parameter specifies the base Docker image to be used for the task, while the username parameter is used to personalize the task's output.

      Here is an example of the hello-world.yaml task template:

      apiVersion: tekton.dev/v1beta1
      kind: Task
      metadata:
        name: hello-world
        labels:
          {{- include "edp-tekton.labels" . | nindent 4 }}
      spec:
        params:
        - name: BASE_IMAGE
          type: string
          default: "alpine"
          description: "The base Docker image to use for the task"
        - name: username
          type: string
          description: "The username to personalize the task's output"
        workspaces:
          - description: A workspace that contains fetched git repo.
            name: source
        steps:
          - name: hello-world
            image: $(params.BASE_IMAGE)
            workingDir: $(workspaces.source.path)
            script: |
              #!/bin/sh
              echo "Hello $(params.username), the repository contains the following files:"
              ls -al

   2. Create a Custom Tekton Pipeline.

      Define a custom Tekton pipeline that uses the hello-world custom task. Since the application for which this pipeline will be used is located in GitHub, has the component type Application, and has the versioning type semver, use the template github-build-semver.yaml in the templates/pipelines directory. First, set the parameters in the spec.params field that will be used in the custom task:

      spec:
        params:
          - default: "World"
            description: "Example parameter"
            name: username
            type: string
          - default: 'alpine:3.18.9'
            name: image-version
            type: string

      Then, in the spec.tasks field, after the get-version task, describe the custom hello-world task. Also, in the subsequent git-tag task, specify the runAfter: hello-world field to maintain the sequence of execution.

      Here is an example of part of the github-build-semver.yaml pipeline template:

      apiVersion: tekton.dev/v1
      kind: Pipeline
      metadata:
        name: github-build-semver
        labels:
          app.edp.epam.com/pipelinetype: build
      spec:
        tasks:
          ...
          - name: hello-world
            taskRef:
              name: hello-world
            runAfter:
              - get-version
            params:
              - name: BASE_IMAGE
                value: "$(params.image-version)"
              - name: username
                value: "$(params.username)"
            workspaces:
              - name: source
                workspace: shared-workspace
                subPath: source
      
          - name: git-tag
            params:
              - name: GIT_USER_EMAIL
                value: edp-ci@edp.ci-user
              - name: GIT_USER_NAME
                value: edp-ci
              - name: GIT_SCRIPT
                value: |
                  git tag -a "$(tasks.get-version.results.VCS_TAG)" -m "Tag is added automatically by CI user"
                  git push --tags
            runAfter:
              - hello-world  # Ensure to update the runAfter field for task that should run after the custom task with the correct task name
            taskRef:
              kind: Task
              name: git-cli
          ...

   3. Create a Custom Tekton Trigger Template.

      Create a Tekton Trigger Template to invoke the custom github-build-semver pipeline. Use the custom-deploy.yaml template in the templates/triggers directory. Modify the .spec.resourcetemplates.spec.pipelineRef.name field to reference the custom pipeline that uses the hello-world custom Tekton task, in this case, custom-deploy.

      Here is the relevant part of the custom-deploy.yaml trigger template:

      apiVersion: triggers.tekton.dev/v1beta1
      kind: TriggerTemplate
      metadata:
        name: custom-deploy
        labels:
          app.edp.epam.com/pipelinetype: deploy
      ...
      spec:
        resourcetemplates:
          - apiVersion: tekton.dev/v1beta1
            kind: PipelineRun
            ...
            spec:
              pipelineRef:
                name: custom-deploy
            ...

   4. Optional: Specify the dnsWildCard field in the values.yaml file.

      If custom Tekton review pipelines have been created, it is necessary to specify the dnsWildCard field in the values.yaml file. This field will be used to form the pipelineUrl variable, which is utilized in merge and pull requests to navigate to the KubeRocketCI Portal.

      Here is an example of the values.yaml file:

      dnsWildCard: "example.com"

5. Commit and push the changes to the Git repository.

   After modifying the pipeline and task templates, commit and push the changes to the Git repository. Use the following commands to commit and push the changes:

   git add .
   git commit -m "Add custom task and pipeline templates"
   git push origin main

Deliver Custom Tekton Pipelines to the Cluster

To deploy custom pipelines to the cluster, you can use Argo CD, which includes a repository with add-ons.

note

Before proceeding, ensure you have added Add-Ons repository and application according to the Install via Add-Ons page. Ensure to add the credentials template in the Argo CD settings to grant Argo CD access to both your Add-Ons and tekton-custom-pipelines repositories.

To deliver custom Tekton pipelines to the cluster, follow these steps:

1. Clone the private repository with add-ons and make the following changes. In the clusters/core/apps/values.yaml file, set the repoUrl and namespace fields to specify the Git URL of the tekton-custom-pipelines repository and the namespace where the KubeRocketCI platform is deployed. Also, set the kuberocketci-pipelines.enabled field to true to enable the deployment of the tekton-custom-pipelines Argo CD Application:

   kuberocketci-pipelines:
     enable: true
     namespace: <krci-namespace>
     repoUrl: ssh://git@github.com:22/<account-name>/tekton-custom-pipelines.git

2. Commit and push the changes to the repository.

   After making the necessary changes, commit and push the changes to the repository. Use the following commands to commit and push the changes:

   git add .
   git commit -m "Enable custom Tekton pipelines deployment"
   git push origin main

3. After pushing the changes, access Argo CD, navigate to the Application that corresponds to the repository with the add-ons, and initiate the Sync process. This will apply the tekton-custom-pipelines Helm Chart to the cluster within the specified namespace:

   

Replace Pipelines With Personalized Versions

This section demonstrates how to replace existing build, review, deploy, and clean pipelines with custom pipelines in KubeRocketCI. Two cases are covered: replacing build and review pipelines for a created component, and replacing deploy and clean pipelines for an existing deployment.

1. To replace existing review or build pipelines for a component with custom pipelines, follow these steps:

   1. Open the KubeRocketCI portal. Navigate to the Projects tab and select the component for which you want to replace the build or review pipeline:

      

   2. In the Branches tab, select the branch for which you want to replace the pipeline. Click the three dots (⋮) menu and click Edit:

      

   3. In the Edit branch dialog, select the required build or review pipeline from the dropdown list:

      

2. To replace existing deploy or clean pipelines for a deployment with custom pipelines, follow these steps:

   1. Open the KubeRocketCI portal. Navigate to the Deployments tab and select the deployment for which you want to replace the deploy or clean pipeline:

      

   2. Choose the required Environment:

      

   3. Click the Edit button to edit the Environment:

      

   4. In the Edit Environment dialog, select the required deploy or clean pipeline from the dropdown list:

      

Related Articles

• Tekton Overview
• Add Application
• Add Deployment
• Argo CD Overview
• Cluster Add-Ons Overview