External Secrets Operator Integration
External Secrets Operator (ESO) can be integrated with KubeRocketCI.
There are multiple Secrets Providers that can be used within ESO. On this page, we detail the integration of KubeRocketCI with the two foremost providers:
KubeRocketCI leverages a variety of secrets to facilitate the integration of different solutions. This document provides a comprehensive overview of the secrets utilized within the KubeRocketCI platform, along with their respective descriptions. Please note that all secrets are encoded in Base64 format.
Secret Name | Fields | Description | Used by |
---|---|---|---|
keycloak | username password | Username and password with specific permissions for the Platform tenant management in Keycloak | keycloak-operator |
ci-defectdojo | token url | DefectDojo token DefectDojo URL | edp-tekton |
kaniko-docker-config | .dockerconfigjson | Serialized JSON that follows docker config patterns | edp-tekton |
regcred | .dockerconfigjson | Serialized JSON that follows docker config patterns | cd-pipeline-operator |
ci-github | id_rsa token secretString | Private key from github repo API token Random string | edp-tekton |
ci-gitlab | id_rsa token secretString | Private key from gitlab repo API token Random string | edp-tekton |
ci-jira | username password | Jira username Jira password | edp-codebase-operator |
ci-sonarqube | token url | SonarQube token SonarQube URL | edp-tekton |
ci-nexus | username password url | Nexus username Nexus password Nexus URL | edp-tekton |
ci-dependency-track | token url | Dependency-Track token Dependency-Track URL | edp-tekton |
oauth2-proxy-cookie-secret | cookie-secret | Secret key for oauth2-proxy | edp-install |
keycloak-client-headlamp-secret | clientSecret | Secret key for keycloak client | keycloak-operator |
ci-argocd | token url | Argo CD token Argo CD URL | edp-tekton |
Platform Core Secretsβ
The list below represents the baseline required for full operation within platform core components:
- kaniko-docker-config: Used for pushing container images to a specific registry.
- ci-sonarqube: Used in the CI process for SonarQube integration.
- ci-nexus: Used for pushing artifacts to the Nexus storage.
These secrets are mandatory for Tekton pipelines to work properly.
Kubernetes Providerβ
All secrets are stored in Kubernetes in pre-defined namespaces. Platform suggests using the following approach for secrets management:
-
EDP_NAMESPACE-vault
, whereEDP_NAMESPACE
is a name of the namespace where KubeRocketCI is deployed, such asedp-vault
. This namespace is used by the platform. Access to secrets in theedp-vault
is granted only forAdministrators
. -
EDP_NAMESPACE-cicd-vault
, whereEDP_NAMESPACE
is a name of the namespace where KubeRocketCI is deployed, such asedp-cicd-vault
. Development team uses secrets in theedp-cicd-vault
for microservices development.
See a diagram below for more details:
In order to Install KubeRocketCI, a list of passwords must be created. Secrets are provided automatically when using ESO.
-
Create a common namespace for secrets and platform:
kubectl create namespace edp-vault
kubectl create namespace edp -
Create secrets in the
edp-vault
namespace:apiVersion: v1
kind: Secret
metadata:
name: keycloak
namespace: edp-vault
data:
password: cGFzcw== # pass in base64
username: dXNlcg== # user in base64
type: Opaque -
In the
edp-vault
namespace, create a Role with a permission to read secrets:apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: edp-vault
name: external-secret-store
rules:
- apiGroups: [""]
resources:
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- authorization.k8s.io
resources:
- selfsubjectrulesreviews
verbs:
- create -
In the
edp-vault
namespace, create a ServiceAccount used bySecretStore
:apiVersion: v1
kind: ServiceAccount
metadata:
name: secret-manager
namespace: edp -
Connect the Role from the
edp-vault
namespace with the ServiceAccount in theedp
namespace:apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: eso-from-edp
namespace: edp-vault
subjects:
- kind: ServiceAccount
name: secret-manager
namespace: edp
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: external-secret-store -
Create a SecretStore in the
edp
namespace, and use ServiceAccount for authentication:apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: edp-vault
namespace: edp
spec:
provider:
kubernetes:
remoteNamespace: edp-vault # namespace with secrets
auth:
serviceAccount:
name: secret-manager
server:
caProvider:
type: ConfigMap
name: kube-root-ca.crt
key: ca.crt -
Each secret must be defined by the
ExternalSecret
object. A code example below creates thekeycloak
secret in theedp
namespace based on a secret with the same name in theedp-vault
namespace:apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: keycloak
namespace: edp
spec:
refreshInterval: 1h
secretStoreRef:
kind: SecretStore
name: edp-vault
# target:
# name: secret-to-be-created # name of the k8s Secret to be created. metadata.name used if not defined
data:
- secretKey: username # key to be created
remoteRef:
key: keycloak # remote secret name
property: username # value will be fetched from this field
- secretKey: password # key to be created
remoteRef:
key: keycloak # remote secret name
property: password # value will be fetched from this field
Apply the same approach for enabling secrets management in the namespaces used for microservices development, such as sit
and qa
on the diagram above.
AWS Systems Manager Parameter Storeβ
AWS SSM Parameter Store can be used as a Secret Provider for ESO. For the platform, it is recommended to use the IAM Roles For Service Accounts approach (see a diagram below).
AWS Parameter Store Scenario in KubeRocketCIβ
In order to Install KubeRocketCI, a list of passwords must be created. Follow the steps below, to get secrets from the SSM:
-
In the AWS, create an AWS IAM policy and an IAM role used by
ServiceAccount
inSecretStore
. The IAM role must have permissions to get values from the SSM Parameter Store.a. Create an IAM policy that allows to get values from the Parameter Store with the
edp/
path. Use yourAWS Region
andAWS Account Id
:{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "ssm:GetParameter*",
"Resource": "arn:aws:ssm:eu-central-1:012345678910:parameter/edp/*"
}
]
}b. Create an AWS IAM role with trust relationships (defined below) and attach the IAM policy. Put your string for
Federated
value (see more on IRSA enablement for EKS Cluster) and AWS region.{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::012345678910:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/XXXXXXXXXXXXXXXXXX"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringLike": {
"oidc.eks.eu-central-1.amazonaws.com/id/XXXXXXXXXXXXXXXXXX:sub": "system:serviceaccount:edp:*"
}
}
}
]
} -
Create a secret in the AWS Parameter Store with the name
/edp/my-json-secret
. This secret is represented as a parameter of type string within the AWS Parameter Store:View: Parameter Store JSON
{
"keycloak":
{
"username": "keycloak-username",
"password": "keycloak-password"
},
"ci-defectdojo":
{
"token": "XXXXXXXXXXXX",
"url": "https://defectdojo.example.com"
},
"kaniko-docker-config":
{
"auths" :
{
"registry.com":
{
"username":"registry-username",
"password":"registry-password",
"auth": "<base64 encoded 'user:secret' string>"
}
}},
"regcred":
{
"auths":
{
"registry.com":
{
"username":"registry-username",
"password":"registry-password",
"auth":"<base64 encoded 'user:secret' string>"
}
}},
"ci-github":
{
"id_rsa": "id-rsa-key",
"token": "github-token",
"secretString": "XXXXXXXXXXXX"
},
"ci-gitlab":
{
"id_rsa": "id-rsa-key",
"token": "gitlab-token",
"secretString": "XXXXXXXXXXXX"
},
"ci-jira":
{
"username": "jira-username",
"password": "jira-password"
},
"ci-sonarqube":
{
"username": "<ci-user>",
"secret": "<secret>"
},
"ci-nexus":
{
"username": "<ci.user>",
"password": "<secret>"
},
"oauth2-proxy-cookie-secret":
{
"cookie-secret": "XXXXXXXXXXXX"
},
"keycloak-client-headlamp-secret": "XXXXXXXXXXXX",
"ci-argocd":
{
"token": "argocd-token",
"url": "https://argocd.example.com"
},
"ci-dependency-track":
{
"token": "dependency-track-token",
"url": "https://dependency-track.example.com"
}
} -
Set External Secret operator enabled by updating the values.yaml file:
externalSecrets:
enabled: true -
Install/upgrade edp-install:
helm upgrade --install edp epamedp/edp-install --wait --timeout=900s \
--version <edp_version> \
--values values.yaml \
--namespace edp \
--atomic