Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,26 @@ make poetry

# Install dependencies without dev tools (used in CI/container)
make poetry-no-dev

# Complete local setup (creates kind cluster + installs all components)
make local-setup # Default: components mode (latest from GitHub)

# Use Helm instead of components mode
KUADRANT_DEPLOY_MODE=helm make local-setup

# Enable Prometheus CRDs for observability testing (ServiceMonitor, PodMonitor)
INSTALL_PROMETHEUS=true make local-setup

# Apply additional manifests during setup (e.g., DNS credentials, secrets)
ADDITIONAL_MANIFESTS=./my-secrets.yaml make local-setup

# Deploy specific versions
AUTHORINO_OPERATOR_VERSION=v0.13.0 \
LIMITADOR_OPERATOR_VERSION=v1.5.0 \
DNS_OPERATOR_VERSION=v0.8.0 \
make local-setup # Components: specific versions
KUADRANT_DEPLOY_MODE=helm KUADRANT_OPERATOR_VERSION=v0.10.0 \
make local-setup # Helm: specific version
```

### Running Tests
Expand Down
5 changes: 5 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,11 @@ test-scale-dnspolicy: kube-burner ## Run DNSPolicy scale tests.
cd scale_test/dnspolicy && $(KUBE_BURNER) init -c ${KUBEBURNER_WORKLOAD} --log-level debug


# Include local environment setup modules
# Load variables first, then all other modules
include ./make/vars.mk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed because of the following line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is importing the vars first as the other make files need it as well. One the next line is actually importing all the other make files, except this one.

include $(filter-out ./make/vars.mk,$(wildcard ./make/*.mk))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the make/vars.mk two times if I understand the wildcard corrently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is importing the vars first as the other make files need it as well. One the next line is actually importing all the other make files, except this one.


##@ Build Dependencies

## Location to install dependencies to
Expand Down
83 changes: 78 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,60 @@ make poetry
> - [Kuadrant Helm Charts](https://github.com/Kuadrant/helm-charts) for any Kubernetes cluster
> - [Deploying Kuadrant via OLM](https://github.com/Kuadrant/helm-charts-olm/blob/main/README.md) for OpenShift (recommended as it also deploys testing tools)

## Local Kind Cluster Setup

For local development and testing, you can set up a complete Kuadrant environment using Kind (Kubernetes in Docker).

### Prerequisites
* [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
* [Helm](https://helm.sh/docs/intro/install/)
* [jq](https://jqlang.github.io/jq/download/) (JSON processor)
* **Red Hat Registry credentials** (optional but recommended for testing tools)
- Username: Your Red Hat account username
- Password: Your Red Hat registry token (from [console.redhat.com/openshift/downloads](https://console.redhat.com/openshift/downloads))
- **Note:** Without these credentials, testing tools (Keycloak, Mockserver, etc.) won't be deployed, but core Kuadrant functionality will still work

> **⚠️ macOS Limitation:**
> MetalLB LoadBalancer services have limited functionality on macOS due to Docker Desktop's VM isolation. While MetalLB will work inside the cluster, LoadBalancer IPs won't be accessible from your Mac host machine. **For macOS users, we recommend running tests in containers** (see [From a Container](#from-a-container) section above) in addition to the local Kind setup.

### Quick Start

Set up a complete local environment with one command:

```bash
# Run the setup (defaults to Istio gateway)
make local-setup

# Optional: Apply additional manifests (e.g., DNS provider credentials, secrets, etc.)
ADDITIONAL_MANIFESTS=./my-secrets.yaml make local-setup

# Optional: Install Prometheus CRDs for observability testing
INSTALL_PROMETHEUS=true make local-setup

# Or specify EnvoyGateway
GATEWAYAPI_PROVIDER=envoygateway make local-setup

# Combine options
GATEWAYAPI_PROVIDER=envoygateway INSTALL_PROMETHEUS=true ADDITIONAL_MANIFESTS=./secrets.yaml make local-setup
```

This will:
1. Create a Kind cluster named `kuadrant-local`
2. Install metrics-server and MetalLB (LoadBalancer support)
3. Install Gateway API CRDs
4. Install cert-manager and create a self-signed ClusterIssuer
5. Install Prometheus CRDs (only if `INSTALL_PROMETHEUS=true`) - ServiceMonitor, PodMonitor, etc.
6. Install Istio or EnvoyGateway (based on `GATEWAYAPI_PROVIDER`)
7. Create test namespaces (`kuadrant`, `kuadrant2`)
8. Apply additional manifests (only if `ADDITIONAL_MANIFESTS` is provided)
9. Deploy Kuadrant Operator and Kuadrant CR
10. Deploy testing tools - Keycloak, Mockserver, etc.

**Cleanup:**
```bash
make local-cleanup # Delete the Kind cluster
```

## Configuration

The Kuadrant testsuite uses [Dynaconf](https://www.dynaconf.com/) for configuration.
Expand Down Expand Up @@ -80,19 +134,38 @@ helm install --values values-tools.yaml --wait --timeout 10m -g charts/tools-ins
<details>
<summary><b>DNS Provider Secret example (click to expand)</b></summary>

Save this as a file (e.g., `additionalManifests.yaml`) and provide it via `ADDITIONAL_MANIFESTS`:
```bash
ADDITIONAL_MANIFESTS=./additionalManifests.yaml make local-setup
```

```yaml
kind: Secret
apiVersion: v1
kind: Secret
metadata:
name: aws-credentials
namespace: kuadrant
annotations:
base_domain: example.com
data:
AWS_ACCESS_KEY_ID: <key>
AWS_REGION: <region>
AWS_SECRET_ACCESS_KEY: <key>
stringData:
AWS_ACCESS_KEY_ID: <your-key>
AWS_REGION: <your-region>
AWS_SECRET_ACCESS_KEY: <your-secret>
type: kuadrant.io/aws
---
# You can include multiple resources in the same file
# For example, GCP credentials:
apiVersion: v1
kind: Secret
metadata:
name: gcp-credentials
namespace: kuadrant
annotations:
base_domain: example.com
stringData:
PROJECT_ID: <your-project-id>
GOOGLE: <base64-encoded-service-account-json>
type: kuadrant.io/gcp
```
</details>

Expand Down
64 changes: 64 additions & 0 deletions make/components.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@

##@ Component Deployment (Direct from GitHub)

# Component versions (when not using Helm)
AUTHORINO_OPERATOR_VERSION ?= latest
LIMITADOR_OPERATOR_VERSION ?= latest
DNS_OPERATOR_VERSION ?= latest

# Convert "latest" to "main" for GitHub refs, otherwise use as-is (e.g., v0.13.0)
AUTHORINO_GITREF = $(if $(filter latest,$(AUTHORINO_OPERATOR_VERSION)),main,$(AUTHORINO_OPERATOR_VERSION))
LIMITADOR_GITREF = $(if $(filter latest,$(LIMITADOR_OPERATOR_VERSION)),main,$(LIMITADOR_OPERATOR_VERSION))
DNS_GITREF = $(if $(filter latest,$(DNS_OPERATOR_VERSION)),main,$(DNS_OPERATOR_VERSION))

.PHONY: deploy-authorino-operator
deploy-authorino-operator: ## Deploy Authorino Operator
@echo "Deploying Authorino Operator ($(AUTHORINO_GITREF)) to $(KUADRANT_NAMESPACE)..."
@mkdir -p /tmp/kuadrant-kustomize-authorino
@printf '%s\n' \
'namespace: $(KUADRANT_NAMESPACE)' \
'resources:' \
'- github.com/Kuadrant/authorino-operator/config/deploy?ref=$(AUTHORINO_GITREF)' \
> /tmp/kuadrant-kustomize-authorino/kustomization.yaml
kubectl apply --server-side -k /tmp/kuadrant-kustomize-authorino
@rm -rf /tmp/kuadrant-kustomize-authorino
@echo "Authorino Operator deployed"

.PHONY: deploy-limitador-operator
deploy-limitador-operator: ## Deploy Limitador Operator
@echo "Deploying Limitador Operator ($(LIMITADOR_GITREF)) to $(KUADRANT_NAMESPACE)..."
@mkdir -p /tmp/kuadrant-kustomize-limitador
@printf '%s\n' \
'namespace: $(KUADRANT_NAMESPACE)' \
'resources:' \
'- github.com/Kuadrant/limitador-operator/config/default?ref=$(LIMITADOR_GITREF)' \
> /tmp/kuadrant-kustomize-limitador/kustomization.yaml
kubectl apply --server-side -k /tmp/kuadrant-kustomize-limitador
@rm -rf /tmp/kuadrant-kustomize-limitador
@echo "Limitador Operator deployed"

.PHONY: deploy-dns-operator
deploy-dns-operator: ## Deploy DNS Operator
@echo "Deploying DNS Operator ($(DNS_GITREF)) to $(KUADRANT_NAMESPACE)..."
@mkdir -p /tmp/kuadrant-kustomize-dns
@printf '%s\n' \
'namespace: $(KUADRANT_NAMESPACE)' \
'resources:' \
'- github.com/kuadrant/dns-operator/config/default?ref=$(DNS_GITREF)' \
> /tmp/kuadrant-kustomize-dns/kustomization.yaml
kubectl apply --server-side -k /tmp/kuadrant-kustomize-dns
@rm -rf /tmp/kuadrant-kustomize-dns
@echo "DNS Operator deployed"

.PHONY: deploy-kuadrant-operator-components
deploy-kuadrant-operator-components: ## Deploy Kuadrant Operator from components
kubectl create namespace $(KUADRANT_NAMESPACE) || true
$(MAKE) deploy-authorino-operator
$(MAKE) deploy-limitador-operator
$(MAKE) deploy-dns-operator
@echo "Deploying Kuadrant Operator ($(KUADRANT_OPERATOR_GITREF)) to $(KUADRANT_NAMESPACE)..."
kubectl apply --server-side -k "github.com/kuadrant/kuadrant-operator/config/deploy?ref=$(KUADRANT_OPERATOR_GITREF)"
@echo "Waiting for all operator deployments to be ready..."
kubectl -n $(KUADRANT_NAMESPACE) wait --timeout=$(KUBECTL_TIMEOUT) --for=condition=Available deployments --all
$(MAKE) patch-kuadrant-operator-env
@echo "All operators deployed in $(KUADRANT_NAMESPACE)"
85 changes: 85 additions & 0 deletions make/dependencies.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@

##@ Core Dependencies

.PHONY: install-metrics-server
install-metrics-server: ## Install metrics-server
@echo "Installing metrics-server..."
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl patch deployment metrics-server -n kube-system --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'
@echo "metrics-server installed"

.PHONY: install-metallb
install-metallb: ## Install MetalLB for LoadBalancer services
@echo "Installing MetalLB $(METALLB_VERSION)..."
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/$(METALLB_VERSION)/config/manifests/metallb-native.yaml
kubectl wait --namespace metallb-system --for=condition=Available deployment/controller --timeout=$(METALLB_TIMEOUT)
kubectl wait --namespace metallb-system --for=condition=ready pod --selector=component=controller --timeout=$(METALLB_TIMEOUT)
@echo "Configuring MetalLB IP pool..."
@printf '%s\n' \
'apiVersion: metallb.io/v1beta1' \
'kind: IPAddressPool' \
'metadata:' \
' name: default' \
' namespace: metallb-system' \
'spec:' \
' addresses:' \
' - 172.18.255.200-172.18.255.250' \
| kubectl apply -f -
@printf '%s\n' \
'apiVersion: metallb.io/v1beta1' \
'kind: L2Advertisement' \
'metadata:' \
' name: default' \
' namespace: metallb-system' \
| kubectl apply -f -
@echo "MetalLB installed with IP pool 172.18.255.200-172.18.255.250"

.PHONY: gateway-api-install
gateway-api-install: ## Install Gateway API CRDs
@echo "Installing Gateway API $(GATEWAY_API_VERSION)..."
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/$(GATEWAY_API_VERSION)/standard-install.yaml
@echo "Gateway API CRDs installed"

.PHONY: install-cert-manager
install-cert-manager: ## Install cert-manager
@echo "Installing cert-manager $(CERT_MANAGER_VERSION)..."
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/$(CERT_MANAGER_VERSION)/cert-manager.yaml
kubectl wait --namespace cert-manager --for=condition=Available deployment/cert-manager --timeout=$(CERT_MANAGER_TIMEOUT)
kubectl wait --namespace cert-manager --for=condition=Available deployment/cert-manager-webhook --timeout=$(CERT_MANAGER_TIMEOUT)
kubectl wait --namespace cert-manager --for=condition=ready pod --selector=app.kubernetes.io/name=webhook --timeout=$(CERT_MANAGER_TIMEOUT)
@echo "cert-manager installed"

.PHONY: create-cluster-issuer
create-cluster-issuer: ## Create self-signed ClusterIssuer for TLS testing
@echo "Creating self-signed ClusterIssuer..."
@printf '%s\n' \
'apiVersion: cert-manager.io/v1' \
'kind: ClusterIssuer' \
'metadata:' \
' name: kuadrant-qe-issuer' \
'spec:' \
' selfSigned: {}' \
| kubectl apply -f -
Comment on lines +55 to +62
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will most likely trigger the gitGuardian and reconcilation with them. I think for kind, self-signed should be good enough. What do you reckon?

@echo "ClusterIssuer 'kuadrant-qe-issuer' created"

.PHONY: install-prometheus-crds
install-prometheus-crds: ## Install only Prometheus Operator CRDs (ServiceMonitor, PodMonitor, etc.)
@echo "Installing Prometheus Operator CRDs $(PROMETHEUS_OPERATOR_VERSION)..."
@curl -sL https://github.com/prometheus-operator/prometheus-operator/releases/download/$(PROMETHEUS_OPERATOR_VERSION)/stripped-down-crds.yaml | \
kubectl apply --server-side -f -
@echo "Prometheus CRDs installed"

.PHONY: apply-additional-manifests
apply-additional-manifests: ## Apply additional manifests from file (if ADDITIONAL_MANIFESTS is set)
@if [ -n "$(ADDITIONAL_MANIFESTS)" ]; then \
if [ -f "$(ADDITIONAL_MANIFESTS)" ]; then \
echo "Applying additional manifests from $(ADDITIONAL_MANIFESTS)..."; \
kubectl apply -f "$(ADDITIONAL_MANIFESTS)"; \
echo "Additional manifests applied"; \
else \
echo "❌ Error: ADDITIONAL_MANIFESTS file '$(ADDITIONAL_MANIFESTS)' not found"; \
exit 1; \
fi; \
else \
echo "⏭️ No additional manifests to apply (ADDITIONAL_MANIFESTS not set)"; \
fi
11 changes: 11 additions & 0 deletions make/envoygateway.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@

##@ EnvoyGateway

.PHONY: envoygateway-install
envoygateway-install: ## Install EnvoyGateway
@echo "Installing EnvoyGateway..."
helm install eg oci://docker.io/envoyproxy/gateway-helm --version $(ENVOYGATEWAY_VERSION) \
--create-namespace \
--namespace envoy-gateway-system \
--wait
@echo "EnvoyGateway installed"
30 changes: 30 additions & 0 deletions make/istio.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@

##@ Istio

.PHONY: istio-install
istio-install: ## Install Istio via SAIL operator
@echo "Installing Sail Operator $(SAIL_OPERATOR_VERSION)..."
helm repo add sail-operator https://istio-ecosystem.github.io/sail-operator --force-update
helm install sail-operator \
--create-namespace \
--namespace istio-system \
--wait \
--timeout=$(HELM_TIMEOUT) \
sail-operator/sail-operator \
--version $(SAIL_OPERATOR_VERSION)
@echo "Creating Istio CR..."
@printf '%s\n' \
'apiVersion: sailoperator.io/v1' \
'kind: Istio' \
'metadata:' \
' name: default' \
'spec:' \
' namespace: istio-system' \
' updateStrategy:' \
' type: InPlace' \
' values:' \
' pilot:' \
' autoscaleMin: 2' \
' version: $(ISTIO_VERSION)' \
| kubectl apply -f -
@echo "Istio $(ISTIO_VERSION) installed via SAIL"
12 changes: 12 additions & 0 deletions make/kind.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

##@ Kind Cluster

.PHONY: kind-create-cluster
kind-create-cluster: ## Create kind cluster
@echo "Creating kind cluster '$(KIND_CLUSTER_NAME)'..."
@kind create cluster --name $(KIND_CLUSTER_NAME) || echo "Cluster already exists"

.PHONY: kind-delete-cluster
kind-delete-cluster: ## Delete kind cluster
@echo "Deleting kind cluster '$(KIND_CLUSTER_NAME)'..."
@kind delete cluster --name $(KIND_CLUSTER_NAME) || true
Loading
Loading