From 969caa75659fe93b3fae2d9c3e88005657d80ec1 Mon Sep 17 00:00:00 2001 From: Matthew Steffen Date: Fri, 7 Jan 2022 00:58:33 -0800 Subject: [PATCH] Add CONTRIBUTING.md and link to it from README.md --- CONTRIBUTING.md | 145 ++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 3 + 2 files changed, 148 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..f32beac --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,145 @@ +# Operator Architecture + +The operator reads a Kubernetes Custom Resource (defined in ???) and uses it to instantiate Pachyderm's helm chart (in ???) to get a Kubernetes manifest, which it then creates. It's based on a kubebuilder template, and many of its files are generated. + +# Repo Structure + +**TODO**: Comment on each of these directories, and any files of interest (remove files where possible. Many of these are generated by Kubebuilder) + +. +├── api +│   └── v1beta1 +│   ├── groupversion_info.go +│   ├── pachydermexport_types.go +│   ├── pachyderm_types.go +│   ├── pachyderm_webhook.go +│   ├── webhook_suite_test.go +│   └── zz_generated.deepcopy.go +├── bundle +│   ├── manifests +│   │   ├── aiml.pachyderm.com_pachydermexports.yaml +│   │   ├── aiml.pachyderm.com_pachyderms.yaml +│   │   ├── pachyderm-operator.clusterserviceversion.yaml +│   │   ├── pachyderm-operator-controller-manager-metrics-service_v1_service.yaml +│   │   ├── pachyderm-operator-manager-config_v1_configmap.yaml +│   │   ├── pachyderm-operator-metrics-reader_rbac.authorization.k8s.io_v1_clusterrole.yaml +│   │   └── pachyderm-operator-webhook-service_v1_service.yaml +│   ├── metadata +│   │   └── annotations.yaml +│   └── tests +│   └── scorecard +│   └── config.yaml +├── bundle.Dockerfile +├── config +│   ├── certmanager +│   │   ├── certificate.yaml +│   │   ├── kustomization.yaml +│   │   └── kustomizeconfig.yaml +│   ├── crd +│   │   ├── bases +│   │   │   ├── aiml.pachyderm.com_pachydermexports.yaml +│   │   │   └── aiml.pachyderm.com_pachyderms.yaml +│   │   ├── kustomization.yaml +│   │   ├── kustomizeconfig.yaml +│   │   └── patches +│   │   ├── cainjection_in_pachydermexports.yaml +│   │   ├── cainjection_in_pachyderms.yaml +│   │   ├── webhook_in_pachydermexports.yaml +│   │   └── webhook_in_pachyderms.yaml +│   ├── default +│   │   ├── kustomization.yaml +│   │   ├── manager_auth_proxy_patch.yaml +│   │   ├── manager_config_patch.yaml +│   │   ├── manager_webhook_patch.yaml +│   │   └── webhookcainjection_patch.yaml +│   ├── manager +│   │   ├── controller_manager_config.yaml +│   │   ├── kustomization.yaml +│   │   └── manager.yaml +│   ├── manifests +│   │   ├── bases +│   │   │   └── pachyderm-operator.clusterserviceversion.yaml +│   │   └── kustomization.yaml +│   ├── prometheus +│   │   ├── kustomization.yaml +│   │   └── monitor.yaml +│   ├── rbac +│   │   ├── auth_proxy_client_clusterrole.yaml +│   │   ├── auth_proxy_role_binding.yaml +│   │   ├── auth_proxy_role.yaml +│   │   ├── auth_proxy_service.yaml +│   │   ├── kustomization.yaml +│   │   ├── leader_election_role_binding.yaml +│   │   ├── leader_election_role.yaml +│   │   ├── pachyderm_editor_role.yaml +│   │   ├── pachydermexport_editor_role.yaml +│   │   ├── pachydermexport_viewer_role.yaml +│   │   ├── pachyderm_viewer_role.yaml +│   │   ├── role_binding.yaml +│   │   ├── role.yaml +│   │   └── service_account.yaml +│   ├── samples +│   │   ├── aiml_v1beta1_pachydermexport.yaml +│   │   ├── aiml_v1beta1_pachyderm.yaml +│   │   └── kustomization.yaml +│   ├── scorecard +│   │   ├── bases +│   │   │   └── config.yaml +│   │   ├── kustomization.yaml +│   │   └── patches +│   │   ├── basic.config.yaml +│   │   └── olm.config.yaml +│   └── webhook +│   ├── kustomization.yaml +│   ├── kustomizeconfig.yaml +│   ├── manifests.yaml +│   └── service.yaml +├── CONTRIBUTING.md +├── controllers +│   ├── backup_job.go +│   ├── diff.go +│   ├── errors.go +│   ├── generators +│   │   ├── images.go +│   │   ├── manifests.go +│   │   └── template.go +│   ├── initialize_database.go +│   ├── pachyderm_controller.go +│   ├── pachydermexport_controller.go +│   └── suite_test.go +├── Dockerfile +├── go.mod +├── go.sum +├── hack +│   ├── boilerplate.go.txt +│   ├── bundle_prep.sh +│   ├── catalogsource.yaml +│   └── charts +│   ├── 2.0.0 +│   │   ├── images.json +│   │   ├── pachyderm-2.0.0-rc.1.tgz +│   │   └── values.yaml +│   └── 2.0.2 +│   ├── images.json +│   ├── pachyderm-2.0.2.tgz +│   └── values.yaml +├── main.go +└── PROJECT + +# Testing the Operator + +**TODO**: Fill this in. Open Questions: +- How are operators installed in a running Kubernetes cluster? +- Is it necessary to install OLM or load the operator into OperatorHub somehow? +- Does the operator need to be tested in an OpenShift cluster? Does this mean that we need to provision an AWS cluster to run tests or can we use Minikube/KinD somehow, for local testing? Could we run automated tests in CircleCI? + +# Releasing the Operator + +**TODO** Fill this in. Specifically, explain how to: +- Upload new Pachyderm and Worker images to our project in the Red Hat Partner Connect portal + - Also explain how to add people to this project, if needed +- Update the operator itself in Red Hat Marketplace (does it need to be updated anywhere else? ODH?) +- Update the Red Hat Marketplace documentation for Pachyderm +- Update the Open Data Hub (ODH) docs. These docs are also used for Pachyderm in RHODS. + + diff --git a/README.md b/README.md index 03a86b2..9b32af2 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,9 @@ The pachyderm operator is an application responsible for installing, watching and managing Pachyderm resources in your Openshift cluster. Pachyderm is the data foundation for machine learning. Pachyderm provides industry leading data versioning, pipelines and lineage that allow data science teams to automate the machine learning lifecycle and optimize their machine learning operations (MLOps). + +To lean about the structure of the operator's architecture, this repo's code, how to test the operator, or how to release the operator, see [CONTRIBUTING.md](CONTRIBUTING.md) + ## Usage **1. Using AWS S3 for Pachd storage**