Added Kubernetes files, Service Mash and change the README.md

This commit is contained in:
Jonas Becher 2021-11-06 14:25:10 +01:00
parent fb89d5e5d0
commit 75e0ee1fac
12 changed files with 426 additions and 131 deletions

1
LIESMICH.md Normal file
View File

@ -0,0 +1 @@
Deutsche README.md

160
README.md
View File

@ -1,5 +1,5 @@
<p align="center"> <p align="center">
<img src="source/logo/aufbruch-fahrrad-logo.png" width="300" alt="Aufbruch Fahrrad" /> <img src="docs/img/aufbruch-fahrrad-logo.png" width="300" alt="Aufbruch Fahrrad" />
</p> </p>
<details> <details>
@ -33,174 +33,72 @@ In addition to the Docker Engine, Docker Compose must also be installed on the t
## Quickstart (Docker) ## Quickstart (Docker)
1. 1. **Clone this repository.**
1. **[Create a Google Cloud Platform project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project)** or use an existing project. Set the `PROJECT_ID` environment variable and ensure the Google Kubernetes Engine and Cloud Operations APIs are enabled.
``` ```
PROJECT_ID="<your-project-id>" git clone https://git.masasana.ai/DamageDetector/DamageDetector.git
gcloud services enable container.googleapis.com --project ${PROJECT_ID} cd DamageDetector
gcloud services enable monitoring.googleapis.com \
cloudtrace.googleapis.com \
clouddebugger.googleapis.com \
cloudprofiler.googleapis.com \
--project ${PROJECT_ID}
``` ```
2. **Clone this repository.** 2. **Deploy the app to your local cluster.**
``` ```
git clone https://github.com/GoogleCloudPlatform/microservices-demo.git bash ./scripts/local_deploy.sh
cd microservices-demo
``` ```
3. **Create a GKE cluster.** 3. **Wait for the stack to be ready.**
- GKE autopilot mode (see [Autopilot
overview](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview)
to learn more):
``` ```
REGION=us-central1 docker stack ps DamageDetector
gcloud container clusters create-auto onlineboutique \
--project=${PROJECT_ID} --region=${REGION}
```
- GKE Standard mode:
```
ZONE=us-central1-b
gcloud container clusters create onlineboutique \
--project=${PROJECT_ID} --zone=${ZONE} \
--machine-type=e2-standard-2 --num-nodes=4
```
4. **Deploy the sample app to the cluster.**
```
kubectl apply -f ./release/kubernetes-manifests.yaml
```
5. **Wait for the Pods to be ready.**
```
kubectl get pods
``` ```
After a few minutes, you should see: After a few minutes, you should see:
``` ```
NAME READY STATUS RESTARTS AGE ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
adservice-76bdd69666-ckc5j 1/1 Running 0 2m58s xy5n7qwqrhp6 DamageDetector_mongo.1 mongo:5.0 .3 localhost Running Running 1 minute ago
cartservice-66d497c6b7-dp5jr 1/1 Running 0 2m59s x8inapxrfsc7 DamageDetector_frontend.1 frontend:latest localhost Running Running 1 minute ago
checkoutservice-666c784bd6-4jd22 1/1 Running 0 3m1s
currencyservice-5d5d496984-4jmd7 1/1 Running 0 2m59s
emailservice-667457d9d6-75jcq 1/1 Running 0 3m2s
frontend-6b8d69b9fb-wjqdg 1/1 Running 0 3m1s
loadgenerator-665b5cd444-gwqdq 1/1 Running 0 3m
paymentservice-68596d6dd6-bf6bv 1/1 Running 0 3m
productcatalogservice-557d474574-888kr 1/1 Running 0 3m
recommendationservice-69c56b74d4-7z8r5 1/1 Running 0 3m1s
redis-cart-5f59546cdd-5jnqf 1/1 Running 0 2m58s
shippingservice-6ccc89f8fd-v686r 1/1 Running 0 2m58s
``` ```
7. **Access the web frontend in a browser** using the frontend's `EXTERNAL_IP`. 5. **Access the web frontend in a browser** using the frontend's `EXTERNAL_IP`.
``` ```
kubectl get service frontend-external | awk '{print $4}' http://localhost:8080/ui
```
*Example output - do not copy*
```
EXTERNAL-IP
<your-ip>
```
**Note**- you may see `<pending>` while GCP provisions the load balancer. If this happens, wait a few minutes and re-run the command.
8. [Optional] **Clean up**:
```
gcloud container clusters delete onlineboutique \
--project=${PROJECT_ID} --zone=${ZONE}
``` ```
## Other Deployment Options ## Other Deployment Options
- **Workload Identity**: [See these instructions.](docs/workload-identity.md)
- **Istio**: [See these instructions.](docs/service-mesh.md) - **Istio**: [See these instructions.](docs/service-mesh.md)
- **Anthos Service Mesh**: ASM requires Workload Identity to be enabled in your GKE cluster. [See the workload identity instructions](docs/workload-identity.md) to configure and deploy the app. Then, use the [service mesh guide](/docs/service-mesh.md). - **Minikube**: see the [Development Guide](/docs/development-guide.md)
- **non-GKE clusters (Minikube, Kind)**: see the [Development Guide](/docs/development-guide.md)
- **Memorystore**: [See these instructions](/docs/memorystore.md) to replace the in-cluster `redis` database with hosted Google Cloud Memorystore (redis).
- **Cymbal Shops Branding**: [See these instructions](/docs/cymbal-shops.md)
## Architecture ## Architecture
**Online Boutique** is composed of 11 microservices written in different **DamageDetector** is composed of 2 microservices written in different
languages that talk to each other over gRPC. See the [Development Principles](/docs/development-principles.md) doc for more information. languages that talk to each other.
[![Architecture of [![Architecture of docker
microservices](./docs/img/architecture-diagram.png)](./docs/img/architecture-diagram.png) microservices](./docs/img/DemageDetector_Docker_Architecture.png)](./docs/img/architecture-diagram.png)
Find **Protocol Buffers Descriptions** at the [`./pb` directory](./pb).
The cloud architecture shown here represents an exemplary network infrastructure that can be adapted by the operator as desired. It makes no claim to completeness or correctness.
[![Architecture of cloud
microservices](./docs/img/DemageDetector_Cloud_Architecture.png)](./docs/img/architecture-diagram.png)
| Service | Language | Description | | Service | Language | Description |
| ---------------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------- | | ---------------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| [frontend](./src/frontend) | Go | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. | | [frontend](./source/frontend) | Python | Exposes an HTTP server to serve the website. Does require basicAuth signup/login to interact with the API |
| [cartservice](./src/cartservice) | C# | Stores the items in the user's shopping cart in Redis and retrieves it. | | [mongodb](./source/mongodb) | C++ | Stores demage notifications from user's. |
| [productcatalogservice](./src/productcatalogservice) | Go | Provides the list of products from a JSON file and ability to search products and get individual products. |
| [currencyservice](./src/currencyservice) | Node.js | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. |
| [paymentservice](./src/paymentservice) | Node.js | Charges the given credit card info (mock) with the given amount and returns a transaction ID. |
| [shippingservice](./src/shippingservice) | Go | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock) |
| [emailservice](./src/emailservice) | Python | Sends users an order confirmation email (mock). |
| [checkoutservice](./src/checkoutservice) | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. |
| [recommendationservice](./src/recommendationservice) | Python | Recommends other products based on what's given in the cart. |
| [adservice](./src/adservice) | Java | Provides text ads based on given context words. |
| [loadgenerator](./src/loadgenerator) | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. |
## Features ## Features
- **[Kubernetes](https://kubernetes.io)/[GKE](https://cloud.google.com/kubernetes-engine/):** - **[CNCF - Cloud Nativ Computing Foundation](https://www.cncf.io):** This application is developed according to the recommendations of the CNCF.
- **[Kubernetes](https://kubernetes.io)/[Docker](https://www.docker.com):**
The app is designed to run on Kubernetes (both locally on "Docker for The app is designed to run on Kubernetes (both locally on "Docker for
Desktop", as well as on the cloud with GKE). Desktop", as well as on the cloud).
- **[gRPC](https://grpc.io):** Microservices use a high volume of gRPC calls to
communicate to each other.
- **[Istio](https://istio.io):** Application works on Istio service mesh. - **[Istio](https://istio.io):** Application works on Istio service mesh.
- **[OpenCensus](https://opencensus.io/) Tracing:** Most services are
instrumented using OpenCensus trace interceptors for gRPC/HTTP.
- **[Cloud Operations (Stackdriver)](https://cloud.google.com/products/operations):** Many services
are instrumented with **Profiling**, **Tracing** and **Debugging**. In
addition to these, using Istio enables features like Request/Response
**Metrics** and **Context Graph** out of the box. When it is running out of
Google Cloud, this code path remains inactive.
- **[Skaffold](https://skaffold.dev):** Application
is deployed to Kubernetes with a single command using Skaffold.
- **Synthetic Load Generation:** The application demo comes with a background
job that creates realistic usage patterns on the website using
[Locust](https://locust.io/) load generator.
## Local Development ## Local Development
If you would like to contribute features or fixes to this app, see the [Development Guide](/docs/development-guide.md) on how to build this demo locally. If you would like to contribute features or fixes to this app, see the [Development Guide](/docs/development-guide.md) on how to build this demo locally.
## Demos featuring Online Boutique
- [From edge to mesh: Exposing service mesh applications through GKE Ingress](https://cloud.google.com/architecture/exposing-service-mesh-apps-through-gke-ingress)
- [Take the first step toward SRE with Cloud Operations Sandbox](https://cloud.google.com/blog/products/operations/on-the-road-to-sre-with-cloud-operations-sandbox)
- [Deploying the Online Boutique sample application on Anthos Service Mesh](https://cloud.google.com/service-mesh/docs/onlineboutique-install-kpt)
- [Anthos Service Mesh Workshop: Lab Guide](https://codelabs.developers.google.com/codelabs/anthos-service-mesh-workshop)
- [KubeCon EU 2019 - Reinventing Networking: A Deep Dive into Istio's Multicluster Gateways - Steve Dake, Independent](https://youtu.be/-t2BfT59zJA?t=982)
- Google Cloud Next'18 SF
- [Day 1 Keynote](https://youtu.be/vJ9OaAqfxo4?t=2416) showing GKE On-Prem
- [Day 3 Keynote](https://youtu.be/JQPOPV_VH5w?t=815) showing Stackdriver
APM (Tracing, Code Search, Profiler, Google Cloud Build)
- [Introduction to Service Management with Istio](https://www.youtube.com/watch?v=wCJrdKdD6UM&feature=youtu.be&t=586)
- [Google Cloud Next'18 London Keynote](https://youtu.be/nIq2pkNcfEI?t=3071)
showing Stackdriver Incident Response Management
---
This is not an official Google project.

59
docs/development-guide.md Normal file
View File

@ -0,0 +1,59 @@
# Development Guide
This doc explains how to build and run the DamageDetector source code locally.
## Prerequisites
- [Docker for Desktop](https://www.docker.com/products/docker-desktop).
- kubectl installed
- [Minikube](https://minikube.sigs.k8s.io/docs/start/) (optional - see Local Cluster)
## Option 1 - Local Cluster
1. Launch a local Kubernetes cluster with one of the following tools:
- To launch **Minikube** (tested with Ubuntu Linux). Please, ensure that the
local Kubernetes cluster has at least:
- 4 CPUs
- 4.0 GiB memory
- 32 GB disk space
```shell
minikube start --cpus=4 --memory 4096 --disk-size 32g
```
- To launch **Docker for Desktop** (tested with Mac/Windows/Linux). Go to Preferences:
- choose “Enable Kubernetes”,
- set CPUs to at least 3, and Memory to at least 6.0 GiB
- on the "Disk" tab, set at least 32 GB disk space
- To launch a **Kind** cluster:
```shell
kind create cluster
```
2. Run `kubectl get nodes` to verify you're connected to the respective control plane.
3. Run `kubectl get pods` to verify the Pods are ready and running.
4. Access the web frontend through your browser
- **Minikube** requires you to run a command to access the frontend service:
```shell
minikube service frontend-external
```
- **Docker For Desktop** should automatically provide the frontend at http://localhost:80
- **Kind** does not provision an IP address for the service.
You must run a port-forwarding process to access the frontend at http://localhost:8080:
```shell
kubectl port-forward deployment/frontend 8080:8080
```
## Cleanup
If you've deployed the application with `skaffold run` command, you can run
`skaffold delete` to clean up the deployed resources.

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

View File

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

36
docs/service-mesh.md Normal file
View File

@ -0,0 +1,36 @@
# Deploying to an Istio-enabled cluster
This repository provides an [`istio-manifests`](/istio-manifests) directory containing ingress resources (an Istio `Gateway` and `VirtualService`) needed to expose the app frontend running inside a Kubernetes cluster.
You can apply these resources to your cluster in addition to the `kubernetes-manifests`, then use the Istio IngressGateway's external IP to view the app frontend. See the following instructions for Istio steps.
## Steps
1. [Install Istio](https://istio.io/latest/docs/setup/getting-started/) on your cluster.
2. Enable Istio sidecar proxy injection in the `default` Kubernetes namespace.
```sh
kubectl label namespace default istio-injection=enabled
```
3. Apply all the manifests in the `/release` directory. This includes the Istio and Kubernetes manifests.
```sh
kubectl apply -f ./release
```
4. Run `kubectl get pods` to see pods are in a healthy and ready state.
5. Find the IP address of your Istio gateway Ingress or Service, and visit the
application frontend in a web browser.
```sh
INGRESS_HOST="$(kubectl -n istio-system get service istio-ingressgateway \
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')"
echo "$INGRESS_HOST"
```
```sh
curl -v "http://$INGRESS_HOST"
```

View File

@ -0,0 +1,44 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI
spec:
hosts:
- "your.account.com" # Used to get token
- "*.youtapiserver.com"
ports:
- number: 80
protocol: HTTP
name: http
- number: 443
protocol: HTTPS
name: https
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI-metadata
spec:
hosts:
- metadata.statsAPI.internal
addresses:
- x.x.x.x # your metadata server
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS

View File

@ -0,0 +1,42 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: frontend-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend-ingress
spec:
hosts:
- "*"
gateways:
- frontend-gateway
http:
- route:
- destination:
host: frontend
port:
number: 80

View File

@ -0,0 +1,25 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend
spec:
hosts:
- "frontend.default.svc.cluster.local"
http:
- route:
- destination:
host: frontend
port:
number: 80

View File

@ -0,0 +1,94 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------
# WARNING: This file is autogenerated. Do not manually edit.
# ----------------------------------------------------------
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: frontend-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend-ingress
spec:
hosts:
- "*"
gateways:
- frontend-gateway
http:
- route:
- destination:
host: frontend
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-egress-googleapis
spec:
hosts:
- "your.account.com" # Used to get token
- "*.statsapi.com"
ports:
- number: 80
protocol: HTTP
name: http
- number: 443
protocol: HTTPS
name: https
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI-metadata
spec:
hosts:
- metadata.statsAPI.internal
addresses:
- x.x.x.x # your metadata server
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend
spec:
hosts:
- "frontend.default.svc.cluster.local"
http:
- route:
- destination:
host: frontend
port:
number: 80
---

View File

@ -0,0 +1,96 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------
# WARNING: This file is autogenerated. Do not manually edit.
# ----------------------------------------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
spec:
serviceAccountName: default
terminationGracePeriodSeconds: 5
containers:
- name: server
image: *insertyourimagehere*
ports:
- containerPort: 8080
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: frontend
spec:
type: ClusterIP
selector:
app: frontend
ports:
- name: http
port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongo
spec:
selector:
matchLabels:
app: mongo
template:
metadata:
labels:
app: mongo
spec:
serviceAccountName: default
containers:
- name: server
image: mongo:5.0.3
ports:
- containerPort: 27017
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: mongo
spec:
type: ClusterIP
selector:
app: mongo
ports:
name: http
port: 27017
targetPort: 27017