DamageDetector
/
DamageDetector
2
0
Fork 0
Code Issues Pull Requests Projects Releases Activity

Added Kubernetes files, Service Mash and change the README.md

This commit is contained in:
Jonas Becher 2021-11-06 14:25:10 +01:00
parent fb89d5e5d0
commit 75e0ee1fac
12 changed files with 426 additions and 131 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
LIESMICH.md Normal file
View File

@ -0,0 +1 @@
Deutsche README.md

160
README.md
View File

@ -1,5 +1,5 @@
<p align="center">
<img src="source/logo/aufbruch-fahrrad-logo.png" width="300" alt="Aufbruch Fahrrad" />
<img src="docs/img/aufbruch-fahrrad-logo.png" width="300" alt="Aufbruch Fahrrad" />
</p>
<details>
@ -33,174 +33,72 @@ In addition to the Docker Engine, Docker Compose must also be installed on the t
## Quickstart (Docker)
1.
1. **[Create a Google Cloud Platform project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project)** or use an existing project. Set the `PROJECT_ID` environment variable and ensure the Google Kubernetes Engine and Cloud Operations APIs are enabled.
1. **Clone this repository.**
```
PROJECT_ID="<your-project-id>"
gcloud services enable container.googleapis.com --project ${PROJECT_ID}
gcloud services enable monitoring.googleapis.com \
cloudtrace.googleapis.com \
clouddebugger.googleapis.com \
cloudprofiler.googleapis.com \
--project ${PROJECT_ID}
git clone https://git.masasana.ai/DamageDetector/DamageDetector.git
cd DamageDetector
```
2. **Clone this repository.**
2. **Deploy the app to your local cluster.**
```
git clone https://github.com/GoogleCloudPlatform/microservices-demo.git
cd microservices-demo
bash ./scripts/local_deploy.sh
```
3. **Create a GKE cluster.**
- GKE autopilot mode (see [Autopilot
overview](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview)
to learn more):
3. **Wait for the stack to be ready.**
```
REGION=us-central1
gcloud container clusters create-auto onlineboutique \
--project=${PROJECT_ID} --region=${REGION}
```
- GKE Standard mode:
```
ZONE=us-central1-b
gcloud container clusters create onlineboutique \
--project=${PROJECT_ID} --zone=${ZONE} \
--machine-type=e2-standard-2 --num-nodes=4
```
4. **Deploy the sample app to the cluster.**
```
kubectl apply -f ./release/kubernetes-manifests.yaml
```
5. **Wait for the Pods to be ready.**
```
kubectl get pods
docker stack ps DamageDetector
```
After a few minutes, you should see:
```
NAME READY STATUS RESTARTS AGE
adservice-76bdd69666-ckc5j 1/1 Running 0 2m58s
cartservice-66d497c6b7-dp5jr 1/1 Running 0 2m59s
checkoutservice-666c784bd6-4jd22 1/1 Running 0 3m1s
currencyservice-5d5d496984-4jmd7 1/1 Running 0 2m59s
emailservice-667457d9d6-75jcq 1/1 Running 0 3m2s
frontend-6b8d69b9fb-wjqdg 1/1 Running 0 3m1s
loadgenerator-665b5cd444-gwqdq 1/1 Running 0 3m
paymentservice-68596d6dd6-bf6bv 1/1 Running 0 3m
productcatalogservice-557d474574-888kr 1/1 Running 0 3m
recommendationservice-69c56b74d4-7z8r5 1/1 Running 0 3m1s
redis-cart-5f59546cdd-5jnqf 1/1 Running 0 2m58s
shippingservice-6ccc89f8fd-v686r 1/1 Running 0 2m58s
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
xy5n7qwqrhp6 DamageDetector_mongo.1 mongo:5.0 .3 localhost Running Running 1 minute ago
x8inapxrfsc7 DamageDetector_frontend.1 frontend:latest localhost Running Running 1 minute ago
```
7. **Access the web frontend in a browser** using the frontend's `EXTERNAL_IP`.
5. **Access the web frontend in a browser** using the frontend's `EXTERNAL_IP`.
```
kubectl get service frontend-external | awk '{print $4}'
```
*Example output - do not copy*
```
EXTERNAL-IP
<your-ip>
```
**Note**- you may see `<pending>` while GCP provisions the load balancer. If this happens, wait a few minutes and re-run the command.
8. [Optional] **Clean up**:
```
gcloud container clusters delete onlineboutique \
--project=${PROJECT_ID} --zone=${ZONE}
http://localhost:8080/ui
```
## Other Deployment Options
- **Workload Identity**: [See these instructions.](docs/workload-identity.md)
- **Istio**: [See these instructions.](docs/service-mesh.md)
- **Anthos Service Mesh**: ASM requires Workload Identity to be enabled in your GKE cluster. [See the workload identity instructions](docs/workload-identity.md) to configure and deploy the app. Then, use the [service mesh guide](/docs/service-mesh.md).
- **non-GKE clusters (Minikube, Kind)**: see the [Development Guide](/docs/development-guide.md)
- **Memorystore**: [See these instructions](/docs/memorystore.md) to replace the in-cluster `redis` database with hosted Google Cloud Memorystore (redis).
- **Cymbal Shops Branding**: [See these instructions](/docs/cymbal-shops.md)
- **Minikube**: see the [Development Guide](/docs/development-guide.md)
## Architecture
**Online Boutique** is composed of 11 microservices written in different
languages that talk to each other over gRPC. See the [Development Principles](/docs/development-principles.md) doc for more information.
**DamageDetector** is composed of 2 microservices written in different
languages that talk to each other.
[![Architecture of
microservices](./docs/img/architecture-diagram.png)](./docs/img/architecture-diagram.png)
[![Architecture of docker
microservices](./docs/img/DemageDetector_Docker_Architecture.png)](./docs/img/architecture-diagram.png)
Find **Protocol Buffers Descriptions** at the [`./pb` directory](./pb).
The cloud architecture shown here represents an exemplary network infrastructure that can be adapted by the operator as desired. It makes no claim to completeness or correctness.
[![Architecture of cloud
microservices](./docs/img/DemageDetector_Cloud_Architecture.png)](./docs/img/architecture-diagram.png)
| Service | Language | Description |
| ---------------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| [frontend](./src/frontend) | Go | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. |
| [cartservice](./src/cartservice) | C# | Stores the items in the user's shopping cart in Redis and retrieves it. |
| [productcatalogservice](./src/productcatalogservice) | Go | Provides the list of products from a JSON file and ability to search products and get individual products. |
| [currencyservice](./src/currencyservice) | Node.js | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. |
| [paymentservice](./src/paymentservice) | Node.js | Charges the given credit card info (mock) with the given amount and returns a transaction ID. |
| [shippingservice](./src/shippingservice) | Go | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock) |
| [emailservice](./src/emailservice) | Python | Sends users an order confirmation email (mock). |
| [checkoutservice](./src/checkoutservice) | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. |
| [recommendationservice](./src/recommendationservice) | Python | Recommends other products based on what's given in the cart. |
| [adservice](./src/adservice) | Java | Provides text ads based on given context words. |
| [loadgenerator](./src/loadgenerator) | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. |
| [frontend](./source/frontend) | Python | Exposes an HTTP server to serve the website. Does require basicAuth signup/login to interact with the API |
| [mongodb](./source/mongodb) | C++ | Stores demage notifications from user's. |
## Features
- **[Kubernetes](https://kubernetes.io)/[GKE](https://cloud.google.com/kubernetes-engine/):**
- **[CNCF - Cloud Nativ Computing Foundation](https://www.cncf.io):** This application is developed according to the recommendations of the CNCF.
- **[Kubernetes](https://kubernetes.io)/[Docker](https://www.docker.com):**
The app is designed to run on Kubernetes (both locally on "Docker for
Desktop", as well as on the cloud with GKE).
- **[gRPC](https://grpc.io):** Microservices use a high volume of gRPC calls to
communicate to each other.
Desktop", as well as on the cloud).
- **[Istio](https://istio.io):** Application works on Istio service mesh.
- **[OpenCensus](https://opencensus.io/) Tracing:** Most services are
instrumented using OpenCensus trace interceptors for gRPC/HTTP.
- **[Cloud Operations (Stackdriver)](https://cloud.google.com/products/operations):** Many services
are instrumented with **Profiling**, **Tracing** and **Debugging**. In
addition to these, using Istio enables features like Request/Response
**Metrics** and **Context Graph** out of the box. When it is running out of
Google Cloud, this code path remains inactive.
- **[Skaffold](https://skaffold.dev):** Application
is deployed to Kubernetes with a single command using Skaffold.
- **Synthetic Load Generation:** The application demo comes with a background
job that creates realistic usage patterns on the website using
[Locust](https://locust.io/) load generator.
## Local Development
If you would like to contribute features or fixes to this app, see the [Development Guide](/docs/development-guide.md) on how to build this demo locally.
## Demos featuring Online Boutique
- [From edge to mesh: Exposing service mesh applications through GKE Ingress](https://cloud.google.com/architecture/exposing-service-mesh-apps-through-gke-ingress)
- [Take the first step toward SRE with Cloud Operations Sandbox](https://cloud.google.com/blog/products/operations/on-the-road-to-sre-with-cloud-operations-sandbox)
- [Deploying the Online Boutique sample application on Anthos Service Mesh](https://cloud.google.com/service-mesh/docs/onlineboutique-install-kpt)
- [Anthos Service Mesh Workshop: Lab Guide](https://codelabs.developers.google.com/codelabs/anthos-service-mesh-workshop)
- [KubeCon EU 2019 - Reinventing Networking: A Deep Dive into Istio's Multicluster Gateways - Steve Dake, Independent](https://youtu.be/-t2BfT59zJA?t=982)
- Google Cloud Next'18 SF
- [Day 1 Keynote](https://youtu.be/vJ9OaAqfxo4?t=2416) showing GKE On-Prem
- [Day 3 Keynote](https://youtu.be/JQPOPV_VH5w?t=815) showing Stackdriver
APM (Tracing, Code Search, Profiler, Google Cloud Build)
- [Introduction to Service Management with Istio](https://www.youtube.com/watch?v=wCJrdKdD6UM&feature=youtu.be&t=586)
- [Google Cloud Next'18 London – Keynote](https://youtu.be/nIq2pkNcfEI?t=3071)
showing Stackdriver Incident Response Management
---
This is not an official Google project.

59
docs/development-guide.md Normal file
View File

@ -0,0 +1,59 @@
# Development Guide
This doc explains how to build and run the DamageDetector source code locally.
## Prerequisites
- [Docker for Desktop](https://www.docker.com/products/docker-desktop).
- kubectl installed
- [Minikube](https://minikube.sigs.k8s.io/docs/start/) (optional - see Local Cluster)
## Option 1 - Local Cluster
1. Launch a local Kubernetes cluster with one of the following tools:
- To launch **Minikube** (tested with Ubuntu Linux). Please, ensure that the
local Kubernetes cluster has at least:
- 4 CPUs
- 4.0 GiB memory
- 32 GB disk space
```shell
minikube start --cpus=4 --memory 4096 --disk-size 32g
```
- To launch **Docker for Desktop** (tested with Mac/Windows/Linux). Go to Preferences:
- choose “Enable Kubernetes”,
- set CPUs to at least 3, and Memory to at least 6.0 GiB
- on the "Disk" tab, set at least 32 GB disk space
- To launch a **Kind** cluster:
```shell
kind create cluster
```
2. Run `kubectl get nodes` to verify you're connected to the respective control plane.
3. Run `kubectl get pods` to verify the Pods are ready and running.
4. Access the web frontend through your browser
- **Minikube** requires you to run a command to access the frontend service:
```shell
minikube service frontend-external
```
- **Docker For Desktop** should automatically provide the frontend at http://localhost:80
- **Kind** does not provision an IP address for the service.
You must run a port-forwarding process to access the frontend at http://localhost:8080:
```shell
kubectl port-forward deployment/frontend 8080:8080
```
## Cleanup
If you've deployed the application with `skaffold run` command, you can run
`skaffold delete` to clean up the deployed resources.

BIN
docs/img/DemageDetector_Cloud_Architecture.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 141 KiB

BIN
docs/img/DemageDetector_Docker_Architecture.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 59 KiB

0
source/logo/aufbruch-fahrrad-logo.png → docs/img/aufbruch-fahrrad-logo.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

36
docs/service-mesh.md Normal file
View File

@ -0,0 +1,36 @@
# Deploying to an Istio-enabled cluster
This repository provides an [`istio-manifests`](/istio-manifests) directory containing ingress resources (an Istio `Gateway` and `VirtualService`) needed to expose the app frontend running inside a Kubernetes cluster.
You can apply these resources to your cluster in addition to the `kubernetes-manifests`, then use the Istio IngressGateway's external IP to view the app frontend. See the following instructions for Istio steps.
## Steps
1. [Install Istio](https://istio.io/latest/docs/setup/getting-started/) on your cluster.
2. Enable Istio sidecar proxy injection in the `default` Kubernetes namespace.
```sh
kubectl label namespace default istio-injection=enabled
```
3. Apply all the manifests in the `/release` directory. This includes the Istio and Kubernetes manifests.
```sh
kubectl apply -f ./release
```
4. Run `kubectl get pods` to see pods are in a healthy and ready state.
5. Find the IP address of your Istio gateway Ingress or Service, and visit the
application frontend in a web browser.
```sh
INGRESS_HOST="$(kubectl -n istio-system get service istio-ingressgateway \
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')"
echo "$INGRESS_HOST"
```
```sh
curl -v "http://$INGRESS_HOST"
```

44
istio-manifests/allow-ingress-statsAPI.yaml Normal file
View File

@ -0,0 +1,44 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI
spec:
hosts:
- "your.account.com" # Used to get token
- "*.youtapiserver.com"
ports:
- number: 80
protocol: HTTP
name: http
- number: 443
protocol: HTTPS
name: https
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI-metadata
spec:
hosts:
- metadata.statsAPI.internal
addresses:
- x.x.x.x # your metadata server
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS

42
istio-manifests/frontend-gateway.yaml Normal file
View File

@ -0,0 +1,42 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: frontend-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend-ingress
spec:
hosts:
- "*"
gateways:
- frontend-gateway
http:
- route:
- destination:
host: frontend
port:
number: 80

25
istio-manifests/frontend.yaml Normal file
View File

@ -0,0 +1,25 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend
spec:
hosts:
- "frontend.default.svc.cluster.local"
http:
- route:
- destination:
host: frontend
port:
number: 80

94
release/istio-manifests.yaml Normal file
View File

@ -0,0 +1,94 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------
# WARNING: This file is autogenerated. Do not manually edit.
# ----------------------------------------------------------
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: frontend-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend-ingress
spec:
hosts:
- "*"
gateways:
- frontend-gateway
http:
- route:
- destination:
host: frontend
port:
number: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-egress-googleapis
spec:
hosts:
- "your.account.com" # Used to get token
- "*.statsapi.com"
ports:
- number: 80
protocol: HTTP
name: http
- number: 443
protocol: HTTPS
name: https
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: allow-ingress-statsAPI-metadata
spec:
hosts:
- metadata.statsAPI.internal
addresses:
- x.x.x.x # your metadata server
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontend
spec:
hosts:
- "frontend.default.svc.cluster.local"
http:
- route:
- destination:
host: frontend
port:
number: 80
---

96
release/kubernetes-manifests.yaml Normal file
View File

@ -0,0 +1,96 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------
# WARNING: This file is autogenerated. Do not manually edit.
# ----------------------------------------------------------
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
spec:
serviceAccountName: default
terminationGracePeriodSeconds: 5
containers:
- name: server
image: *insertyourimagehere*
ports:
- containerPort: 8080
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: frontend
spec:
type: ClusterIP
selector:
app: frontend
ports:
- name: http
port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongo
spec:
selector:
matchLabels:
app: mongo
template:
metadata:
labels:
app: mongo
spec:
serviceAccountName: default
containers:
- name: server
image: mongo:5.0.3
ports:
- containerPort: 27017
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: mongo
spec:
type: ClusterIP
selector:
app: mongo
ports:
name: http
port: 27017
targetPort: 27017