Merge pull request #10 from MetaCell/feature/1

#1 chore: Add portal application

.gitignore

@@ -0,0 +1,4 @@
+.idea
+.vscode
+cloud-harness
+skaffold.yaml

applications/portal/.dockerignore

@@ -0,0 +1,2 @@
+node_modules
+dist

applications/portal/Dockerfile

@@ -0,0 +1,17 @@
+ARG CLOUDHARNESS_DJANGO
+
+FROM $CLOUDHARNESS_DJANGO
+
+WORKDIR ${APP_DIR}
+RUN mkdir -p ${APP_DIR}/static/www
+
+COPY backend/requirements.txt ${APP_DIR}
+RUN pip3 install --no-cache-dir -r requirements.txt
+
+COPY backend/requirements.txt backend/setup.py ${APP_DIR}
+RUN python3 -m pip install -e .
+
+COPY backend ${APP_DIR}
+RUN python3 manage.py collectstatic --noinput
+
+ENTRYPOINT uvicorn --workers ${WORKERS} --host 0.0.0.0 --port ${PORT} main:app

applications/portal/README.md

@@ -0,0 +1,108 @@
+# portal
+
+FastAPI/Django/React-based web application.
+This application is constructed to be deployed inside a cloud-harness Kubernetes.
+It can be also run locally for development and test purpose.
+
+The code is generated with the script `harness-application` and is in part automatically generated 
+from [openapi definition](./api/openapi.yaml).
+
+## Configuration
+
+### Accounts
+
+The CloudHarness Django application template comes with a configuration that can retrieve user account updates from Keycloak (accounts)
+To enable this feature:
+* log in into the accounts admin interface
+* select in the left sidebar Events
+* select the `Config` tab
+* enable "metacell-admin-event-listener" under the `Events Config` - `Event Listeners`
+
+An other option is to enable the "metacell-admin-event-listener" through customizing the Keycloak realm.json from the CloudHarness repository.
+
+## Develop
+
+This application is composed of a FastAPI Django backend and a React frontend.
+
+### Backend
+
+Backend code is inside the *backend* directory.
+See [backend/README.md#Develop]
+
+### Frontend
+
+Backend code is inside the *frontend* directory.
+
+Frontend is by default generated as a React web application, but no constraint about this specific technology.
+
+#### Call the backend apis
+All the api stubs are automatically generated in the [frontend/rest](frontend/rest) directory by `harness-application`
+and `harness-generate`.
+
+#### Update the backend apis from openapi.yaml
+THe backend openapi models and main.py can be updated using the `genapi.sh` from the api folder.
+
+## Local build & run
+
+### Install dependencies 
+1 - Clone cloud-harness into your project root folder 
+
+2 - Install cloud-harness requirements
+```
+cd cloud-harness
+bash install.sh
+```
+
+3 - Install cloud-harness common library
+```
+cd libraries/cloudharness-common
+pip install -e .
+```
+
+4 - Install cloud-harness django library
+```
+cd ../../infrastructure/common-images/cloudharness-django/libraries/cloudharness-django
+pip install -e .
+```
+
+5 - Install cloud-harness fastapi requirements
+```
+cd ../fastapi
+pip install -r requirements.txt
+```
+
+### Prepare backend
+
+Create a Django local superuser account, this you only need to do on initial setup
+```bash
+cd backend
+python3 manage.py migrate # to sync the database with the Django models
+python3 manage.py collectstatic --noinput # to copy all assets to the static folder
+python3 manage.py createsuperuser
+# link the frontend dist to the django static folder, this is only needed once, frontend updates will automatically be applied
+cd static/www
+ln -s ../../../frontend/dist dist
+```
+
+### Build frontend
+
+Compile the frontend
+```bash
+cd frontend
+npm install
+npm run build
+```
+
+### Run backend application
+
+start the FastAPI server
+```bash
+uvicorn --workers 2 --host 0.0.0.0 --port 8000 main:app
+```
+
+
+### Running local with port forwardings to a kubernetes cluster
+When you create port forwards to microservices in your k8s cluster you want to forced your local backend server to initialize
+the AuthService and EventService services.
+This can be done by setting the `KUBERNETES_SERVICE_HOST` environment variable to a dummy or correct k8s service host.
+The `KUBERNETES_SERVICE_HOST` switch will activate the creation of the keycloak client and client roles of this microservice.

applications/portal/api/config.json

@@ -0,0 +1,3 @@
+{
+  "packageName": "portal"
+}

applications/portal/api/genapi.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+fastapi-codegen --input openapi.yaml --output app -t templates && mv app/main.py ../backend/ && mv app/models.py ../backend/openapi/
+rm -rf app
+
+echo Generated new models and main.py

applications/portal/api/openapi.yaml

@@ -0,0 +1,61 @@
+components:
+  schemas:
+    Valid:
+      properties:
+        response: {type: string}
+      type: object
+  securitySchemes:
+    bearerAuth: {bearerFormat: JWT, scheme: bearer, type: http, x-bearerInfoFunc: cloudharness.auth.decode_token}
+info:
+  contact: {email: [email protected]}
+  description: portal
+  license: {name: UNLICENSED}
+  title: portal
+  version: 0.1.0
+openapi: 3.0.0
+paths:
+  /live:
+    get:
+      operationId: live
+      responses:
+        '200':
+          content:
+            application/json:
+              schema: {type: string}
+          description: Healthy
+        '500': {description: Application is not healthy}
+      security: []
+      summary: Test if application is healthy
+      tags: [test]
+  /ping:
+    get:
+      operationId: ping
+      responses:
+        '200':
+          content:
+            application/json:
+              schema: {type: string}
+          description: What we want
+        '500': {description: This shouldn't happen}
+      security: []
+      summary: Test the application is up
+      tags: [test]
+  /ready:
+    get:
+      operationId: ready
+      responses:
+        '200':
+          content:
+            application/json:
+              schema: {type: string}
+          description: Ready
+        '500': {description: Application is not ready}
+      security: []
+      summary: Test if application is ready
+      tags: [test]
+security:
+- bearerAuth: []
+servers:
+- {url: /api}
+tags:
+- {name: test}

applications/portal/api/templates/main.jinja2

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import os
+
+from fastapi import APIRouter, Depends, FastAPI, HTTPException, Request
+from fastapi.security import HTTPBearer, HTTPBasicCredentials
+
+from django.conf import settings
+from django.apps import apps
+from django.core.asgi import get_asgi_application
+
+from starlette.middleware.cors import CORSMiddleware
+from fastapi.staticfiles import StaticFiles
+
+{{imports | replace(".","openapi.")}}
+
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "portal.settings")
+apps.populate(settings.INSTALLED_APPS)
+
+# migrate the Django models
+os.system("python manage.py migrate")
+
+from api.controllers import *
+
+app = FastAPI(
+    {% if info %}
+    {% for key,value in info.items() %}
+    {% if key != 'servers' %}
+    {{ key }} = {% if value is string %}"{{ value }}"{% else %}{{ value }}{% endif %},
+    {% endif %}
+    {% endfor %}
+    {% endif %}
+    debug=settings.DEBUG
+    )
+
+{% if info %}
+{% for key,value in info.items() %}
+{% if key == 'servers' %}
+prefix_router = APIRouter(prefix="{{value[0].url}}")
+{% endif %}
+{% endfor %}
+{% endif %}
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=settings.ALLOWED_HOSTS or ["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+from cloudharness.middleware import set_authentication_token
+@app.middleware("http")
+async def add_process_time_header(request: Request, call_next):
+    # retrieve the bearer token from the header
+    # and save it for use in the AuthClient
+    authorization = request.headers.get('Authorization')
+    if authorization:
+        set_authentication_token(authorization)
+
+    return await call_next(request)
+
+if os.environ.get('KUBERNETES_SERVICE_HOST', None):
+    # init the auth service when running in/for k8s
+    from cloudharness_django.services import init_services, get_auth_service
+    init_services()
+    # start the kafka event listener when running in/for k8s
+    import cloudharness_django.services.events
+
+# enable the Bearer Authentication
+security = HTTPBearer()
+
+async def has_access(credentials: HTTPBasicCredentials = Depends(security)):
+    """
+    Function that is used to validate the token in the case that it requires it
+    """
+    if not os.environ.get('KUBERNETES_SERVICE_HOST', None):
+        return {}
+    token = credentials.credentials
+
+    try:
+        payload = get_auth_service().get_auth_client().decode_token(token)
+    except Exception as e:  # catches any exception
+        raise HTTPException(
+            status_code=401,
+        )
+
+PROTECTED = [Depends(has_access)]
+
+# Operations
+
+{% for operation in operations %}
+@prefix_router.{{operation.type}}('{{operation.snake_case_path}}', response_model={{operation.response}}, tags={{operation["tags"]}}{% if operation.security %}, dependencies=PROTECTED{% endif %})
+def {{operation.function_name}}({{operation.snake_case_arguments}}) -> {{operation.response}}:
+    {%- if operation.summary %}
+    """
+    {{ operation.summary }}
+    """
+    {%- endif %}
+    {% if operation["tags"] -%}
+    return {{operation["tags"][0]}}_controller.{{operation.function_name}}(
+    {% else %}
+    return api_controller.{{operation.function_name}}(
+    {% endif %}
+    {% for params in operation.snake_case_arguments.split(",") -%}
+    {% if params and ':' in params %}
+        {{params.split(":")[0]}}={{params.split(":")[0]}},
+    {% endif %}
+    {% endfor -%})
+{% endfor %}
+
+
+app.include_router(prefix_router)
+
+app.mount("/static", StaticFiles(directory=settings.STATIC_ROOT), name="static")
+app.mount("/media", StaticFiles(directory=settings.MEDIA_ROOT), name="media")
+app.mount("/", get_asgi_application())

applications/portal/backend/README.md

@@ -0,0 +1,22 @@
+# OpenAPI generated server
+
+## Overview
+This server was generated by the [OpenAPI Generator](https://openapi-generator.tech) project. By using the
+[OpenAPI-Spec](https://openapis.org) from a remote server, you can easily generate a server stub.  This
+is an example of building a OpenAPI-enabled Django FastAPI server.
+
+This example uses the [Django](https://www.djangoproject.com/) library on top of [FastAPI](https://fastapi.tiangolo.com/).
+
+## Requirements
+Python >= 3
+
+## Local backend development
+```
+# store the accounts api admin password on the local disk
+
+mkdir -p /opt/cloudharness/resources/auth/
+kubectl -n sckan get secrets accounts -o yaml|grep api_user_password|cut -d " " -f 4|base64 -d > /opt/cloudharness/resources/auth/api_user_password
+
+# Make the cloudharness application configuration available on your local machine
+cp deployment/helm/values.yaml /opt/cloudharness/resources/allvalues.yaml
+```

applications/portal/backend/api/admin.py

@@ -0,0 +1,3 @@
+from django.contrib import admin
+
+# Register your models here.

applications/portal/backend/api/apps.py

@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class ApiConfig(AppConfig):
+    default_auto_field = 'django.db.models.BigAutoField'
+    name = 'api'

applications/portal/backend/api/controllers/__init__.py

@@ -0,0 +1 @@
+import api.controllers.test as test_controller

applications/portal/backend/api/controllers/test.py

@@ -0,0 +1,31 @@
+def live():  # noqa: E501
+    """Test if application is healthy
+
+     # noqa: E501
+
+
+    :rtype: str
+    """
+    return "I'm alive!"
+
+
+def ping():  # noqa: E501
+    """test the application is up
+
+     # noqa: E501
+
+
+    :rtype: str
+    """
+    return "Ping!"
+
+
+def ready():  # noqa: E501
+    """Test if application is ready to take requests
+
+     # noqa: E501
+
+
+    :rtype: str
+    """
+    return "I'm READY!"