Agent prefill (#402)

* Squashed commit of the following:

commit c4a6556
Merge: 5b25b6a c4fef70
Author: Judith <[email protected]>
Date:   Wed Apr 6 16:02:16 2022 +0200

    Merge branch 'development' into 372-agents-json

commit 5b25b6a
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 15:09:15 2022 +0200

    Updated comment

commit 0433736
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 13:43:15 2022 +0200

    Recommerce :)

commit 53b88d9
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 10:20:27 2022 +0200

    Added migration

commit 5742f71
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 09:42:27 2022 +0200

    Added comment

commit 2b2536e
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 09:24:51 2022 +0200

    Updated docstring

commit 4e072d2
Author: NikkelM <[email protected]>
Date:   Wed Apr 6 09:23:14 2022 +0200

    Small refactoring

commit 4e8f7d4
Merge: 0768372 545825a
Author: Nikkel Mollenhauer <[email protected]>
Date:   Wed Apr 6 09:20:41 2022 +0200

    Merge branch 'development' into 372-agents-json

commit 0768372
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 18:11:54 2022 +0200

    Fixed tests

commit ac6fa2e
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 17:52:35 2022 +0200

    Fixed merge for agents

commit 76a01e4
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 17:36:41 2022 +0200

    Fixed typo

commit ceeae81
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 17:36:12 2022 +0200

    Adapted to new agents-format, upload still not working

commit 39a33dd
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 16:33:40 2022 +0200

    Adapted to new agents-format in json

commit d72da1f
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 16:09:19 2022 +0200

    Adapted to new agents-config format in json

commit 673cdb4
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 16:03:52 2022 +0200

    Fixed tests

commit b272fe8
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 15:51:54 2022 +0200

    Adapted to new agent-structure in json

commit f4fb7c6
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 15:39:38 2022 +0200

    Adapted to new agent format in json

commit 007f7d4
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 14:32:47 2022 +0200

    changed test-data

commit e8e4cc6
Author: NikkelM <[email protected]>
Date:   Tue Apr 5 14:29:56 2022 +0200

    changed jsons to new format

* check button

* prefill for agents works

* Squashed commit of the following:

commit 70bc36d
Author: Nikkel Mollenhauer <[email protected]>
Date:   Thu Apr 7 10:39:56 2022 +0200

    Include API status for docker, make docker API more failsafe (#401)

    * stop polling! on click api health only

    * Added differentiation between API unavailable and docker unavailable

    * api health button with checked time

    * Removed unnecessary print statement

    Co-authored-by: Judith <[email protected]>

commit 01da9ac
Author: Nikkel Mollenhauer <[email protected]>
Date:   Wed Apr 6 16:52:57 2022 +0200

    Refactored agents-key in configuration files (#396)

    * changed jsons to new format

    * changed test-data

    * Adapted to new agent format in json

    * Adapted to new agent-structure in json

    * Fixed tests

    * Adapted to new agents-config format in json

    * Adapted to new agents-format in json

    * Adapted to new agents-format, upload still not working

    * Fixed typo

    * Fixed merge for agents

    * Fixed tests

    * Small refactoring

    * Updated docstring

    * Added comment

    * Added migration

    * Recommerce :)

    * Updated comment

    * Changed interval for updating API indicator

    Co-authored-by: Judith <[email protected]>

* notice field for check button

* footer

* remove debug code

* Updated some texts

* Fixed wrong access to agent-class

* Reverted type of comparison check

* only one notice field

* fixed template error

* fix more template error

* automatic template indentation

Co-authored-by: NikkelM <[email protected]>

.pre-commit-config.yaml

@@ -44,3 +44,10 @@ repos:
       args: [--config=pyproject.toml]
       # Use this line instead if you want to update the badge locally
       # args: [--config=pyproject.toml, --generate-badge=./badges/docstring_coverage.svg, --badge-style=flat]
+  - repo: https://github.com/rtts/djhtml
+    rev: 'v1.5.0'  # replace with the latest tag on GitHub
+    hooks:
+      - id: djhtml
+        # Indent only HTML files in template directories
+        files: .*/templates/.*\.html$
+

recommerce/configuration/environment_config.py

@@ -130,7 +130,7 @@ def check_types(cls, config: dict, task: str = 'None', single_agent: bool = Fals
 						try:
 							get_class(agent['agent_class'])
 						except Exception as error:
-							raise AssertionError(f'This agent could not be parsed to a valid class: "{config["agents"][agent]["agent_class"]}"') from error
+							raise AssertionError(f'This agent could not be parsed to a valid class: "{agent["agent_class"]}"') from error
 				# make sure the marketplace class can be parsed/is valid
 				elif key == 'marketplace':
 					try:

webserver/alpha_business_app/buttons.py

@@ -269,6 +269,9 @@ def _pre_fill(self) -> HttpResponse:
 			return self._decide_rendering()
 		merger = ConfigMerger()
 		final_dict, error_dict = merger.merge_config_objects(post_request['config_id'])
+		# set an id for each agent (necessary for view)
+		for agent_index in range(len(final_dict['environment']['agents'])):
+			final_dict['environment']['agents'][agent_index]['id'] = agent_index
 		return render(self.request, self.view_to_render,
 			{'prefill': final_dict, 'error_dict': error_dict, 'all_configurations': Config.objects.all(), **self._params_for_selection()})
 

webserver/alpha_business_app/static/css/buttons.css

@@ -86,3 +86,17 @@
 .bc-xs {
 	height: min-content !important;
 }
+
+.footer {
+	position: relative;
+	bottom: 0;
+	width: 100%;
+	/* Set the fixed height of the footer here */
+	height: 60px;
+	line-height: 20px; /* Vertically center the text there */
+	background-color: #ffffff;
+}
+
+.bc-footer-item {
+	color: rgba(0, 0, 0, 0.5)
+}

webserver/alpha_business_app/static/js/custom.js

@@ -9,7 +9,6 @@ $(document).ready(function() {
 		});
 	});
 
-
 	function updateAPIHealth() {
 		// replaces the element by the element returned by ajax (html) and adds this click event to it
 		var statusButton = $("button.replace-me")
@@ -36,4 +35,41 @@ $(document).ready(function() {
 			});
 		}
 	}).trigger('change');
+
+	function getCookie(name) {
+		let cookieValue = null;
+		if (document.cookie && document.cookie !== '') {
+			const cookies = document.cookie.split(';');
+			for (let i = 0; i < cookies.length; i++) {
+				const cookie = cookies[i].trim();
+				// Does this cookie string begin with the name we want?
+				if (cookie.substring(0, name.length + 1) === (name + '=')) {
+					cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
+					break;
+				}
+			}
+		}
+		return cookieValue;
+	}
+
+	$("button.form-check").click(function () {
+		$("table.config-status-display").remove();
+
+		var self = $(this);
+		var form = $("form.config-form");
+		var formdata = form.serializeArray();
+
+		const csrftoken = getCookie('csrftoken');
+		$.ajax({
+			type: "POST",
+			url: self.data("url"),
+			data: {
+				csrfmiddlewaretoken: csrftoken,
+				formdata
+			},
+			success: function (data) {
+				$("p.notice-field").replaceWith(data);
+			}
+		});
+	});
 });

webserver/alpha_business_app/urls.py

@@ -12,5 +12,6 @@
 	path('delete_config/<int:config_id>', views.delete_config, name='delete_config'),
 	# AJAX relevant url's
 	path('agent', views.agent, name='agent'),
-	path('api_availability', views.api_availability, name='api_availability')
+	path('api_availability', views.api_availability, name='api_availability'),
+	path('validate_config', views.config_validation, name='config_validation')
 ]

webserver/alpha_business_app/views.py

@@ -5,7 +5,10 @@
 from django.http import Http404, HttpResponse
 from django.shortcuts import render
 
+from recommerce.configuration.config_validation import validate_config
+
 from .buttons import ButtonHandler
+from .config_parser import ConfigFlatDictParser
 from .forms import UploadFileForm
 from .handle_files import handle_uploaded_file
 from .handle_requests import DOCKER_API
@@ -80,3 +83,28 @@ def api_availability(request):
 	if api_is_available.status_code == 200:
 		return render(request, 'api_buttons/api_health_button.html', {'api_success': f'API available - {current_time}'})
 	return render(request, 'api_buttons/api_health_button.html', {'api_docker_timeout': f'Docker  unavailable - {current_time}'})
+
+
+def config_validation(request):
+	if request.method == 'POST':
+		post_request = request.POST
+		# convert formdata dict to normal form dict
+		resulting_dict = {
+			'environment-agents-name': [],
+			'environment-agents-agent_class': [],
+			'environment-agents-argument': []
+		}
+		for index in range(len(post_request) // 2):
+			current_name = post_request[f'formdata[{index}][name]']
+			current_value = post_request[f'formdata[{index}][value]']
+			if 'agents' in current_name:
+				resulting_dict[current_name] += [current_value]
+			else:
+				resulting_dict[current_name] = [current_value]
+
+		config_dict = ConfigFlatDictParser().flat_dict_to_hierarchical_config_dict(resulting_dict)
+
+		validate_status, validate_data = validate_config(config=config_dict, config_is_final=True)
+		if not validate_status:
+			return render(request, 'notice_field.html', {'error': validate_data})
+	return render(request, 'notice_field.html', {'success': 'This config is valid'})

webserver/templates/api_buttons/api_health_button.html

@@ -1,9 +1,9 @@
-<button class="btn btn-sm mt-1 bc-xs replace-me {% if api_success %}bc-outline-lightgreen{% elif api_docker_timeout %}bc-outline-orange{% elif api_timeout %}bc-outline-red{% else %}bc-outline-lightgray{% endif %}" 
-	type="button"
-	data-url="/api_availability">
-	{% if api_success or api_docker_timeout or api_timeout %}
-		{{api_success}}{{api_docker_timeout}}{{api_timeout}}
-	{% else %}
-		unknown
-	{% endif %}
+<button class="btn btn-sm mt-1 bc-xs replace-me {% if api_success %}bc-outline-lightgreen{% elif api_docker_timeout %}bc-outline-orange{% elif api_timeout %}bc-outline-red{% else %}bc-outline-lightgray{% endif %}"
+    type="button"
+    data-url="/api_availability">
+    {% if api_success or api_docker_timeout or api_timeout %}
+        {{api_success}}{{api_docker_timeout}}{{api_timeout}}
+    {% else %}
+        unknown
+    {% endif %}
 </button>

webserver/templates/api_buttons/delete_button.html

@@ -1,6 +1,6 @@
 <form method="post" enctype="multipart/form-data">
-	{% csrf_token %}
-	<input type="hidden" name="action", value="delete">
-	<button class="btn btn-outline-danger btn-sm" title="deletes all data we stored about the container"
-		name="container_id" value="{{ container.id }}" type="submit">delete</button>
+    {% csrf_token %}
+    <input type="hidden" name="action", value="delete">
+    <button class="btn btn-outline-danger btn-sm" title="deletes all data we stored about the container"
+        name="container_id" value="{{ container.id }}" type="submit">delete</button>
 </form>

webserver/templates/api_buttons/download_button.html

@@ -1,32 +1,32 @@
 <div class="d-flex">
-	<form method="post" enctype="multipart/form-data">
-		{% csrf_token %}
-		<input type="hidden" name="action" value="data">
-		<input type="hidden" name="file_type" value="zip">
-		<button class="btn btn-sm mx-auto {% if container.is_archived%} btn-outline-dark {% else %} bc-outline-download{% endif %}"
-			name="container_id" value="{{ container.id }}"
-			title="downloads data from running container as zip"
-			type="{% if container.is_archived%}button{% else %}submit{% endif %}">
-				{% if container.is_archived%}	
-					not available
-				{% else %}
-					zip
-				{% endif %}
-		</button>
-	</form>
-	<form method="post" enctype="multipart/form-data">
-		{% csrf_token %}
-		<input type="hidden" name="action" value="data">
-		<input type="hidden" name="file_type" value="tar">
-		<button class="btn btn-sm {% if container.is_archived%} btn-outline-dark {% else %}bc-outline-download{% endif %}"
-			name="container_id" value="{{ container.id }}"
-			title="downloads data from running container as tar"
-			type="{% if container.is_archived%}button{% else %}submit{% endif %}">
-				{% if container.is_archived%}	
-					not available
-				{% else %}
-					tar
-				{% endif %}
-		</button>
-	</form>
+    <form method="post" enctype="multipart/form-data">
+        {% csrf_token %}
+        <input type="hidden" name="action" value="data">
+        <input type="hidden" name="file_type" value="zip">
+        <button class="btn btn-sm mx-auto {% if container.is_archived%} btn-outline-dark {% else %} bc-outline-download{% endif %}"
+            name="container_id" value="{{ container.id }}"
+            title="downloads data from running container as zip"
+            type="{% if container.is_archived%}button{% else %}submit{% endif %}">
+            {% if container.is_archived%}
+                not available
+            {% else %}
+                zip
+            {% endif %}
+        </button>
+    </form>
+    <form method="post" enctype="multipart/form-data">
+        {% csrf_token %}
+        <input type="hidden" name="action" value="data">
+        <input type="hidden" name="file_type" value="tar">
+        <button class="btn btn-sm {% if container.is_archived%} btn-outline-dark {% else %}bc-outline-download{% endif %}"
+            name="container_id" value="{{ container.id }}"
+            title="downloads data from running container as tar"
+            type="{% if container.is_archived%}button{% else %}submit{% endif %}">
+            {% if container.is_archived%}
+                not available
+            {% else %}
+                tar
+            {% endif %}
+        </button>
+    </form>
 </div>

webserver/templates/api_buttons/health_button.html

@@ -1,7 +1,7 @@
 <form method="post" enctype="multipart/form-data">
-	{% csrf_token %}
-	<input type="hidden" name="action", value="health">
-	<button class="btn btn-sm bc-outline-lightgreen" 
-		title="check health of your container"
-		name="container_id" value="{{ container.id }}" type="submit">check health</button>
+    {% csrf_token %}
+    <input type="hidden" name="action", value="health">
+    <button class="btn btn-sm bc-outline-lightgreen"
+        title="check health of your container"
+        name="container_id" value="{{ container.id }}" type="submit">check health</button>
 </form>

webserver/templates/api_buttons/logs_button.html

@@ -1,9 +1,9 @@
 <form method="post" enctype="multipart/form-data">
-	{% csrf_token %}
-	<input type="hidden" name="action", value="logs">
-	<button class="btn btn-sm bc-outline-logs" 
-		name="container_id"
-		value="{{ container.id }}"
-		title="gets the logs of your container"
-		type="submit">get logs</button>
+    {% csrf_token %}
+    <input type="hidden" name="action", value="logs">
+    <button class="btn btn-sm bc-outline-logs"
+        name="container_id"
+        value="{{ container.id }}"
+        title="gets the logs of your container"
+        type="submit">get logs</button>
 </form>