Remote Code Invoker Design Doc #37

AlexMaxHorkun · 2018-10-26T15:13:30Z

Design for remote code execution as a part of distributed Magento

antonkril · 2018-10-29T21:15:50Z

design-documents/remote-invoker.md

+* Other application state spoofing may be required for seamless remote execution of services
+### Functional requirements
+* Invoker's code is a part of Magento\Framework namespace
+* Invoker is able to call any class' method


Don't think that such freedom is acceptable

@antonkril Updated the doc

antonkril · 2018-10-29T21:17:06Z

design-documents/remote-invoker.md

+proper deserialization by the caller
+* Remote call response can provide either serialized method call result or serialized exception to be
+rethrow in order to emulate local method call
+* Remote calls are done in a synchronous manner because in order for


Such limitation will introduce misleading abstraction of local call. Also this contradicts the main design document

@antonkril I've added explanations for this issue

I understood your explanation, but the conversation is about making new services asynchronous, not existing ones.

You mean you want our new service contracts look like below and client code always using other services in asynchronous manner?

interface NewEntityRepositoryInterface { public function save(NewEntityInterface $entity): Promise; }

Local implementations would almost always be executed in a synchronous manner and using them would just severely complicate our code. PHP does not support async code execution in it's core, and making some of the code async will most likely force all the client code working with such services be async. I mean look at node.js and it's callback hell, do we want that in Magento?

AlexMaxHorkun · 2018-11-01T14:16:25Z

I have added "Explanations" section to the document where some of the controversial points are being explained

antonkril · 2018-11-05T20:55:05Z

design-documents/remote-invoker.md

+proper deserialization by the caller
+* Remote call response can provide either serialized method call result or serialized exception to be
+rethrow in order to emulate local method call
+* Remote calls are done in a synchronous manner because in order for


I understood your explanation, but the conversation is about making new services asynchronous, not existing ones.

buskamuza · 2018-11-09T14:05:42Z

design-documents/remote-invoker.md

+If the remote call to _ProductRepositoryInterface_ service was done via RESTful (or GraphQL) web API the _incrOrdederedCount_ would have
+to be exposed as Web API and could be requested via HTTP by anyone and independently from context which we cannot afford since we can't just increase the counter without an order happening so only another service responsible for orders may call _incrOrdederedCount_.
+
+If done via the Invoker the _incrOrdederedCount_ wouldn't be exposed to be requested by HTTP clients via web API and wouldn't show up on


There is no difference because Invoker makes the method exposed to all services, if I understand the proposal correctly.
Keep in mind that client-facing Web APIs are not the same as service-facing (inter-network) Web APIs. Only limited list of Web APIs (explicitly declared) will be exposed to the client on the BFF side. All other APIs will be available for services only.

With RPC style communication there would be no need to re-declare our web API endpoints and there would be a clear differentiation between web API endpoints for outside world and inner-network

buskamuza · 2018-11-09T14:11:27Z

design-documents/remote-invoker.md

+If the remote call to _ProductRepositoryInterface::decrOrderedCount_ was done via RESTful web API the exposed endpoint would have
+ACL configured for it so it requires _Catalog edit access_ from admins, but our current admin only has access to _orders_ and the call would fail. However if the Catalog domain modules were not remote and the _ProductRepositoryInterface::decrOrderedCount_ call was made then no ACL validation would occur and we would save the order successsfully.
+
+With Invoker such ACL validations wouldn't occur and the remote call would be completed just as if it was local.


ACL for Web API should be revisited. It's not necessary to introduce another type of APIs (invoker-based) to modify access.
Authentication and authorization can be done on API Gateway level, and in-network calls between services can be done w/o additional validation. Or for better security, there can be additional validation on each service, which will be probably similar to what you propose for the invoker. It depends on how we want to implement it.

buskamuza · 2018-11-09T14:16:56Z

design-documents/remote-invoker.md

+HTTP method/URL pair for external systems - we don't need that when writing a proxy service since we
+would always already have class and method's names, the only other thing existing RESTful Web API mechanism
+provides is it's serializer and it will be reused in the _Remote Code Invoker_
+* When an exception occurs during a service call via RESTful API the response does not contain


Looks more like an issue with error reporting for Web APIs.

not really, clientside and 3rd-party systems wouldn't care for exception classes in responses of RESTful APIs

… remote_code_invoker

melnikovi · 2018-11-12T19:53:38Z

design-documents/remote-invoker.md

+
+Depending on whether _BModule_ or _BModuleProxy_ is deployed on the local node _ServiceBInterface_ will either
+work locally or remotely.
+### Why not just use cURL and existing Web APIs


I was thinking that service mesh will handle retries

melnikovi · 2018-11-12T19:54:23Z

design-documents/remote-invoker.md

+* Multiple remote sources can be configured and assigned unique IDs
+* Each HTTP request to remote sources is signed using unique source's key, timestamp and request's attributes
+* Each remote call must be given unique ID to be used by retries mechanism
+* Timeout limit and retries count can be configured globally and for each call


I think service mesh should handle these things.

service mash is a separate system which is acting as a proxy between inter-service communications and, yes, provides retries and data consistency. I haven't found a mention of such system in Anton's docs and nor do we need such system - they only make sense when different services use different languages/frameworks etc. And we are going to use PHP + Magento on all of our services.

If what you were meaning to say is that we need a separate entity in our codebase to handle retries, data consistency etc - this document is in fact describing such entity (invoker = service mesh)

Service mesh can solve many problems related to deployment and communication for us: service registration, routing, load balancing, circuit breaking, retries, statistics and monitoring, etc. Retries logic is complicated and I would suggest not to implement it ourselves.

melnikovi · 2018-11-14T16:17:58Z

design-documents/remote-invoker.md

+        $promise->then(function (\Magento\ModuleNameApi\Data\ResultDTOInterface $result) use (&$remoteResult) {
+            $remoteResult = $result;
+        });
+        $promise->wait();


I wonder if we need to have different interfaces for synchronous and asynchronous communication. Developer would know how communication happens. And we will not have to wait for result of asynchronous operation, one of the benefits of async operations is that we don't need to wait for the response and it can be performed quickly.

melnikovi · 2018-11-14T16:29:49Z

design-documents/remote-invoker.md

+and in an asynchronous manner for new services to be able to interact with other services with remote invoking in mind
+* Each Magento instance that can be used for remote calls (_Magento server_)
+has it's own unique key for signing requests
+* Remote calls can be disabled in a Magento server


What is Magento server, instance/installation?

melnikovi · 2018-11-14T16:30:24Z

design-documents/remote-invoker.md

+* Each Magento instance that can be used for remote calls (_Magento server_)
+has it's own unique key for signing requests
+* Remote calls can be disabled in a Magento server
+* Authenticated user's information (ID and type) is passed from caller


Will it be added by the remote implementation of the service?

melnikovi · 2018-11-14T16:36:45Z

design-documents/remote-invoker.md

+Response body will contain serialized service call result or a serialized thrown exception encoded in json.
+
+#### Why RESTful web API should not be used as the network gateway for the invoker
+* Not all service contracts are exposed as web API and nor should they be (for security and workflow reasons)


We will have BFF that would expose public API. API of the services can be exposed and will be protected by private network.

buskamuza · 2019-01-03T19:50:42Z

Requires tech vision for inter services communication first.

melnikovi · 2019-02-04T17:42:55Z

Closing as topic is covered in #50.

Remote Code Invoker Design Doc

603f8a7

antonkril suggested changes Oct 31, 2018

View reviewed changes

Updates to the design doc

6f2342d

antonkril suggested changes Nov 5, 2018

View reviewed changes

Oleksandr Gorkun added 2 commits November 7, 2018 18:54

Promises, remote code invoker as an abstraction, explanations

5f46d3f

More detailed examples

dd1d769

buskamuza reviewed Nov 9, 2018

View reviewed changes

buskamuza mentioned this pull request Nov 9, 2018

November 14, 2018 #43

Closed

Oleksandr Gorkun added 4 commits November 12, 2018 15:32

Merge branch 'master' of https://github.com/magento/architecture into…

ddd91aa

… remote_code_invoker

Invoker based on RESTful API

e2e8aac

formatting

bc29193

RESTful APIs

09d6bd5

melnikovi reviewed Nov 12, 2018

View reviewed changes

melnikovi reviewed Nov 14, 2018

View reviewed changes

updaed markdown

28dc65e

melnikovi reviewed Nov 14, 2018

View reviewed changes

melnikovi closed this Feb 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remote Code Invoker Design Doc #37

Remote Code Invoker Design Doc #37

AlexMaxHorkun commented Oct 26, 2018 •

edited

Loading

antonkril Oct 29, 2018

AlexMaxHorkun Nov 1, 2018

antonkril Oct 29, 2018

AlexMaxHorkun Nov 1, 2018

antonkril Nov 5, 2018

AlexMaxHorkun Nov 6, 2018

AlexMaxHorkun commented Nov 1, 2018

antonkril Nov 5, 2018

buskamuza Nov 9, 2018

AlexMaxHorkun Nov 9, 2018

buskamuza Nov 9, 2018

buskamuza Nov 9, 2018

AlexMaxHorkun Nov 9, 2018

melnikovi Nov 12, 2018

melnikovi Nov 12, 2018

AlexMaxHorkun Nov 14, 2018

AlexMaxHorkun Nov 14, 2018

melnikovi Nov 14, 2018

melnikovi Nov 14, 2018

melnikovi Nov 14, 2018

melnikovi Nov 14, 2018

melnikovi Nov 14, 2018

buskamuza commented Jan 3, 2019

melnikovi commented Feb 4, 2019

Remote Code Invoker Design Doc #37

Remote Code Invoker Design Doc #37

Conversation

AlexMaxHorkun commented Oct 26, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexMaxHorkun commented Nov 1, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

buskamuza commented Jan 3, 2019

melnikovi commented Feb 4, 2019

AlexMaxHorkun commented Oct 26, 2018 •

edited

Loading