-
Notifications
You must be signed in to change notification settings - Fork 2
Design choices
- For most topics in this document different issues are being created to allow discussions for achieving optimal design choices. This document is subject of regular updates based on those discussions within corespondent issues
- All headers/chapters having attached a link to it will point you to the right issue. If any content needs a separate topic of disscusion, a link labeled as 'Comments at' will be provided
- we need a list of all available pairs and their state (liquidity / price)
- we need to listen to dex transactions and update our pair state accordingly
Q(1): what part of pair's state we need to persist (allowances,balances, etc) ?
- we need to listen to the factory when new pairs are created
- we need an endpoint where i can pass a pair and it returns me all pairs that could be swapped in two or less pairs (A -> WAE -> B)
- we need an endpoint that lists all tokens that are swappable
From the requirements topic it's easily to understand that we need at least one hosted agent which listens for some changes in the relevant contracts (Factory/Router/Pairs) and updates the Dex-backend state.
Q(2.1): How do we listen? Are we going to scan the middleware?
Based on the behaviour the worker might have at initialisation step, at least two different approaches we can identify:
(1) We create a resynchronisation process to achieve a one-to-one state with the actual state of the contracts.
Q(2.2): How do we get that that synced state? Are we going to make dry-runs over the contracts, or to query the middleware from the point when the agent shut-downed ?
OR
(2)
- We are erasing the state
- We await any calls for certain informations (get pair info, get allowances, etc)
- We cache the answer and from that point on we listen for events which could modify those related informations (adding/removing liquidity, swaps, etc)
Q(2.3): Which approach should we implement (1) or (2) ?
Since there is a lot of experience through Aeternity contributors with Javascript, it would make sense to remain within Javascript/Node.JS domain for API and Worker development
Dealing with known data and their transformations a static type checker could bring some improvement on discovering bugs before runtime Most known alternatives are Typescript and FlowJs. Typescript has a lot more adoption in the last years, and might make the natural choice to make if we make such a choice
Q(3.2.1) Are we going to use a static type-checker?
Remains to be discussed and decided on separate topic. But if we need an ORM like library Prisma could be a good choice.
The project should embed the deployment scripts for any perquisite containers needed for running the solution:
- SQL/NoSQL GraphDB storage software
- Any other third party software needed run Dex-Backend
can be achieved with https://www.npmjs.com/package/express-jsdoc-swagger
For a better scalability a serverless approach could be a good option. Though we have to take in consideration some constraints regarding this approach might impact the Persistence/ State management decisions. For example, any in-process state management has to be avoided
Q(4.3): Do we wish for a serverless support in the api layer?
One of the main requirements is to provide the user the pairs information and eventually a path of posible pairs for swapping actions. Any pair could be seen as an undirected relation between two tokens. The whole pairs collection could be seen as an undirected graph, with nodes being the tokens and the edges being represented by the pairs.
There are two possible alternatives to store and process such a graph: A Graph Database or a traditional one (RDBMS/NoSQL).
Let's discuss main pros and cons for both of them.
-
pros: A Graph database could be the best choice for that. The main advantage of it is that has builtin mechanism for modeling and iterate for the Pairs graph that we need.
-
cons: Being such a simple graph, this might be an over-engineered solution and unnecessarily dependency
Q(5.1.2) Is Neo4J a good solution if we choose Graph Database?
We could store these pair relations in a traditional table/document way, and use an already known library, like graphlib, to operate against those
In order to maintain and process this graph, we have two options:
- whenever wee need to iterate through it (data needed by the swap view) we have to fetch the whole relation collection, to transform it into a graph, and then to extract the needed subgraphs (paths)
- We maintain in-process a synced copy of the graph. But this will cancel the the Serverless approach.
- pros: a straight well known solution of database - orm - in-process iteration
- cons: the main problem here will be scalability since multiple nodes will require state-full atchitecture
Q(5.1) Which database should we use for the pairs graph: Graph / SQL or NoSQL ?
There is state information within the Pairs as allowances or balances that might fit into a regular kind of storage (RDBMS/NoSQL). Since the information is well defined and no complex relations are involved an SQL server engine could be the right choice
Q(5.2.1): what kind of database should we use for Pairs extra information? SQL, NoSQL or should we stay with GraphDB ( if that is also the Pair Relation choice)
Q(5.3.1): do we care for any additional faster caching solution as Redis or Memcached
By monitoring Dex-backend's activity could bring some useful information. We are splitting the monitoring subject into the following topics:
We could use StatsD metrics within API nodes and Worker node to monitor the regular activity defining some metrics.
Q(6.1.1): Should we use such monitoring technique as StatsD?
Q(6.1.1): Which platform for storing and viewing data should we use (Datadoghq, Graphana, Other) ?
Q(6.1.1): Do we need an error tracking solution ? Is sentry a good choice?