Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(compiler): Abilities #731

Merged
merged 35 commits into from
Jul 18, 2023
Merged

feat(compiler): Abilities #731

merged 35 commits into from
Jul 18, 2023

Conversation

DieMyst
Copy link
Member

@DieMyst DieMyst commented Jun 5, 2023

No description provided.

DieMyst added 4 commits April 24, 2023 15:16
# Conflicts:
#	aqua-src/antithesis.aqua
#	parser/src/main/scala/aqua/parser/lexer/ValueToken.scala
#	parser/src/test/scala/aqua/parser/StructValueExprSpec.scala
#	semantics/src/main/scala/aqua/semantics/expr/ArrowTypeSem.scala
#	semantics/src/main/scala/aqua/semantics/expr/DataStructSem.scala
#	semantics/src/main/scala/aqua/semantics/expr/FieldTypeSem.scala
#	semantics/src/main/scala/aqua/semantics/expr/ScopeSem.scala
#	semantics/src/main/scala/aqua/semantics/rules/abilities/AbilitiesAlgebra.scala
#	semantics/src/main/scala/aqua/semantics/rules/abilities/AbilitiesInterpreter.scala
#	semantics/src/main/scala/aqua/semantics/rules/abilities/AbilitiesState.scala
#	semantics/src/main/scala/aqua/semantics/rules/locations/DummyLocationsInterpreter.scala
#	semantics/src/main/scala/aqua/semantics/rules/locations/LocationsState.scala
#	semantics/src/main/scala/aqua/semantics/rules/types/TypesAlgebra.scala
#	semantics/src/main/scala/aqua/semantics/rules/types/TypesInterpreter.scala
#	semantics/src/main/scala/aqua/semantics/rules/types/TypesState.scala
@linear
Copy link

linear bot commented Jun 5, 2023

LNG-123 Introduce `scope` expression

On the early days of Fluence development, Aqua programs were very low-level, taking every service, every topology move as a first class citizen.

Now we have Workers, Clients, CLI, builtins like Registry, Spells, and so on. It brings new requirements on the table:

  • Services are usually resolved by-alias, but available only on particular peers, e.g. Init peer or a Worker of the specified Subnet
  • Service resolution is too verbose to do manually every time
  • Services do not capture their location, so to call a service developer needs to wrap this service call with a closure, which might be hard to consider and takes a lot of boilerplate to do
  • Topology with on … via … is very fragile, especially with workers: hard to understand whether you need to have this via or not?
  • Complex behaviors, involving worker, host, init peer, and emergent systems like Registry, is being done on function level (with func …), which is again not easy wrt topology, service resolution

This proposal provides one more approach to solve better composability, help organizing libraries and reusing emergent behaviors, and simplify developers live.

Let's make two new top-level expressions, scope and lib, to describe the scope in the sense of topology, service resolutions, and local behaviors.

Example might look like the following:

-- Lib wraps some functions that needs topology provided
lib SpellLib:
  -- Need Spell service defined in scope
  Spell

  -- Closure
  run = ():
    -- Can use services, functions of the lib
    Spell.run_script()

-- Libraries may extend other libraries
lib WorkerSpellLib extends SpellLib:
  -- Resolve a service
  Spell "worker-spell"

  -- Require another Spell service to be resolved as OtherSpell
  -- So we can use two instances of the same service type
  Spell as OtherSpell

  -- Adding another closure
  stop = ():
    OtherSpell.kill_worker()

-- Scope definition may not be abstract: all services must be resolved, topology must be defined
scope Worker(peer: PeerId)
  -- May extend libs
  extends WorkerSpellLib
  -- Topology resolution is required
  on peer via HOST_PEER_ID:
    -- Unresolved services must be resolved
    OtherSpell "other-spell"

scope OtherWorker(peer: PeerId)
  -- May extend 0 or more scopes
  -- If extends a scope, topology resolution is delegated to that scope
  -- If extends more than one scope, topology resolutions of all scopes must be the same
  extends Worker(peer), OtherLib: -- extend a lib as well

    -- A scope may reference other scopes
    parent = PeerRootWorker(HOST_PEER_ID)

-- Usage of a scope from a detached function
func smth(peer: PeerId):
  -- Worker is a new data type that falls into the same namespace as arrows and variables
  worker = Worker(peer)

  -- OtherService is resolved in Worker, function is called on peer via HOST_PEER_ID
  worker.OtherSpell.callIt()

  -- closures capture worker's position
  worker.doLocally(peer)

  -- on ... via ... should be considered a super low level thing, so just no need to use it
  -- no need to do service resolutions; it should be eventually deprecated in service definitions and outside of scopes
  -- later, may do `func smth() on worker: Worker` or smth like that to move execution into given scope

DieMyst added 14 commits June 26, 2023 13:59
# Conflicts:
#	model/inline/src/main/scala/aqua/model/inline/RawValueInliner.scala
#	model/inline/src/main/scala/aqua/model/inline/TagInliner.scala
#	model/inline/src/main/scala/aqua/model/inline/raw/CallArrowRawInliner.scala
#	semantics/src/main/scala/aqua/semantics/rules/ValuesAlgebra.scala
# Conflicts:
#	model/inline/src/main/scala/aqua/model/inline/ArrowInliner.scala
# Conflicts:
#	build.sbt
#	model/inline/src/main/scala/aqua/model/inline/TagInliner.scala
#	model/inline/src/main/scala/aqua/model/inline/raw/ApplyPropertiesRawInliner.scala
#	semantics/src/main/scala/aqua/semantics/rules/types/TypesInterpreter.scala
@DieMyst DieMyst marked this pull request as ready for review July 11, 2023 12:42
@DieMyst DieMyst added the e2e Run e2e workflow label Jul 11, 2023
@DieMyst DieMyst changed the title WIP: scopes feat: Abilities Jul 13, 2023
(ops, rets) = opsAndRets
} yield SeqModel.wrap(ops.reverse: _*) -> rets.reverse
)
): State[S, (SeqModel.Tree, List[ValueModel], Map[String, ValueModel], Map[String, FuncArrow])] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add comment on what this function returns. Also I think it would be better to define a case class for return type at this point.

Comment on lines 80 to 81
Exports[S].exports.flatMap { oldExports =>
getOutsideStreamNames.flatMap { outsideDeclaredStreams =>
Copy link
Contributor

@InversionSpaces InversionSpaces Jul 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
(Exports[S].exports, getOutsideStreamNames).flatMapN { (oldExports, outsideDeclaredStreams) =>
(with import cats.syntax.apply)

strInFunc2 <- SomeService.getStr(s)
<- strInFunc, strInFunc2

func handleSecAb {SomeAb, SecondAb}() -> string, string, string, u32:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to have several abilities of the same type passed to a single function?

strInFunc2 <- SomeService.getStr(s)
<- strInFunc, strInFunc2

func handleSecAb {SomeAb, SecondAb}() -> string, string, string, u32:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to pass an array of abilities?

@@ -136,25 +136,37 @@ object ArrowInliner extends Logging {
arrowsToSave <- Arrows[S].pickArrows(returnedArrows)
} yield {
val (valsFromAbilities, arrowsFromAbilities) = returnedFromAbilities
InlineResult(SeqModel.wrap(ops.reverse: _*), rets.reverse, valsFromAbilities, arrowsFromAbilities ++ arrowsToSave)
InlineResult(
SeqModel.wrap(ops.reverse: _*),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SeqModel.wrap(ops.reverse: _*),
SeqModel.wrap(ops.reverse),

@DieMyst DieMyst enabled auto-merge (squash) July 18, 2023 17:09
@DieMyst DieMyst merged commit 63a9f42 into main Jul 18, 2023
@DieMyst DieMyst deleted the LNG-123-scopes branch July 18, 2023 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
e2e Run e2e workflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants