Make REST API use Twenty ORM direclty

[FelixMalfait]

Objective: Enhance the reliability and performance of the REST API.

Technical Context

Currently, our REST API (documentation here) is exposed via two dynamic endpoint sets:

• rest/...
• rest/batch/...
  These endpoints are handled by two controllers: rest-api-core.controller and rest-api-batch-core.controller, both backed by RestApiCoreService.

The RestApiCoreService currently parses REST payloads, converts them into GraphQL payloads, and calls the GraphQL API—an inefficient process. This approach was initially necessary because querying workspaceSchemas in Twenty required pg_graphql.

However, we recently introduced TwentyORM (soon to be renamed WorkspaceORM), a lightweight layer on top of TypeORM, which can directly query workspaceSchemas.

Goal: Refactor the REST API to leverage TwentyORM directly.

Note: The GraphQL API is also being migrated to TwentyORM (replacing pg_graphql), so you can draw inspiration from the GraphqlQueryRunnerService.

Key Goals:

Refactor the REST API to use TwentyORM directly.
Ensure the REST API is fully covered by integration tests.

Technical inputs

We should take a lot of inspiration from what has been done in the new GraphqlQueryRunnerService. Advices:

1. split the code in services / utils to keep small and well scoped functional pieces
2. Overall logic: a) parse REST payload into TypeORM format, b) send TypeORM query, c) parse result back into REST response
3. All the pieces are already there: the current code is able to parse REST into graphql and Graphql into twentyORM, we "only" need to skip one step. It will likely mean to write everything from scratch but this should still speed a lot the process

[charlesBochet]

Yes please!

If any experienced contributor wants to tackle this task, I'm happy to give a hand here. There is a lot of work but the engineering and the product is very well defined so it should be doable by the community

[charlesBochet]

@FelixMalfait It's not that difficult anymore I think as most of the complexity is within twenty-orm and we have quite good example of parsing in the current rest api and in our graphql implementation (we know how to parse rest input to graphql and graphql to orm already, we need to do rest input to orm direclty)

[charlesBochet]

I'm editing the description to add more details

[Faisal-imtiyaz123]

@charlesBochet I am ready to work. You may please assign me.

[FelixMalfait]

@Faisal-imtiyaz123 let me know if we should unassign you. Thanks!

[Faisal-imtiyaz123]

@FelixMalfait Yes you may unassign me. I talked to @charlesBochet regarding this.

[charlesBochet]

/oss.gg 3000

[oss-gg]

Thanks for opening an issue! It's live on oss.gg!

[DeepaPrasanna]

I have working experience in nestjs, May I work on this? @charlesBochet

[charlesBochet]

sure, thank you @DeepaPrasanna!
You can type: /assign to get assigned to it

[charlesBochet]

It's not an easy one, feel free to ping me if you need help!

[DeepaPrasanna]

sure! Thank you

[DeepaPrasanna]

/assign

[oss-gg]

Assigned to @DeepaPrasanna! Please open a draft PR linking this issue within 48h ⚠️ If we can't detect a PR from you linking this issue in 48h, you'll be unassigned automatically 🕹️ Excited to have you ship this 🚀

[Tanmayshi]

/assign

[oss-gg]

This issue is already assigned to another person. Please find more issues here.

[oss-gg]

@DeepaPrasanna, Just a little reminder: Please open a draft PR linking this issue within 12 hours. If we can't detect a PR in 12h, you will be unassigned automatically.

[DeepaPrasanna]

hello @charlesBochet , I am working on this. I have an interview scheduled for today, that is why I paused for some time. I plan to resume my work again tomorrow.

[DeepaPrasanna]

/assign

[oss-gg]

Assigned to @DeepaPrasanna! Please open a draft PR linking this issue within 48h ⚠️ If we can't detect a PR from you linking this issue in 48h, you'll be unassigned automatically 🕹️ Excited to have you ship this 🚀

[devhiteshk]

/assign

[oss-gg]

This issue is already assigned to another person. Please find more issues here.

[DeepaPrasanna]

I will be creating a draft PR today. Pls don't unassign me.

[FelixMalfait]

Thanks @DeepaPrasanna, great to create a PR to get intermediate feedback and make sure you go in the right direction since this is a big PR, thanks a lot!

[DeepaPrasanna]

Hello @charlesBochet @FelixMalfait , I have tried to refactor the rest/batch/... endpoint first. I have raised a draft PR. If I am moving in the right direction, I will refactor the remaining APIs soon. Requesting feedback :) Thank u

[charlesBochet]

Thank you @DeepaPrasanna, I will take a look today

[DeepaPrasanna]

Just curious, how the distribution of the points will work if I am unable to complete it everything before 31st. I do understand and respect your time for reviews. :)

[charlesBochet]

Awarded points on the PR :) We will take a deep look at your PR shortly, we are a bit overwhelmed by the volume of PRs right now!

[DeepaPrasanna]

Awarded points on the PR :) We will take a deep look at your PR shortly, we are a bit overwhelmed by the volume of PRs right now!

No worries! Thank you :)

[Nabhag8848]

@FelixMalfait @charlesBochet this is interesting. working on it, getting back with first version to get initial feedback on implementation, will start by making it work for batch endpoints as we will get enough idea for other endpoints with that implementation.

[DeepaPrasanna]

Hey @Nabhag8848, I wasn't able to finish the PR due to my work commitments. You can refer to the incomplete PR to get a head start.

[FelixMalfait]

Thanks a lot @Nabhag8848 this is a big one and an impactful one for the overall quality of the codebase (current solution for the api is hacky)! Ping us when you need feedback!

[Nabhag8848]

still working, have better idea about overall codebase 🙌🏼.

[martmull]

Hey @Nabhag8848 are you still working on that issue? @pacyL2K19 is interested to work on it

[pacyL2K19]

Can you assign this task to me @martmull?
The first PR is under review and I will be working on the second PR for the POST /rest/* endpoint from tomorrow

[martmull]

Done