Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ocean archive indexer and api server #2724

Merged
merged 207 commits into from
Sep 28, 2024
Merged
Show file tree
Hide file tree
Changes from 196 commits
Commits
Show all changes
207 commits
Select commit Hold shift + click to select a range
91282a7
Ocean endpoints scaffold
Jouzo Nov 23, 2023
8956bbb
Model scaffold
Jouzo Nov 24, 2023
fc0f48c
Start ocean server
Jouzo Nov 27, 2023
42b01a1
Remove storage changes
Jouzo Nov 27, 2023
cd00f58
Ocean list pagination + json req/res (#2737)
canonbrother Dec 6, 2023
d0cd25d
Ocean indexing scaffold
Jouzo Dec 7, 2023
641ed94
Ocean API: rocks db implementation for ocean API (#2728)
nagarajm22 Dec 8, 2023
1865191
WIP: Ocean test suite CI (#2734)
canonbrother Dec 11, 2023
35362b2
Fix manger typo
Jouzo Dec 11, 2023
b3aee67
Add -oceanarchive flag
Jouzo Dec 11, 2023
5a067b0
Ocean API MN typing (#2744)
canonbrother Dec 13, 2023
c2885f3
Adds cached crate
Jouzo Dec 19, 2023
491362b
Fix format
Jouzo Dec 20, 2023
4c5b090
WIP: Ocean API database implementation to all modules (#2748)
nagarajm22 Dec 20, 2023
b675fea
Enhance Ocean database implementation (#2762)
Jouzo Dec 27, 2023
6937cd1
Fix ocean ci docker not found (#2759)
canonbrother Dec 27, 2023
a73860a
BlockService
Jouzo Dec 27, 2023
9a7b078
Finish Masternode indexing impl
Jouzo Dec 29, 2023
0acc887
Use serde as default Column trait implementation (#2765)
Jouzo Jan 2, 2024
1d56342
Fix build
Jouzo Jan 4, 2024
274beca
Add 404 handler
Jouzo Jan 5, 2024
9f27d77
Cleanup db iter method
Jouzo Jan 5, 2024
8cad7f9
Remove need for nightly
Jouzo Jan 8, 2024
fd5932f
Index auction (#2771)
Jouzo Jan 9, 2024
0f99a2f
Use single block context (#2773)
Jouzo Jan 9, 2024
915a79d
Fix fmt
Jouzo Jan 9, 2024
c391fe8
Ocean: block indexing (#2777)
canonbrother Jan 11, 2024
9f8770d
Index swap and swap result (#2778)
Jouzo Jan 12, 2024
0fe7ce7
Ocean: Genesis block indexed + missing mn info update (#2791)
canonbrother Jan 24, 2024
6b6272e
Ocean: RPC client (#2793)
Jouzo Jan 24, 2024
2639bf2
Ocean: Highest block endpoint (#2799)
Jouzo Jan 25, 2024
4ad383c
Accept height in /blocks/:id
Jouzo Jan 25, 2024
03f03af
WIP: Implementation for ocean-API transaction module (#2781)
nagarajm22 Jan 25, 2024
11b4860
Log indexing time
Jouzo Jan 25, 2024
d9598bb
Ocean: Fix Transaction index typing (#2801)
Jouzo Jan 25, 2024
b9ff0c8
Ocean: Governance API (#2805)
Jouzo Jan 29, 2024
aea2a96
Format cpp
Jouzo Jan 30, 2024
d69bf28
Ocean CI Refinement (#2764)
canonbrother Jan 31, 2024
45a3a89
update ocean ci target branch
canonbrother Jan 31, 2024
2c24380
WIP : Ocean Fixing transaction Indexing bugs (#2802)
nagarajm22 Feb 1, 2024
20a283f
Fix: Ocean resp get (#2811)
canonbrother Feb 1, 2024
4da1982
Fix RPC port (#2815)
Jouzo Feb 6, 2024
24a9275
pagination size serde_with displayfromstr
canonbrother Feb 6, 2024
62ddeb1
deserialize undefined to none
canonbrother Feb 7, 2024
e204056
Ocean: async RPC client (#2817)
Jouzo Feb 7, 2024
27578f6
Format rs
Jouzo Feb 8, 2024
2c3cf35
Stats API initial impl (#2819)
Jouzo Feb 9, 2024
6d98c23
Ocean: Masternode API (#2820)
Jouzo Feb 11, 2024
5337d8c
Correct API Decimal float serialization
Jouzo Feb 11, 2024
280c9f1
Ocean: Ocean loan API initial impl (#2823)
Jouzo Feb 19, 2024
668b897
typo stats/reward/distribution
canonbrother Feb 20, 2024
d2e4d63
Fix list block API (#2826)
canonbrother Feb 21, 2024
dce747f
Ocean: Initial Scan support (#2829)
Jouzo Feb 22, 2024
d4b9bb3
Fix block indexing
Jouzo Feb 23, 2024
3c123d2
Ocean: fix ocean api tx get (#2831)
canonbrother Feb 23, 2024
4db943b
Revert Column default impl
Jouzo Feb 23, 2024
e567c13
Revert cache changes
Jouzo Feb 23, 2024
f4679ad
Revert code cache changes
Jouzo Feb 23, 2024
544ac6b
Clippy cleanup
Jouzo Feb 23, 2024
bba99a6
Fix TransactionId visibility
Jouzo Feb 23, 2024
4b4e871
dftx-rs as crate (#2832)
Jouzo Feb 23, 2024
5d010c3
Merge branch 'master' into feature/ocean-archive
Jouzo Feb 23, 2024
b7a951d
Restore clippy
Jouzo Feb 23, 2024
a618882
Clippy
Jouzo Feb 23, 2024
5af42b5
Clippy tests
Jouzo Feb 24, 2024
c054019
Fix RepositoryOps trait arg names
Jouzo Feb 26, 2024
42c7898
Fix: ocean api tx getvins (#2833)
canonbrother Feb 26, 2024
dbb21cc
Ocean: Use custom axum path extractor (#2836)
canonbrother Feb 27, 2024
025640b
Ocean: Cleanup data acess (#2837)
Jouzo Feb 27, 2024
95d7b3f
Disable cache on build-dev
sieniven Feb 28, 2024
851d0c5
Cleanup ocean deps
Jouzo Feb 28, 2024
1c9b6a0
Re-enable CI caching
sieniven Feb 28, 2024
3bd491d
Fix: ocean api tx get vouts (#2834)
canonbrother Mar 1, 2024
79798d9
Merge branch 'master' into feature/ocean-archive
Jouzo Mar 4, 2024
246b26f
Set MACOSX_DEPLOYMENT_TARGET in macos cross compile
Jouzo Mar 4, 2024
700d235
Ocean cli (#2844)
Jouzo Mar 5, 2024
dae2d12
Update rustc version to 1.76
Jouzo Mar 5, 2024
a95cecc
Add todo fallback poolswap comment
Jouzo Mar 5, 2024
30c645b
Clippy
Jouzo Mar 5, 2024
9182131
Add RUST_DEFAULT_VERSION and MACOSX_DEPLOYMENT_TARGET to ci_export_vars
Jouzo Mar 5, 2024
53a3ece
Overwrite RUST_DEFAULT_VERSION
Jouzo Mar 5, 2024
6a62631
Merge branch 'master' into feature/ocean-archive
Jouzo Mar 6, 2024
4df2c81
Use defich/rust-defichain-rpc
Jouzo Mar 6, 2024
80706e7
Merge branch 'master' into feature/ocean-archive
Jouzo Mar 10, 2024
5aff3e8
Network Type for ocean API (#2861)
nagarajm22 Mar 18, 2024
f94ebe5
Merge branch master into feature/ocean-archive
Jouzo Apr 9, 2024
fd0539d
Implementation of oracle indexing (#2862)
nagarajm22 Apr 9, 2024
60f5acd
Clippy
Jouzo Apr 9, 2024
ed0267f
Restore txid in Transaction
Jouzo Apr 10, 2024
afb1d43
Update index mapper and oracle invalidate interval (#2888)
nagarajm22 Apr 11, 2024
7b20110
Merge branch 'master' into feature/ocean-archive
Jouzo Apr 12, 2024
4f87db4
Handle masternode state (#2889)
Jouzo Apr 15, 2024
81e256c
Fix doc
Jouzo Apr 15, 2024
25c19d5
Add support for mocknet
Jouzo Apr 16, 2024
d656a19
Parse 'main' as mainnet network
Jouzo Apr 16, 2024
5ca1db5
Trigger lint on feature/** branches
Jouzo Apr 25, 2024
489a8a7
Fix missing ethsubscription doc
Jouzo Apr 25, 2024
c7c78af
Merge branch 'master' into feature/ocean-archive
Jouzo Apr 25, 2024
6e46734
Merge branch master into feature/ocean-archive
Jouzo May 3, 2024
ada8b10
Ocean: Api PoolPair (#2838)
canonbrother May 13, 2024
7f02b0f
Ocean: Remove poolpair API unwraps (#2913)
Jouzo May 13, 2024
d62bb52
Merge branch 'master' of github.com:DeFiCh/ain into feature/ocean-arc…
Jouzo May 14, 2024
fed50af
Restore GetByteArray()
Jouzo May 14, 2024
580ca18
Merge branch master into feature/ocean-archive
Jouzo May 22, 2024
5325100
Ocean: list pool swap aggregates (#2918)
canonbrother May 27, 2024
34b6201
Impl new in repository macro
Jouzo May 28, 2024
e5a556b
Ocean: list dex prices api (#2923)
canonbrother May 29, 2024
b892a42
ocean indexing active price (#2892)
nagarajm22 May 29, 2024
0fe051a
Ocean: poolpair apr & volume (#2925)
canonbrother May 31, 2024
88719d4
Ocean list pool swap verbose api (#2926)
canonbrother Jun 6, 2024
dd360ce
Expose skipped txs to ocean (#2931)
Jouzo Jun 7, 2024
0e12703
fix split account by @ (#2934)
canonbrother Jun 10, 2024
86e1d43
Ocean: Fix cli linking issue. Add rust native skipped_txs
Jouzo Jun 10, 2024
0d733d0
Ocean: poolswap fallback handling (#2932)
canonbrother Jun 11, 2024
5751b16
Ocean: Index createpoolpair result (#2936)
Jouzo Jun 12, 2024
ebcc7fd
Ocean: add error field in ApiError (#2938)
canonbrother Jun 12, 2024
88a7d77
Merge branch master into feature/ocean-archive
Jouzo Jun 12, 2024
488f9ad
Remove cli indexer
Jouzo Jun 12, 2024
38e27f2
Flatten dftx check
Jouzo Jun 14, 2024
ae6d0a3
Ocean: fix `PoolCreationHeight` sorting & add `CompositeSwap` Aggrega…
canonbrother Jun 14, 2024
4e3902b
RawTx API (#2921)
nagarajm22 Jun 28, 2024
a284782
Ocean: fix governance APIs (listProposals, getProposal, listProposalV…
nagarajm22 Jul 15, 2024
5700a1d
Ocean: address indexers + apis (#2946)
canonbrother Jul 15, 2024
1fc3b4e
Ocean: api address list tokens (#2960)
canonbrother Jul 15, 2024
051b569
Ocean: api address get account history (#2967)
canonbrother Jul 15, 2024
4186f2b
Ocean: list auctions api (#2942)
canonbrother Jul 16, 2024
9d79115
Ocean: vault get + list apis (#2958)
canonbrother Jul 16, 2024
53835b1
Ocean: finishing get/list loan token api (#2968)
canonbrother Jul 16, 2024
feb799e
Ocean: api list vaults by address (#2974)
canonbrother Jul 17, 2024
6b5b7dd
Ocean: Implements `index_block_end ` (#2969)
nagarajm22 Jul 17, 2024
47a8f1b
Ocean: Fixed oracle invalidate methods (#2973)
nagarajm22 Jul 18, 2024
a9defef
Ocean: Error handling cleanup (#2978)
Jouzo Jul 18, 2024
cad2fa4
Ocean: fix decimal division by zero (#2977)
canonbrother Jul 18, 2024
8249b2f
Remove unwrap (#2979)
Jouzo Jul 19, 2024
2a25a87
Ocean: `checked_div` default value as 1 (#2981)
canonbrother Jul 24, 2024
8e6811d
Ocean: set loopback + cors (#2983)
canonbrother Jul 29, 2024
5fca655
Merge branch master into feature/ocean-archive
Jouzo Aug 1, 2024
c32acb7
Merge branch 'master' of github.com:DeFiCh/ain into feature/ocean-arc…
Jouzo Aug 5, 2024
1726a74
Ocean: revamp setoracledata indexer (#2987)
canonbrother Aug 5, 2024
68e4295
Merge branch 'feature/ocean-archive' of github.com:DeFiCh/ain into fe…
Jouzo Aug 5, 2024
6f0a9b1
Ocean: fix todo on get_stats (#2989)
canonbrother Aug 8, 2024
fb9c235
weightage u8 (#2992)
canonbrother Aug 8, 2024
f523e19
Auto ocean port
Jouzo Aug 8, 2024
0e287ea
fix typo (#2995)
canonbrother Aug 13, 2024
d62c177
Ocean: cache refined (#2999)
canonbrother Aug 16, 2024
9d95333
Ocean: fix loan token indexer (#2998)
canonbrother Aug 19, 2024
8feb418
Ocean: CollateralToken.active_price + typing amount: decimal (#2990)
canonbrother Aug 20, 2024
79dce23
Ocean: cache list pool pairs api (#3000)
canonbrother Aug 20, 2024
347f632
check_evm_tx within vin loop (#3004)
canonbrother Aug 21, 2024
6cd0242
Ocean: refactor tables (#3014)
Jouzo Aug 29, 2024
bb24bd8
Ocean: fix rm oracle indexer, oracle & price apis (#3013)
canonbrother Aug 29, 2024
fb42f7e
Ocean: feat Snafu (#3017)
canonbrother Sep 4, 2024
e1b3dbe
Merge branch master into feature/ocean-archive
Jouzo Sep 4, 2024
e85b43d
Clippy
Jouzo Sep 4, 2024
59f00ee
Granular ocean flags
Jouzo Sep 4, 2024
f921757
Cleanup logs
Jouzo Sep 4, 2024
4858dbb
Remove unused macro
Jouzo Sep 4, 2024
409fac8
Fix clippy
Jouzo Sep 4, 2024
ca75ae5
Remove unused imports
Jouzo Sep 4, 2024
87a2a6f
Remove unused deps
Jouzo Sep 4, 2024
073eeff
Ocean: fix map loan collateral active price (#3027)
canonbrother Sep 5, 2024
337f937
Ocean: feat. indexer invalidate (#3028)
canonbrother Sep 6, 2024
da1665b
Merge branch master into feature/ocean-archive
Jouzo Sep 10, 2024
48a383c
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 10, 2024
912e382
Ocean: Reduce disk usage (#3045)
Jouzo Sep 13, 2024
6f581bc
Ocean: fix outdated pool (#3046)
canonbrother Sep 14, 2024
ec48975
Add /debug/dumptables endpoint (#3049)
Jouzo Sep 16, 2024
6d7ab3a
Ci/sync with ocena activate
Jouzo Sep 17, 2024
fc0791f
Revert "Ci/sync with ocena activate"
Jouzo Sep 17, 2024
9e3d137
Ocean: refactor consolidate index script loop (#3052)
canonbrother Sep 17, 2024
c4d795d
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 18, 2024
9bf64f4
Ocean: cleaup pricefeeds (#3054)
Jouzo Sep 18, 2024
7019739
Ocean: refactor `from_script` (#3056)
canonbrother Sep 18, 2024
32dae60
Cleanup tokio
Jouzo Sep 19, 2024
20ca25d
get to_amount from index (#3055)
canonbrother Sep 19, 2024
e76de13
Ocean: chore pool model cleanup (#3057)
canonbrother Sep 20, 2024
c41fc25
Log txs
Jouzo Sep 20, 2024
4739a39
Log index script and remove clones
Jouzo Sep 20, 2024
8bb6d98
Cleanup clones
Jouzo Sep 20, 2024
67792e5
Cleanup ScriptAggregation
Jouzo Sep 20, 2024
1c59be3
Checked_add weightage
Jouzo Sep 23, 2024
0e044a2
Accept "test" as testnet name
Jouzo Sep 23, 2024
29b79c3
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 23, 2024
ae8be4e
invalidate order correction (#3064)
canonbrother Sep 23, 2024
3f9fd76
Ocean feat: replace `rpc::get_token` by `ffi::get_token` (#3061)
canonbrother Sep 23, 2024
5ec692f
Add ocean_index_block logs
Jouzo Sep 23, 2024
d422a1b
Ocean model weightage as u32
Jouzo Sep 24, 2024
12985df
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 24, 2024
d4a6ab5
Ocean: chore oracle/price model cleanup (#3067)
canonbrother Sep 26, 2024
3748eb9
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 26, 2024
9a6ce3a
Ocean: cache pool utils (#3069)
canonbrother Sep 26, 2024
b73ec36
Rename LoanCount to camelCase
Jouzo Sep 26, 2024
8c9ec8e
handle diff usdt pool naming (#3070)
canonbrother Sep 27, 2024
939cd54
Merge branch 'master' into feature/ocean-archive
Jouzo Sep 27, 2024
420c38c
Debug ocean endpoint error
Jouzo Sep 27, 2024
3648669
Terser logs
Jouzo Sep 27, 2024
19fea19
Ocean: fix get apr default if any zero (#3075)
canonbrother Sep 27, 2024
995116b
Rename ocean rest* to server
Jouzo Sep 27, 2024
5d47699
Update gArgs access
Jouzo Sep 27, 2024
c573c31
Grammar
Jouzo Sep 27, 2024
02f7f95
Fix guard
Jouzo Sep 27, 2024
04716fa
Cleanup -oceanarchive default
Jouzo Sep 27, 2024
34145e3
Fix client host
Jouzo Sep 27, 2024
5b3525d
Cleanup logs
Jouzo Sep 27, 2024
9224e8a
Cache with sync_writes
Jouzo Sep 27, 2024
c5ea970
Revert "Cleanup -oceanarchive default"
Jouzo Sep 27, 2024
e853878
Sort poolpairs by id
Jouzo Sep 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/build-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ concurrency:
env:
BUILD_VERSION: latest # Computed
DOCKER_HUB_USER: defi
# Note: We do not use debug build on CI for dev as well, since debug builds
# take 5x the amount of space and run the GitHub CI workers out of space.
# Note: We do not use debug build on CI for dev as well, since debug builds
# take 5x the amount of space and run the GitHub CI workers out of space.
# make.sh still defaults to debug for local builds
MAKE_DEBUG: 0
GIT_VERSION: 1
Expand Down Expand Up @@ -115,7 +115,7 @@ jobs:
with:
name: defichain-${{ env.BUILD_VERSION }}-${{ env.TARGET }}
path: ./build/

- name: Unpack binaries
run: tar -xvzf ./build/defichain-${{ env.BUILD_VERSION }}-${{ env.TARGET }}.${{ env.PKG_TYPE }} -C ./build/

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
pull_request:
branches:
- master
- feature/**

concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
Expand Down
94 changes: 94 additions & 0 deletions .github/workflows/tests-ocean.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Tests - Jellyfish Whale Apis

on:
workflow_dispatch:
pull_request:
branches:
- master
- feature/ocean-archive # TODO(): remove before merge to master

concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}

env:
TARGET: x86_64-pc-linux-gnu
MAKE_DEBUG: 0
GIT_VERSION: 1

jobs:
build:
runs-on: ubuntu-latest
container: defi/ain-builder:latest
env:
CARGO_INCREMENTAL: 0

steps:
- uses: actions/checkout@v4
- run: git config --global --add safe.directory '*'

- name: Populate environment
run: ./make.sh ci-export-vars

- name: Setup dependencies
run: ./make.sh ci-setup-deps

- name: Setup user dependencies
run: ./make.sh ci-setup-user-deps

- name: Restore cpp build cache
id: cpp-cache-restore
uses: actions/cache/restore@v3
with:
path: |
./build/depends
./build/src
~/.ccache
key: cpp-${{ env.TARGET }}-${{ env.BUILD_TYPE }}

- uses: Swatinem/rust-cache@v2
with:
workspaces: lib -> ../build/lib/target
save-if: ${{ github.ref == 'refs/heads/master' }}
shared-key: ${{ env.TARGET }}

- name: Build binaries
run: ./make.sh build

- name: Upload binaries
uses: actions/upload-artifact@v3
with:
name: defi-bins
path: build/src/defid

test:
name: Jellyfish Whale Api tests
runs-on: ubuntu-latest
needs: [build]
strategy:
fail-fast: false

steps:
- uses: actions/checkout@v4
with:
repository: birthdayresearch/jellyfishsdk
# ref: 'main'
ref: 'canonbrother/whale-defid' # TODO(): remove before merge to master

- name: Download binaries
uses: actions/download-artifact@v3
with:
name: defi-bins

- name: Setup permissions
run: |
chmod uog+x "$(pwd)/defid"

- uses: actions/setup-node@v3
with:
node-version: '18'

- name: Run tests
run: |
npm ci
DEFID="$(pwd)/defid" npm run defid apps/whale-api/src/module.api/__defid__
7 changes: 3 additions & 4 deletions doc/build-quick.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Quick build notes

DeFiChain is built with the same process as Bitcoin, but provides certain convenience steps to
DeFiChain is built with the same process as Bitcoin, but provides certain convenience steps to
build it easily with the `./make.sh` file in the root directory.

```
Expand Down Expand Up @@ -89,7 +89,6 @@ an environment with correct arch and pre-requisites configured.
(most pre-requisites can be installed with pkg-* commands).
```


## `TARGET` values

### Tier 1
Expand Down Expand Up @@ -145,7 +144,7 @@ but receives little to no testing.
PYTHON_VENV_DIR=${PYTHON_VENV_DIR:-"${BUILD_DIR}/pyenv"}

CLANG_DEFAULT_VERSION=${CLANG_DEFAULT_VERSION:-"15"}
RUST_DEFAULT_VERSION=${RUST_DEFAULT_VERSION:-"1.72"}
RUST_DEFAULT_VERSION=${RUST_DEFAULT_VERSION:-"1.76"}

MAKE_DEBUG=${MAKE_DEBUG:-"1"}
MAKE_USE_CLANG=${MAKE_USE_CLANG:-"$(get_default_use_clang)"}
Expand Down Expand Up @@ -177,5 +176,5 @@ but receives little to no testing.

Please read the `./make.sh` file for more details on the build helpers.
[UNIX build process](./build-unix.md) should also have more info though using
`./make.sh` is recommended as it builds out-of-tree by default and supports
`./make.sh` is recommended as it builds out-of-tree by default and supports
multiple targets.
Loading
Loading