Code

Initial server commit.#1

Code

Closed

arik-so wants to merge 90 commits intomainlightningdevkit/rapid-gossip-sync-server:mainfrom server_additionlightningdevkit/rapid-gossip-sync-server:server_additionCopy head branch name to clipboard

+2,198

Commits

Commits on Jan 11, 2022

• Configure SAST in .gitlab-ci.yml, creating this file if it does not already exist

  

  Arik Sosman

  committed
  c34851a

  Copy full SHA for c34851a

• relocation commit

  arik-so

  committed
  fb538a4

  Copy full SHA for fb538a4

Commits on Jan 13, 2022

• create naïve downloader and server functionality

  arik-so

  committed
  f5e64f6

  Copy full SHA for f5e64f6

Commits on Jan 14, 2022

• remove redundant imports and move db commands to config

  arik-so

  committed
  f399fb6

  Copy full SHA for f399fb6

• offload the compression to flate2 for better benchmarking opportunities, efficacy measurement and response length visibility

  arik-so

  committed
  64bd942

  Copy full SHA for 64bd942

Commits on Jan 15, 2022

• add channel update delta tallies, and omit updates that have no delta

  arik-so

  committed
  1cfce08

  Copy full SHA for 1cfce08

• Update the server full gossip data once the downloader is caught up with the gossip.

  arik-so

  committed
  8d637cd

  Copy full SHA for 8d637cd

• Cache the warp response type directly.

  arik-so

  committed
  b3de771

  Copy full SHA for b3de771

• Dynamic gossip update (#1)

  Show description for 7d9289d
  arik-so

  authored
  7d9289d

  Copy full SHA for 7d9289d

• Make response pending first load return status 503.

  arik-so

  committed
  81eb1dd

  Copy full SHA for 81eb1dd

• Serve both full and dynamic gossip.

  arik-so

  committed
  3ea8f12

  Copy full SHA for 3ea8f12

Commits on Jan 16, 2022

• Change output type to first list all announcements, and then all updates.

  arik-so

  committed
  47f1dc1

  Copy full SHA for 47f1dc1

• Fix bug where type data was serialized upon timestamp zeroing.

  arik-so

  committed
  836175a

  Copy full SHA for 836175a

Commits on Jan 17, 2022

• Output incremental updates where possible

  arik-so

  committed
  a76d45f

  Copy full SHA for a76d45f

Commits on Jan 18, 2022

• Encode incremental channel updates using unused bit flags in the channel flags field

  arik-so

  committed
  691b10b

  Copy full SHA for 691b10b

Commits on Mar 11, 2022

• use updated serialziation format with default disclosure for non-incremental channel updates

  arik-so

  committed
  8f7aa97

  Copy full SHA for 8f7aa97

Commits on May 16, 2022

• create separate server path for snapshotted data

  arik-so

  committed
  08accf1

  Copy full SHA for 08accf1

Commits on May 17, 2022

• properly store seen timestamps and solely rely on them for dynamic updates

  arik-so

  committed
  849b1ca

  Copy full SHA for 849b1ca

Commits on May 18, 2022

• create mechanism for calculating set deltas efficiently, discerning whether intermediate updates should be included, and preparing an easily traversable serialization set.

  arik-so

  committed
  b5967ba

  Copy full SHA for b5967ba

• use new serialization mechanism in dynamic server response calculation

  arik-so

  committed
  e5581c1

  Copy full SHA for e5581c1

• remove experiment setup constants

  arik-so

  committed
  b8ae6ef

  Copy full SHA for b8ae6ef

• Remove sample module.

  arik-so

  committed
  6bc4af7

  Copy full SHA for 6bc4af7

• Merge branch 'main' of github.com:arik-so/rust-ln-sync into arik_main

  Show description for 8b4f3e6
  arik-so

  committed
  8b4f3e6

  Copy full SHA for 8b4f3e6

• add some config and rationale comments

  arik-so

  committed
  9f824b5

  Copy full SHA for 9f824b5

• Remove unused compression module

  arik-so

  committed
  e8c3844

  Copy full SHA for e8c3844

• remove unused functions in serialization

  arik-so

  committed
  24b835d

  Copy full SHA for 24b835d

• add snapshotting template

  arik-so

  committed
  726d87c

  Copy full SHA for 726d87c

• refactor server response serialization and compression into a separate method

  arik-so

  committed
  b36dece

  Copy full SHA for b36dece

• store created snapshots

  arik-so

  committed
  1d32613

  Copy full SHA for 1d32613

• Await sync completion prior to initializing snapshotting service and activating dynamic response route.

  arik-so

  committed
  cbe3109

  Copy full SHA for cbe3109

• Serve snapshots if they are available.

  arik-so

  committed
  4fa237a

  Copy full SHA for 4fa237a

• Create readme.

  arik-so

  committed
  6a0b888

  Copy full SHA for 6a0b888

• Fix snapshot cleanup logging.

  arik-so

  committed
  5a3df01

  Copy full SHA for 5a3df01

Commits on May 19, 2022

• expand readme

  arik-so

  committed
  7a63fd3

  Copy full SHA for 7a63fd3

• configure db connection credentials using env vars

  arik-so

  committed
  447aa49

  Copy full SHA for 447aa49

• fix cargo.toml dependencies

  arik-so

  committed
  529bb69

  Copy full SHA for 529bb69

Commits on May 20, 2022

• Merge branch 'main' of github.com:arik-so/rust-ln-sync into server_addition

  arik-so

  committed
  5e269c5

  Copy full SHA for 5e269c5

• separate rapid sync server from processor

  arik-so

  committed
  9b885a4

  Copy full SHA for 9b885a4

Commits on May 30, 2022

• start snapshotting service as soon as initial sync is persisted

  arik-so

  committed
  6526a18

  Copy full SHA for 6526a18

Commits on Jul 1, 2022

• convert code to use version 108

  arik-so

  committed
  62877fa

  Copy full SHA for 62877fa

• Remove unsigned fields from db, and specify queried columns explicitly.

  arik-so

  committed
  f266083

  Copy full SHA for f266083

• Remove unused timestamp fields.

  arik-so

  committed
  3f1bfa6

  Copy full SHA for 3f1bfa6

• Add indices.

  arik-so

  committed
  96b8c0f

  Copy full SHA for 96b8c0f

• Use 2018 cargo package edition and initialize network graph from cache.

  arik-so

  committed
  f6678fd

  Copy full SHA for f6678fd

• don't parallelize snapshotting service

  arik-so

  committed
  ca2497f

  Copy full SHA for ca2497f

Commits on Jul 2, 2022

• Print when network graph is being cached.

  arik-so

  committed
  07af1c9

  Copy full SHA for 07af1c9

• combine latest and intermediate update queries into one

  arik-so

  committed
  8e9d94f

  Copy full SHA for 8e9d94f

• simplify serialization logic pertaining to intermediate update checks

  arik-so

  committed
  6935a66

  Copy full SHA for 6935a66

• force older tokio version

  arik-so

  committed
  29c1b94

  Copy full SHA for 29c1b94

• stop storing intermediate updates at all, and instead determine the mutated properties when iterating through the query responses

  arik-so

  committed
  61c4243

  Copy full SHA for 61c4243

• Update arguments for new on-the-fly calculated mutation sets.

  arik-so

  committed
  9cbe8c2

  Copy full SHA for 9cbe8c2

• Dual license project with Apache/MIT.

  arik-so

  committed
  8bf52a3

  Copy full SHA for 8bf52a3

Commits on Jul 13, 2022

• Don't block on network graph persistence because it can occasionally take longer than the maximum ping response time, resulting in peer disconnects.

  arik-so

  committed
  0dfb67a

  Copy full SHA for 0dfb67a

• Always name snapshot files using rounded timestamps.

  arik-so

  committed
  a5319ab

  Copy full SHA for a5319ab

Commits on Jul 14, 2022

• Fix server sync mpsc send causing blockage due to its low buffer coupled with the receiver only awaiting once.

  arik-so

  committed
  34b1818

  Copy full SHA for 34b1818

• Ensure that the network graph is not cached too frequently.

  arik-so

  committed
  5aecdf3

  Copy full SHA for 5aecdf3

Commits on Aug 11, 2022

• Make server run on older Rust versions for Debian compatibility, and simplify various config methods into static properties.

  arik-so

  committed
  c5fe73f

  Copy full SHA for c5fe73f

Commits on Aug 16, 2022

• Fix in-place mutation borrower issues.

  arik-so

  committed
  9f1c0dc

  Copy full SHA for 9f1c0dc

Commits on Aug 19, 2022

• Use chain::Access with REST as opposed to RPC, obviating the need for credentials.

  arik-so

  committed
  7da1b30

  Copy full SHA for 7da1b30

• Improve README

  arik-so

  committed
  46f0c32

  Copy full SHA for 46f0c32

• Remove some unused stuff

  arik-so

  committed
  ecdfa9a

  Copy full SHA for ecdfa9a

• Throwing an error if a peer connection fails, and aborting if all fail.

  arik-so

  committed
  1aee43d

  Copy full SHA for 1aee43d

• Replace .retain() with hideous cloned keys approach.

  arik-so

  committed
  201d655

  Copy full SHA for 201d655

• Remove most server build warnings.

  arik-so

  committed
  a134dce

  Copy full SHA for a134dce

Commits on Aug 20, 2022

• Add todo.

  arik-so

  committed
  5642f3c

  Copy full SHA for 5642f3c

• remove remaining commented out code

  arik-so

  committed
  061347d

  Copy full SHA for 061347d

• mandate working intra-doc links

  arik-so

  committed
  54c4a76

  Copy full SHA for 54c4a76

• Rip out server, remove compression.

  arik-so

  committed
  3594e11

  Copy full SHA for 3594e11

• Drop chrono feature on tokio postgres.

  arik-so

  committed
  54be347

  Copy full SHA for 54be347

• Remove RPC client feature on lightning block sync.

  arik-so

  committed
  d18f4d4

  Copy full SHA for d18f4d4

• Make returned timestamps be rounded down to the nearest multiple of 24 hours.

  arik-so

  committed
  126c92b

  Copy full SHA for 126c92b

• Remove caching support from chain verifier.

  arik-so

  committed
  9b9b2df

  Copy full SHA for 9b9b2df

• Create symlinks with intermediate values.

  arik-so

  committed
  56e6a4f

  Copy full SHA for 56e6a4f

• Add todo for full sync snapshots.

  arik-so

  committed
  3bf42b7

  Copy full SHA for 3bf42b7

• Use BufReader and BufWriter for network graph access.

  arik-so

  committed
  564062c

  Copy full SHA for 564062c

• Insert schema version into config table.

  arik-so

  committed
  5944d7b

  Copy full SHA for 5944d7b

• Don't immediately persist network graph.

  arik-so

  committed
  09d3658

  Copy full SHA for 09d3658

• Fix tokio sleep behavior.

  arik-so

  committed
  835dab1

  Copy full SHA for 835dab1

Commits on Aug 21, 2022

• fix config table schema versioning

  arik-so

  committed
  d2b9b07

  Copy full SHA for d2b9b07

• Now only reconnecting to peers upon disconnect, instead of every twelve hours.

  arik-so

  committed
  990a616

  Copy full SHA for 990a616

• Make Postgres apply timestamps to stored messages.

  arik-so

  committed
  f914de1

  Copy full SHA for f914de1

• Store blobs as blobs.

  arik-so

  committed
  6894747

  Copy full SHA for 6894747

Commits on Aug 22, 2022

• Improve symlink generation.

  arik-so

  committed
  60e6b5f

  Copy full SHA for 60e6b5f

• fix config changes

  arik-so

  committed
  0ec0e4d

  Copy full SHA for 0ec0e4d

• Restore executor-based connection initialization.

  arik-so

  committed
  189de38

  Copy full SHA for 189de38

• Adjust caught-up-threshold.

  arik-so

  committed
  242c2f3

  Copy full SHA for 242c2f3

• First attempt at CI.

  arik-so

  committed
  7d3484f

  Copy full SHA for 7d3484f

• Ci attempt 02

  arik-so

  committed
  b96de63

  Copy full SHA for b96de63

• Ci attempt 03

  arik-so

  committed
  ec4ca18

  Copy full SHA for ec4ca18

• CI attempt 04

  arik-so

  committed
  eaaf993

  Copy full SHA for eaaf993