Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pick the default indexer ingest topic based on network name #8510

Closed
5 of 15 tasks
masih opened this issue Apr 19, 2022 · 3 comments · Fixed by #8526
Closed
5 of 15 tasks

Pick the default indexer ingest topic based on network name #8510

masih opened this issue Apr 19, 2022 · 3 comments · Fixed by #8526
Assignees
@masih
Labels
kind/enhancement Kind: Enhancement P1 P1: Must be resolved

Comments

@masih
Copy link
Member

masih commented Apr 19, 2022

Checklist

  • This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
  • This is not a new feature request. If it is, please file a feature request instead.
  • This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
  • I have a specific, actionable, and well motivated improvement to propose.

Lotus component

  • lotus daemon - chain sync
  • lotus miner - mining and block production
  • lotus miner/worker - sealing
  • lotus miner - proving(WindowPoSt)
  • lotus miner/market - storage deal
  • lotus miner/market - retrieval deal
  • lotus miner/market - data transfer
  • lotus client
  • lotus JSON-RPC API
  • lotus message management (mpool)
  • Other

Improvement Suggestion

The indexer ingest topic in default config is fixed to /indexer/ingest/mainnet. But the allowed subscriptions topic name for the indexer is constructed based on the network name. For example, a miner running on calibnet is only allowed to contact to /indexer/ingest/calibnet and would fail to join the topic if the default config is not overridden to reflect the network name.

Ideally the default topic name should be selected based on network name unless explicitly specified by the user.
@masih masih added kind/enhancement Kind: Enhancement need/triage labels Apr 19, 2022
@jennijuju
Copy link
Member

yes we need this, currently lotus node cannot start base on default settings (index provider enabled) on testnets and devnets. @masih are you planning to contribute a PR on this by any chance?

@masih
Copy link
Member Author

masih commented Apr 20, 2022

Happy to; please feel free to assign this one to me thanks 🙏

@jennijuju
Copy link
Member

Happy to; please feel free to assign this one to me thanks 🙏

Thank YOU

@jennijuju jennijuju added P1 P1: Must be resolved and removed need/triage labels Apr 20, 2022
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
7fe4213 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
@masih masih mentioned this issue Apr 21, 2022
Merged
5 tasks
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
1f7569d 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
c42bdc1 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
34eb836 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
fbb57e8 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
masih added a commit to masih/lotus that referenced this issue Apr 21, 2022
@masih
Infer index provider topic from network name by default
f273a44 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes filecoin-project#8510
jennijuju pushed a commit that referenced this issue Apr 22, 2022
@masih @jennijuju
Infer index provider topic from network name by default
d044e30 
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes #8510
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@masih masih

Labels
kind/enhancement Kind: Enhancement P1 P1: Must be resolved
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: market: Infer index provider topic from network name by default masih/lotus
2 participants
@masih @jennijuju