Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux: service does not start with a custom base-path #759

Closed
Krixa opened this issue Oct 27, 2022 · 13 comments
Closed

Linux: service does not start with a custom base-path #759

Krixa opened this issue Oct 27, 2022 · 13 comments

Comments

@Krixa
Copy link

Krixa commented Oct 27, 2022

Hello !

When running with a different base-path than /home/polkadot the service does not start because of the ProtectSystem=strict.

To fix this I had to add the flag ReadWritePaths=/mypath on the systemd configuration file. Maybe a new variable can be created on the /etc/default/polkadot file for this case or maybe a bit of documentation to avoid the head scratching as to why the service does not start, here are the logs when I had the issue:

Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 Parity Polkadot
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 ✌️ version 0.9.30-064536093f5
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 ❤️ by Parity Technologies [email protected], 2017-2022
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 📋 Chain specification: Polkadot
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 🏷 Node name:
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 👤 Role: FULL
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 💾 Database: RocksDb at /mnt/nvme/polkadot/chains/polkadot/db/full
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 ⛓ Native runtime: polkadot-9300 (parity-polkadot-0.tx15.au0)
Oct 27 18:46:25 pve polkadot[15651]: Error:
Oct 27 18:46:25 pve polkadot[15651]: 0: Read-only file system (os error 30)
Oct 27 18:46:25 pve polkadot[15651]: Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Oct 27 18:46:25 pve polkadot[15651]: Run with RUST_BACKTRACE=full to include source snippets.
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 Failed to run the sequential write disk benchmark: failed to create a test file: No such file or directory (os error 2)
Oct 27 18:46:25 pve polkadot[15651]: 2022-10-27 18:46:25 Failed to run the random write disk benchmark: failed to create a test file: No such file or directory (os error 2)
Oct 27 18:46:25 pve systemd[1]: polkadot.service: Main process exited, code=exited, status=1/FAILURE

@bkchr
Copy link
Member

bkchr commented Oct 30, 2022

Oct 27 18:46:25 pve polkadot[15651]: 0: Read-only file system (os error 30)

I mean the node is telling you exactly what the problem is. The path needs to be writable.

What kind of documentation would you have expected?

@Krixa
Copy link
Author

Krixa commented Nov 1, 2022

The thing is the systemd configuration is shipped with the deb/rpm package.

The hardening of the systemd (ProtectSystem=strict) configuration makes it impossible to run polkadot outside of default base-path without modifying the systemd configuration file.

I am trying to use a documented option of the program that is incompatible with the service configuration so I figured this a bug ?

@bkchr
Copy link
Member

bkchr commented Nov 1, 2022

Maybe something that should be documented in the polkadot wiki?

@Krixa
Copy link
Author

Krixa commented Nov 3, 2022

I think the wiki does not have a doc about installing polkadot from the parity debian repository, only manually running the binary.

@bkchr
Copy link
Member

bkchr commented Nov 3, 2022

I think the wiki does not have a doc about installing polkadot from the parity debian repository, only manually running the binary.

Maybe could still be added? Could you maybe create some pr to the polkadot wiki? Or some docs to the polkadot Readme.

@rfuegen
Copy link

rfuegen commented Nov 5, 2022

I'm also struggling with this issue on RockyLinux 8.5. I tried to add

ReadWritePaths=/mnt/blockchain/polkadot

to polkadot.service, but it doesn't fix the problem. I'm not fluent enough with systemd to fix this issue; helpful wiki entry highly appreciated.

@Krixa
Copy link
Author

Krixa commented Nov 5, 2022

I'm also struggling with this issue on RockyLinux 8.5. I tried to add

ReadWritePaths=/mnt/blockchain/polkadot

to polkadot.service, but it doesn't fix the problem. I'm not fluent enough with systemd to fix this issue; helpful wiki entry highly appreciated.

Did you issue a systemctl daemon-reload after you modification ?

@rfuegen
Copy link

rfuegen commented Nov 5, 2022

Did you issue a systemctl daemon-reload after you modification ?

yes, but still not working. here's the service file:

`
[Unit]
Description=Polkadot Validator

[Service]
User=polkadot
Group=polkadot
ReadWritePaths=/mnt/blockchain/polkadot

ExecStart=/home/polkadot/bin/polkadot --base-path /mnt/blockchain/polkadot/ --ws-external --rpc-external --rpc-cors all

Restart=always
RestartSec=2400

[Install]
WantedBy=multi-user.target
`

result:

Nov 05 16:21:09 host systemd[1]: Reloading. Nov 05 16:21:15 host systemd[1]: Started Polkadot Validator. Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Parity Polkadot Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ✌️ version 0.9.30-064536093f5 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ❤️ by Parity Technologies <[email protected]>, 2017-2022 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 📋 Chain specification: Polkadot Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 🏷 Node name: magical-smash-0654 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 👤 Role: FULL Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 💾 Database: RocksDb at /mnt/blockchain/polkadot/chains/polkadot/db/full Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ⛓ Native runtime: polkadot-9300 (parity-polkadot-0.tx15.au0) Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Failed to run the sequential write disk benchmark: failed to create a test file: Permission denied (os error 13) Nov 05 16:21:15 host polkadot[2071]: Error: Nov 05 16:21:15 host polkadot[2071]: 0: Backend error: Io error: Permission denied (os error 13) Nov 05 16:21:15 host polkadot[2071]: Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Nov 05 16:21:15 host polkadot[2071]: Run with RUST_BACKTRACE=full to include source snippets. Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Failed to run the random write disk benchmark: failed to create a test file: Permission denied (os error 13) Nov 05 16:21:15 host systemd[1]: polkadot.service: Main process exited, code=exited, status=1/FAILURE Nov 05 16:21:15 host systemd[1]: polkadot.service: Failed with result 'exit-code'.

I just downloaded the polkadot binary from here, not installed as RPM package.

@Krixa
Copy link
Author

Krixa commented Nov 5, 2022

Did you issue a systemctl daemon-reload after you modification ?

yes, but still not working. here's the service file:

` [Unit] Description=Polkadot Validator

[Service] User=polkadot Group=polkadot ReadWritePaths=/mnt/blockchain/polkadot

ExecStart=/home/polkadot/bin/polkadot --base-path /mnt/blockchain/polkadot/ --ws-external --rpc-external --rpc-cors all

Restart=always RestartSec=2400

[Install] WantedBy=multi-user.target `

result:

Nov 05 16:21:09 host systemd[1]: Reloading. Nov 05 16:21:15 host systemd[1]: Started Polkadot Validator. Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Parity Polkadot Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ✌️ version 0.9.30-064536093f5 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ❤️ by Parity Technologies <[email protected]>, 2017-2022 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 📋 Chain specification: Polkadot Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 🏷 Node name: magical-smash-0654 Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 👤 Role: FULL Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 💾 Database: RocksDb at /mnt/blockchain/polkadot/chains/polkadot/db/full Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 ⛓ Native runtime: polkadot-9300 (parity-polkadot-0.tx15.au0) Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Failed to run the sequential write disk benchmark: failed to create a test file: Permission denied (os error 13) Nov 05 16:21:15 host polkadot[2071]: Error: Nov 05 16:21:15 host polkadot[2071]: 0: Backend error: Io error: Permission denied (os error 13) Nov 05 16:21:15 host polkadot[2071]: Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Nov 05 16:21:15 host polkadot[2071]: Run with RUST_BACKTRACE=full to include source snippets. Nov 05 16:21:15 host polkadot[2071]: 2022-11-05 16:21:15 Failed to run the random write disk benchmark: failed to create a test file: Permission denied (os error 13) Nov 05 16:21:15 host systemd[1]: polkadot.service: Main process exited, code=exited, status=1/FAILURE Nov 05 16:21:15 host systemd[1]: polkadot.service: Failed with result 'exit-code'.

I just downloaded the polkadot binary from here, not installed as RPM package.

What are the permissions on /mnt/ /mnt/blockchain and /mnt/blockchain/polkadot/ ? can you provide the output of an ls

@rfuegen
Copy link

rfuegen commented Nov 5, 2022

[polkadot@host mnt]$ ls -lh
total 4.0K
drwxr-xr-x. 3 root root 4.0K Nov 3 17:22 blockchain
[polkadot@host mnt]$ ls -lh blockchain/
total 4.0K
drwxr-xr-x. 4 polkadot polkadot 4.0K Oct 31 15:48 polkadot

works perfectly when started as user polkadot in its $HOME directory.

@Krixa
Copy link
Author

Krixa commented Nov 5, 2022

[polkadot@host mnt]$ ls -lh total 4.0K drwxr-xr-x. 3 root root 4.0K Nov 3 17:22 blockchain [polkadot@host mnt]$ ls -lh blockchain/ total 4.0K drwxr-xr-x. 4 polkadot polkadot 4.0K Oct 31 15:48 polkadot

works perfectly when started as user polkadot in its $HOME directory.

Do you have selinux enabled ? (sestatus if I remeber correctly), did you try installing the rpm ?

Actually you dont need to add ReadWritePaths given you are not using the package systemd hardened service file that contains the ProtectSystem=strict entry

@Krixa
Copy link
Author

Krixa commented Nov 5, 2022

I think the wiki does not have a doc about installing polkadot from the parity debian repository, only manually running the binary.

Maybe could still be added? Could you maybe create some pr to the polkadot wiki? Or some docs to the polkadot Readme.

I have created a PR on the wiki w3f/polkadot-wiki#4012

@rfuegen
Copy link

rfuegen commented Nov 5, 2022

ooops! disabling selinux after an installation is pretty much the first thing I usually do, but in this case I forgot ... now it works, thanks!

I tried to install from the rpm package, but dnf/rpm did not accept the PGP key.

@Sophia-Gold Sophia-Gold transferred this issue from paritytech/polkadot Aug 24, 2023
@bkchr bkchr closed this as completed Aug 25, 2023
claravanstaden pushed a commit to Snowfork/polkadot-sdk that referenced this issue Dec 8, 2023
* Remove apps & incentivized channel from parachain (paritytech#753)

* Remove apps & incentivized channel from parachain

* Fix formatting

* Remove incentivized channel & channel ID from relayer (paritytech#756)

* Remove incentivized channel from relayer

Will regenerate the bindings in contracts/ in the Solidity PR.

Still TODO:
- Remove mapping in:
    - relayer/cmd/fetch_messages.go
    - relayer/relays/ethereum/ethereum-listener.go
- Remove redundant imports of github.com/ethereum/go-ethereum/core/types
- Remove channel ID in relayer/relays/parachain/query_events.go
- Remove other occurrences of channelId:
rg -i channelid | sed -r 's/([^:]*):.*/\1/' | uniq | rg -v ^contracts/
- Replace Events with BasicChannelEvent in
  relayer/relays/parachain/query_events.go
- Remove unused relays/parachain/storage_key.go

* Remove address-method mapping for basic channel

* Remove redundant import

* Remove unused module storage_key

* Remove channelID

* Replace Events struct with BasicChannelEvent

* Remove incentivized channel, apps, channel ID & sundry from contracts (paritytech#757)

* Remove incentivized channel

* Remove deprecated prettier option

Deprecated in prettier-plugin-solidity 1.0.0:
NomicFoundation/hardhat-vscode#367 (comment)

The comment above indicates that solhint might have a rule for this, but
it doesn't look like there is one yet:
https://protofire.github.io/solhint/

* Remove apps

* Remove channel id

* Remove unused OpaqueProof

* Regenerate contract bindings

* Replace necessary contracts

* Fix generated contract bindings

Remove incentivized channel bindings & keep generating OpaqueProof.

* Move log & mining settings to Hardhat config

* Remove other references to apps & custom token

* Remove incentivized channel and apps from test & cleanup  (paritytech#759)

* Remove app and incentivized channel from tests

This also removes all existing tests, since they all depend on the apps.

* Remove channelID

* Remove some remaining references

Still need to decide on the docs/ and core/packages/api/ directories.

* Remove asset-registry pallet

Not removing the assets pallet yet, as it's still used by the XCM
pallet.

* Reorder params to match other runtimes

Makes diffs between the runtimes cleaner.

* Remove detect-fork script

* Remove helper code in src/

* Keep src/ directory

* Convert MessageId to struct
helin6 pushed a commit to boolnetwork/polkadot-sdk that referenced this issue Feb 5, 2024
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.137...v1.0.138)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants