feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934

mattdibi · 2023-10-30T16:09:31Z

This PR follows #4894 and adds the endpoint for Eclipse Marketplace's Package Descriptor retrieval.

Can be tested using:

export ADDRESS=<addr>
export USERNAME=<user>
export PASSWORD=<pass>

and then:

curl -X PUT -k -u $USERNAME:$PASSWORD \
       -H 'Content-Type: application/json' \
       -d '{ "url": "http://marketplace.eclipse.org/marketplace-client-intro?mpc_install=5514714" }' \
       https://$ADDRESS/services/deploy/v2/_packageDescriptor

Roadmap:

Contribute deploy/v2 in its current form (feat(rest.packages.provider): add deploy/v2 Rest API #4868)
Add endpoint for Deployment Package upload using multipart/form-data (feat(rest.packages.provider): add endpoint for .dp upload #4894)
Add Depoloyment Package descriptor download and checking to install endpoint (feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934)

Changelog

I mainly repurposed the code in:

Creating a new POJO (i.e. MarketplacePackageDescriptor) for representing the info contained in the downloaded descriptor and basically using everything in the rest.packages.provider.

mattdibi · 2023-10-31T13:58:16Z

Setting this as ready for review since the only thing missing are tests at this point

WIP

…nterface

WIP

STILL WIP: we should at least have a dedicated request type instead of recycling InstallRequest

...deployment.agent/src/main/java/org/eclipse/kura/deployment/agent/DeploymentAgentService.java

nicolatimeus · 2023-11-10T09:50:00Z

Please update the exported package version to 1.1.0 here https://github.com/mattdibi/kura/blob/73adee3550e2d4febf49bfe16abe7207aa4078dc/kura/org.eclipse.kura.deployment.agent/META-INF/MANIFEST.MF#L9

nicolatimeus · 2023-11-10T09:55:56Z

kura/org.eclipse.kura.web2/src/main/java/org/eclipse/kura/web/server/GwtPackageServiceImpl.java

-
-            mpUrl = new URL(String.format(MARKETPLACE_URL, nodeId));
-            connection = (HttpsURLConnection) mpUrl.openConnection();
-            connection.setSSLSocketFactory(this.sslManagerService.getSSLSocketFactory());


Currently the console has a configuration property that allows to specify the SslManagerService that is used to fetch deployment packages from the marketlplace, this change would make it use the SslManagerService configured in the deployment agent instead.. We could add another getMarketplacePackageDescriptor(String, SslManagerService) to the DeploymentAgent that allows to specify the SslManagerService to restore the old behaviour

Addressed in 08eca41

nicolatimeus · 2023-11-10T09:56:47Z

Please also update copyright headers in all changed files

nicolatimeus · 2023-11-10T09:59:33Z

Please update the exported package version to 1.1.0 here https://github.com/mattdibi/kura/blob/73adee3550e2d4febf49bfe16abe7207aa4078dc/kura/org.eclipse.kura.deployment.agent/META-INF/MANIFEST.MF#L9

We should also update the import declarations of this package to use version range [1.1,2.0) in the bundles that use the new methods (console)

mattdibi · 2023-11-10T10:20:30Z

Please update the exported package version to 1.1.0 here https://github.com/mattdibi/kura/blob/73adee3550e2d4febf49bfe16abe7207aa4078dc/kura/org.eclipse.kura.deployment.agent/META-INF/MANIFEST.MF#L9

Addressed in d0dee4c

We should also update the import declarations of this package to use version range [1.1,2.0) in the bundles that use the new methods (console)

Addressed in d0dee4c

Please also update copyright headers in all changed files

Addressed in 98cbbaa

…onnection

mattdibi · 2023-11-11T14:11:58Z

Update to mockserver v 5.15.0. Newly added dependency is ✅

echo "maven/mavencentral/org.mock-server/mockserver-netty-no-dependencies/5.15.0" | java -jar ~/Downloads/org.eclipse.dash.licenses-1.0.3-20231111.065112-172.jar -
[main] INFO Querying Eclipse Foundation for license data for 1 items.
[main] INFO Found 1 items.
[main] INFO Vetted license information was found for all content. No further investigation is required.

echo "maven/mavencentral/org.mock-server/mockserver-client-java-no-dependencies/5.15.0" | java -jar ~/Downloads/org.eclipse.dash.licenses-1.0.3-20231111.065112-172.jar -
[main] INFO Querying Eclipse Foundation for license data for 1 items.
[main] INFO Found 1 items.
[main] INFO Vetted license information was found for all content. No further investigation is required.

maven/mavencentral/org.mock-server/mockserver-client-java-no-dependencies/5.15.0, Apache-2.0 AND LGPL-3.0-only, approved, #11514
maven/mavencentral/org.mock-server/mockserver-netty-no-dependencies/5.15.0, Apache-2.0 AND LGPL-3.0-only AND OFL-1.1 AND (BSD-3-Clause AND MIT), approved, #11515

See:

github-actions · 2023-11-13T10:05:45Z

The backport to release-5.4.0 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-release-5.4.0 release-5.4.0
# Navigate to the new working tree
cd .worktrees/backport-release-5.4.0
# Create a new branch
git switch --create backport-4934-to-release-5.4.0
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 5b7b71cd83bbfb37209e39998bde732916586fe3
# Push it to GitHub
git push --set-upstream origin backport-4934-to-release-5.4.0
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-release-5.4.0

Then, create a pull request where the base branch is release-5.4.0 and the compare/head branch is backport-4934-to-release-5.4.0.

…rieval (#4934) * feat: add MarketplacePackageDescriptor.java * feat: add barebone getMarketplacePackageDescriptor method implementation WIP * feat: add getMarketplacePackageDescriptor to DeploymentAgentService interface * feat: add isEclipseMarketplaceUrl check WIP * style: fix variable name * feat: add quick&dirty descriptor check endpoint STILL WIP: we should at least have a dedicated request type instead of recycling InstallRequest * feat: return json object * debug: add debugging log * feat: actually perform check on input url * feat: add proper Kura version retrival * refactor: move instantiation * refactor: use proper exceptions * style: add copyright header * feat: use QueryParam for passing the URL to be checked * feat: provide equals and hashCode overrides * fix: do not log user-controlled data * fix: replace character class by the character itself * test: add unit tests for MarketplacePackageDescriptor * style: add copyright header * style: refactor into gerkhin style * refactor: move given/when/then methods in their sections * refactor: use the builder for gods sake * style: add forgotten about files * test: add DeploymentRestServiceUnitTest for marketplace descriptor * refactor: move URL checks outside DeploymentAgentService to allow for testing * test: update unit tests * refactor: move method into its section * test: add integration tests * fix: remove commented out code * test: add some more tests * test: add test case for failing getMarketplacePackageDescriptor call * docs: add Javadocs for DeploymentRestService * docs: add javadocs for DeploymentAgentService * test: add DeploymentAgentTest:getMarketplacePackageDescriptor tests (#11) * ci(debug): set envvar to run testcontainers with podman * refactor: user raw Mockserver instead of going through Testcontainers * refactor: remove unnecessary dependencies * ci: roll-back changes * test: use random free port * test: add null URL test * test: reset MockServer status after each test * feat: GET -> PUT * test: GET -> PUT * refactor(web2): modify UI so that it uses the newly introduced method * build: downgrade mockserver 5.15.0 -> 5.14.0 * build: I'm a dum dum * docs: add "since" annotation for newly introduced method * style: update copyright headers * chore: deployment agent service version bump to 1.1.0 * feat: add method for overriding SslManagerService used to establish connection * style: restore formatting * style: update copyright headers * style: update copyright header * build: switched to mockserver v 5.15.0 * docs: remove newly added test deps from NOTICE file

…rieval [backport release-5.4.0] (#4968) feat(rest.packages.provider): add endpoint for Package Descriptor retrieval (#4934) * feat: add MarketplacePackageDescriptor.java * feat: add barebone getMarketplacePackageDescriptor method implementation WIP * feat: add getMarketplacePackageDescriptor to DeploymentAgentService interface * feat: add isEclipseMarketplaceUrl check WIP * style: fix variable name * feat: add quick&dirty descriptor check endpoint STILL WIP: we should at least have a dedicated request type instead of recycling InstallRequest * feat: return json object * debug: add debugging log * feat: actually perform check on input url * feat: add proper Kura version retrival * refactor: move instantiation * refactor: use proper exceptions * style: add copyright header * feat: use QueryParam for passing the URL to be checked * feat: provide equals and hashCode overrides * fix: do not log user-controlled data * fix: replace character class by the character itself * test: add unit tests for MarketplacePackageDescriptor * style: add copyright header * style: refactor into gerkhin style * refactor: move given/when/then methods in their sections * refactor: use the builder for gods sake * style: add forgotten about files * test: add DeploymentRestServiceUnitTest for marketplace descriptor * refactor: move URL checks outside DeploymentAgentService to allow for testing * test: update unit tests * refactor: move method into its section * test: add integration tests * fix: remove commented out code * test: add some more tests * test: add test case for failing getMarketplacePackageDescriptor call * docs: add Javadocs for DeploymentRestService * docs: add javadocs for DeploymentAgentService * test: add DeploymentAgentTest:getMarketplacePackageDescriptor tests (#11) * ci(debug): set envvar to run testcontainers with podman * refactor: user raw Mockserver instead of going through Testcontainers * refactor: remove unnecessary dependencies * ci: roll-back changes * test: use random free port * test: add null URL test * test: reset MockServer status after each test * feat: GET -> PUT * test: GET -> PUT * refactor(web2): modify UI so that it uses the newly introduced method * build: downgrade mockserver 5.15.0 -> 5.14.0 * build: I'm a dum dum * docs: add "since" annotation for newly introduced method * style: update copyright headers * chore: deployment agent service version bump to 1.1.0 * feat: add method for overriding SslManagerService used to establish connection * style: restore formatting * style: update copyright headers * style: update copyright header * build: switched to mockserver v 5.15.0 * docs: remove newly added test deps from NOTICE file

mattdibi marked this pull request as ready for review October 31, 2023 13:58

mattdibi force-pushed the refactor/deployment_packages_download branch from e89e70e to 4988ad4 Compare October 31, 2023 13:59

mattdibi requested a review from nicolatimeus October 31, 2023 15:34

mattdibi added the backport release-5.4.0 label Nov 2, 2023

mattdibi added 24 commits November 6, 2023 15:26

feat: add MarketplacePackageDescriptor.java

4a72816

feat: add barebone getMarketplacePackageDescriptor method implementation

28ce696

WIP

feat: add getMarketplacePackageDescriptor to DeploymentAgentService i…

40f2d94

…nterface

feat: add isEclipseMarketplaceUrl check

f76e3a5

WIP

style: fix variable name

6b99f60

feat: add quick&dirty descriptor check endpoint

c6d6006

STILL WIP: we should at least have a dedicated request type instead of recycling InstallRequest

feat: return json object

9862950

debug: add debugging log

d8b7a2a

feat: actually perform check on input url

575949e

feat: add proper Kura version retrival

1579ea4

refactor: move instantiation

3b6920f

refactor: use proper exceptions

892dd90

style: add copyright header

3f6647f

feat: use QueryParam for passing the URL to be checked

f642533

feat: provide equals and hashCode overrides

87d9bb6

fix: do not log user-controlled data

738b1db

fix: replace character class by the character itself

ae7acde

test: add unit tests for MarketplacePackageDescriptor

3234ed7

style: add copyright header

c43c882

style: refactor into gerkhin style

812858f

refactor: move given/when/then methods in their sections

b424881

refactor: use the builder for gods sake

50b6096

style: add forgotten about files

d890cd8

test: add DeploymentRestServiceUnitTest for marketplace descriptor

4aa99d0

This comment was marked as outdated.

Sign in to view

build: I'm a dum dum

73adee3

nicolatimeus reviewed Nov 10, 2023

View reviewed changes

...deployment.agent/src/main/java/org/eclipse/kura/deployment/agent/DeploymentAgentService.java Show resolved Hide resolved

nicolatimeus reviewed Nov 10, 2023

View reviewed changes

mattdibi added 3 commits November 10, 2023 11:03

docs: add "since" annotation for newly introduced method

280d140

style: update copyright headers

98cbbaa

chore: deployment agent service version bump to 1.1.0

d0dee4c

mattdibi added 2 commits November 10, 2023 11:59

feat: add method for overriding SslManagerService used to establish c…

08eca41

…onnection

style: restore formatting

91545f5

mattdibi force-pushed the refactor/deployment_packages_download branch from e2ccf4e to 91545f5 Compare November 10, 2023 11:22

mattdibi added 3 commits November 10, 2023 12:22

style: update copyright headers

617a5ae

style: update copyright header

5c9da39

build: switched to mockserver v 5.15.0

883b381

docs: remove newly added test deps from NOTICE file

808b7fd

nicolatimeus approved these changes Nov 13, 2023

View reviewed changes

mattdibi merged commit 5b7b71c into eclipse-kura:develop Nov 13, 2023

mattdibi deleted the refactor/deployment_packages_download branch November 13, 2023 10:04

mattdibi mentioned this pull request Nov 13, 2023

feat(rest.packages.provider): add endpoint for Package Descriptor retrieval [backport release-5.4.0] #4968

Merged

This was referenced Nov 23, 2023

fix(linux.net): prefer dnsmasq over udhcpd #5017

Closed

fix(nm): load dhcp server configuration after change #5018

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934

feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934

mattdibi commented Oct 30, 2023 •

edited

Loading

mattdibi commented Oct 31, 2023

This comment was marked as outdated.

nicolatimeus commented Nov 10, 2023

nicolatimeus Nov 10, 2023

mattdibi Nov 10, 2023

nicolatimeus commented Nov 10, 2023

nicolatimeus commented Nov 10, 2023

mattdibi commented Nov 10, 2023

mattdibi commented Nov 11, 2023

github-actions bot commented Nov 13, 2023

feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934

feat(rest.packages.provider): add endpoint for Package Descriptor retrieval #4934

Conversation

mattdibi commented Oct 30, 2023 • edited Loading

Changelog

mattdibi commented Oct 31, 2023

This comment was marked as outdated.

nicolatimeus commented Nov 10, 2023

nicolatimeus Nov 10, 2023

Choose a reason for hiding this comment

mattdibi Nov 10, 2023

Choose a reason for hiding this comment

nicolatimeus commented Nov 10, 2023

nicolatimeus commented Nov 10, 2023

mattdibi commented Nov 10, 2023

mattdibi commented Nov 11, 2023

github-actions bot commented Nov 13, 2023

mattdibi commented Oct 30, 2023 •

edited

Loading