-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dev plugin visibility #268
Merged
Merged
Changes from 37 commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
d871e65
Step 1: Add reading and default behavior of visibility flag from the …
klai95 c7428c1
Code Review Updates
klai95 04cb04f
STEP 2 and STEP 3
klai95 4032b97
Test Error and Spacing Issue
klai95 3491e51
Added default behavior for white_list parameter in filter_excluded_pl…
klai95 801a754
Refactored code
klai95 7855f6d
Updated s3 file and refactored parameter name
klai95 3fb9c6e
Added comments and refactored read/write operations in endpoints
klai95 1b0f499
Added code in update_index() to update and write plugins.json and exc…
klai95 5f66e01
Fixed test errors
klai95 4e873f9
Refactored methods and wrote comments
klai95 d4004da
Modify test file to change version to visibility parameter
klai95 3bdd831
In process of fixing test error from assert len(result) == 2 due to g…
klai95 8883fc9
Modified test_get_plugins() to have expected test code due to changes…
klai95 a9927ac
Merge branch 'main' into hidden-plugin-feature
klai95 068a532
Some PR comment changes
klai95 7298de2
Merge branch 'hidden-plugin-feature' of github.com:chanzuckerberg/nap…
klai95 22c5c7f
Refactored comment and added todo per pr comment
klai95 b7ea3b8
update plugin visibility feature
3091cd9
correct boto3 api
1efdd63
correct visibility check
bd7ef33
correct exclusion update
d4bea94
ignore exclusion stats when visibility not present
b608781
add no value return
0f6199a
refactored code structure
dac8d76
address import structure
95e6bc8
address import structure
b885e80
address project url override
fbfe761
cache plugin metadata after build
4598376
correct github repo url
9d64572
reformat
c93be46
update documentation
58cbfe3
update documentation
87af444
fix github project url reading
1fad480
remove unused variable
4bcc6f0
update documentation
7eef38f
address comments
22f725e
use slack url
67a45db
update s3 to use client only
4f55728
update s3 to use client only
6b960c0
reinstate bucket path
fd8b531
correct s3 API
6247a91
use bucket path to separate data in dev environment
07bf5ac
also verify empty url
e40a2a8
add alert for exceptions
367fce3
address comments on citation dict
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,4 +36,5 @@ jobs: | |
pip install pytest | ||
|
||
- name: Run tests | ||
run : pytest | ||
working-directory: backend | ||
run : python -m pytest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
import unittest | ||
from unittest.mock import patch | ||
|
||
from github import get_github_repo_url, get_license, get_citations | ||
from test_utils import FakeResponse, license_response, no_license_response, citation_string | ||
|
||
|
||
class TestGithub(unittest.TestCase): | ||
|
||
def test_github_get_url(self): | ||
project_urls = {"Source Code": "test1"} | ||
assert ("test1" == get_github_repo_url(project_urls)) | ||
|
||
project_urls = {"Random": "https://random.com"} | ||
assert (get_github_repo_url(project_urls) is None) | ||
|
||
project_urls = {"Random": "https://github.com/org"} | ||
assert (get_github_repo_url(project_urls) is None) | ||
|
||
project_urls = {"Random": "https://github.com/org/repo/random"} | ||
assert ("https://github.com/org/repo" == get_github_repo_url(project_urls)) | ||
|
||
@patch( | ||
'requests.get', return_value=FakeResponse(data=license_response) | ||
) | ||
def test_github_license(self, mock_get): | ||
result = get_license("test_website") | ||
assert result == "BSD-3-Clause" | ||
|
||
@patch( | ||
'requests.get', return_value=FakeResponse(data=no_license_response) | ||
) | ||
def test_github_no_assertion_license(self, mock_get): | ||
result = get_license("test_website") | ||
assert result is None | ||
|
||
def test_valid_citation(self): | ||
citation = get_citations(citation_string) | ||
assert citation['APA'] == "Fa G.N., Family G. (2019). testing (version 0.0.0). " \ | ||
"DOI: 10.0000/something.0000000 URL: https://example.com/example\n" | ||
assert citation['BibTex'] == "@misc{YourReferenceHere,\nauthor = {Fa, Gi N. and Family, Given},\n" \ | ||
"doi = {10.0000/something.0000000},\nmonth = {11},\ntitle = {testing},\n" \ | ||
"url = {https://example.com/example},\nyear = {2019}\n}\n" | ||
assert citation['RIS'] == "TY - GEN\nAB - Test\nAU - Fa, Gi N.\nAU - Family, Given\nDA - 2019-11-12\n" \ | ||
"DO - 10.0000/something.0000000\nKW - citation\nKW - test\nKW - cff\n" \ | ||
"KW - CITATION.cff\nPY - 2019\nTI - testing\n" \ | ||
"UR - https://example.com/example\nER\n" | ||
assert citation['citation'] == "# YAML 1.2\n---\nabstract: \"Test\"\nauthors:\n -\n " \ | ||
"affiliation: \"Test Center\"\n family-names: Fa\n " \ | ||
"given-names: Gi N.\n orcid: https://orcid.org/0000-0000-0000-0000\n " \ | ||
"-\n affiliation: \"Test Center 2\"\n family-names: Family\n " \ | ||
"given-names: Given\ncff-version: \"1.0.3\"\ndate-released: 2019-11-12\n" \ | ||
"doi: 10.0000/something.0000000\nkeywords:\n - \"citation\"\n - \"test\"\n" \ | ||
" - \"cff\"\n - \"CITATION.cff\"\nlicense: Apache-2.0\n" \ | ||
"message: \"If you use this software, please cite it using these metadata.\"\n" \ | ||
"repository-code: \"https://example.com/example\"\ntitle: testing\n" \ | ||
"version: \"0.0.0\"\n" | ||
|
||
def test_invalid_citation(self): | ||
citation_str = """Ha? What is this?""" | ||
citations = get_citations(citation_str) | ||
assert citations['citation'] is None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
import unittest | ||
from unittest.mock import patch | ||
|
||
from requests import HTTPError | ||
|
||
from pypi import query_pypi, get_plugin_pypi_metadata | ||
from test_utils import FakeResponse, plugin, plugin_list | ||
|
||
|
||
class TestPypi(unittest.TestCase): | ||
|
||
@patch( | ||
'requests.get', return_value=FakeResponse(data=plugin_list) | ||
) | ||
def test_query_pypi(self, mock_get): | ||
result = query_pypi() | ||
assert len(result) == 2 | ||
assert result['package1'] == "0.2.7" | ||
assert result['package2'] == "0.1.0" | ||
|
||
@patch( | ||
'requests.get', return_value=FakeResponse(data=plugin) | ||
) | ||
def test_get_plugin_pypi_metadata(self, mock_request_get): | ||
result = get_plugin_pypi_metadata("test", "0.0.1") | ||
assert (result["name"] == "test") | ||
assert (result["summary"] == "A test plugin") | ||
assert (result["description"] == "# description [example](http://example.com)") | ||
assert (result["description_content_type"] == "") | ||
assert (result["authors"] == [{'email': '[email protected]', 'name': 'Test Author'}]) | ||
assert (result["license"] == "BSD-3") | ||
assert (result["python_version"] == ">=3.6") | ||
assert (result["operating_system"] == ['Operating System :: OS Independent']) | ||
assert (result["release_date"] == '2020-04-13T03:37:20.169990Z') | ||
assert (result["version"] == "0.0.1") | ||
assert (result["first_released"] == "2020-04-13T03:37:20.169990Z") | ||
assert (result["development_status"] == ['Development Status :: 4 - Beta']) | ||
assert (result["requirements"] is None) | ||
assert (result["project_site"] == "https://github.com/test/test") | ||
assert (result["documentation"] == "") | ||
assert (result["support"] == "") | ||
assert (result["report_issues"] == "") | ||
assert (result["twitter"] == "") | ||
|
||
@patch( | ||
'requests.get', side_effect=HTTPError() | ||
) | ||
def test_get_plugin_error(self, mock_get): | ||
assert ({} == get_plugin_pypi_metadata("test", "0.0.1")) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
import unittest | ||
from unittest.mock import patch | ||
|
||
|
||
@patch( | ||
'model.get_valid_plugins', return_value={"package1": "0.0.1"} | ||
) | ||
class TestShield(unittest.TestCase): | ||
|
||
def test_get_shield(self, mock_get_valid_plugins): | ||
from shield import get_shield | ||
result = get_shield('package1') | ||
assert result['message'] == 'package1' | ||
assert 'label' in result | ||
assert 'schemaVersion' in result | ||
assert 'color' in result | ||
|
||
def test_get_shield_for_non_plugin(self, mock_get_valid_plugins): | ||
from shield import get_shield | ||
result = get_shield('not-a-package') | ||
assert result['message'] == 'plugin not found' | ||
assert 'label' in result | ||
assert 'schemaVersion' in result | ||
assert 'color' in result |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,6 @@ | ||
from unittest import mock | ||
import requests | ||
from requests.exceptions import HTTPError | ||
|
||
from backend.napari import get_plugin, get_shield | ||
from backend.napari import get_plugins | ||
from backend.napari import get_download_url | ||
from backend.napari import get_license | ||
from backend.napari import get_citations | ||
|
||
|
||
class FakeResponse: | ||
def __init__(self, *, data: str): | ||
|
@@ -97,86 +90,6 @@ def raise_for_status(self): | |
"upload_time_iso_8601":"2020-04-20T15:28:53.386281Z", | ||
"url":"","yanked":false,"yanked_reason":null}]}}""" | ||
|
||
@mock.patch( | ||
'requests.get', return_value=FakeResponse(data=plugin_list) | ||
) | ||
def test_get_plugins(mock_get): | ||
result = get_plugins() | ||
assert len(result) == 2 | ||
assert result['package1'] == "0.2.7" | ||
assert result['package2'] == "0.1.0" | ||
|
||
@mock.patch( | ||
'requests.get', return_value=FakeResponse(data=plugin) | ||
) | ||
@mock.patch( | ||
'backend.napari.get_plugins', return_value={'test': '0.0.1'} | ||
) | ||
def test_get_plugin(mock_get, mock_plugins): | ||
result = get_plugin("test") | ||
assert(result["name"] == "test") | ||
assert(result["summary"] == "A test plugin") | ||
assert(result["description"] == "# description [example](http://example.com)") | ||
assert(result["description_text"] == "description example") | ||
assert(result["description_content_type"] == "") | ||
assert(result["authors"] == [{'email': '[email protected]', 'name': 'Test Author'}]) | ||
assert(result["license"] == "BSD-3") | ||
assert(result["python_version"] == ">=3.6") | ||
assert(result["operating_system"] == ['Operating System :: OS Independent']) | ||
assert(result["release_date"] == '2020-04-13T03:37:20.169990Z') | ||
assert(result["version"] == "0.0.1") | ||
assert(result["first_released"] == "2020-04-13T03:37:20.169990Z") | ||
assert(result["development_status"] == ['Development Status :: 4 - Beta']) | ||
assert(result["requirements"] is None) | ||
assert(result["project_site"] == "https://github.com/test/test") | ||
assert(result["documentation"] == "") | ||
assert(result["support"] == "") | ||
assert(result["report_issues"] == "") | ||
assert(result["twitter"] == "") | ||
assert(result["code_repository"] == "https://github.com/test/test") | ||
|
||
|
||
@mock.patch( | ||
'requests.get', return_value=FakeResponse(data=plugin) | ||
) | ||
@mock.patch( | ||
'backend.napari.get_plugins', return_value={'not_test': '0.0.1'} | ||
) | ||
def test_get_invalid_plugin(mock_get, mock_plugins): | ||
assert({} == get_plugin("test")) | ||
|
||
|
||
@mock.patch( | ||
'backend.napari.get_plugins', return_value=plugin_list | ||
) | ||
def test_get_shield(mock_plugins): | ||
result = get_shield('package1') | ||
assert result['message'] == 'package1' | ||
assert 'label' in result | ||
assert 'schemaVersion' in result | ||
assert 'color' in result | ||
|
||
result = get_shield('not-a-package') | ||
assert result['message'] == 'plugin not found' | ||
assert 'label' in result | ||
assert 'schemaVersion' in result | ||
assert 'color' in result | ||
|
||
|
||
def test_github_get_url(): | ||
plugins = {"info": {"project_urls": {"Source Code": "test1"}}} | ||
assert("test1" == get_download_url(plugins)) | ||
|
||
plugins = {"info": {"project_urls": {"Random": "https://random.com"}}} | ||
assert(get_download_url(plugins) is None) | ||
|
||
plugins = {"info": {"project_urls": {"Random": "https://github.com/org"}}} | ||
assert(get_download_url(plugins) is None) | ||
|
||
plugins = {"info": {"project_urls": {"Random": "https://github.com/org/repo/random"}}} | ||
assert("https://github.com/org/repo" == get_download_url(plugins)) | ||
|
||
|
||
license_response = """ | ||
{ | ||
"name": "LICENSE", | ||
|
@@ -190,15 +103,6 @@ def test_github_get_url(): | |
} | ||
""" | ||
|
||
|
||
@mock.patch( | ||
'requests.get', return_value=FakeResponse(data=license_response) | ||
) | ||
def test_github_license(mock_get): | ||
result = get_license("test_website") | ||
assert result == "BSD-3-Clause" | ||
|
||
|
||
no_license_response = """ | ||
{ | ||
"name": "LICENSE", | ||
|
@@ -212,17 +116,7 @@ def test_github_license(mock_get): | |
} | ||
""" | ||
|
||
|
||
@mock.patch( | ||
'requests.get', return_value=FakeResponse(data=no_license_response) | ||
) | ||
def test_github_no_assertion_license(mock_get): | ||
result = get_license("test_website") | ||
assert result is None | ||
|
||
|
||
def test_valid_citation(): | ||
citation_string = """# YAML 1.2 | ||
citation_string = """# YAML 1.2 | ||
--- | ||
abstract: "Test" | ||
authors: | ||
|
@@ -249,39 +143,3 @@ def test_valid_citation(): | |
title: testing | ||
version: "0.0.0" | ||
""" | ||
citation = get_citations(citation_string) | ||
assert citation['citation'] == citation_string | ||
assert citation['RIS'] == """TY - COMP | ||
AU - Fa, Gi N. | ||
AU - Family, Given | ||
DO - 10.0000/something.0000000 | ||
KW - citation | ||
KW - test | ||
KW - cff | ||
KW - CITATION.cff | ||
M3 - software | ||
PB - GitHub Inc. | ||
PP - San Francisco, USA | ||
PY - 2019/11/12 | ||
T1 - testing | ||
UR - https://example.com/example | ||
ER - | ||
""" | ||
assert citation['BibTex'] == """@misc{YourReferenceHere, | ||
author = { | ||
Gi N. Fa and | ||
Given Family | ||
}, | ||
title = {testing}, | ||
month = {11}, | ||
year = {2019}, | ||
doi = {10.0000/something.0000000}, | ||
url = {https://example.com/example} | ||
} | ||
""" | ||
|
||
|
||
def test_invalid_citation(): | ||
citation_str = """Ha? What is this?""" | ||
citations = get_citations(citation_str) | ||
assert citations['citation'] is None |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we refactoring the
path
for the sake of simplicity?"There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
originally the update is only updating the index file, however right now the update is responsible for keeping content up to date across all resources, so this change is to would reflect the actual logical level of the endpoint