Pre-release merge for search v0.5.6, search-api v0.2 #272

mhl10 · 2020-02-21T16:43:44Z

Release notes - search-api-0.2, search-0.5.6

The vast majority of changes are related to search APIs, particularly the classic search API reimplementation. Most files have been touched since the last release because of Black reformatting (now part of a pre-commit hook), so it may not be useful to review all files; I have highlighted a few with comments.

Many thanks to @alefnula for his big push on the classic API reimplementation and other helpful contributions!

New Feature

[ARXIVNG-1896] - Both new and legacy APIs should support max page size of 2,000 results
[ARXIVNG-1897] - Support for legacy API query parameter
[ARXIVNG-1898] - Implement routes module for legacy API

Task

[ARXIVNG-2965] - update base release and fix wsgi.py
[ARXIVNG-1516] - Reimplement Atom/XML serialization for metadata API
[ARXIVNG-1518] - Update OpenAPI schema with Atom/XML examples, Accept header
[ARXIVNG-1893] - Add a `query` parameter to the JSON-based API Schema
[ARXIVNG-1895] - Add start and size to openapi schema of JSON API
[ARXIVNG-1971] - Implement boolean search in Search API
[ARXIVNG-1972] - Support for parenthetical boolean operations and search
[ARXIVNG-2043] - Handle author affiliations in Atom search results
[ARXIVNG-2668] - Stage search API and classic API on beta.arxiv.org
[ARXIVNG-2904] - Classic arXiv API error handling
[ARXIVNG-2905] - Classic arXiv API query search_query and id_list logic
[ARXIVNG-2906] - Classic arXiv API start and max_results paging
[ARXIVNG-2907] - Classic arXiv API sort order for return results
[ARXIVNG-2972] - Fix the ordering issue for the classical API

…arch.process.transform

Co-Authored-By: JaimieMurdock <[email protected]>

Atom/XML Serializer

Initial boolean support ARXIVNG-1971

ARXIVNG-2224 ARXIVNG-2225 added base app tests, fixed Elasticsearch config

… recursive descent

Co-Authored-By: Martin Lessmeister <[email protected]>

Feature/#258 search query and id list

Remove all references to deprecated arxiv.status module.

* mypy fixes * pydocstyle fixes, including relaxation of D202 * move linstats.sh to travis script section

claassistantio · 2020-02-21T16:44:02Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 5 committers have signed the CLA.

✅ erickpeirson
✅ JaimieMurdock
✅ alefnula
✅ mhl10
❌ dependabot[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

mhl10 · 2020-02-21T19:38:13Z

search/domain/classic_api/query_parser.py

+"""
+Utility module for Classic API Query parsing.
+
+Uses lark-parser (EBNF parser) [1].


The classic API reimplementation has been updated to use lark-parser.

mhl10 · 2020-02-21T19:39:39Z

.pre-commit-config.yaml

@@ -0,0 +1,35 @@
+- repo: https://github.com/ambv/black


New pre-commit hooks. Installation instructions are in the readme.

SBBCornell

I cant comment on the code, but it spins up, looks as expected, and the footer links and new colors from base are correct!

bdc34

This seems good to me.

bdc34 · 2020-02-21T21:09:32Z

search/domain/classic_api/query_parser.py

+"""
+Utility module for Classic API Query parsing.
+
+Uses lark-parser (EBNF parser) [1].


bdc34 · 2020-02-21T21:10:15Z

search/domain/classic_api/query_parser.py

+        return tokens[0]  # type:ignore
+
+
+QUERY_PARSER = Lark(


JaimieMurdock and others added 30 commits March 7, 2019 00:19

Adding notes and boolean support to the _parse_search_query function

29412db

initial boolean support by chaninging _get_fielded_params

8568891

utilizing the multidict better

c496331

upgrading mypy, adding some type: ignores

2d58f5a

Correcting mypy errors for tests

9a1389d

Merge branch 'develop' into task/ARXIVNG-1971

d50adc1

Atom serializer

6ee11de

Adding stubs for where arxiv:affiliation needs to be handled

afef5bb

Adding author affiliation handling to entries. Still no support in se…

a30916b

…arch.process.transform

Adding note about not supporting serialize_document

79efeab

Apply suggestions from code review

24ab5f9

Co-Authored-By: JaimieMurdock <[email protected]>

Merge pull request #239 from arXiv/task/ARXIVNG-1516

b1848d4

Atom/XML Serializer

Merge pull request #234 from arXiv/task/ARXIVNG-1971

f74a4db

Initial boolean support ARXIVNG-1971

adding a TODO

9098718

ARXIVNG-2224 added base app_tests to test suite

fac1c40

ARXIVNG-2224 run base tests separately

204825e

ARXIVNG-2225 renamed elasticsearch config params

c285e33

Merge pull request #248 from arXiv/task/ARXIVNG-2224

ddb1dfa

ARXIVNG-2224 ARXIVNG-2225 added base app tests, fixed Elasticsearch config

working on classic API query parsing

f757177

ARXIVNG-1972 most of the way to parsing query strings

0262736

ARXIVNG-1972 fixed last querystring parsing test

09a9da0

ARXIVNG-1972 cleanup

ca4a3c6

adding test for first arg disjunct

bc933e1

adding a test with dual phrases

858e437

adding a new tokenizer that avoids some of the issues with parens via…

b85c9a7

… recursive descent

recursive tokenizer

080027b

code reorganization

21620dc

skeleton for classic search api in service.index

d8315d7

phrase->DSL QueryString conversion

ddb4e61

classic queries!

3bba91f

alefnula and others added 11 commits February 20, 2020 10:30

Refactor query parser test cases for easier readability.

1f1ef30

Implemented new parsing of classic query string

c1606c2

disable generator entry

41059a2

Update search/domain/classic_api/query_parser.py

fc7911f

Co-Authored-By: Martin Lessmeister <[email protected]>

Fix paper_id version matching.

7b44512

Add documentation.

c39a18e

Empty query handling.

fc700ef

Merge pull request #269 from arXiv/feature/#258-search-query-and-id-list

342ff00

Feature/#258 search query and id list

Remove all references to deprecated arxiv.status module.

0ed5b58

Merge pull request #270 from arXiv/feature/http-status-refactor

d8145b5

Remove all references to deprecated arxiv.status module.

mypy and pydocstyle fixes (#271)

9fe9f9d

* mypy fixes * pydocstyle fixes, including relaxation of D202 * move linstats.sh to travis script section

mhl10 changed the title ~~Pre-release merge for search v0.5.6~~ Pre-release merge for search v0.5.6, search-api v0.2 Feb 21, 2020

mhl10 commented Feb 21, 2020

View reviewed changes

mhl10 added 5 commits February 21, 2020 14:40

bump version

d45f9c5

minor change to pre-commit instructions

76d8c70

update release notes URL, text

77341a9

instructions for running classic API

10624dc

update feedback collector component

d68c614

mhl10 requested review from alefnula, bdc34, DavidLFielding and SBBCornell February 21, 2020 19:58

mhl10 marked this pull request as ready for review February 21, 2020 19:58

Merge branch 'master' into develop

eb725b1

SBBCornell approved these changes Feb 21, 2020

View reviewed changes

bdc34 approved these changes Feb 21, 2020

View reviewed changes

alefnula approved these changes Feb 24, 2020

View reviewed changes

mhl10 merged commit 0797c67 into master Feb 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pre-release merge for search v0.5.6, search-api v0.2 #272

Pre-release merge for search v0.5.6, search-api v0.2 #272

mhl10 commented Feb 21, 2020 •

edited

Loading

claassistantio commented Feb 21, 2020 •

edited

Loading

mhl10 Feb 21, 2020

bdc34 Feb 21, 2020

mhl10 Feb 21, 2020

SBBCornell left a comment

bdc34 left a comment

bdc34 Feb 21, 2020

bdc34 Feb 21, 2020

Pre-release merge for search v0.5.6, search-api v0.2 #272

Pre-release merge for search v0.5.6, search-api v0.2 #272

Conversation

mhl10 commented Feb 21, 2020 • edited Loading

Release notes - search-api-0.2, search-0.5.6

New Feature

Task

claassistantio commented Feb 21, 2020 • edited Loading

mhl10 Feb 21, 2020

Choose a reason for hiding this comment

bdc34 Feb 21, 2020

Choose a reason for hiding this comment

mhl10 Feb 21, 2020

Choose a reason for hiding this comment

SBBCornell left a comment

Choose a reason for hiding this comment

bdc34 left a comment

Choose a reason for hiding this comment

bdc34 Feb 21, 2020

Choose a reason for hiding this comment

bdc34 Feb 21, 2020

Choose a reason for hiding this comment

mhl10 commented Feb 21, 2020 •

edited

Loading

claassistantio commented Feb 21, 2020 •

edited

Loading