-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Search Service - Public Preview #1
Comments
Hi we are using the API at the Library of Congress and would like to see the ability:
|
Keyword search capability would be a terrific feature and would significantly enhance the value of the API. |
@eheitfield -thanks for the input. We're looking to enable similar/equivalent search functionality as is available via the UI. Requires some additional planning and design for consistency's sake. From an api user standpoint, would it be preferable to keep the syntax for searches to be similar to the UI -- allowing field operators, filters, etc.? |
A search service as proposed by jonquandt would be a big plus. A syntax similar to the existing UI is fine. |
Yes, please keep the syntax similar to the UI because it'll make research easier. It would be great to be able to search |
The keyword search would be great. Is it available now? |
@yanbinghan - not yet. We're looking at prioritizing this into a future release. In the meantime, you may be interested in looking at our recent feature allowing for CSV/JSON exports of search results on the UI. any additional feedback or thoughts on what you would want to see in the search service would be very valuable. |
@jonquandt Thank you for your quick response. Could you please refer me to the link on how to use the recent feature allowing for CSV/JSON exports of search results on the UI? |
@yanbinghan -- You just need to perform a search. Then in the top right of the search results there is a series of icons that include social media, email, and a link icon. To the right is a download button. Click on that and then choose Here's an image that demonstrates the feature: |
I'd really like to be able to get an API call for this. Am I correct in that it's not available yet, but it's something you're looking to add? Something like:
Etc. |
@krisrobinson - yes, that's correct. The search service is still something that's on our roadmap, and any additional examples will be helpful as we develop it. |
Additional example from a user - they would like to be able to pull specific USCOURTS packages by the Nature of Suit metadata.
|
We have a goal to deliver some additional information for public feedback over the summer, including:
Our initial goal is to provide capabilities that allows the use of a large number of fields currently available in the user interface, with a default set of fields returned for responsive items in the search query. As we receive feedback, we may make tweaks to the set of fields or other items to address community questions. We are currently targeting Fall 2023 for an initial launch of search functionality in the API. |
I'm building a tool that will regularly look for any record that fulfills a given Boolean string. Is the planned search function on the API going to enable this to be done? I will need retrieve all the data of records that come up for the search. A manual version of the type of search is below: |
@RogueSergeant - I am adding this to our list of test cases, but it appears that we will be able to support that kind of request, yes. |
Hi @jonquandt thanks for that, do you have an update on how development is coming along and when we can expect a launch? |
@RogueSergeant - We are in the midst of internal testing now. A Public preview version of the search will be available next week for initial feedback. We will post more information on GitHub and GovInfo when it's available. |
@RogueSergeant - Quick sample search and response from a limited dataset test environment. I will provide additional examples when we go live, along with some additional documentation as part of an overview article. To get the next page of results, you would send the same post request, but pull the offsetMark value from the response and replace
Response: {
"results": [
{
"title": "Polyvinyl alcohol film.",
"packageId": "CFR-2018-title21-vol3",
"granuleId": "CFR-2018-title21-vol3-sec177-1670",
"lastModified": "2023-08-07T05:32:59Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2018-04-01",
"collectionCode": "CFR",
"resultLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/granules/CFR-2018-title21-vol3-sec177-1670/summary",
"dateIngested": "2023-02-09",
"download": {
"premisLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/premis",
"xmlLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/granules/CFR-2018-title21-vol3-sec177-1670/xml",
"zipLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/zip",
"modsLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/granules/CFR-2018-title21-vol3-sec177-1670/mods",
"pdfLink": "https://api.govinfo.gov/packages/CFR-2018-title21-vol3/granules/CFR-2018-title21-vol3-sec177-1670/pdf"
}
},
{
"title": "Polyvinyl Alcohol From Taiwan: Preliminary Results of Antidumping Duty Administrative Review; 2010-2012",
"packageId": "FR-2013-04-08",
"granuleId": "2013-08110",
"lastModified": "2023-08-06T14:24:07Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2013-04-08",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2013-04-08/granules/2013-08110/summary",
"dateIngested": "2023-02-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2013-04-08/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2013-04-08/granules/2013-08110/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2013-04-08/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2013-04-08/granules/2013-08110/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2013-04-08/granules/2013-08110/pdf"
}
},
{
"title": "Polyvinyl Alcohol From China and Japan; Institution of Five-Year Reviews",
"packageId": "FR-2020-04-01",
"granuleId": "2020-06718",
"lastModified": "2023-08-04T21:15:49Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2020-04-01",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2020-04-01/granules/2020-06718/summary",
"dateIngested": "2023-02-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2020-04-01/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2020-04-01/granules/2020-06718/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2020-04-01/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2020-04-01/granules/2020-06718/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2020-04-01/granules/2020-06718/pdf"
}
},
{
"title": "Polyvinyl Alcohol From China, Japan, and Korea; Revised Schedule for Full Five-Year Reviews",
"packageId": "FR-2015-03-12",
"granuleId": "2015-05599",
"lastModified": "2023-08-06T10:50:34Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2015-03-12",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2015-03-12/granules/2015-05599/summary",
"dateIngested": "2023-02-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2015-03-12/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2015-03-12/granules/2015-05599/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2015-03-12/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2015-03-12/granules/2015-05599/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2015-03-12/granules/2015-05599/pdf"
}
},
{
"title": "Polyvinyl Alcohol From China, Japan, and Korea: Revised Schedule for Full Five-Year Reviews",
"packageId": "FR-2015-02-05",
"granuleId": "2015-02286",
"lastModified": "2023-08-06T11:35:46Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2015-02-05",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2015-02-05/granules/2015-02286/summary",
"dateIngested": "2023-02-08",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2015-02-05/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2015-02-05/granules/2015-02286/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2015-02-05/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2015-02-05/granules/2015-02286/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2015-02-05/granules/2015-02286/pdf"
}
},
{
"title": "Polyvinyl Alcohol from Taiwan: Investigation No. 731-Ta-1088 (Final)",
"packageId": "GOVPUB-ITC1-PURL-gpo15352",
"granuleId": null,
"lastModified": "2023-06-07T12:48:18Z",
"governmentAuthor": [
"International Trade Commission"
],
"dateIssued": "2011-01-01",
"collectionCode": "GOVPUB",
"resultLink": "https://api.govinfo.gov/packages/GOVPUB-ITC1-PURL-gpo15352/summary",
"dateIngested": "2023-06-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/GOVPUB-ITC1-PURL-gpo15352/premis",
"zipLink": "https://api.govinfo.gov/packages/GOVPUB-ITC1-PURL-gpo15352/zip",
"modsLink": "https://api.govinfo.gov/packages/GOVPUB-ITC1-PURL-gpo15352/mods",
"pdfLink": "https://api.govinfo.gov/packages/GOVPUB-ITC1-PURL-gpo15352/pdf"
}
},
{
"title": "Polyvinyl Alcohol (PVA); TSCA Section 21 Petition for Rulemaking; Reasons for Agency Response; Denial of Requested Rulemaking",
"packageId": "FR-2023-04-27",
"granuleId": "2023-08864",
"lastModified": "2023-08-09T15:47:19Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2023-04-27",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2023-04-27/granules/2023-08864/summary",
"dateIngested": "2023-08-09",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2023-04-27/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2023-04-27/granules/2023-08864/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2023-04-27/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2023-04-27/granules/2023-08864/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2023-04-27/granules/2023-08864/pdf"
}
},
{
"title": "Polyvinyl Alcohol From the People's Republic of China: Rescission of Antidumping Duty Administrative Review; 2013-2014",
"packageId": "FR-2015-04-01",
"granuleId": "2015-07471",
"lastModified": "2023-08-05T08:37:16Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2015-04-01",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2015-04-01/granules/2015-07471/summary",
"dateIngested": "2023-02-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2015-04-01/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2015-04-01/granules/2015-07471/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2015-04-01/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2015-04-01/granules/2015-07471/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2015-04-01/granules/2015-07471/pdf"
}
},
{
"title": "Polyvinyl Alcohol From the People's Republic of China and Japan: Continuation of Antidumping Duty Orders",
"packageId": "FR-2021-04-09",
"granuleId": "2021-07303",
"lastModified": "2023-08-05T18:29:52Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2021-04-09",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2021-04-09/granules/2021-07303/summary",
"dateIngested": "2023-02-07",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2021-04-09/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2021-04-09/granules/2021-07303/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2021-04-09/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2021-04-09/granules/2021-07303/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2021-04-09/granules/2021-07303/pdf"
}
},
{
"title": "Notice of Initiation of Antidumping Duty Investigations: Polyvinyl Alcohol From Germany, Japan, the People's Republic of China, the Republic of Korea, and Singapore",
"packageId": "FR-2002-10-01",
"granuleId": "02-24928",
"lastModified": "2023-08-06T19:01:54Z",
"governmentAuthor": [
"National Archives and Records Administration",
"Office of the Federal Register"
],
"dateIssued": "2002-10-01",
"collectionCode": "FR",
"resultLink": "https://api.govinfo.gov/packages/FR-2002-10-01/granules/02-24928/summary",
"dateIngested": "2023-02-08",
"download": {
"premisLink": "https://api.govinfo.gov/packages/FR-2002-10-01/premis",
"txtLink": "https://api.govinfo.gov/packages/FR-2002-10-01/granules/02-24928/htm",
"zipLink": "https://api.govinfo.gov/packages/FR-2002-10-01/zip",
"modsLink": "https://api.govinfo.gov/packages/FR-2002-10-01/granules/02-24928/mods",
"pdfLink": "https://api.govinfo.gov/packages/FR-2002-10-01/granules/02-24928/pdf"
}
}
],
"offsetMark": "AoIIRFp5lDZGUi0yMDAyLTEwLTAxLTAyLTI0OTI4",
"count": 568
} |
Good morning @RogueSergeant @relarson1 @chrisbabayans @yanbinghan @krisrobinson @rvschlichtegroll @eheitfield - the new Search Service is now available as a Public Preview! This means that while it is feature complete as it stands, there is the potential for significant changes based on your feedback. Read more here: I also updated the readme and added a new Postman collection with example requests and responses. Hopefully, this initial release will meet most of your needs. If you have feedback on the search service as implemented, please open a new issue so we can triage. As issues and enhancements are triaged, we will add them to milestones and update as changes occur with notice ahead of deployment windows. At some point in the coming months, we will make the call when the search service is fully production-ready; at that point, we will announce the change in status, and that will signal that the search service request and response formats are essentially stabilized, with only minor additive enhancements anticipated in the future. |
Hi @jonquandt, just been playing around with the API - looks incredibly promising, however, I have started getting a 401 status error. Any idea what that's from? I switched to the DEMO key and was still getting the same error, so it's not authorization. It could be that I've reached my limit but wouldn't that give me a different error code and shouldn't it then work when I switch to the DEMO key? |
@RogueSergeant - let me look to see if I can identify the issue from logs. I'm not able to reproduce using any of the examples that I've tried. Can you provide an example curl or similar of what you're sending? Are you supplying the curl --location 'https://api.govinfo.gov/search' \
--header 'Content-Type: application/json' \
--header 'x-api-key: DEMO_KEY' \
--data '{
"query":"(PVOH OR '\''polyvinyl alcohol'\'' OR (PVAL AND (chemical* OR compound* OR substance OR molecule*)) OR polyethenol OR '\''alcoholysis resin'\'' OR '\''alcohol-soluble resin'\'' OR '\''ethanol homopolymer'\'' OR elvanol OR vinol OR gelvatol)",
"resultLevel":"default",
"pageSize":"1000",
"offsetMark":"*",
"sorts":[
{
"field":"score",
"sortOrder":"DESC"
}
]
}' or as a query parameter? curl --location 'https://api.govinfo.gov/search?api_key=DEMO_KEY' \
--header 'Content-Type: application/json' \
--data '{
"query":"(PVOH OR '\''polyvinyl alcohol'\'' OR (PVAL AND (chemical* OR compound* OR substance OR molecule*)) OR polyethenol OR '\''alcoholysis resin'\'' OR '\''alcohol-soluble resin'\'' OR '\''ethanol homopolymer'\'' OR elvanol OR vinol OR gelvatol)",
"resultLevel":"default",
"pageSize":"1000",
"offsetMark":"*",
"sorts":[
{
"field":"score",
"sortOrder":"DESC"
}
]
}' |
@jonquandt - thanks! I'm using a python script which WAS working earlier, and so I'm not sure why it's not working now. I also tried visiting the URL search direct with the following and got a similar response: https://api.govinfo.gov/search?API_KEY=DEMO_KEY&query=%22test%22 {"message":"Oops, Something went wrong, Please contact govinfo team for further assistance"} Here's the script:
|
@RogueSergeant - you need to set the api key header with
You could also have the failure return the response body as part of your else block, which would return more details: which will return: Also, It looks like the remainingCalls variable will throw an error because the X-RateLImit-Remaining is part of the response headers and not the content. |
If that's something that's desirable, please create a new issue and we can triage if there's enough community support for it. |
@jonquandt Got it working! Not sure how I managed to change the api parameter between runs...! As for the GET requests - not necessary at all, just wanted to check. Thanks for the help |
I am closing this issue now, as the service is now available. Please create new issues if you see any problems or potential enhancements to the search service. |
Here's an upcoming service we're thinking about. We'd love to have feedback on any additional features within this service.
Story
As a govinfo api user, I want to be able to get a limited subset of packages or granules based on a query.
Acceptance Criteria
Examples
The text was updated successfully, but these errors were encountered: