Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: pitchfork extractor #439

Merged
merged 8 commits into from
Jun 26, 2019

Conversation

mgeraci
Copy link
Contributor

@mgeraci mgeraci commented Jun 11, 2019

Hi there!

This PR adds a custom extractor for reviews on Pitchfork, a music review site. It's a pretty simple extractor, with an author, date, image, dek, and content. I did make one editorial choice here: Pitchfork assigns each album reviewed a score (a number between 0 and 10), and I chose to prepend this score as a sentence to the start of the review. Please do let me know if there's a better way to tackle something like this!

Thanks for this great tool, and for making contributing so simple; adding a new extractor was a cinch.

Here's a screenshot of a Pitchfork review showing the score:
Screenshot 2019-06-11 15 36 25

@postlight-org
Copy link
Collaborator

🤖 Automated Parsing Preview 🤖

Commit: prepend the score to the content

Screenshot of fixture (this embed should work after repo is public)

Original Article | HTML Fixture | Parsed Content Preview

Parsed JSON
{
  "title": "Lust for Youth: Lust for Youth Album Review",
  "content": "<div><div><span class=\"score\">Score: 6.2. </span><div class=\"review-detail__text clearfix\"><div class=\"contents dropcap\"><p>Coldwave never cared about you. The minimalistic, machine-driven sound that bubbled up twice in the past half-century&#x2014;once in <a href=\"https://pitchfork.com/reviews/albums/13843-wierd-presents-cold-waves-and-minimal-electronics-vol-1/\">the midst of post-punk&apos;s late-1970s heyday</a>, and again in <a href=\"http://wierdrecords.com/\">early-2010s Brooklyn</a>&#x2014;was largely reliant on keeping its audience at bay, all but ensuring its limited shelf life. With synth lines so brittle that you could snap them over your pinky finger, hissy drum machines, and vocals that frequently bordered on atonality, coldwave was purposefully alienating. A lot of DIY or home-recorded music from this decade gained recognition from <a href=\"https://pitchfork.com/reviews/albums/19254-boring-ecstasy-the-bedroom-pop-of-orchid-tapes/\">its</a> <a href=\"https://pitchfork.com/reviews/albums/19462-alex-g-dsu/\">warmth</a> <a href=\"https://pitchfork.com/reviews/albums/14385-swung-from-the-branches/\">and</a> <a href=\"https://pitchfork.com/reviews/albums/13931-causers-of-this/\">inviting textures</a>; in stark opposition, coldwave&apos;s practitioners rarely seemed interested in whether anyone was listening at all.</p>\n<p>The subgenre&apos;s aesthetic limitations are baked in, and practically a virtue, which makes Lust for Youth&apos;s trajectory over the last three years all the more fascinating. When Swedish synth aesthete Hannes Norrvide&apos;s project emerged with his 2011 debut <em>Solar Flare</em>, all the marks of coldwave&apos;s sub-zero sound were present and accounted for, and it was still mostly business as usual when Loke Rahbek (who also founded the aptly-named Posh Isolation label) joined up for the following year&apos;s Sacred Bones debut <a href=\"https://pitchfork.com/reviews/albums/17366-growing-seeds/\"><em>Growing Seeds</em></a>.</p>\n<p>But as Lust for Youth persisted, there emerged signs of life after coldwave. <em>Pomegranate</em> from 2013&#x2014;billed as a collaboration between Norrvide and Rahbek&apos;s <a href=\"https://pitchfork.com/artists/33972-croatian-armor/\">Croatian Amor</a> project&#x2014;explored the opalescent clouds of new age; <a href=\"https://pitchfork.com/reviews/albums/18122-lust-for-youth-perfect-view/\"><em>Perfect View</em></a> from the same year incorporated conversational vocal samples not unlike something from the Swedish imprint <a href=\"https://pitchfork.com/features/article/9236-sincerely-yours/\">Sincerely Yours</a>. When <em>International</em> arrived in 2014&#x2014;the fifth Lust for Youth release in the span of four years, evidence of coldwave&#x2019;s one-take prolificacy&#x2014;the group had expanded to a trio and tilted towards the explicitly poppier sounds of new wave, a journey they continued on 2016&apos;s <em>Compassion.</em></p>\n<p>Their latest, a self-titled effort, finds Lust for Youth back to a two-person setup, Rahbek having departed while Norrvide and Malthe Fischer soldier on. The paring back of personnel isn&apos;t reflected in the record&apos;s sound, though; instead, Norrvide and Fischer pick up where <em>Compassion</em> left off, with gleaming synths and crisp drum patterns lovingly recalling the dour, sweet-and-sour synth-pop of <a href=\"https://pitchfork.com/artists/3031-new-order/\">New Order</a>, <a href=\"https://pitchfork.com/artists/1063-depeche-mode/\">Depeche Mode</a>, and their thousands of imitators.</p>\n<p>Norrvide&apos;s vocals are front and center, and his voice often possesses a lovely ugliness that&apos;s alluring when cast in the right light, as when he scales the jagged pop mountains of &#x201C;Adrift&#x201D; or drapes himself over &#x201C;Great Concerns.&#x201D; <em>Lust for Youth</em> is at its most engaging when Norrvide and Fischer throw curveballs. They find an able collaborator in Danish musician and philosopher Soho Rezanejad, whose vocals add a glowing and gloomy texture to the slow-dance duskiness of &#x201C;Fifth Terrace&#x201D;; with wavering atmospherics lying under a cumulus of spoken-word, &#x201C;Imola&#x201D; suggests a sparser take on the starry-eyed synth fantasias of <a href=\"https://pitchfork.com/artists/2625-m83/\">M83</a>.</p>\n<p>Otherwise, <em>Lust for Youth</em> trades in one kind of over-familiarity for another. The gloriously mopey sound of new wave might be novel to Norrvide and Fischer, but there&apos;s not much here that stands out in synth-pop&apos;s always-crowded field. In a sense, that&apos;s fine; Lust for Youth wear this sound well. But <em>Lust for Youth</em> shows they might have escaped coldwave&#x2019;s dead end only to settle into a rut.</p>\n</div><a class=\"end-mark-container\" href=\"https://pitchfork.com/\"><span class=\"sr-only\">Back to home</span><svg class=\"end-mark\"><g><path/><path/></g></svg></a></div></div></div>",
  "author": "Larry Fitzmaurice",
  "date_published": "2019-06-07T00:00:00.000Z",
  "lead_image_url": "https://media.pitchfork.com/photos/5cefef2693a53659ed1ee6b8/1:1/w_320/LustForYouth_LustForYouth.jpg",
  "dek": "Hannes Norrvide's long-running coldwave synth project breaks into the greener pastures of Depeche Mode-style new wave.",
  "next_page_url": null,
  "url": "https://pitchfork.com/reviews/albums/lust-for-youth-lust-for-youth/",
  "domain": "pitchfork.com",
  "word_count": 519,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

null fields

  • next_page_url
1 failed test 😱

PitchforkComExtractor initial test case returns the date_published

See what went wrong
AssertionError [ERR_ASSERTION]: '2019-06-07T00:00:00.000Z' == '2019-06-07T04:00:00.000Z'
    at Object.equal (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:46:14)
    at tryCatch (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:288:22)
    at Generator.prototype.(anonymous function) [as next] (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:114:21)
    at asyncGeneratorStep (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:17:103)
    at _next (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:19:194)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)


@mgeraci
Copy link
Contributor Author

mgeraci commented Jun 11, 2019

I noticed that after making this PR the new test I added is failing. I can't reproduce this locally yet, but since it's related to the time I'd guess it's a timezone issue. I'll take a look!

@postlight-org
Copy link
Collaborator

🤖 Automated Parsing Preview 🤖

Commit: get the date from the datetime attribute

Screenshot of fixture (this embed should work after repo is public)

Original Article | HTML Fixture | Parsed Content Preview

Parsed JSON
{
  "title": "Lust for Youth: Lust for Youth Album Review",
  "content": "<div><div><span class=\"score\">Score: 6.2. </span><div class=\"review-detail__text clearfix\"><div class=\"contents dropcap\"><p>Coldwave never cared about you. The minimalistic, machine-driven sound that bubbled up twice in the past half-century&#x2014;once in <a href=\"https://pitchfork.com/reviews/albums/13843-wierd-presents-cold-waves-and-minimal-electronics-vol-1/\">the midst of post-punk&apos;s late-1970s heyday</a>, and again in <a href=\"http://wierdrecords.com/\">early-2010s Brooklyn</a>&#x2014;was largely reliant on keeping its audience at bay, all but ensuring its limited shelf life. With synth lines so brittle that you could snap them over your pinky finger, hissy drum machines, and vocals that frequently bordered on atonality, coldwave was purposefully alienating. A lot of DIY or home-recorded music from this decade gained recognition from <a href=\"https://pitchfork.com/reviews/albums/19254-boring-ecstasy-the-bedroom-pop-of-orchid-tapes/\">its</a> <a href=\"https://pitchfork.com/reviews/albums/19462-alex-g-dsu/\">warmth</a> <a href=\"https://pitchfork.com/reviews/albums/14385-swung-from-the-branches/\">and</a> <a href=\"https://pitchfork.com/reviews/albums/13931-causers-of-this/\">inviting textures</a>; in stark opposition, coldwave&apos;s practitioners rarely seemed interested in whether anyone was listening at all.</p>\n<p>The subgenre&apos;s aesthetic limitations are baked in, and practically a virtue, which makes Lust for Youth&apos;s trajectory over the last three years all the more fascinating. When Swedish synth aesthete Hannes Norrvide&apos;s project emerged with his 2011 debut <em>Solar Flare</em>, all the marks of coldwave&apos;s sub-zero sound were present and accounted for, and it was still mostly business as usual when Loke Rahbek (who also founded the aptly-named Posh Isolation label) joined up for the following year&apos;s Sacred Bones debut <a href=\"https://pitchfork.com/reviews/albums/17366-growing-seeds/\"><em>Growing Seeds</em></a>.</p>\n<p>But as Lust for Youth persisted, there emerged signs of life after coldwave. <em>Pomegranate</em> from 2013&#x2014;billed as a collaboration between Norrvide and Rahbek&apos;s <a href=\"https://pitchfork.com/artists/33972-croatian-armor/\">Croatian Amor</a> project&#x2014;explored the opalescent clouds of new age; <a href=\"https://pitchfork.com/reviews/albums/18122-lust-for-youth-perfect-view/\"><em>Perfect View</em></a> from the same year incorporated conversational vocal samples not unlike something from the Swedish imprint <a href=\"https://pitchfork.com/features/article/9236-sincerely-yours/\">Sincerely Yours</a>. When <em>International</em> arrived in 2014&#x2014;the fifth Lust for Youth release in the span of four years, evidence of coldwave&#x2019;s one-take prolificacy&#x2014;the group had expanded to a trio and tilted towards the explicitly poppier sounds of new wave, a journey they continued on 2016&apos;s <em>Compassion.</em></p>\n<p>Their latest, a self-titled effort, finds Lust for Youth back to a two-person setup, Rahbek having departed while Norrvide and Malthe Fischer soldier on. The paring back of personnel isn&apos;t reflected in the record&apos;s sound, though; instead, Norrvide and Fischer pick up where <em>Compassion</em> left off, with gleaming synths and crisp drum patterns lovingly recalling the dour, sweet-and-sour synth-pop of <a href=\"https://pitchfork.com/artists/3031-new-order/\">New Order</a>, <a href=\"https://pitchfork.com/artists/1063-depeche-mode/\">Depeche Mode</a>, and their thousands of imitators.</p>\n<p>Norrvide&apos;s vocals are front and center, and his voice often possesses a lovely ugliness that&apos;s alluring when cast in the right light, as when he scales the jagged pop mountains of &#x201C;Adrift&#x201D; or drapes himself over &#x201C;Great Concerns.&#x201D; <em>Lust for Youth</em> is at its most engaging when Norrvide and Fischer throw curveballs. They find an able collaborator in Danish musician and philosopher Soho Rezanejad, whose vocals add a glowing and gloomy texture to the slow-dance duskiness of &#x201C;Fifth Terrace&#x201D;; with wavering atmospherics lying under a cumulus of spoken-word, &#x201C;Imola&#x201D; suggests a sparser take on the starry-eyed synth fantasias of <a href=\"https://pitchfork.com/artists/2625-m83/\">M83</a>.</p>\n<p>Otherwise, <em>Lust for Youth</em> trades in one kind of over-familiarity for another. The gloriously mopey sound of new wave might be novel to Norrvide and Fischer, but there&apos;s not much here that stands out in synth-pop&apos;s always-crowded field. In a sense, that&apos;s fine; Lust for Youth wear this sound well. But <em>Lust for Youth</em> shows they might have escaped coldwave&#x2019;s dead end only to settle into a rut.</p>\n</div><a class=\"end-mark-container\" href=\"https://pitchfork.com/\"><span class=\"sr-only\">Back to home</span><svg class=\"end-mark\"><g><path/><path/></g></svg></a></div></div></div>",
  "author": "Larry Fitzmaurice",
  "date_published": "2019-06-07T05:00:00.000Z",
  "lead_image_url": "https://media.pitchfork.com/photos/5cefef2693a53659ed1ee6b8/1:1/w_320/LustForYouth_LustForYouth.jpg",
  "dek": "Hannes Norrvide's long-running coldwave synth project breaks into the greener pastures of Depeche Mode-style new wave.",
  "next_page_url": null,
  "url": "https://pitchfork.com/reviews/albums/lust-for-youth-lust-for-youth/",
  "domain": "pitchfork.com",
  "word_count": 519,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

null fields

  • next_page_url
1 failed test 😱

PitchforkComExtractor initial test case returns the date_published

See what went wrong
AssertionError [ERR_ASSERTION]: '2019-06-07T05:00:00.000Z' == '2019-06-07T09:00:00.000Z'
    at Object.equal (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:46:14)
    at tryCatch (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:288:22)
    at Generator.prototype.(anonymous function) [as next] (/home/circleci/project/node_modules/regenerator-runtime/runtime.js:114:21)
    at asyncGeneratorStep (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:17:103)
    at _next (/home/circleci/project/src/extractors/custom/pitchfork.com/index.test.js:19:194)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)


@postlight-org
Copy link
Collaborator

🤖 Automated Parsing Preview 🤖

Commit: mangle this test a little, but just a little (it does work properly)

Screenshot of fixture (this embed should work after repo is public)

Original Article | HTML Fixture | Parsed Content Preview

Parsed JSON
{
  "title": "Lust for Youth: Lust for Youth Album Review",
  "content": "<div><div><span class=\"score\">Score: 6.2. </span><div class=\"review-detail__text clearfix\"><div class=\"contents dropcap\"><p>Coldwave never cared about you. The minimalistic, machine-driven sound that bubbled up twice in the past half-century&#x2014;once in <a href=\"https://pitchfork.com/reviews/albums/13843-wierd-presents-cold-waves-and-minimal-electronics-vol-1/\">the midst of post-punk&apos;s late-1970s heyday</a>, and again in <a href=\"http://wierdrecords.com/\">early-2010s Brooklyn</a>&#x2014;was largely reliant on keeping its audience at bay, all but ensuring its limited shelf life. With synth lines so brittle that you could snap them over your pinky finger, hissy drum machines, and vocals that frequently bordered on atonality, coldwave was purposefully alienating. A lot of DIY or home-recorded music from this decade gained recognition from <a href=\"https://pitchfork.com/reviews/albums/19254-boring-ecstasy-the-bedroom-pop-of-orchid-tapes/\">its</a> <a href=\"https://pitchfork.com/reviews/albums/19462-alex-g-dsu/\">warmth</a> <a href=\"https://pitchfork.com/reviews/albums/14385-swung-from-the-branches/\">and</a> <a href=\"https://pitchfork.com/reviews/albums/13931-causers-of-this/\">inviting textures</a>; in stark opposition, coldwave&apos;s practitioners rarely seemed interested in whether anyone was listening at all.</p>\n<p>The subgenre&apos;s aesthetic limitations are baked in, and practically a virtue, which makes Lust for Youth&apos;s trajectory over the last three years all the more fascinating. When Swedish synth aesthete Hannes Norrvide&apos;s project emerged with his 2011 debut <em>Solar Flare</em>, all the marks of coldwave&apos;s sub-zero sound were present and accounted for, and it was still mostly business as usual when Loke Rahbek (who also founded the aptly-named Posh Isolation label) joined up for the following year&apos;s Sacred Bones debut <a href=\"https://pitchfork.com/reviews/albums/17366-growing-seeds/\"><em>Growing Seeds</em></a>.</p>\n<p>But as Lust for Youth persisted, there emerged signs of life after coldwave. <em>Pomegranate</em> from 2013&#x2014;billed as a collaboration between Norrvide and Rahbek&apos;s <a href=\"https://pitchfork.com/artists/33972-croatian-armor/\">Croatian Amor</a> project&#x2014;explored the opalescent clouds of new age; <a href=\"https://pitchfork.com/reviews/albums/18122-lust-for-youth-perfect-view/\"><em>Perfect View</em></a> from the same year incorporated conversational vocal samples not unlike something from the Swedish imprint <a href=\"https://pitchfork.com/features/article/9236-sincerely-yours/\">Sincerely Yours</a>. When <em>International</em> arrived in 2014&#x2014;the fifth Lust for Youth release in the span of four years, evidence of coldwave&#x2019;s one-take prolificacy&#x2014;the group had expanded to a trio and tilted towards the explicitly poppier sounds of new wave, a journey they continued on 2016&apos;s <em>Compassion.</em></p>\n<p>Their latest, a self-titled effort, finds Lust for Youth back to a two-person setup, Rahbek having departed while Norrvide and Malthe Fischer soldier on. The paring back of personnel isn&apos;t reflected in the record&apos;s sound, though; instead, Norrvide and Fischer pick up where <em>Compassion</em> left off, with gleaming synths and crisp drum patterns lovingly recalling the dour, sweet-and-sour synth-pop of <a href=\"https://pitchfork.com/artists/3031-new-order/\">New Order</a>, <a href=\"https://pitchfork.com/artists/1063-depeche-mode/\">Depeche Mode</a>, and their thousands of imitators.</p>\n<p>Norrvide&apos;s vocals are front and center, and his voice often possesses a lovely ugliness that&apos;s alluring when cast in the right light, as when he scales the jagged pop mountains of &#x201C;Adrift&#x201D; or drapes himself over &#x201C;Great Concerns.&#x201D; <em>Lust for Youth</em> is at its most engaging when Norrvide and Fischer throw curveballs. They find an able collaborator in Danish musician and philosopher Soho Rezanejad, whose vocals add a glowing and gloomy texture to the slow-dance duskiness of &#x201C;Fifth Terrace&#x201D;; with wavering atmospherics lying under a cumulus of spoken-word, &#x201C;Imola&#x201D; suggests a sparser take on the starry-eyed synth fantasias of <a href=\"https://pitchfork.com/artists/2625-m83/\">M83</a>.</p>\n<p>Otherwise, <em>Lust for Youth</em> trades in one kind of over-familiarity for another. The gloriously mopey sound of new wave might be novel to Norrvide and Fischer, but there&apos;s not much here that stands out in synth-pop&apos;s always-crowded field. In a sense, that&apos;s fine; Lust for Youth wear this sound well. But <em>Lust for Youth</em> shows they might have escaped coldwave&#x2019;s dead end only to settle into a rut.</p>\n</div><a class=\"end-mark-container\" href=\"https://pitchfork.com/\"><span class=\"sr-only\">Back to home</span><svg class=\"end-mark\"><g><path/><path/></g></svg></a></div></div></div>",
  "author": "Larry Fitzmaurice",
  "date_published": "2019-06-07T05:00:00.000Z",
  "lead_image_url": "https://media.pitchfork.com/photos/5cefef2693a53659ed1ee6b8/1:1/w_320/LustForYouth_LustForYouth.jpg",
  "dek": "Hannes Norrvide's long-running coldwave synth project breaks into the greener pastures of Depeche Mode-style new wave.",
  "next_page_url": null,
  "url": "https://pitchfork.com/reviews/albums/lust-for-youth-lust-for-youth/",
  "domain": "pitchfork.com",
  "word_count": 519,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

null fields

  • next_page_url

✅ All tests passed

@adampash
Copy link
Contributor

@mgeraci This is great, thanks!

I did make one editorial choice here: Pitchfork assigns each album reviewed a score (a number between 0 and 10), and I chose to prepend this score as a sentence to the start of the review. Please do let me know if there's a better way to tackle something like this!

Actually, we recently added a feature that allows you to extend the fields in a custom parser beyond the defaults. It's not actually implemented on any of the parsers yet, but this seems like a perfect first case! You can see how it works here:

https://github.com/postlight/mercury-parser/blob/262dda94b30cf94c180972cbcd7671758c65c9a4/src/mercury.test.js#L130-L149

@postlight-org
Copy link
Collaborator

🤖 Automated Parsing Preview 🤖

Commit: move from prepending the score to the review text to adding it as a custom field in the extractor

Screenshot of fixture (this embed should work after repo is public)

Original Article | HTML Fixture | Parsed Content Preview

Parsed JSON
{
  "title": "Lust for Youth: Lust for Youth Album Review",
  "content": "<div><div class=\"review-detail__text clearfix\"><div class=\"contents dropcap\"><p>Coldwave never cared about you. The minimalistic, machine-driven sound that bubbled up twice in the past half-century&#x2014;once in <a href=\"https://pitchfork.com/reviews/albums/13843-wierd-presents-cold-waves-and-minimal-electronics-vol-1/\">the midst of post-punk&apos;s late-1970s heyday</a>, and again in <a href=\"http://wierdrecords.com/\">early-2010s Brooklyn</a>&#x2014;was largely reliant on keeping its audience at bay, all but ensuring its limited shelf life. With synth lines so brittle that you could snap them over your pinky finger, hissy drum machines, and vocals that frequently bordered on atonality, coldwave was purposefully alienating. A lot of DIY or home-recorded music from this decade gained recognition from <a href=\"https://pitchfork.com/reviews/albums/19254-boring-ecstasy-the-bedroom-pop-of-orchid-tapes/\">its</a> <a href=\"https://pitchfork.com/reviews/albums/19462-alex-g-dsu/\">warmth</a> <a href=\"https://pitchfork.com/reviews/albums/14385-swung-from-the-branches/\">and</a> <a href=\"https://pitchfork.com/reviews/albums/13931-causers-of-this/\">inviting textures</a>; in stark opposition, coldwave&apos;s practitioners rarely seemed interested in whether anyone was listening at all.</p>\n<p>The subgenre&apos;s aesthetic limitations are baked in, and practically a virtue, which makes Lust for Youth&apos;s trajectory over the last three years all the more fascinating. When Swedish synth aesthete Hannes Norrvide&apos;s project emerged with his 2011 debut <em>Solar Flare</em>, all the marks of coldwave&apos;s sub-zero sound were present and accounted for, and it was still mostly business as usual when Loke Rahbek (who also founded the aptly-named Posh Isolation label) joined up for the following year&apos;s Sacred Bones debut <a href=\"https://pitchfork.com/reviews/albums/17366-growing-seeds/\"><em>Growing Seeds</em></a>.</p>\n<p>But as Lust for Youth persisted, there emerged signs of life after coldwave. <em>Pomegranate</em> from 2013&#x2014;billed as a collaboration between Norrvide and Rahbek&apos;s <a href=\"https://pitchfork.com/artists/33972-croatian-armor/\">Croatian Amor</a> project&#x2014;explored the opalescent clouds of new age; <a href=\"https://pitchfork.com/reviews/albums/18122-lust-for-youth-perfect-view/\"><em>Perfect View</em></a> from the same year incorporated conversational vocal samples not unlike something from the Swedish imprint <a href=\"https://pitchfork.com/features/article/9236-sincerely-yours/\">Sincerely Yours</a>. When <em>International</em> arrived in 2014&#x2014;the fifth Lust for Youth release in the span of four years, evidence of coldwave&#x2019;s one-take prolificacy&#x2014;the group had expanded to a trio and tilted towards the explicitly poppier sounds of new wave, a journey they continued on 2016&apos;s <em>Compassion.</em></p>\n<p>Their latest, a self-titled effort, finds Lust for Youth back to a two-person setup, Rahbek having departed while Norrvide and Malthe Fischer soldier on. The paring back of personnel isn&apos;t reflected in the record&apos;s sound, though; instead, Norrvide and Fischer pick up where <em>Compassion</em> left off, with gleaming synths and crisp drum patterns lovingly recalling the dour, sweet-and-sour synth-pop of <a href=\"https://pitchfork.com/artists/3031-new-order/\">New Order</a>, <a href=\"https://pitchfork.com/artists/1063-depeche-mode/\">Depeche Mode</a>, and their thousands of imitators.</p>\n<p>Norrvide&apos;s vocals are front and center, and his voice often possesses a lovely ugliness that&apos;s alluring when cast in the right light, as when he scales the jagged pop mountains of &#x201C;Adrift&#x201D; or drapes himself over &#x201C;Great Concerns.&#x201D; <em>Lust for Youth</em> is at its most engaging when Norrvide and Fischer throw curveballs. They find an able collaborator in Danish musician and philosopher Soho Rezanejad, whose vocals add a glowing and gloomy texture to the slow-dance duskiness of &#x201C;Fifth Terrace&#x201D;; with wavering atmospherics lying under a cumulus of spoken-word, &#x201C;Imola&#x201D; suggests a sparser take on the starry-eyed synth fantasias of <a href=\"https://pitchfork.com/artists/2625-m83/\">M83</a>.</p>\n<p>Otherwise, <em>Lust for Youth</em> trades in one kind of over-familiarity for another. The gloriously mopey sound of new wave might be novel to Norrvide and Fischer, but there&apos;s not much here that stands out in synth-pop&apos;s always-crowded field. In a sense, that&apos;s fine; Lust for Youth wear this sound well. But <em>Lust for Youth</em> shows they might have escaped coldwave&#x2019;s dead end only to settle into a rut.</p>\n</div><a class=\"end-mark-container\" href=\"https://pitchfork.com/\"><span class=\"sr-only\">Back to home</span><svg class=\"end-mark\"><g><path/><path/></g></svg></a></div></div>",
  "author": "Larry Fitzmaurice",
  "date_published": "2019-06-07T05:00:00.000Z",
  "lead_image_url": "https://media.pitchfork.com/photos/5cefef2693a53659ed1ee6b8/1:1/w_320/LustForYouth_LustForYouth.jpg",
  "dek": "Hannes Norrvide's long-running coldwave synth project breaks into the greener pastures of Depeche Mode-style new wave.",
  "next_page_url": null,
  "url": "https://pitchfork.com/reviews/albums/lust-for-youth-lust-for-youth/",
  "domain": "pitchfork.com",
  "word_count": 519,
  "direction": "ltr",
  "score": "6.2",
  "total_pages": 1,
  "rendered_pages": 1
}

null fields

  • next_page_url

✅ All tests passed

@mgeraci
Copy link
Contributor Author

mgeraci commented Jun 17, 2019

@adampash Awesome, that was a really easy change; thanks for pointing it out!

When adding a custom field like this, would you recommend leaving it as is, score: '6.2', or making the output more descriptive like I had when prepending to the review content, score: 'Score: 6.2'?

@adampash
Copy link
Contributor

When adding a custom field like this, would you recommend leaving it as is, score: '6.2', or making the output more descriptive like I had when prepending to the review content, score: 'Score: 6.2'?

I like the way you've done it. Thanks again @mgeraci!

Copy link
Contributor

@adampash adampash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@adampash adampash merged commit 571a913 into postlight:master Jun 26, 2019
@mgeraci
Copy link
Contributor Author

mgeraci commented Jun 26, 2019

🎉 Thanks for the feedback, and for merging!

@postlight-org
Copy link
Collaborator

🤖 Automated Parsing Preview 🤖

Commit: Merge branch 'master' into feat-pitchfork-extractor

Screenshot of fixture (this embed should work after repo is public)

Original Article | HTML Fixture | Parsed Content Preview

Parsed JSON
{
  "title": "Lust for Youth: Lust for Youth Album Review",
  "content": "<div><div class=\"review-detail__text clearfix\"><div class=\"contents dropcap\"><p>Coldwave never cared about you. The minimalistic, machine-driven sound that bubbled up twice in the past half-century&#x2014;once in <a href=\"https://pitchfork.com/reviews/albums/13843-wierd-presents-cold-waves-and-minimal-electronics-vol-1/\">the midst of post-punk&apos;s late-1970s heyday</a>, and again in <a href=\"http://wierdrecords.com/\">early-2010s Brooklyn</a>&#x2014;was largely reliant on keeping its audience at bay, all but ensuring its limited shelf life. With synth lines so brittle that you could snap them over your pinky finger, hissy drum machines, and vocals that frequently bordered on atonality, coldwave was purposefully alienating. A lot of DIY or home-recorded music from this decade gained recognition from <a href=\"https://pitchfork.com/reviews/albums/19254-boring-ecstasy-the-bedroom-pop-of-orchid-tapes/\">its</a> <a href=\"https://pitchfork.com/reviews/albums/19462-alex-g-dsu/\">warmth</a> <a href=\"https://pitchfork.com/reviews/albums/14385-swung-from-the-branches/\">and</a> <a href=\"https://pitchfork.com/reviews/albums/13931-causers-of-this/\">inviting textures</a>; in stark opposition, coldwave&apos;s practitioners rarely seemed interested in whether anyone was listening at all.</p>\n<p>The subgenre&apos;s aesthetic limitations are baked in, and practically a virtue, which makes Lust for Youth&apos;s trajectory over the last three years all the more fascinating. When Swedish synth aesthete Hannes Norrvide&apos;s project emerged with his 2011 debut <em>Solar Flare</em>, all the marks of coldwave&apos;s sub-zero sound were present and accounted for, and it was still mostly business as usual when Loke Rahbek (who also founded the aptly-named Posh Isolation label) joined up for the following year&apos;s Sacred Bones debut <a href=\"https://pitchfork.com/reviews/albums/17366-growing-seeds/\"><em>Growing Seeds</em></a>.</p>\n<p>But as Lust for Youth persisted, there emerged signs of life after coldwave. <em>Pomegranate</em> from 2013&#x2014;billed as a collaboration between Norrvide and Rahbek&apos;s <a href=\"https://pitchfork.com/artists/33972-croatian-armor/\">Croatian Amor</a> project&#x2014;explored the opalescent clouds of new age; <a href=\"https://pitchfork.com/reviews/albums/18122-lust-for-youth-perfect-view/\"><em>Perfect View</em></a> from the same year incorporated conversational vocal samples not unlike something from the Swedish imprint <a href=\"https://pitchfork.com/features/article/9236-sincerely-yours/\">Sincerely Yours</a>. When <em>International</em> arrived in 2014&#x2014;the fifth Lust for Youth release in the span of four years, evidence of coldwave&#x2019;s one-take prolificacy&#x2014;the group had expanded to a trio and tilted towards the explicitly poppier sounds of new wave, a journey they continued on 2016&apos;s <em>Compassion.</em></p>\n<p>Their latest, a self-titled effort, finds Lust for Youth back to a two-person setup, Rahbek having departed while Norrvide and Malthe Fischer soldier on. The paring back of personnel isn&apos;t reflected in the record&apos;s sound, though; instead, Norrvide and Fischer pick up where <em>Compassion</em> left off, with gleaming synths and crisp drum patterns lovingly recalling the dour, sweet-and-sour synth-pop of <a href=\"https://pitchfork.com/artists/3031-new-order/\">New Order</a>, <a href=\"https://pitchfork.com/artists/1063-depeche-mode/\">Depeche Mode</a>, and their thousands of imitators.</p>\n<p>Norrvide&apos;s vocals are front and center, and his voice often possesses a lovely ugliness that&apos;s alluring when cast in the right light, as when he scales the jagged pop mountains of &#x201C;Adrift&#x201D; or drapes himself over &#x201C;Great Concerns.&#x201D; <em>Lust for Youth</em> is at its most engaging when Norrvide and Fischer throw curveballs. They find an able collaborator in Danish musician and philosopher Soho Rezanejad, whose vocals add a glowing and gloomy texture to the slow-dance duskiness of &#x201C;Fifth Terrace&#x201D;; with wavering atmospherics lying under a cumulus of spoken-word, &#x201C;Imola&#x201D; suggests a sparser take on the starry-eyed synth fantasias of <a href=\"https://pitchfork.com/artists/2625-m83/\">M83</a>.</p>\n<p>Otherwise, <em>Lust for Youth</em> trades in one kind of over-familiarity for another. The gloriously mopey sound of new wave might be novel to Norrvide and Fischer, but there&apos;s not much here that stands out in synth-pop&apos;s always-crowded field. In a sense, that&apos;s fine; Lust for Youth wear this sound well. But <em>Lust for Youth</em> shows they might have escaped coldwave&#x2019;s dead end only to settle into a rut.</p>\n</div><a class=\"end-mark-container\" href=\"https://pitchfork.com/\"><span class=\"sr-only\">Back to home</span><svg class=\"end-mark\"><g><path/><path/></g></svg></a></div></div>",
  "author": "Larry Fitzmaurice",
  "date_published": "2019-06-07T05:00:00.000Z",
  "lead_image_url": "https://media.pitchfork.com/photos/5cefef2693a53659ed1ee6b8/1:1/w_320/LustForYouth_LustForYouth.jpg",
  "dek": "Hannes Norrvide's long-running coldwave synth project breaks into the greener pastures of Depeche Mode-style new wave.",
  "next_page_url": null,
  "url": "https://pitchfork.com/reviews/albums/lust-for-youth-lust-for-youth/",
  "domain": "pitchfork.com",
  "word_count": 519,
  "direction": "ltr",
  "score": "6.2",
  "total_pages": 1,
  "rendered_pages": 1
}

null fields

  • next_page_url

✅ All tests passed

@samuelclay
Copy link
Contributor

Thanks for adding this field, but I want to see the score in the story content and not in a field. The point of this library is that I get the story's full content in the content field and not in single-use fields that are per-site. Since the website shows the score at top I feel it would be appropriate to have the score at top of the article content. Can we move it there so users can benefit from this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants