Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After update: Exception during INSERT: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80 #1165

Closed
3 tasks done
Jonny007-MKD opened this issue Feb 11, 2021 · 11 comments

Comments

@Jonny007-MKD
Copy link

  • I have read the CONTRIBUTING.md and followed the provided tips
  • I accept that the issue will be closed without comment if I do not check here
  • I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

Explain the Problem

Since I updated NextCloud and also the News app most of my feeds aren't updated anymore. This seems to be caused by this feed: http://www.windytan.com/feeds/posts/default
It is a valid Atom 1.0 feed. feed-io failed for me, because Firefox denied the "consume" request with "CORS Missing Allow Origin". But i think fetching is fine, as an exception is logged in nextcloud.log only when inserting the data to the database. This probably leads to all following feeds not being updated.

This started with my update of NextCloud and News. This is a grep on "read access" in the log showing this clearly:

... even more feeds in this cron call
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://foonathan.net/post/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://github.com/kimai/kimai/releases.atom into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.linuxmint.com/?feed=rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"iYCN1BkNdPgjQ86n5CSj","level":0,"time":"2021-01-23T21:34:48+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : https://blog.gitea.io/index.xml into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.4.0"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.tagesschau.de/xml/rss2 into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"KGIXbcHcmeEDp796Z9Mf","level":0,"time":"2021-01-23T23:34:27+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}

System Information

  • News app version: 15.3.1
  • Nextcloud version: 20.0.5
  • Cron type: systemd cron.php
  • PHP version: 7.4.14
  • Database and version: MariaDB 10.3.27
  • Browser and version: Firefox 78.7.0
  • OS and version: Debian 10
Contents of nextcloud/data/nextcloud.log
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"read access : http://www.windytan.com/feeds/posts/default into a feed instance (feed class : FeedIo\\Feed)","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"start reading http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"no 'modifiedSince' parameter given, setting it to 01/01/1970","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:20+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"hitting http://www.windytan.com/feeds/posts/default","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"response ok, now turning it into a document","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":1,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"the stream is modified, parsing it","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"accurate parser : FeedIo\\Parser\\XmlParser","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Feed http://www.windytan.com/feeds/posts/default was modified since last fetch. #25 items","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Plotting patterns in music with a fantasy record player for feed absorptions publishdate: 1608189512","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Capturing PAL video with an SDR (and a few dead-ends) for feed absorptions publishdate: 1589710223","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":0,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"news","method":"","url":"--","message":"Added item Beeps and melodies in two-way radio for feed absorptions publishdate: 1566587018","userAgent":"--","version":"20.0.5.2"}
{"reqId":"1AscWZy8T7CHNOyvh5Mo","level":3,"time":"2021-01-23T22:34:21+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"<p>Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/en.wikipedia.org\\/wiki\\/Miscellaneous_Symbols_and_Pictographs\\\" title=\\\"Wikipedia: Miscellaneous Symbols and Pictographs\\\">Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p> <p>This Perl script does just that:<\\/p> <pre><code><br \\/>@emoji = qw( \\ud83c\\udf00  \\ud83c\\udf02  \\ud83c\\udf05  \\ud83c\\udf08  \\ud83c\\udf19  \\ud83c\\udf1e  \\ud83c\\udf1f  \\ud83c\\udf20  \\ud83c\\udf30  \\ud83c\\udf31  \\ud83c\\udf32  \\ud83c\\udf33  \\ud83c\\udf34  \\ud83c\\udf35  \\ud83c\\udf37  \\ud83c\\udf38<br \\/>             \\ud83c\\udf39  \\ud83c\\udf3a  \\ud83c\\udf3b  \\ud83c\\udf3c  \\ud83c\\udf3d  \\ud83c\\udf3e  \\ud83c\\udf3f  \\ud83c\\udf40  \\ud83c\\udf41  \\ud83c\\udf42  \\ud83c\\udf43  \\ud83c\\udf44  \\ud83c\\udf45  \\ud83c\\udf46  \\ud83c\\udf47  \\ud83c\\udf48<br \\/>             \\ud83c\\udf49  \\ud83c\\udf4a  \\ud83c\\udf4b  \\ud83c\\udf4c  \\ud83c\\udf4d  \\ud83c\\udf4e  \\ud83c\\udf4f  \\ud83c\\udf50  \\ud83c\\udf51  \\ud83c\\udf52  \\ud83c\\udf53  \\ud83c\\udf54  \\ud83c\\udf55  \\ud83c\\udf56  \\ud83c\\udf57  \\ud83c\\udf58<br \\/>             \\ud83c\\udf5c  \\ud83c\\udf5d  \\ud83c\\udf5e  \\ud83c\\udf5f  \\ud83c\\udf60  \\ud83c\\udf61  \\ud83c\\udf62  \\ud83c\\udf63  \\ud83c\\udf64  \\ud83c\\udf65  \\ud83c\\udf66  \\ud83c\\udf67  \\ud83c\\udf68  \\ud83c\\udf69  \\ud83c\\udf6a  \\ud83c\\udf6b<br \\/>             \\ud83c\\udf6c  \\ud83c\\udf6d  \\ud83c\\udf6e  \\ud83c\\udf6f  \\ud83c\\udf70  \\ud83c\\udf71  \\ud83c\\udf72  \\ud83c\\udf73  \\ud83c\\udf74  \\ud83c\\udf75  \\ud83c\\udf76  \\ud83c\\udf77  \\ud83c\\udf78  \\ud83c\\udf79  \\ud83c\\udf7a  \\ud83c\\udf7b<br \\/>             \\ud83c\\udf7c  \\ud83c\\udf80  \\ud83c\\udf81  \\ud83c\\udf82  \\ud83c\\udf83  \\ud83c\\udf84  \\ud83c\\udf85  \\ud83c\\udf88  \\ud83c\\udf89  \\ud83c\\udf8a  \\ud83c\\udf8b  \\ud83c\\udf8c  \\ud83c\\udf8d  \\ud83c\\udf8e  \\ud83c\\udf8f  \\ud83c\\udf92<br \\/>             \\ud83c\\udf93  \\ud83c\\udfa0  \\ud83c\\udfa1  \\ud83c\\udfa2  \\ud83c\\udfa3  \\ud83c\\udfa4  \\ud83c\\udfa5  \\ud83c\\udfa6  \\ud83c\\udfa7  \\ud83c\\udfa8  \\ud83c\\udfa9  \\ud83c\\udfaa  \\ud83c\\udfab  \\ud83c\\udfac  \\ud83c\\udfad  \\ud83c\\udfae<br \\/>             \\ud83c\\udfaf  \\ud83c\\udfb0  \\ud83c\\udfb1  \\ud83c\\udfb2  \\ud83c\\udfb3  \\ud83c\\udfb4  \\ud83c\\udfb5  \\ud83c\\udfb7  \\ud83c\\udfb8  \\ud83c\\udfb9  \\ud83c\\udfba  \\ud83c\\udfbb  \\ud83c\\udfbd  \\ud83c\\udfbe  \\ud83c\\udfbf  \\ud83c\\udfc0<br \\/>             \\ud83c\\udfc1  \\ud83c\\udfc2  \\ud83c\\udfc3  \\ud83c\\udfc4  \\ud83c\\udfc6  \\ud83c\\udfc7  \\ud83c\\udfc8  \\ud83c\\udfc9  \\ud83c\\udfca  \\ud83d\\udc00  \\ud83d\\udc01  \\ud83d\\udc02  \\ud83d\\udc03  \\ud83d\\udc04  \\ud83d\\udc05  \\ud83d\\udc06<br \\/>             \\ud83d\\udc07  \\ud83d\\udc08  \\ud83d\\udc09  \\ud83d\\udc0a  \\ud83d\\udc0b  \\ud83d\\udc0c  \\ud83d\\udc0d  \\ud83d\\udc0e  \\ud83d\\udc0f  \\ud83d\\udc10  \\ud83d\\udc11  \\ud83d\\udc12  \\ud83d\\udc13  \\ud83d\\udc14  \\ud83d\\udc15  \\ud83d\\udc16<br \\/>             \\ud83d\\udc17  \\ud83d\\udc18  \\ud83d\\udc19  \\ud83d\\udc1a  \\ud83d\\udc1b  \\ud83d\\udc1c  \\ud83d\\udc1d  \\ud83d\\udc1e  \\ud83d\\udc1f  \\ud83d\\udc20  \\ud83d\\udc21  \\ud83d\\udc22  \\ud83d\\udc23  \\ud83d\\udc24  \\ud83d\\udc25  \\ud83d\\udc26<br \\/>             \\ud83d\\udc27  \\ud83d\\udc28  \\ud83d\\udc29  \\ud83d\\udc2a  \\ud83d\\udc2b  \\ud83d\\udc2c  \\ud83d\\udc2d  \\ud83d\\udc2e  \\ud83d\\udc2f  \\ud83d\\udc30  \\ud83d\\udc31  \\ud83d\\udc32  \\ud83d\\udc33  \\ud83d\\udc34  \\ud83d\\udc35  \\ud83d\\udc36<br \\/>             \\ud83d\\udc37  \\ud83d\\udc38  \\ud83d\\udc39  \\ud83d\\udc3a  \\ud83d\\udc3b  \\ud83d\\udc3c  \\ud83d\\udc3d  \\ud83d\\udc3e  \\ud83d\\udc40  \\ud83d\\udc42  \\ud83d\\udc43  \\ud83d\\udc44  \\ud83d\\udc45  \\ud83d\\udc46  \\ud83d\\udc47  \\ud83d\\udc48<br \\/>             \\ud83d\\udc49  \\ud83d\\udc4a  \\ud83d\\udc4b  \\ud83d\\udc4c  \\ud83d\\udc4d  \\ud83d\\udc4e  \\ud83d\\udc4f  \\ud83d\\udc50  \\ud83d\\udc51  \\ud83d\\udc52  \\ud83d\\udc53  \\ud83d\\udc54  \\ud83d\\udc55  \\ud83d\\udc56  \\ud83d\\udc57  \\ud83d\\udc58<br \\/>             \\ud83d\\udc59  \\ud83d\\udc5a  \\ud83d\\udc5b  \\ud83d\\udc5c  \\ud83d\\udc5d  \\ud83d\\udc5e  \\ud83d\\udc5f  \\ud83d\\udc60  \\ud83d\\udc61  \\ud83d\\udc62  \\ud83d\\udc63  \\ud83d\\udc64  \\ud83d\\udc65  \\ud83d\\udc66  \\ud83d\\udc67  \\ud83d\\udc68<br \\/>             \\ud83d\\udc69  \\ud83d\\udc6a  \\ud83d\\udc6e  \\ud83d\\udc6f  \\ud83d\\udc7a  \\ud83d\\udc7b  \\ud83d\\udc7c  \\ud83d\\udc7d  \\ud83d\\udc7e  \\ud83d\\udc7f  \\ud83d\\udc80  \\ud83d\\udc81  \\ud83d\\udc82  \\ud83d\\udc83  \\ud83d\\udc84  \\ud83d\\udc85 );<br \\/><br \\/>while (&lt;&gt;) {<br \\/>  if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {<br \\/>    ($b, $m, $a) = ($`, $&amp;, $');<br \\/>    print $b.join(\\\"  \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;<br \\/>  }<br \\/>}<\\/code><\\/pre> <p>What's happening here? First we create a 256-element array containing a hand-picked collection of emoji. Naturally, they're all assigned an index from <span>0x00<\\/span> to <span>0xff<\\/span>. Then we'll loop through standard input and look for lines containing colon-separated hex bytes. Each hex value is replaced with an emoji from the array.<\\/p> <p>Here's the output:<\\/p> <div><img border=\\\"0\\\" src=\\\"https:\\/\\/2.bp.blogspot.com\\/-fx-QGLb65NQ\\/VFF5VuQ6V9I\\/AAAAAAAADhg\\/F52MqWiLGJg\\/s480\\/rsa.png\\\" alt=\\\"[Image: Terminal screenshot showing a PGP key fingerprint and the same with all hex numbers replaced with emoji.]\\\" \\/><\\/div> <p>The script could easily be extended to support output from other hex-formatted sources as well, such as xxd:<\\/p> <div><img border=\\\"0\\\" src=\\\"https:\\/\\/2.bp.blogspot.com\\/-cGSKJPcr16g\\/VFH-95vNJVI\\/AAAAAAAADh8\\/PvryEVbLX_M\\/s480\\/korvista.png\\\" alt=\\\"[Image: Terminal screenshot showing a hex dump of a poem and the same with all hex numbers replaced with emoji. kissofoni; tassun kynsi neulana \\/ musa korvista kajahtaa]\\\" \\/><\\/div> <p>Some additional methods for visualizing hex dumps and key fingerprints, from the comments section:<\\/p> <ul>  <li><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"http:\\/\\/user.xmission.com\\/~atoponce\\/art\\/\\\" title=\\\"PGP Art\\\">PGP Strong Set Top 50 Fingerprint Art<\\/a><\\/li>  <li><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"http:\\/\\/sebsauvage.net\\/wiki\\/doku.php?id=php:vizhash_gd\\\" title=\\\"php:vizhash_gd [sebsauvage]\\\">VizHash GD - a visual hash<\\/a><\\/li><\\/ul>\", \"memorizing ssh public key fingerprints can be difficult; they're just long random numbers displayed in base 16. there are some terminal-friendly solutions, like openssh's randomart. but because i use a unicode terminal, i like to map the individual bytes into characters in the miscellaneous symbols and pictographs block. this perl script does just that: @emoji = qw( \\ud83c\\udf00  \\ud83c\\udf02  \\ud83c\\udf05  \\ud83c\\udf08  \\ud83c\\udf19  \\ud83c\\udf1e  \\ud83c\\udf1f  \\ud83c\\udf20  \\ud83c\\udf30  \\ud83c\\udf31  \\ud83c\\udf32  \\ud83c\\udf33  \\ud83c\\udf34  \\ud83c\\udf35  \\ud83c\\udf37  \\ud83c\\udf38             \\ud83c\\udf39  \\ud83c\\udf3a  \\ud83c\\udf3b  \\ud83c\\udf3c  \\ud83c\\udf3d  \\ud83c\\udf3e  \\ud83c\\udf3f  \\ud83c\\udf40  \\ud83c\\udf41  \\ud83c\\udf42  \\ud83c\\udf43  \\ud83c\\udf44  \\ud83c\\udf45  \\ud83c\\udf46  \\ud83c\\udf47  \\ud83c\\udf48             \\ud83c\\udf49  \\ud83c\\udf4a  \\ud83c\\udf4b  \\ud83c\\udf4c  \\ud83c\\udf4d  \\ud83c\\udf4e  \\ud83c\\udf4f  \\ud83c\\udf50  \\ud83c\\udf51  \\ud83c\\udf52  \\ud83c\\udf53  \\ud83c\\udf54  \\ud83c\\udf55  \\ud83c\\udf56  \\ud83c\\udf57  \\ud83c\\udf58             \\ud83c\\udf5c  \\ud83c\\udf5d  \\ud83c\\udf5e  \\ud83c\\udf5f  \\ud83c\\udf60  \\ud83c\\udf61  \\ud83c\\udf62  \\ud83c\\udf63  \\ud83c\\udf64  \\ud83c\\udf65  \\ud83c\\udf66  \\ud83c\\udf67  \\ud83c\\udf68  \\ud83c\\udf69  \\ud83c\\udf6a  \\ud83c\\udf6b             \\ud83c\\udf6c  \\ud83c\\udf6d  \\ud83c\\udf6e  \\ud83c\\udf6f  \\ud83c\\udf70  \\ud83c\\udf71  \\ud83c\\udf72  \\ud83c\\udf73  \\ud83c\\udf74  \\ud83c\\udf75  \\ud83c\\udf76  \\ud83c\\udf77  \\ud83c\\udf78  \\ud83c\\udf79  \\ud83c\\udf7a  \\ud83c\\udf7b             \\ud83c\\udf7c  \\ud83c\\udf80  \\ud83c\\udf81  \\ud83c\\udf82  \\ud83c\\udf83  \\ud83c\\udf84  \\ud83c\\udf85  \\ud83c\\udf88  \\ud83c\\udf89  \\ud83c\\udf8a  \\ud83c\\udf8b  \\ud83c\\udf8c  \\ud83c\\udf8d  \\ud83c\\udf8e  \\ud83c\\udf8f  \\ud83c\\udf92             \\ud83c\\udf93  \\ud83c\\udfa0  \\ud83c\\udfa1  \\ud83c\\udfa2  \\ud83c\\udfa3  \\ud83c\\udfa4  \\ud83c\\udfa5  \\ud83c\\udfa6  \\ud83c\\udfa7  \\ud83c\\udfa8  \\ud83c\\udfa9  \\ud83c\\udfaa  \\ud83c\\udfab  \\ud83c\\udfac  \\ud83c\\udfad  \\ud83c\\udfae             \\ud83c\\udfaf  \\ud83c\\udfb0  \\ud83c\\udfb1  \\ud83c\\udfb2  \\ud83c\\udfb3  \\ud83c\\udfb4  \\ud83c\\udfb5  \\ud83c\\udfb7  \\ud83c\\udfb8  \\ud83c\\udfb9  \\ud83c\\udfba  \\ud83c\\udfbb  \\ud83c\\udfbd  \\ud83c\\udfbe  \\ud83c\\udfbf  \\ud83c\\udfc0             \\ud83c\\udfc1  \\ud83c\\udfc2  \\ud83c\\udfc3  \\ud83c\\udfc4  \\ud83c\\udfc6  \\ud83c\\udfc7  \\ud83c\\udfc8  \\ud83c\\udfc9  \\ud83c\\udfca  \\ud83d\\udc00  \\ud83d\\udc01  \\ud83d\\udc02  \\ud83d\\udc03  \\ud83d\\udc04  \\ud83d\\udc05  \\ud83d\\udc06             \\ud83d\\udc07  \\ud83d\\udc08  \\ud83d\\udc09  \\ud83d\\udc0a  \\ud83d\\udc0b  \\ud83d\\udc0c  \\ud83d\\udc0d  \\ud83d\\udc0e  \\ud83d\\udc0f  \\ud83d\\udc10  \\ud83d\\udc11  \\ud83d\\udc12  \\ud83d\\udc13  \\ud83d\\udc14  \\ud83d\\udc15  \\ud83d\\udc16             \\ud83d\\udc17  \\ud83d\\udc18  \\ud83d\\udc19  \\ud83d\\udc1a  \\ud83d\\udc1b  \\ud83d\\udc1c  \\ud83d\\udc1d  \\ud83d\\udc1e  \\ud83d\\udc1f  \\ud83d\\udc20  \\ud83d\\udc21  \\ud83d\\udc22  \\ud83d\\udc23  \\ud83d\\udc24  \\ud83d\\udc25  \\ud83d\\udc26             \\ud83d\\udc27  \\ud83d\\udc28  \\ud83d\\udc29  \\ud83d\\udc2a  \\ud83d\\udc2b  \\ud83d\\udc2c  \\ud83d\\udc2d  \\ud83d\\udc2e  \\ud83d\\udc2f  \\ud83d\\udc30  \\ud83d\\udc31  \\ud83d\\udc32  \\ud83d\\udc33  \\ud83d\\udc34  \\ud83d\\udc35  \\ud83d\\udc36             \\ud83d\\udc37  \\ud83d\\udc38  \\ud83d\\udc39  \\ud83d\\udc3a  \\ud83d\\udc3b  \\ud83d\\udc3c  \\ud83d\\udc3d  \\ud83d\\udc3e  \\ud83d\\udc40  \\ud83d\\udc42  \\ud83d\\udc43  \\ud83d\\udc44  \\ud83d\\udc45  \\ud83d\\udc46  \\ud83d\\udc47  \\ud83d\\udc48             \\ud83d\\udc49  \\ud83d\\udc4a  \\ud83d\\udc4b  \\ud83d\\udc4c  \\ud83d\\udc4d  \\ud83d\\udc4e  \\ud83d\\udc4f  \\ud83d\\udc50  \\ud83d\\udc51  \\ud83d\\udc52  \\ud83d\\udc53  \\ud83d\\udc54  \\ud83d\\udc55  \\ud83d\\udc56  \\ud83d\\udc57  \\ud83d\\udc58             \\ud83d\\udc59  \\ud83d\\udc5a  \\ud83d\\udc5b  \\ud83d\\udc5c  \\ud83d\\udc5d  \\ud83d\\udc5e  \\ud83d\\udc5f  \\ud83d\\udc60  \\ud83d\\udc61  \\ud83d\\udc62  \\ud83d\\udc63  \\ud83d\\udc64  \\ud83d\\udc65  \\ud83d\\udc66  \\ud83d\\udc67  \\ud83d\\udc68             \\ud83d\\udc69  \\ud83d\\udc6a  \\ud83d\\udc6e  \\ud83d\\udc6f  \\ud83d\\udc7a  \\ud83d\\udc7b  \\ud83d\\udc7c  \\ud83d\\udc7d  \\ud83d\\udc7e  \\ud83d\\udc7f  \\ud83d\\udc80  \\ud83d\\udc81  \\ud83d\\udc82  \\ud83d\\udc83  \\ud83d\\udc84  \\ud83d\\udc85 );while (<>) {  if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {    ($b, $m, $a) = ($`, $&, $');    print $b.join(\\\"  \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;  }} what's happening here? first we create a 256-element array containing a hand-picked collection of emoji. naturally, they're all assigned an index from 0x00 to 0xff. then we'll loop through standard input and look for lines containing colon-separated hex bytes. each hex value is replaced with an emoji from the array. here's the output:  the script could easily be extended to support output from other hex-formatted sources as well, such as xxd:  some additional methods for visualizing hex dumps and key fingerprints, from the comments section:   pgp strong set top 50 fingerprint art  vizhash gd - a visual hashoona r\\u00e4is\\u00e4nenvisualizing hex dumps with unicode emojihttp:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"812c550a581428a95bc4d897f721b2a9\", \"fa053fe2305b1c1592ea0651a9d5a943\", \"4\", 55]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80  ...' for column `newown_nordlichter_brv_de`.`oc_news_items`.`body` at row 1","Code":"22007","Trace":[{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":127,"function":"execute","class":"PDOStatement","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/public/AppFramework/Db/QBMapper.php","line":211,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Db/NewsMapperV2.php","line":64,"function":"update","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/ItemServiceV2.php","line":97,"function":"update","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/FeedServiceV2.php","line":311,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/FeedServiceV2.php","line":347,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Cron/UpdaterJob.php","line":68,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->"},{"file":"/var/www/vhosts/2newown.nordlichter-brv.de/web/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->"}],"File":"/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":127}},"CustomMessage":"Error while running background job (class: OCA\\News\\Cron\\UpdaterJob, arguments: )"},"userAgent":"--","version":"20.0.5.2"}
@Grotax
Copy link
Member

Grotax commented Feb 12, 2021

I reported the feed-io issue alexdebril/feed-io#325

And you are right, fetching the feed is no problem I tried it on cli with feed-io.
The issue happens when news tries to insert the data into the DB, I have to say though that I don't understand the error message.

SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80  ...' for column `newown_nordlichter_brv_de`.`oc_news_items`.`body` at row 1",

The full error, isolated:

{
    "reqId": "1AscWZy8T7CHNOyvh5Mo",
    "level": 3,
    "time": "2021-01-23T22:34:21+00:00",
    "remoteAddr": "",
    "user": "--",
    "app": "core",
    "method": "",
    "url": "--",
    "message": {
        "Exception": "Doctrine\\DBAL\\Exception\\DriverException",
        "Message": "An exception occurred while executing 'UPDATE `oc_news_items` SET `unread` = ?, `url` = ?, `guid` = ?, `guid_hash` = ?, `pub_date` = ?, `last_modified` = ?, `title` = ?, `author` = ?, `body` = ?, `search_index` = ?, `fingerprint` = ?, `content_hash` = ?, `feed_id` = ? WHERE `id` = ?' with params [false, \"http:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"tag:blogger.com,1999:blog-5096278891763426276.post-7039745451118618656\", \"a47b16a321e3f78a36d0ab89e53e505b\", 1519239491, \"1611441261931408\", \"Visualizing hex dumps with Unicode emoji\", \"Oona R\\u00e4is\\u00e4nen\", \"<p>Memorizing SSH public key fingerprints can be difficult; they're just long random numbers displayed in base 16. There are some terminal-friendly solutions, like OpenSSH's randomart. But because I use a Unicode terminal, I like to map the individual bytes into characters in the <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/en.wikipedia.org\\/wiki\\/Miscellaneous_Symbols_and_Pictographs\\\" title=\\\"Wikipedia: Miscellaneous Symbols and Pictographs\\\">Miscellaneous Symbols and Pictographs<\\/a> block.<\\/p> <p>This Perl script does just that:<\\/p> <pre><code><br \\/>@emoji = qw( \\ud83c\\udf00  \\ud83c\\udf02  \\ud83c\\udf05  \\ud83c\\udf08  \\ud83c\\udf19  \\ud83c\\udf1e  \\ud83c\\udf1f  \\ud83c\\udf20  \\ud83c\\udf30  \\ud83c\\udf31  \\ud83c\\udf32  \\ud83c\\udf33  \\ud83c\\udf34  \\ud83c\\udf35  \\ud83c\\udf37  \\ud83c\\udf38<br \\/>             \\ud83c\\udf39  \\ud83c\\udf3a  \\ud83c\\udf3b  \\ud83c\\udf3c  \\ud83c\\udf3d  \\ud83c\\udf3e  \\ud83c\\udf3f  \\ud83c\\udf40  \\ud83c\\udf41  \\ud83c\\udf42  \\ud83c\\udf43  \\ud83c\\udf44  \\ud83c\\udf45  \\ud83c\\udf46  \\ud83c\\udf47  \\ud83c\\udf48<br \\/>             \\ud83c\\udf49  \\ud83c\\udf4a  \\ud83c\\udf4b  \\ud83c\\udf4c  \\ud83c\\udf4d  \\ud83c\\udf4e  \\ud83c\\udf4f  \\ud83c\\udf50  \\ud83c\\udf51  \\ud83c\\udf52  \\ud83c\\udf53  \\ud83c\\udf54  \\ud83c\\udf55  \\ud83c\\udf56  \\ud83c\\udf57  \\ud83c\\udf58<br \\/>             \\ud83c\\udf5c  \\ud83c\\udf5d  \\ud83c\\udf5e  \\ud83c\\udf5f  \\ud83c\\udf60  \\ud83c\\udf61  \\ud83c\\udf62  \\ud83c\\udf63  \\ud83c\\udf64  \\ud83c\\udf65  \\ud83c\\udf66  \\ud83c\\udf67  \\ud83c\\udf68  \\ud83c\\udf69  \\ud83c\\udf6a  \\ud83c\\udf6b<br \\/>             \\ud83c\\udf6c  \\ud83c\\udf6d  \\ud83c\\udf6e  \\ud83c\\udf6f  \\ud83c\\udf70  \\ud83c\\udf71  \\ud83c\\udf72  \\ud83c\\udf73  \\ud83c\\udf74  \\ud83c\\udf75  \\ud83c\\udf76  \\ud83c\\udf77  \\ud83c\\udf78  \\ud83c\\udf79  \\ud83c\\udf7a  \\ud83c\\udf7b<br \\/>             \\ud83c\\udf7c  \\ud83c\\udf80  \\ud83c\\udf81  \\ud83c\\udf82  \\ud83c\\udf83  \\ud83c\\udf84  \\ud83c\\udf85  \\ud83c\\udf88  \\ud83c\\udf89  \\ud83c\\udf8a  \\ud83c\\udf8b  \\ud83c\\udf8c  \\ud83c\\udf8d  \\ud83c\\udf8e  \\ud83c\\udf8f  \\ud83c\\udf92<br \\/>             \\ud83c\\udf93  \\ud83c\\udfa0  \\ud83c\\udfa1  \\ud83c\\udfa2  \\ud83c\\udfa3  \\ud83c\\udfa4  \\ud83c\\udfa5  \\ud83c\\udfa6  \\ud83c\\udfa7  \\ud83c\\udfa8  \\ud83c\\udfa9  \\ud83c\\udfaa  \\ud83c\\udfab  \\ud83c\\udfac  \\ud83c\\udfad  \\ud83c\\udfae<br \\/>             \\ud83c\\udfaf  \\ud83c\\udfb0  \\ud83c\\udfb1  \\ud83c\\udfb2  \\ud83c\\udfb3  \\ud83c\\udfb4  \\ud83c\\udfb5  \\ud83c\\udfb7  \\ud83c\\udfb8  \\ud83c\\udfb9  \\ud83c\\udfba  \\ud83c\\udfbb  \\ud83c\\udfbd  \\ud83c\\udfbe  \\ud83c\\udfbf  \\ud83c\\udfc0<br \\/>             \\ud83c\\udfc1  \\ud83c\\udfc2  \\ud83c\\udfc3  \\ud83c\\udfc4  \\ud83c\\udfc6  \\ud83c\\udfc7  \\ud83c\\udfc8  \\ud83c\\udfc9  \\ud83c\\udfca  \\ud83d\\udc00  \\ud83d\\udc01  \\ud83d\\udc02  \\ud83d\\udc03  \\ud83d\\udc04  \\ud83d\\udc05  \\ud83d\\udc06<br \\/>             \\ud83d\\udc07  \\ud83d\\udc08  \\ud83d\\udc09  \\ud83d\\udc0a  \\ud83d\\udc0b  \\ud83d\\udc0c  \\ud83d\\udc0d  \\ud83d\\udc0e  \\ud83d\\udc0f  \\ud83d\\udc10  \\ud83d\\udc11  \\ud83d\\udc12  \\ud83d\\udc13  \\ud83d\\udc14  \\ud83d\\udc15  \\ud83d\\udc16<br \\/>             \\ud83d\\udc17  \\ud83d\\udc18  \\ud83d\\udc19  \\ud83d\\udc1a  \\ud83d\\udc1b  \\ud83d\\udc1c  \\ud83d\\udc1d  \\ud83d\\udc1e  \\ud83d\\udc1f  \\ud83d\\udc20  \\ud83d\\udc21  \\ud83d\\udc22  \\ud83d\\udc23  \\ud83d\\udc24  \\ud83d\\udc25  \\ud83d\\udc26<br \\/>             \\ud83d\\udc27  \\ud83d\\udc28  \\ud83d\\udc29  \\ud83d\\udc2a  \\ud83d\\udc2b  \\ud83d\\udc2c  \\ud83d\\udc2d  \\ud83d\\udc2e  \\ud83d\\udc2f  \\ud83d\\udc30  \\ud83d\\udc31  \\ud83d\\udc32  \\ud83d\\udc33  \\ud83d\\udc34  \\ud83d\\udc35  \\ud83d\\udc36<br \\/>             \\ud83d\\udc37  \\ud83d\\udc38  \\ud83d\\udc39  \\ud83d\\udc3a  \\ud83d\\udc3b  \\ud83d\\udc3c  \\ud83d\\udc3d  \\ud83d\\udc3e  \\ud83d\\udc40  \\ud83d\\udc42  \\ud83d\\udc43  \\ud83d\\udc44  \\ud83d\\udc45  \\ud83d\\udc46  \\ud83d\\udc47  \\ud83d\\udc48<br \\/>             \\ud83d\\udc49  \\ud83d\\udc4a  \\ud83d\\udc4b  \\ud83d\\udc4c  \\ud83d\\udc4d  \\ud83d\\udc4e  \\ud83d\\udc4f  \\ud83d\\udc50  \\ud83d\\udc51  \\ud83d\\udc52  \\ud83d\\udc53  \\ud83d\\udc54  \\ud83d\\udc55  \\ud83d\\udc56  \\ud83d\\udc57  \\ud83d\\udc58<br \\/>             \\ud83d\\udc59  \\ud83d\\udc5a  \\ud83d\\udc5b  \\ud83d\\udc5c  \\ud83d\\udc5d  \\ud83d\\udc5e  \\ud83d\\udc5f  \\ud83d\\udc60  \\ud83d\\udc61  \\ud83d\\udc62  \\ud83d\\udc63  \\ud83d\\udc64  \\ud83d\\udc65  \\ud83d\\udc66  \\ud83d\\udc67  \\ud83d\\udc68<br \\/>             \\ud83d\\udc69  \\ud83d\\udc6a  \\ud83d\\udc6e  \\ud83d\\udc6f  \\ud83d\\udc7a  \\ud83d\\udc7b  \\ud83d\\udc7c  \\ud83d\\udc7d  \\ud83d\\udc7e  \\ud83d\\udc7f  \\ud83d\\udc80  \\ud83d\\udc81  \\ud83d\\udc82  \\ud83d\\udc83  \\ud83d\\udc84  \\ud83d\\udc85 );<br \\/><br \\/>while (&lt;&gt;) {<br \\/>  if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {<br \\/>    ($b, $m, $a) = ($`, $&amp;, $');<br \\/>    print $b.join(\\\"  \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;<br \\/>  }<br \\/>}<\\/code><\\/pre> <p>What's happening here? First we create a 256-element array containing a hand-picked collection of emoji. Naturally, they're all assigned an index from <span>0x00<\\/span> to <span>0xff<\\/span>. Then we'll loop through standard input and look for lines containing colon-separated hex bytes. Each hex value is replaced with an emoji from the array.<\\/p> <p>Here's the output:<\\/p> <div><img border=\\\"0\\\" src=\\\"https:\\/\\/2.bp.blogspot.com\\/-fx-QGLb65NQ\\/VFF5VuQ6V9I\\/AAAAAAAADhg\\/F52MqWiLGJg\\/s480\\/rsa.png\\\" alt=\\\"[Image: Terminal screenshot showing a PGP key fingerprint and the same with all hex numbers replaced with emoji.]\\\" \\/><\\/div> <p>The script could easily be extended to support output from other hex-formatted sources as well, such as xxd:<\\/p> <div><img border=\\\"0\\\" src=\\\"https:\\/\\/2.bp.blogspot.com\\/-cGSKJPcr16g\\/VFH-95vNJVI\\/AAAAAAAADh8\\/PvryEVbLX_M\\/s480\\/korvista.png\\\" alt=\\\"[Image: Terminal screenshot showing a hex dump of a poem and the same with all hex numbers replaced with emoji. kissofoni; tassun kynsi neulana \\/ musa korvista kajahtaa]\\\" \\/><\\/div> <p>Some additional methods for visualizing hex dumps and key fingerprints, from the comments section:<\\/p> <ul>  <li><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"http:\\/\\/user.xmission.com\\/~atoponce\\/art\\/\\\" title=\\\"PGP Art\\\">PGP Strong Set Top 50 Fingerprint Art<\\/a><\\/li>  <li><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"http:\\/\\/sebsauvage.net\\/wiki\\/doku.php?id=php:vizhash_gd\\\" title=\\\"php:vizhash_gd [sebsauvage]\\\">VizHash GD - a visual hash<\\/a><\\/li><\\/ul>\", \"memorizing ssh public key fingerprints can be difficult; they're just long random numbers displayed in base 16. there are some terminal-friendly solutions, like openssh's randomart. but because i use a unicode terminal, i like to map the individual bytes into characters in the miscellaneous symbols and pictographs block. this perl script does just that: @emoji = qw( \\ud83c\\udf00  \\ud83c\\udf02  \\ud83c\\udf05  \\ud83c\\udf08  \\ud83c\\udf19  \\ud83c\\udf1e  \\ud83c\\udf1f  \\ud83c\\udf20  \\ud83c\\udf30  \\ud83c\\udf31  \\ud83c\\udf32  \\ud83c\\udf33  \\ud83c\\udf34  \\ud83c\\udf35  \\ud83c\\udf37  \\ud83c\\udf38             \\ud83c\\udf39  \\ud83c\\udf3a  \\ud83c\\udf3b  \\ud83c\\udf3c  \\ud83c\\udf3d  \\ud83c\\udf3e  \\ud83c\\udf3f  \\ud83c\\udf40  \\ud83c\\udf41  \\ud83c\\udf42  \\ud83c\\udf43  \\ud83c\\udf44  \\ud83c\\udf45  \\ud83c\\udf46  \\ud83c\\udf47  \\ud83c\\udf48             \\ud83c\\udf49  \\ud83c\\udf4a  \\ud83c\\udf4b  \\ud83c\\udf4c  \\ud83c\\udf4d  \\ud83c\\udf4e  \\ud83c\\udf4f  \\ud83c\\udf50  \\ud83c\\udf51  \\ud83c\\udf52  \\ud83c\\udf53  \\ud83c\\udf54  \\ud83c\\udf55  \\ud83c\\udf56  \\ud83c\\udf57  \\ud83c\\udf58             \\ud83c\\udf5c  \\ud83c\\udf5d  \\ud83c\\udf5e  \\ud83c\\udf5f  \\ud83c\\udf60  \\ud83c\\udf61  \\ud83c\\udf62  \\ud83c\\udf63  \\ud83c\\udf64  \\ud83c\\udf65  \\ud83c\\udf66  \\ud83c\\udf67  \\ud83c\\udf68  \\ud83c\\udf69  \\ud83c\\udf6a  \\ud83c\\udf6b             \\ud83c\\udf6c  \\ud83c\\udf6d  \\ud83c\\udf6e  \\ud83c\\udf6f  \\ud83c\\udf70  \\ud83c\\udf71  \\ud83c\\udf72  \\ud83c\\udf73  \\ud83c\\udf74  \\ud83c\\udf75  \\ud83c\\udf76  \\ud83c\\udf77  \\ud83c\\udf78  \\ud83c\\udf79  \\ud83c\\udf7a  \\ud83c\\udf7b             \\ud83c\\udf7c  \\ud83c\\udf80  \\ud83c\\udf81  \\ud83c\\udf82  \\ud83c\\udf83  \\ud83c\\udf84  \\ud83c\\udf85  \\ud83c\\udf88  \\ud83c\\udf89  \\ud83c\\udf8a  \\ud83c\\udf8b  \\ud83c\\udf8c  \\ud83c\\udf8d  \\ud83c\\udf8e  \\ud83c\\udf8f  \\ud83c\\udf92             \\ud83c\\udf93  \\ud83c\\udfa0  \\ud83c\\udfa1  \\ud83c\\udfa2  \\ud83c\\udfa3  \\ud83c\\udfa4  \\ud83c\\udfa5  \\ud83c\\udfa6  \\ud83c\\udfa7  \\ud83c\\udfa8  \\ud83c\\udfa9  \\ud83c\\udfaa  \\ud83c\\udfab  \\ud83c\\udfac  \\ud83c\\udfad  \\ud83c\\udfae             \\ud83c\\udfaf  \\ud83c\\udfb0  \\ud83c\\udfb1  \\ud83c\\udfb2  \\ud83c\\udfb3  \\ud83c\\udfb4  \\ud83c\\udfb5  \\ud83c\\udfb7  \\ud83c\\udfb8  \\ud83c\\udfb9  \\ud83c\\udfba  \\ud83c\\udfbb  \\ud83c\\udfbd  \\ud83c\\udfbe  \\ud83c\\udfbf  \\ud83c\\udfc0             \\ud83c\\udfc1  \\ud83c\\udfc2  \\ud83c\\udfc3  \\ud83c\\udfc4  \\ud83c\\udfc6  \\ud83c\\udfc7  \\ud83c\\udfc8  \\ud83c\\udfc9  \\ud83c\\udfca  \\ud83d\\udc00  \\ud83d\\udc01  \\ud83d\\udc02  \\ud83d\\udc03  \\ud83d\\udc04  \\ud83d\\udc05  \\ud83d\\udc06             \\ud83d\\udc07  \\ud83d\\udc08  \\ud83d\\udc09  \\ud83d\\udc0a  \\ud83d\\udc0b  \\ud83d\\udc0c  \\ud83d\\udc0d  \\ud83d\\udc0e  \\ud83d\\udc0f  \\ud83d\\udc10  \\ud83d\\udc11  \\ud83d\\udc12  \\ud83d\\udc13  \\ud83d\\udc14  \\ud83d\\udc15  \\ud83d\\udc16             \\ud83d\\udc17  \\ud83d\\udc18  \\ud83d\\udc19  \\ud83d\\udc1a  \\ud83d\\udc1b  \\ud83d\\udc1c  \\ud83d\\udc1d  \\ud83d\\udc1e  \\ud83d\\udc1f  \\ud83d\\udc20  \\ud83d\\udc21  \\ud83d\\udc22  \\ud83d\\udc23  \\ud83d\\udc24  \\ud83d\\udc25  \\ud83d\\udc26             \\ud83d\\udc27  \\ud83d\\udc28  \\ud83d\\udc29  \\ud83d\\udc2a  \\ud83d\\udc2b  \\ud83d\\udc2c  \\ud83d\\udc2d  \\ud83d\\udc2e  \\ud83d\\udc2f  \\ud83d\\udc30  \\ud83d\\udc31  \\ud83d\\udc32  \\ud83d\\udc33  \\ud83d\\udc34  \\ud83d\\udc35  \\ud83d\\udc36             \\ud83d\\udc37  \\ud83d\\udc38  \\ud83d\\udc39  \\ud83d\\udc3a  \\ud83d\\udc3b  \\ud83d\\udc3c  \\ud83d\\udc3d  \\ud83d\\udc3e  \\ud83d\\udc40  \\ud83d\\udc42  \\ud83d\\udc43  \\ud83d\\udc44  \\ud83d\\udc45  \\ud83d\\udc46  \\ud83d\\udc47  \\ud83d\\udc48             \\ud83d\\udc49  \\ud83d\\udc4a  \\ud83d\\udc4b  \\ud83d\\udc4c  \\ud83d\\udc4d  \\ud83d\\udc4e  \\ud83d\\udc4f  \\ud83d\\udc50  \\ud83d\\udc51  \\ud83d\\udc52  \\ud83d\\udc53  \\ud83d\\udc54  \\ud83d\\udc55  \\ud83d\\udc56  \\ud83d\\udc57  \\ud83d\\udc58             \\ud83d\\udc59  \\ud83d\\udc5a  \\ud83d\\udc5b  \\ud83d\\udc5c  \\ud83d\\udc5d  \\ud83d\\udc5e  \\ud83d\\udc5f  \\ud83d\\udc60  \\ud83d\\udc61  \\ud83d\\udc62  \\ud83d\\udc63  \\ud83d\\udc64  \\ud83d\\udc65  \\ud83d\\udc66  \\ud83d\\udc67  \\ud83d\\udc68             \\ud83d\\udc69  \\ud83d\\udc6a  \\ud83d\\udc6e  \\ud83d\\udc6f  \\ud83d\\udc7a  \\ud83d\\udc7b  \\ud83d\\udc7c  \\ud83d\\udc7d  \\ud83d\\udc7e  \\ud83d\\udc7f  \\ud83d\\udc80  \\ud83d\\udc81  \\ud83d\\udc82  \\ud83d\\udc83  \\ud83d\\udc84  \\ud83d\\udc85 );while (<>) {  if (\\/[a-f0-9:]+:[a-f0-9:]+\\/) {    ($b, $m, $a) = ($`, $&, $');    print $b.join(\\\"  \\\", map { $emoji[$_] } map hex, split \\/:\\/, $m).\\\" \\\".$a;  }} what's happening here? first we create a 256-element array containing a hand-picked collection of emoji. naturally, they're all assigned an index from 0x00 to 0xff. then we'll loop through standard input and look for lines containing colon-separated hex bytes. each hex value is replaced with an emoji from the array. here's the output:  the script could easily be extended to support output from other hex-formatted sources as well, such as xxd:  some additional methods for visualizing hex dumps and key fingerprints, from the comments section:   pgp strong set top 50 fingerprint art  vizhash gd - a visual hashoona r\\u00e4is\\u00e4nenvisualizing hex dumps with unicode emojihttp:\\/\\/www.windytan.com\\/2014\\/10\\/visualizing-hex-bytes-with-unicode-emoji.html\", \"812c550a581428a95bc4d897f721b2a9\", \"fa053fe2305b1c1592ea0651a9d5a943\", \"4\", 55]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8C\\x80  ...' for column `newown_nordlichter_brv_de`.`oc_news_items`.`body` at row 1",
        "Code": "22007",
        "Trace": [
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php",
                "line": 127,
                "function": "execute",
                "class": "PDOStatement",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php",
                "line": 1062,
                "function": "execute",
                "class": "Doctrine\\DBAL\\Driver\\PDOStatement",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/DB/Connection.php",
                "line": 222,
                "function": "executeUpdate",
                "class": "Doctrine\\DBAL\\Connection",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php",
                "line": 209,
                "function": "executeUpdate",
                "class": "OC\\DB\\Connection",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/DB/QueryBuilder/QueryBuilder.php",
                "line": 217,
                "function": "execute",
                "class": "Doctrine\\DBAL\\Query\\QueryBuilder",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/public/AppFramework/Db/QBMapper.php",
                "line": 211,
                "function": "execute",
                "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Db/NewsMapperV2.php",
                "line": 64,
                "function": "update",
                "class": "OCP\\AppFramework\\Db\\QBMapper",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/ItemServiceV2.php",
                "line": 97,
                "function": "update",
                "class": "OCA\\News\\Db\\NewsMapperV2",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/FeedServiceV2.php",
                "line": 311,
                "function": "insertOrUpdate",
                "class": "OCA\\News\\Service\\ItemServiceV2",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/FeedServiceV2.php",
                "line": 347,
                "function": "fetch",
                "class": "OCA\\News\\Service\\FeedServiceV2",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Service/UpdaterService.php",
                "line": 55,
                "function": "fetchAll",
                "class": "OCA\\News\\Service\\FeedServiceV2",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/apps/news/lib/Cron/UpdaterJob.php",
                "line": 68,
                "function": "update",
                "class": "OCA\\News\\Service\\UpdaterService",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/BackgroundJob/Job.php",
                "line": 52,
                "function": "run",
                "class": "OCA\\News\\Cron\\UpdaterJob",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/lib/private/BackgroundJob/TimedJob.php",
                "line": 59,
                "function": "execute",
                "class": "OC\\BackgroundJob\\Job",
                "type": "->"
            },
            {
                "file": "/var/www/vhosts/2newown.nordlichter-brv.de/web/cron.php",
                "line": 127,
                "function": "execute",
                "class": "OC\\BackgroundJob\\TimedJob",
                "type": "->"
            }
        ],
        "File": "/var/www/vhosts/2newown.nordlichter-brv.de/web/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php",
        "Line": 127
    }
},
"CustomMessage": "Error while running background job (class: OCA\\News\\Cron\\UpdaterJob, arguments: )"
},
"userAgent": "--",
"version": "20.0.5.2"
}

@Jonny007-MKD
Copy link
Author

Probably the source of the error lies in the body of the post "Animated line drawings with OpenCV", as the previous posts were added (according to the nextcloud.log).
For me it seems to be valid UTF-8, I could not find any chars apart [-a-zA-Z0-9<>,.;:-_"\/\\+#\'&(){}*%$!|] in it. Perhaps something is interpreted incorrectly due to some \s?

@anoymouserver
Copy link
Contributor

anoymouserver commented Feb 12, 2021

It happens in the Visualizing hex dumps with Unicode emoji article.

The escape sequences are exactly the ones of the first emoji:

🌀                🌂                ...
\xF0\x9F\x8C\x80  \xF0\x9F\x8C\x82

@Jonny007-MKD
Copy link
Author

As a temporary workaround I set the prevent_update flag in the oc_news_feeds to 1 for this feed. Now all the other feeds are successfully fetched again.

@Jdogzz
Copy link

Jdogzz commented Feb 23, 2021

I can add a couple extra datapoints with these two rss feeds yielding the invalid datetime format error:
https://www.reddit.com/r/LaptopDeals/.rss
https://www.reddit.com/r/PCDeals/.rss

They give, for example, this error in my logs:

{"reqId":"uatNiR4c0l1uWj2bT5fY","level":3,"time":"2021-02-23T15:30:03+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":169,"function":"convertException","class":"Doctrine\\DBAL\\Driver\\AbstractMySQLDriver","type":"->","args":["An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1",{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php","line":149,"function":"wrapException","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"An exception occurred while executing 'INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [true, \"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"t3_lqglv5\", \"ae4638099ccceaabee9bb6828307d5a1\", 1614081022, \"1614094203478929\", \"\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 Guess what's coming up? We are on the verge of amassing 65,000 subscribers. Upon achieving that milestone, we will be having an Amazon gift card giveaway party!\", \"\\/u\\/techstar2000\", \"<div><p>The title says it all! Be on the look out for this event; you could be one of the twenty (20) lucky winners! <\\/p> <p>Regards, <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"\\/r\\/Laptopdeals\\\">r\\/Laptopdeals<\\/a> Moderating Team.<\\/p> <\\/div>   submitted by   <a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/user\\/techstar2000\\\"> \\/u\\/techstar2000 <\\/a> <br \\/><span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[link]<\\/a><\\/span>   <span><a target=\\\"_blank\\\" rel=\\\"noreferrer\\\" href=\\\"https:\\/\\/www.reddit.com\\/r\\/LaptopDeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\\\">[comments]<\\/a><\\/span>\", \"the title says it all! be on the look out for this event; you could be one of the twenty (20) lucky winners!  regards, r\\/laptopdeals moderating team.    submitted by    \\/u\\/techstar2000   [link]   [comments]\\/u\\/techstar2000\\u23f1\\ufe0f \\u23f3 \\ud83c\\udf89 guess what's coming up? we are on the verge of amassing 65,000 subscribers. upon achieving that milestone, we will be having an amazon gift card giveaway party!https:\\/\\/www.reddit.com\\/r\\/laptopdeals\\/comments\\/lqglv5\\/guess_whats_coming_up_we_are_on_the_verge_of\\/\", \"ebc0cc539e5679038838aad0d9d9151e\", \"4727f6b68615e6246faa85929eefc36a\", 2]:\n\nSQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1071,"function":"driverExceptionDuringQuery","class":"Doctrine\\DBAL\\DBALException","type":"::","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"},{"errorInfo":["22007",1366,"Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1"],"__class__":"Doctrine\\DBAL\\Driver\\PDOException"},"INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php","Line":106,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":129,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\\xF0\\x9F\\x8E\\x89 G...' for column `nextcloud`.`oc_news_items`.`title` at row 1","Code":"22007","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","line":127,"function":"execute","class":"PDOStatement","type":"->","args":[null]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1062,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDOStatement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":222,"function":"executeUpdate","class":"Doctrine\\DBAL\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",[true,"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","t3_lqglv5","ae4638099ccceaabee9bb6828307d5a1",1614081022,"And 8 more entries, set log level to debug to see all entries"],[5,2,2,2,1,"And 8 more entries, set log level to debug to see all entries"]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":209,"function":"executeUpdate","class":"OC\\DB\\Connection","type":"->","args":["INSERT INTO `oc_news_items` (`unread`, `url`, `guid`, `guid_hash`, `pub_date`, `last_modified`, `title`, `author`, `body`, `search_index`, `fingerprint`, `content_hash`, `feed_id`) VALUES(:dcValue1, :dcValue2, :dcValue3, :dcValue4, :dcValue5, :dcValue6, :dcValue7, :dcValue8, :dcValue9, :dcValue10, :dcValue11, :dcValue12, :dcValue13)",{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":true,"dcValue2":"https://www.reddit.com/r/LaptopDeals/comments/lqglv5/guess_whats_coming_up_we_are_on_the_verge_of/","dcValue3":"t3_lqglv5","dcValue4":"ae4638099ccceaabee9bb6828307d5a1","dcValue5":1614081022},{"0":"And 8 more entries, set log level to debug to see all entries","dcValue1":5,"dcValue2":2,"dcValue3":2,"dcValue4":2,"dcValue5":1}]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php","line":135,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Db/NewsMapperV2.php","line":76,"function":"insert","class":"OCP\\AppFramework\\Db\\QBMapper","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/ItemServiceV2.php","line":102,"function":"insert","class":"OCA\\News\\Db\\NewsMapperV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":302,"function":"insertOrUpdate","class":"OCA\\News\\Service\\ItemServiceV2","type":"->","args":[{"id":null,"__class__":"OCA\\News\\Db\\Item"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedServiceV2.php","line":341,"function":"fetch","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[{"items":[],"id":2,"__class__":"OCA\\News\\Db\\Feed"}]},{"file":"/var/www/nextcloud/apps/news/lib/Service/UpdaterService.php","line":55,"function":"fetchAll","class":"OCA\\News\\Service\\FeedServiceV2","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/news/lib/Cron/UpdaterJob.php","line":71,"function":"update","class":"OCA\\News\\Service\\UpdaterService","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":52,"function":"run","class":"OCA\\News\\Cron\\UpdaterJob","type":"->","args":[null]},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":59,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->","args":[{"__class__":"OC\\BackgroundJob\\JobList"},{"__class__":"OC\\Log"}]}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php","Line":127}},"CustomMessage":"Error while running background job (class: OCA\\News\\Cron\\UpdaterJob, arguments: )"},"userAgent":"--","version":"20.0.7.1","id":"60352c86ed578"}

@anoymouserver
Copy link
Contributor

anoymouserver commented Feb 23, 2021

All three feeds work for me (NC 21, News 15.3.2, MariaDB 10.1.48). Did you enable the MySQL 4-byte support (utf8mb4) for your server?

Screenshot with the emojis

grafik

@Jdogzz
Copy link

Jdogzz commented Feb 24, 2021

@anoymouserver Thanks for letting me know about that, I went through the process but it still threw errors in my log. However, I took one extra step and added a line to my database config file as noted here:

nextcloud/server#20883 (comment)

After that I added a test feed back in again and no more errors were generated. I will report back in the event the feeds stop updating again and errors return.

@Jonny007-MKD
Copy link
Author

I also enabled utf8mb4 as described and it works now. Thanks!

@LinAGKar
Copy link

This can still happen even with the database set to utf8mb4, if a feed contains invalid UTF-8.

@MCMic
Copy link

MCMic commented Aug 16, 2022

I have the same issue, and indeed I do not have utf8mb4 yet, that's a limitation of yunohost package currently. Would it be possible to at least catch the error so that only the problematic feed do not update? Also having a list of feeds with errors somewhere would be useful.

@nextcloud nextcloud locked and limited conversation to collaborators Feb 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants
@MCMic @Jonny007-MKD @Jdogzz @LinAGKar @Grotax @anoymouserver and others