Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on C Warnings #32163

Merged
merged 13 commits into from
Mar 19, 2020
Merged

Error on C Warnings #32163

merged 13 commits into from
Mar 19, 2020

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Feb 21, 2020

Using Clang and latest numpy locally looks like we can use -Werror. WIP as I think there may still be things with gcc and older numpy versions to sort out

Note that there is also a warning emitted by cython for unreachable code in groupby, but -Werror wouldn't affect the "cythonization" step.

@jbrockmendel
Copy link
Member

on the py37 builds

In file included from pandas/_libs/src/ujson/python/date_conversions.h:7:0,
                 from pandas/_libs/src/ujson/python/date_conversions.c:4:
/home/vsts/miniconda3/envs/pandas-dev/include/python3.7m/datetime.h:204:25: error: ‘PyDateTimeAPI’ defined but not used [-Werror=unused-variable]
 static PyDateTime_CAPI *PyDateTimeAPI = NULL;
                         ^
cc1: all warnings being treated as errors

is this something we can handle on our end?

@@ -192,7 +192,7 @@ cdef class StringVector:

append_data_string(self.data, x)

cdef extend(self, ndarray[:] x):
cdef extend(self, ndarray[object] x):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure this is accurate, might be a string dtype? IIRC this class isnt used

@gfyoung gfyoung added the Build Library building on various platforms label Feb 22, 2020
@alimcmaster1
Copy link
Member

on the py37 builds

In file included from pandas/_libs/src/ujson/python/date_conversions.h:7:0,
                 from pandas/_libs/src/ujson/python/date_conversions.c:4:
/home/vsts/miniconda3/envs/pandas-dev/include/python3.7m/datetime.h:204:25: error: ‘PyDateTimeAPI’ defined but not used [-Werror=unused-variable]
 static PyDateTime_CAPI *PyDateTimeAPI = NULL;
                         ^
cc1: all warnings being treated as errors

is this something we can handle on our end?

Potentially missing the PyDateTime_IMPORT macro which should initialise this. https://docs.python.org/3/c-api/datetime.html#datetime-objects

@WillAyd
Copy link
Member Author

WillAyd commented Feb 23, 2020

Yea that and get_blkno_indexers throws a lot of errors in GCC. Haven't been able to figure those out yet but getting close...

@WillAyd
Copy link
Member Author

WillAyd commented Mar 14, 2020

I think the issue here in internals might be a problem with the code Cython generates for generator functions. I opened cython/cython#3430 there so will see (@MomIsBestFriend I believe you were looking at that as well)

@@ -67,7 +67,7 @@ npy_datetime NpyDateTimeToEpoch(npy_datetime dt, NPY_DATETIMEUNIT base) {
}

/* Convert PyDatetime To ISO C-string. mutates len */
char *PyDateTimeToIso(PyDateTime_Date *obj, NPY_DATETIMEUNIT base,
char *PyDateTimeToIso(PyObject *obj, NPY_DATETIMEUNIT base,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently PyDateTime_Date is not considered part of the public API and attempts have been made in the past to privatize:

python/cpython#10238 (comment)

So PyObject seems the right way to go here and gets rid of warnings

if n == 0:
return

result = list()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are some "bugs" in Cython with generator functions that were yielding build warnings. I opened an issue on the tracker for it here:

cython/cython#3430

But for now figure OK to just convert to a list instead of a generator func anyway

@WillAyd WillAyd marked this pull request as ready for review March 19, 2020 02:53
@WillAyd
Copy link
Member Author

WillAyd commented Mar 19, 2020

OK this should be ready for review. w.r.t. the internals change I ran benchmarks for concat and reshape again and got the following:

       before           after         ratio
     [80078ac0]       [5fc220c0]
     <master>         <no-warnings>
-      3.81±0.2ms      3.18±0.02ms     0.83  reshape.Explode.time_explode(1000, 5)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

(which I don't think is related)

@jreback jreback added this to the 1.1 milestone Mar 19, 2020
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. @jbrockmendel

how are the error messages if something is warning? intelligable?

@WillAyd
Copy link
Member Author

WillAyd commented Mar 19, 2020 via email

@jreback
Copy link
Contributor

jreback commented Mar 19, 2020

Maybe misreading your question but compilation warnings will now show as error messages, so CI will fail * note that Cython warnings are separate from compilation warnings, so those wont fail CI

Sent from my iPad
On Mar 19, 2020, at 2:13 PM, Jeff Reback @.***> wrote:  @jreback approved this pull request. lgtm. @jbrockmendel how are the error messages if something is warning? intelligable? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

ok great.

@jreback jreback merged commit aee8e11 into pandas-dev:master Mar 19, 2020
@jreback
Copy link
Contributor

jreback commented Mar 19, 2020

thanks

@WillAyd WillAyd deleted the no-warnings branch March 19, 2020 21:31
SeeminSyed pushed a commit to CSCD01-team01/pandas that referenced this pull request Mar 22, 2020
jbrockmendel pushed a commit to jbrockmendel/pandas that referenced this pull request Mar 23, 2020
jbrockmendel pushed a commit to jbrockmendel/pandas that referenced this pull request Mar 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Library building on various platforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants