In v2, quoted NaN values force a column to character

[slodge]

Between v1.4 and v2 it seems like we've hit an issue with the way "NaN" is read and interpreted.

Previously if a quoted numeric column included "NaN" values, then values would be read as numeric - but now values are being read as character.

A reproducible example is the "Value" column below - it's parsed as dbl in v1.4 and chr in 2.0.1

Is this something that can be changed in readr? Or is this something we need to code around as users?

text <- '"Key","Id","DataDate","ReleaseDate","Value"
"First","-2147483648","1900-01-01","1900-01-01","0.5"
"Second","543","2021-08-13","2021-08-13","NaN"
"First","730","2021-08-13","2021-08-13","0"
'
readr::read_csv(text)

Output readr v1.4.0

> text <- '"Key","Id","DataDate","ReleaseDate","Value"
+ "First","-2147483648","1900-01-01","1900-01-01","0.5"
+ "Second","543","2021-08-13","2021-08-13","NaN"
+ "First","730","2021-08-13","2021-08-13","0"
+ '
> readr::read_csv(text)
# A tibble: 3 x 5
  Key             Id DataDate   ReleaseDate Value
  <chr>        <dbl> <date>     <date>      <dbl>
1 First  -2147483648 1900-01-01 1900-01-01    0.5
2 Second         543 2021-08-13 2021-08-13  NaN  
3 First          730 2021-08-13 2021-08-13    0  

Output readr v2.0.1

> text <- '"Key","Id","DataDate","ReleaseDate","Value"
+ "First","-2147483648","1900-01-01","1900-01-01","0.5"
+ "Second","543","2021-08-13","2021-08-13","NaN"
+ "First","730","2021-08-13","2021-08-13","0"
+ '
> readr::read_csv(text)
Rows: 3 Columns: 5                                                                                                                
-- Column specification -----------------------------------------------------------------------
Delimiter: ","
chr  (2): Key, Value
dbl  (1): Id
date (2): DataDate, ReleaseDate

i Use `spec()` to retrieve the full column specification for this data.
i Specify the column types or set `show_col_types = FALSE` to quiet this message.
# A tibble: 3 x 5
  Key             Id DataDate   ReleaseDate Value
  <chr>        <dbl> <date>     <date>      <chr>
1 First  -2147483648 1900-01-01 1900-01-01  0.5  
2 Second         543 2021-08-13 2021-08-13  NaN  
3 First          730 2021-08-13 2021-08-13  0    

Tested on R 4.0.2 on Windows

[slodge]

Possibly linked to #1225 (although in this case we don't specify the column types)

[jimhester]

Thanks for opening the issue and including a reproducible example!

The double parser in vroom did not handle NaN values specially. This should now be fixed.

Fixed by tidyverse/vroom@f520d37

[slodge]

Thanks!