categorical variables are not detected

[earnaud]

As reported in this pull request from the MetaShARK repository (issue #9 ), the template_categorical_variables return the following error:

Templating categorical variables ...
Warning in utils::read.table(file = paste0(path, "/", templates[i]), header = TRUE,  :
  cols = 1 != length(data) = 7
Warning in utils::read.table(file = paste0(path, "/", templates[i]), header = TRUE,  :
  cols = 1 != length(data) = 7
No categorical variables found.
No categorical variables found.
Done.

(Note that this is not an internal error in the function).
In fact, there are actually two problems:

• the dimension of the read table (aka the attributes_*.txt files) seem to be mistaken
• the categorical variables are not found

[earnaud]

Here are views of the content of the "attributes_*.txt" files built from provided ones:
nitrogen.txt

decomp.csv

[earnaud]

Additional info:

• In the table processing, NA are replaced by "" character vectors.
• "categorical" is recognized among "class" column from a browser() interactive session ran just before template_categorical_variables
• values for the "unit" and "dateTimeFormatString" have been filled automatically for development purposes (thus are all identical), regardless of their meaning.

[clnsmth]

Thanks for reporting this issue @earnaud. The attributes template looks good, but I suspect the problem is related to the data tables it's referencing. What files do your directories at path and data.path contain?

[clnsmth]

Is this still an issue @earnaud?

[earnaud]

The issue is not met anymore in MetaShARK.

[earnaud]

I do not want to open a new issue as this is linked:

I get some variables which shall be detected as "character" which are indeed detected as "categorical". I would suggest this metric to guess the actual nature of a variable:

• character if diversity (number of different terms) > modal occurrence (maximum occurrence of one term)
• categorical else

which could be translated in R as:

type <- ifelse( length(unique(var)) > max(table(var)), # alternatively, mean() could be used. Any way, it is just an approximation
  "character", # high diversity
  "categorical" # high repetitions
)

[clnsmth]

This would be a nice Improvement @earnaud . Would you mind refactoring template_table_attributes() (beginning at line 223) and send as a pull request?

[earnaud]

I will do it with the #77 review

[clnsmth]

Done