One test fails on PowerPC (Fixed) #7

barracuda156 · 2023-06-21T07:01:51Z

@dipterix Could you please look into this?

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

> library(testthat)
> 
> library(filearray)
OpenMP not detected. Using single thread only.
> 
> test_check("filearray")
BIBINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib:
BSTINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst:
CCACHE_DIR              /opt/local/var/macports/build/.ccache
COLUMNS                 80
COMMAND_MODE            legacy
DEVELOPER_DIR           /Developer
DISPLAY                 /tmp/launch-zLBRd4/:0
DYLD_FALLBACK_LIBRARY_PATH
                        /opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib
DYLD_LIBRARY_PATH       /opt/local/lib/libgcc:/opt/local/lib/libgcc:/opt/local/lib/libgcc
EDITOR                  vi
HOME                    /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home
LANG                    en_US.UTF-8
LANGUAGE                C
LC_COLLATE              C
LINES                   24
LN_S                    ln -s
MAKE                    make
NO_PROXY                *.local,169.254/16
PAGER                   /opt/local/bin/less
PATH                    /opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
PWD                     /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-0.1.5/filearray.Rcheck/tests
R_ARCH                  
R_BATCH                 
R_BROWSER               /usr/bin/open
R_BZIPCMD               /opt/local/bin/bzip2
R_CMD                   /opt/local/Library/Frameworks/R.framework/Resources/bin/Rcmd
R_DEFAULT_PACKAGES      
R_DOC_DIR               /opt/local/Library/Frameworks/R.framework/Resources/doc
R_ENVIRON               
R_ENVIRON_USER          
R_GZIPCMD               /opt/local/bin/gzip
R_HOME                  /opt/local/Library/Frameworks/R.framework/Resources
R_INCLUDE_DIR           /opt/local/Library/Frameworks/R.framework/Resources/include
R_LIBS                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-0.1.5/filearray.Rcheck
R_LIBS_SITE             /opt/local/Library/Frameworks/R.framework/Resources/site-library
R_LIBS_USER             /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home/Library/R/Power
                        Macintosh/4.3/library
R_OSTYPE                unix
R_PAPERSIZE             letter
R_PAPERSIZE_USER        letter
R_PDFVIEWER             /usr/bin/open
R_PLATFORM              powerpc-apple-darwin10.0.0d2
R_PRINTCMD              lpr
R_PROFILE               
R_PROFILE_USER          
R_RD4PDF                times,inconsolata,hyper
R_SESSION_TMPDIR        /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp/Rtmp8SJbon
R_SHARE_DIR             /opt/local/Library/Frameworks/R.framework/Resources/share
R_STRIP_SHARED_LIB      strip -x
R_STRIP_STATIC_LIB      strip -S
R_SYSTEM_ABI            macos,gcc,gxx,gfortran,gfortran
R_TESTS                 
R_TEXI2DVICMD           /opt/local/bin/texi2dvi
R_UNZIPCMD              /opt/local/bin/unzip
R_VERSION               4.3.1
R_ZIPCMD                /opt/local/bin/zip
SED                     /usr/bin/sed
SHLVL                   5
TAR                     /opt/local/bin/gtar
TESTTHAT                true
TESTTHAT_IS_CHECKING    true
TESTTHAT_PKG            filearray
TEXINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex:
TMPDIR                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp
USER                    root
_R_CHECK_INTERNALS2_    1
_R_CHECK_LICENSE_       TRUE
_R_CHECK_PACKAGE_NAME_
                        filearray
_R_SHLIB_BUILD_OBJECTS_SYMBOL_TABLES_
                        TRUE
__CF_USER_TEXT_ENCODING
                        0x0:0:0
[ FAIL 1 | WARN 0 | SKIP 3 | PASS 133 ]

══ Skipped tests (3) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-collapse.R:87:5', 'test-collapse.R:198:5',
  'test-collapse.R:310:5'

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-map.R:52:5'): map arrays ─────────────────────────────────────
output[] (`actual`) not equal to `b` (`expected`).

actual vs expected
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 470.729
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ -146.253
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 732.162
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 19.434
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 13947.981
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 330.581
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 811.361
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 715.993
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ -1089.565
- 1968084071671644348527897663031879476955617889699603688900237419077151061474164088289218211399543970587333969889174470151015299072781666953738662667618568086126156341729585445016368989335574697735956656806382217189919174243337515561254819725312.000
+ 2764.628
and 3371190 more ...

[ FAIL 1 | WARN 0 | SKIP 3 | PASS 133 ]
Error: Test failures
Execution halted

P. S. Generally speaking, usual suspects for causes for errors in such cases are assumed little-endianness and assumed 64-bitness. Less common are 4-byte bool, IBM format for long double (non-IEEE) and PPC-specific rounding.

The text was updated successfully, but these errors were encountered:

barracuda156 · 2023-06-21T07:17:30Z

By the way, also OpenMP detection does not work: “OpenMP not detected. Using single thread only”. GCC does support OpenMP, however.

dipterix · 2023-06-21T16:11:56Z

Oh wow. Thanks a lot for reporting this bug. I'll look into this. Would you mind showing me what can I do to reproduce this error? Also if this is too hard, I would be really thankful if you could help me test the devel version on your machine as I might have fixed fmap functions.

Also the devel version of filearray is not using OpenMP anymore. It's using built-in tinythread Instead.

Thanks!

barracuda156 · 2023-06-21T16:44:25Z

@dipterix Thank you for responding to the issue.

From the latest commit tests results are: [ FAIL 253 | WARN 11 | SKIP 3 | PASS 672 ]
I will attach a log now, too big to quote.

UPD.
filearray_34c7dcf1223dcc94a7204a465c78516d2820dc92.txt

Any ideas?

barracuda156 · 2023-06-21T16:52:01Z

@dipterix I suppose that errors which look very wrong may be due to ignored endianness.

Also, does it use long doubles? PowerPC BE uses IBM format for it (same for macOS, AIX, *BSD, Linux – as long as those are Big-endian).

Re reproducing: I do not think these errors have anything to do with macOS specifics, so presumably this can be reproduced on any PowerPC platform, though 64-bit ones may have fewer failures. Common versions of BSD and some versions of Linux support PowerPC, both 32- and 64-bit. If no hardware is available, Qemu may be used, perhaps.

But I can test on my end whatever you suggest, that may be easier.

dipterix · 2023-06-22T18:41:58Z

OK I will try to fix 64-bit big-endian and see what I can do with 32-bits. I don't have machines to test on 32-bits. filearray does assume 64-bits because just like package bit64, it's using R 64-bit double to store integer64, which allows us to index seriously large tensor arrays (2^31 is too small for indexing). As long as package bit64 works on 32-bit system, filearray should. It's not doing anything else special.

In theory filearray should work on big-endian systems. However, when I flipped the internal endian sign, I immediately received errors, that means I was wrong and endianess is not handled properly.

barracuda156 · 2023-06-22T23:00:55Z

Thank you very much for working on this!

For sure, doubles are supported on 32-bit, and even 128-bit long doubles are (with a caveat of being composed of two doubles, IBM format). This should work in R: I believe, many packages are using these, and so far whatever we test on PowerPC, usually passes 100% tests or occasionally have rounding-related issues. I mean, the platform itself is perfectly functional, and all R-related software should work fine.

dipterix · 2023-06-23T00:23:26Z

Hi @barracuda156 I just pushed a patch to the Github. Would you mind helping me test the devel version?

I think 64-bit should be fixed. I gue→↘↗ss, not so sure because I don't have big-endian machine by my hands. All I can do is to flip the endian flag in my code to force the file and machine to have different endianness.

Again, thanks a lot for helping me.

barracuda156 · 2023-06-23T02:03:27Z

@dipterix You have fixed everything:


R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(filearray)
R CMD check mode. Using 2 threads.
> 
> # Sys.setenv("FILEARRAY_SKIP_COLLAPSE" = "TRUE")
> 
> cat(utils::capture.output({
+     print(Sys.getenv())
+ }), sep = "\n", file = stderr())
BIBINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib:
BSTINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst:
CCACHE_DIR              /opt/local/var/macports/build/.ccache
COLUMNS                 80
COMMAND_MODE            legacy
DEVELOPER_DIR           /Developer
DISPLAY                 /tmp/launch-gQ9IMC/:0
DYLD_FALLBACK_LIBRARY_PATH
                        /opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib
DYLD_LIBRARY_PATH       /opt/local/lib/libgcc:/opt/local/lib/libgcc:/opt/local/lib/libgcc
EDITOR                  vi
FILEARRAY_NUM_THREADS   2
HOME                    /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home
LANG                    en_US.UTF-8
LANGUAGE                C
LC_COLLATE              C
LINES                   24
LN_S                    ln -s
MAKE                    make
NO_PROXY                *.local,169.254/16
PAGER                   /opt/local/bin/less
PATH                    /opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
PWD                     /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck/tests
R_ARCH                  
R_BATCH                 
R_BROWSER               /usr/bin/open
R_BZIPCMD               /opt/local/bin/bzip2
R_CMD                   /opt/local/Library/Frameworks/R.framework/Resources/bin/Rcmd
R_DEFAULT_PACKAGES      
R_DOC_DIR               /opt/local/Library/Frameworks/R.framework/Resources/doc
R_ENVIRON               
R_ENVIRON_USER          
R_GZIPCMD               /opt/local/bin/gzip
R_HOME                  /opt/local/Library/Frameworks/R.framework/Resources
R_INCLUDE_DIR           /opt/local/Library/Frameworks/R.framework/Resources/include
R_LIBS                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck
R_LIBS_SITE             /opt/local/Library/Frameworks/R.framework/Resources/site-library
R_LIBS_USER             /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home/Library/R/Power
                        Macintosh/4.3/library
R_OSTYPE                unix
R_PAPERSIZE             letter
R_PAPERSIZE_USER        letter
R_PDFVIEWER             /usr/bin/open
R_PLATFORM              powerpc-apple-darwin10.0.0d2
R_PRINTCMD              lpr
R_PROFILE               
R_PROFILE_USER          
R_RD4PDF                times,hyper
R_SESSION_TMPDIR        /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp/RtmpEuO8rp
R_SHARE_DIR             /opt/local/Library/Frameworks/R.framework/Resources/share
R_STRIP_SHARED_LIB      strip -x
R_STRIP_STATIC_LIB      strip -S
R_SYSTEM_ABI            macos,gcc,gxx,gfortran,gfortran
R_TESTS                 startup.Rs
R_TEXI2DVICMD           /opt/local/bin/texi2dvi
R_UNZIPCMD              /opt/local/bin/unzip
R_VERSION               4.3.0
R_ZIPCMD                /opt/local/bin/zip
SED                     /usr/bin/sed
SHLVL                   5
TAR                     /opt/local/bin/gtar
TEXINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex:
TMPDIR                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp
USER                    root
_R_CHECK_INTERNALS2_    1
_R_CHECK_LICENSE_       TRUE
_R_CHECK_PACKAGE_NAME_
                        filearray
_R_SHLIB_BUILD_OBJECTS_SYMBOL_TABLES_
                        TRUE
__CF_USER_TEXT_ENCODING
                        0x0:0:0
> 
> test_check("filearray")
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]

══ Skipped tests (3) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-collapse.R:85:5', 'test-collapse.R:196:5',
  'test-collapse.R:308:5'

[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]
> 
> filearray:::clear_cache()
> 
> proc.time()
   user  system elapsed 
 70.335   6.858  79.813

Thank you!

dipterix · 2023-06-23T12:38:54Z

The fix 0.1.6 has been submitted to CRAN

barracuda156 · 2023-06-23T12:40:49Z

Awesome! It is already merged to Macports too: https://ports.macports.org/port/R-filearray/details

barracuda156 closed this as completed Jun 23, 2023

barracuda156 changed the title ~~One test fails on PowerPC [ FAIL 1 | WARN 0 | SKIP 3 | PASS 133 ]~~ One test fails on PowerPC (Fixed) Jun 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

One test fails on PowerPC (Fixed) #7

One test fails on PowerPC (Fixed) #7

barracuda156 commented Jun 21, 2023

barracuda156 commented Jun 21, 2023

dipterix commented Jun 21, 2023

barracuda156 commented Jun 21, 2023 •

edited

Loading

barracuda156 commented Jun 21, 2023 •

edited

Loading

dipterix commented Jun 22, 2023

barracuda156 commented Jun 22, 2023

dipterix commented Jun 23, 2023

barracuda156 commented Jun 23, 2023

dipterix commented Jun 23, 2023

barracuda156 commented Jun 23, 2023

One test fails on PowerPC (Fixed) #7

One test fails on PowerPC (Fixed) #7

Comments

barracuda156 commented Jun 21, 2023

barracuda156 commented Jun 21, 2023

dipterix commented Jun 21, 2023

barracuda156 commented Jun 21, 2023 • edited Loading

barracuda156 commented Jun 21, 2023 • edited Loading

dipterix commented Jun 22, 2023

barracuda156 commented Jun 22, 2023

dipterix commented Jun 23, 2023

barracuda156 commented Jun 23, 2023

dipterix commented Jun 23, 2023

barracuda156 commented Jun 23, 2023

barracuda156 commented Jun 21, 2023 •

edited

Loading

barracuda156 commented Jun 21, 2023 •

edited

Loading