Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIxedInputStream raises false error when reading only part of a file #5360

Closed
bentsherman opened this issue Oct 3, 2024 · 2 comments
Closed
Assignees
Labels

Comments

@bentsherman
Copy link
Member

FixedInputStream was added to detect truncated files:

@Override
void close() throws IOException {
if( bytesRead != length )
throw new IOException("Read data length does not match expected size - bytes read: ${bytesRead}; expected: ${length}")
super.close()
}

However this does not handle the case where only part of a file is read. Consider this example from sarek:

// Get first line of a FASTQ file
def readFirstLineOfFastq(path) {
    def line = null
    try {
        path.withInputStream {
            InputStream gzipStream = new java.util.zip.GZIPInputStream(it)
            Reader decoder = new InputStreamReader(gzipStream, 'ASCII')
            BufferedReader buffered = new BufferedReader(decoder)
            line = buffered.readLine()
            assert line.startsWith('@')
        }
    } catch (Exception e) {
        log.warn "FASTQ file(${path}): Error streaming"
        log.warn "${e.message}"
    }
    return line
}

It will log this error every time:

Read data length does not match expected size - bytes read: 2756; expected: 16013759

cc @maxulysse

@bentsherman bentsherman added the bug label Oct 3, 2024
@pditommaso
Copy link
Member

pditommaso commented Oct 11, 2024

Related to #5275 and #5214

@pditommaso
Copy link
Member

Solved by #5405

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants