Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically delete cache if zero-byte or corrupted files found #2

Closed
asimihsan opened this issue Feb 18, 2016 · 2 comments
Closed
Assignees
Labels

Comments

@asimihsan
Copy link
Owner

If the cache contains zero-byte or corrupted files we can no longer unpack tree pack sets, so we'll be unable to list directories or recover files. How to reproduce:

  • Delete the cache directory
  • Run:
arqinator \
    --backup-type s3 \
    --s3-region us-west-2 \
    --s3-bucket-name arq-akiajmthnhpkz2ixzrxq-us-west-2 \
    --verbose list-directory-contents \
    --backup-set-uuid 98DB38F8-B9C6-4296-9385-3C1BF858ED5D \
    --folder-uuid 8D4FAD2A-9E08-46F7-829D-E9601A65455D \
    --path /Users/ai/.ssh
  • While it is caching tree pack sets CTRL-C it.
  • Run the command again. You get the following output:
INFO[0006] Cached tree pack sets.
PANI[0006] Failed to mmap index file:  invalid argument
panic: (*logrus.Entry) (0x71c880,0xc8200112c0)

goroutine 1 [running]:
github.com/Sirupsen/logrus.Entry.log(0xc820072180, 0xc8203044b0, 0x0, 0x0, 0x0, 0xc820423b00, 0x0, 0x0, 0x0, 0xc820304510, ...)
    /Users/ai/Programming/golang/src/github.com/Sirupsen/logrus/entry.go:113 +0x618
github.com/Sirupsen/logrus.(*Entry).Panic(0xc820722550, 0xc8207224e0, 0x1, 0x1)
    /Users/ai/Programming/golang/src/github.com/Sirupsen/logrus/entry.go:158 +0x99
github.com/Sirupsen/logrus.(*Entry).Panicln(0xc820722550, 0xc820722750, 0x2, 0x2)
    /Users/ai/Programming/golang/src/github.com/Sirupsen/logrus/entry.go:253 +0x120
github.com/Sirupsen/logrus.(*Logger).Panicln(0xc820072180, 0xc820722750, 0x2, 0x2)
    /Users/ai/Programming/golang/src/github.com/Sirupsen/logrus/logger.go:204 +0xe7
github.com/Sirupsen/logrus.Panicln(0xc820722750, 0x2, 0x2)
    /Users/ai/Programming/golang/src/github.com/Sirupsen/logrus/exported.go:187 +0x41
github.com/asimihsan/arqinator/arq.(*ArqPackSetIndex).GetTreePackFile.func1(0xc820722870, 0xc8207229f0, 0xc8207227f8, 0xc820722860)
    /Users/ai/Programming/golang/src/github.com/asimihsan/arqinator/arq/pack_set_index.go:277 +0x3da

I've confirmed that there are zero-byte files in the cache directory.

I think we should be doing the following:

  1. Scan the cache for zero-byte files. If any exist delete the cache directory and retry.
  2. Scan the cache for corrupted files (how?). If any exist delete the cache directory and retry.
@asimihsan asimihsan added the bug label Feb 18, 2016
@asimihsan asimihsan self-assigned this Feb 18, 2016
@asimihsan
Copy link
Owner Author

This reproduces the symptoms of #1 but I'm still not sure if this is the actual root cause of that issue.

@asimihsan
Copy link
Owner Author

Resolved by 8df12bf, which is released in v0.1.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant