Detect when pub-cache seems corrupted?

[DartBot]

<img src="https://avatars.githubusercontent.com/u/2049220?v=3" align="left" width="96" height="96"hspace="10"> Issue by sigmundch
Originally opened as dart-lang/sdk#19958

---

Not sure if there is anything we can do, but recently I've heard of several users, and I myself got into a situation where my pub-cache got corrupted.

Today I was running pub build and got this odd error:
Loading source assets...
Loading polymer/src/build/mirrors_remover, observe and smoke/src/default_transformer transformers... (1.1s)Uncaught Error: Failure getting http://localhost:57723/packages/path/path.dart: 404 Not Found
Stack Trace:

­0 _asyncLoadError (dart:_builtin:337)

­1 _httpGet.<anonymous closure>.<anonymous closure> (dart:_builtin:128)

­2 _RootZone.runGuarded (dart:async/zone.dart:1015)

­3 _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:393)

­4 _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:402)

­5 _BufferingStreamSubscription._close (dart:async/stream_impl.dart:290)

­6 _ForwardingStream._handleDone (dart:async/stream_pipe.dart:94)

­7 _handleDone (dart:async/stream_pipe.dart:162)

­8 _RootZone.runGuarded (dart:async/zone.dart:1015)

­9 _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:393)

­10 _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:402)

­11 _DelayedDone.perform (dart:async/stream_impl.dart:617)

­12 _StreamImplEvents.handleNext (dart:async/stream_impl.dart:714)

­13 _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:674)

­14 _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)

­15 _asyncRunCallback (dart:async/schedule_microtask.dart:32)

­16 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:126)

Loading polymer/src/build/mirrors_remover, observe and smoke/src/default_transformer transformers... (1.1s)
'Unhandled exception:
Failure getting http://localhost:57723/packages/path/path.dart: 404 Not Found

­0 _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:826)

­1 _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)

­2 _asyncRunCallback (dart:async/schedule_microtask.dart:32)

­3 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:126)'

dart:isolate-patch/isolate_patch.dart 256 Isolate.spawnUri
/mnt/data/b/build/slave/dart-editor-linux-dev/build/dart/sdk/lib/_internal/pub/lib/src/dart.dart 179 _isolateBuffer
dart:isolate-patch/isolate_patch.dart 219 _startIsolate.isolateStartHandler
dart:isolat

So it took me a while to realize what the problem was. Looking at my cache, only the 'lib/' directories were deleted, so it could be that this is the result of a bug in a transformer or something else.

I am creating this bug to see if we can determine when corruption happens, and provide better diagnostics. One idea would be to check for timestamps and compare to the time when that cache entry was populated: in my case the 'lib' folder said it was modified. I search for my history logs and the only thing that run around that time was a build.py command from the Dart repo, which I doubt was what disturbed my cache, but who knows.

Another idea is to do this check before & after a pub-build command, so we can at least report if a transformer is misbehaving.

Thoughts?

[DartBot]

<img src="https://avatars.githubusercontent.com/u/188?v=3" align="left" width="48" height="48"hspace="10"> Comment by nex3

---

It looks like this corruption is being caused by tools incorrectly following package symlinks when recursively deleting files. Since we're planning on removing symlinks entirely (issue #727), I don't think it makes sense to put much effort into this.

---

Added NotPlanned label.