Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lookup the package config location for Dart and Flutter test targets #7941

Merged
merged 16 commits into from
Jul 9, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion packages/devtools_app_shared/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
## 0.2.0
## 0.2.2-wip
* Lookup the connected app package root from an expression evaluation when
the connected app is a Dart or Flutter test.
* Added a field `logExceptions` to `EvalOnDartLibrary` that defaults to true but
can be disabled to prevent exceptions from being logged to console.

## 0.2.1
* Add `navigateToCode` utility method for jumping to code in IDEs.
* Add `FlutterEvent` and `DeveloperServiceEvent` constants.
* Add `connectedAppPackageRoot`, `rootPackageDirectoryForMainIsolate`, and
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ class EvalOnDartLibrary extends DisposableController
ValueListenable<IsolateRef?>? isolate,
this.disableBreakpoints = true,
this.oneRequestAtATime = false,
this.logExceptions = true,
}) : _clientId = Random().nextInt(1000000000) {
_libraryRef = Completer<LibraryRef>();

Expand All @@ -60,6 +61,9 @@ class EvalOnDartLibrary extends DisposableController
/// Whether to disable breakpoints triggered while evaluating expressions.
final bool disableBreakpoints;

/// Whether to log exceptions to stdout on failed evaluations.
final bool logExceptions;

/// An ID unique to this instance, so that [asyncEval] keeps working even if
/// the devtool is opened on multiple tabs at the same time.
final int _clientId;
Expand Down Expand Up @@ -261,7 +265,7 @@ class EvalOnDartLibrary extends DisposableController
}

void _handleError(Object e, StackTrace stack) {
if (_disposed) return;
if (_disposed || !logExceptions) return;

if (e is RPCError) {
_log.shout('RPCError: $e', e, stack);
Expand Down
88 changes: 70 additions & 18 deletions packages/devtools_app_shared/lib/src/service/service_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import 'package:vm_service/vm_service.dart' hide Error;
import '../utils/utils.dart';
import 'connected_app.dart';
import 'dtd_manager.dart';
import 'eval_on_dart_library.dart' hide SentinelException;
import 'flutter_version.dart';
import 'isolate_manager.dart';
import 'isolate_state.dart';
Expand Down Expand Up @@ -521,30 +522,81 @@ class ServiceManager<T extends VmService> {
// VM service connection spawned from `dart test` or `flutter test`).
if (packageRootUriString?.endsWith('.dart') ?? false) {
final rootLibrary = await _mainIsolateRootLibrary();
final testTargetFileUriString =
(rootLibrary?.dependencies ?? <LibraryDependency>[])
.firstWhereOrNull((dep) => dep.prefix == 'test')
?.target
?.uri;
if (testTargetFileUriString != null) {
if (rootLibrary != null) {
packageRootUriString =
(await _lookupPackageConfigByEval(rootLibrary)) ??
// TODO(kenz): remove this fallback once all test bootstrap
// generators include the `packageConfigLocation` constant we
// can evaluate.
await _lookupTestLibraryByPrefix(rootLibrary, dtdManager);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakemac53 @natebosch @bkonyi is this backwards compatibility required? DevTools is pinned to the Dart SDK version. Is package:test also? Or is it possible for a user to be on a new Dart SDK with these DevTools changes, but an old version of package:test that does not have the generated constant we are trying to eval?

Copy link
Contributor

@natebosch natebosch Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's possible for a new SDK to be used with an old package:test. We should not rely on these always being present. We also should be resilient if the strings have the content 'null' in case these code paths are used in places where Isolate.packageConfig returns null.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. The current code covers that case, so we should be good to go. In the case you described where Isolate.packageConfig returns 'null`, would we still want to fallback to the case where we are checking the test import prefix? From the DevTools perspective, I don't mind more fallbacks that if it allows us to provide a more robustness experience to the user.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would we still want to fallback to the case where we are checking the test import prefix? From the DevTools perspective

I don't know of any concrete reasons for getting a null, so it's hard to say whether the normal fallback behavior would help or not. In any case I don't think it hurts to try a fallback approach for this situation. I mentioned it because the API is nullable and we are explicitly not doing anything special with it there. If it would be useful to omit the variable entirely over writing the string 'null' we could do that too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave the variable even if it writes null. If the variable is missing, the evaluation would throw, so I think it is better to just rely on our string comparison to cover this case.

}
}
_log.fine(
'[connectedAppPackageRoot] package root for test target: '
'$packageRootUriString',
);
return packageRootUriString == null
? null
: Uri.parse(packageRootUriString);
}

Future<String?> _lookupPackageConfigByEval(Library rootLibrary) async {
final eval = EvalOnDartLibrary(
rootLibrary.uri!,
this.service! as VmService,
serviceManager: this,
// Swallow exceptions since this evaluation may be called on an older
// version of package:test where we do not expect the evaluation to
// succeed.
logExceptions: false,
);
final evalDisposable = Disposable();
try {
final packageConfig = (await eval.evalInstance(
'packageConfigLocationn',
isAlive: evalDisposable,
))
.valueAsString;

if (packageConfig?.endsWith(packageConfigIdentifier) ?? false) {
_log.fine(
'[connectedAppPackageRoot] detected test library from root library '
'imports: $testTargetFileUriString',
'[connectedAppPackageRoot] detected test package config from root '
'library eval: $packageConfig.',
);
packageRootUriString = await packageRootFromFileUriString(
testTargetFileUriString,
dtd: dtdManager.connection.value,
);
_log.fine(
'[connectedAppPackageRoot] package root for test target: '
'$packageRootUriString',
return packageConfig!.substring(
jakemac53 marked this conversation as resolved.
Show resolved Hide resolved
0,
// Minus 1 to remove the trailing slash.
packageConfig.length - packageConfigIdentifier.length - 1,
);
}
} catch (_) {
// Fail gracefully if the evaluation fails.
} finally {
evalDisposable.dispose();
}
return null;
}

return packageRootUriString == null
? null
: Uri.parse(packageRootUriString);
Future<String?> _lookupTestLibraryByPrefix(
Library rootLibrary,
DTDManager dtdManager,
) async {
final testTargetFileUriString =
(rootLibrary.dependencies ?? <LibraryDependency>[])
.firstWhereOrNull((dep) => dep.prefix == 'test')
?.target
?.uri;
if (testTargetFileUriString != null) {
_log.fine(
'[connectedAppPackageRoot] detected test library from root library '
'imports: $testTargetFileUriString',
);
return await packageRootFromFileUriString(
testTargetFileUriString,
dtd: dtdManager.connection.value,
);
}
return null;
}

Future<Library?> _mainIsolateRootLibrary() async {
Expand Down
15 changes: 15 additions & 0 deletions packages/devtools_shared/lib/src/common.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,20 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:path/path.dart' as path;

/// Describes an instance of the Dart Tooling Daemon.
typedef DTDConnectionInfo = ({String? uri, String? secret});

/// The name of the Directory where a Dart application's package config file is
/// stored.
const dartToolDirectoryName = '.dart_tool';

/// The name of the package config file for a Dart application.
const packageConfigFileName = 'package_config.json';

/// The path identifier for the package config URI for a Dart application.
///
/// The package config file lives at '.dart_tool/package_config.json'.
final packageConfigIdentifier =
Copy link
Contributor

@jakemac53 jakemac53 Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the name/location of this file is not guaranteed to be consistent. This is just the default path, and it could be anything in practice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider also the pub workspaces feature, I am not sure if this will work well with that or not.

But, in the future it will be common externally to have a .dart_tool/package_config.json file which lives in a directory above the actual package root (not sure if that will matter for this or not).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is an example where the package config will not be named package_config.json? It is my understanding that much of our tooling system, pub, etc. rely on this assumption.

Even with a pub workspace, shouldn't the URI to the package config still end with .dart_tool/package_config.json? @sigurdm

Generally, I filed #7944 to track ensuring DevTools extensions work well with pub workspaces.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any user or tool for any reason can pass any path they want for the --packages argument. Yes, most of the time this doesn't happen, but it is supported.

Specifically, build systems might create a file by any name, and pass the path to it that way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with a pub workspace, shouldn't the URI to the package config still end with .dart_tool/package_config.json? @sigurdm

Yes with workspaces it will - but also what @jakemac53 said!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed these shared constants and filed #7944 to track moving away from the assumptions around .dart_tool/package_config.json

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also worth noting that in the workspace case, the package config will contain all packages depended on by any package in the workspace, not just the current package.

I don't know exactly all the details here, but if DevTools is looking in that file for certain dependencies, assuming they are dependencies of the current package, then that assumption won't hold up (it will contain dependencies which are not transitive deps of the current package).

I don't know enough to suggest a fix for that though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also worth noting that in the workspace case, the package config will contain all packages depended on by any package in the workspace, not just the current package.

Noted. This will take some more thought and design as pub workspaces roll out.

path.join(dartToolDirectoryName, packageConfigFileName);
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,8 @@ class ExtensionsManager {
// may be an incorrect assumption for monorepos.
final packageConfigPath = path.posix.join(
rootFileUriString,
'.dart_tool',
'package_config.json',
dartToolDirectoryName,
packageConfigFileName,
);
extensions = await findExtensions(
'devtools',
Expand Down
4 changes: 3 additions & 1 deletion packages/devtools_shared/lib/src/utils/file_utils.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import 'package:dtd/dtd.dart';
import 'package:meta/meta.dart';

import '../common.dart';

const _fileUriPrefix = 'file://';

/// Attempts to detect the package root of [fileUriString], which is expected to
Expand Down Expand Up @@ -52,7 +54,7 @@ Future<String> packageRootFromFileUriString(
try {
final directoryContents = await dtd.listDirectoryContents(uri);
final containsDartToolDirectory = (directoryContents.uris ?? const [])
.any((uri) => uri.path.endsWith('.dart_tool/'));
.any((uri) => uri.path.endsWith('$dartToolDirectoryName/'));
if (containsDartToolDirectory) {
final uriAsString = uri.toString();
return _assertUriFormatAndReturn(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import 'dart:io';

import 'package:devtools_shared/devtools_shared.dart';
import 'package:path/path.dart' as p;
import 'package:test/test.dart';

Expand Down Expand Up @@ -109,10 +110,7 @@ class ExtensionTestManager {
);
}

final packageConfigFile = File(
p.join(packageRoot.path, '.dart_tool', 'package_config.json'),
);
expect(packageConfigFile.existsSync(), isTrue);
expect(File(packageConfigIdentifier).existsSync(), isTrue);
}
}

Expand Down
Loading