Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for Dart/Flutter SDK at custom sdkPath #288

Merged
merged 15 commits into from
May 9, 2022

Conversation

acoutts
Copy link
Contributor

@acoutts acoutts commented Apr 29, 2022

Description

This PR adds support for specifying the path to a Dart/Flutter SDK, instead of relying on what's in the path.
This makes melos compatible with flutter version manager (fvm).

The SDK path can be specified though the following mechanisms, in order of highest to lowest precedence:

  1. --sdk-path command line option
  2. MELOS_SDK_PATH environment variable
  3. sdkPath in melos.yaml

Closes #81

Type of Change

  • feat -- New feature (non-breaking change which adds functionality)
  • 🛠️ fix -- Bug fix (non-breaking change which fixes an issue)
  • ! -- Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 refactor -- Code refactor
  • ci -- Build configuration change
  • 📝 docs -- Documentation
  • 🗑️ chore -- Chore

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Something seems off - it seems to be doing dart pub get instead of flutter pub get in my project now.

melos bootstrap
   └> /Users/andrew.coutts/Projects/app

Running "/Users/andrew.coutts/Projects/app/.fvm/flutter_sdk/bin/flutter pub get" in workspace packages...
  ✓ app_monorepo
    └> .
  ✓ ui
    └> packages/scope/example/modules/ui
[_root]: Resolving dependencies...
[_root]:   _fe_analyzer_shared 38.0.0 (39.0.0 available)
[_root]:   analyzer 3.4.1 (4.0.0 available)
[_root]:   args 2.1.0 (2.3.0 available)
[_root]:   async 2.8.2 (2.9.0 available)
[_root]:   built_collection 5.0.0 (5.1.1 available)
[_root]:   built_value 8.0.6 (8.2.3 available)
[_root]:   code_builder 4.0.0 (4.1.0 available)
[_root]:   convert 3.0.0 (3.0.1 available)
[_root]:   crypto 3.0.1 (3.0.2 available)
[_root]:   ffi 1.0.0 (1.1.2 available)
[_root]:   file 6.1.0 (6.1.2 available)
[_root]:   glob 2.0.1 (2.0.2 available)
[_root]:   json_annotation 4.4.0 (4.5.0 available)
[_root]:   logging 1.0.1 (1.0.2 available)
[_root]:   material_color_utilities 0.1.4 (0.1.5 available)
[_root]:   package_config 2.0.0 (2.0.2 available)
[_root]:   path_provider_linux 2.0.0 (2.1.5 available)
[_root]:   path_provider_platform_interface 2.0.1 (2.0.3 available)
[_root]:   path_provider_windows 2.0.1 (2.0.5 available)
[_root]:   pedantic 1.11.0 (1.11.1 available)
[_root]:   platform 3.0.0 (3.1.0 available)
[_root]:   plugin_platform_interface 2.0.0 (2.1.2 available)
[_root]:   process 4.2.1 (4.2.4 available)
[_root]:   pub_semver 2.0.0 (2.1.1 available)
[_root]:   shared_preferences_macos 2.0.0 (2.0.3 available)
[_root]:   shared_preferences_web 2.0.0 (2.0.3 available)
[_root]:   source_span 1.8.2 (1.9.0 available)
[_root]:   watcher 1.0.0 (1.0.1 available)
[_root]:   win32 2.0.5 (2.5.2 available)
[_root]:   xdg_directories 0.2.0 (0.2.0+1 available)
[_root]: Downloading shared_preferences 2.0.13...
[_root]: Downloading mobx 2.0.7...
[_root]: Downloading source_gen 1.2.2...
[_root]: Downloading dart_style 2.2.3...
[_root]: Downloading code_builder 4.0.0...
[_root]: Downloading build 2.3.0...
[_root]: Downloading analyzer 3.4.1...
[_root]: Downloading shared_preferences_windows 2.1.0...
[_root]: Downloading shared_preferences_web 2.0.0...
[_root]: Downloading shared_preferences_macos 2.0.0...
[_root]: Downloading shared_preferences_linux 2.1.0...
[_root]: Downloading shared_preferences_ios 2.1.0...
[_root]: Downloading shared_preferences_android 2.0.11...
[_root]: Downloading glob 2.0.1...
[_root]: Downloading pub_semver 2.0.0...
[_root]: Downloading args 2.1.0...
[_root]: Downloading built_value 8.0.6...
[_root]: Downloading built_collection 5.0.0...
[_root]: Downloading logging 1.0.1...
[_root]: Downloading convert 3.0.0...
[_root]: Downloading watcher 1.0.0...
[_root]: Downloading package_config 2.0.0...
[_root]: Downloading _fe_analyzer_shared 38.0.0...
[_root]: Downloading path_provider_windows 2.0.1...
[_root]: Downloading path_provider_platform_interface 2.0.1...
[_root]: Downloading file 6.1.0...
[_root]: Downloading path_provider_linux 2.0.0...
[_root]: Downloading pedantic 1.11.0...
[_root]: Downloading win32 2.0.5...
[_root]: Downloading ffi 1.0.0...
[_root]: Downloading plugin_platform_interface 2.0.0...
[_root]: Downloading platform 3.0.0...
[_root]: Downloading process 4.2.1...
[_root]: Got dependencies!
[_app]: Resolving dependencies...
[_app]:   _fe_analyzer_shared 38.0.0 (39.0.0 available)
[_app]:   analyzer 3.4.1 (4.0.0 available)
[_app]:   args 2.1.0 (2.3.0 available)
[_app]:   async 2.8.2 (2.9.0 available)
[_app]:   built_collection 5.0.0 (5.1.1 available)
[_app]:   built_value 8.0.6 (8.2.3 available)
[_app]:   code_builder 4.0.0 (4.1.0 available)
[_app]:   convert 3.0.0 (3.0.1 available)
[_app]:   crypto 3.0.1 (3.0.2 available)
[_app]:   ffi 1.0.0 (1.1.2 available)
[_app]:   file 6.1.0 (6.1.2 available)
[_app]:   glob 2.0.1 (2.0.2 available)
[_app]:   json_annotation 4.4.0 (4.5.0 available)
[_app]:   logging 1.0.1 (1.0.2 available)
[_app]:   material_color_utilities 0.1.4 (0.1.5 available)
[_app]:   package_config 2.0.0 (2.0.2 available)
[_app]:   path_provider_linux 2.0.0 (2.1.5 available)
[_app]:   path_provider_platform_interface 2.0.1 (2.0.3 available)
[_app]:   path_provider_windows 2.0.1 (2.0.5 available)
[_app]:   pedantic 1.11.0 (1.11.1 available)
[_app]:   platform 3.0.0 (3.1.0 available)
[_app]:   plugin_platform_interface 2.0.0 (2.1.2 available)
[_app]:   process 4.2.1 (4.2.4 available)
[_app]:   pub_semver 2.0.0 (2.1.1 available)
[_app]:   shared_preferences_macos 2.0.0 (2.0.3 available)
[_app]:   shared_preferences_web 2.0.0 (2.0.3 available)
[_app]:   source_span 1.8.2 (1.9.0 available)
[_app]:   watcher 1.0.0 (1.0.1 available)
[_app]:   win32 2.0.5 (2.5.2 available)
[_app]:   xdg_directories 0.2.0 (0.2.0+1 available)
[_app]: Downloading shared_preferences 2.0.13...
[_app]: Downloading mobx 2.0.7...
[_app]: Downloading source_gen 1.2.2...
[_app]: Downloading dart_style 2.2.3...
[_app]: Downloading code_builder 4.0.0...
[_app]: Downloading build 2.3.0...
[_app]: Downloading analyzer 3.4.1...
[_app]: Downloading shared_preferences_windows 2.1.0...
[_app]: Downloading shared_preferences_macos 2.0.0...
[_app]: Downloading shared_preferences_ios 2.1.0...
[_app]: Downloading shared_preferences_android 2.0.11...
[_app]: Downloading glob 2.0.1...
[_app]: Downloading pub_semver 2.0.0...
[_app]: Downloading args 2.1.0...
[_app]: Downloading built_value 8.0.6...
[_app]: Downloading built_collection 5.0.0...
[_app]: Downloading logging 1.0.1...
[_app]: Downloading convert 3.0.0...
[_app]: Downloading watcher 1.0.0...
[_app]: Downloading package_config 2.0.0...
[_app]: Downloading _fe_analyzer_shared 38.0.0...
[_app]: Downloading path_provider_windows 2.0.1...
[_app]: Downloading path_provider_platform_interface 2.0.1...
[_app]: Downloading file 6.1.0...
[_app]: Downloading path_provider_linux 2.0.0...
[_app]: Downloading pedantic 1.11.0...
[_app]: Downloading win32 2.0.5...
[_app]: Downloading ffi 1.0.0...
[_app]: Downloading plugin_platform_interface 2.0.0...
[_app]: Downloading platform 3.0.0...
[_app]: Downloading process 4.2.1...
[_app]: Got dependencies!
[fund]: Resolving dependencies...
[fund]:   _fe_analyzer_shared 31.0.0 (39.0.0 available)
[fund]:   analyzer 2.8.0 (4.0.0 available)
[fund]:   archive 3.2.0 (3.3.0 available)
[fund]:   async 2.8.2 (2.9.0 available)
[fund]:   build 2.2.1 (2.3.0 available)
[fund]:   build_daemon 3.0.1 (3.1.0 available)
[fund]:   build_resolvers 2.0.6 (2.0.8 available)
[fund]:   build_runner 2.1.7 (2.1.10 available)
[fund]:   built_value 8.1.4 (8.2.3 available)
[fund]:   crypto 3.0.1 (3.0.2 available)
[fund]:   dart_style 2.2.1 (2.2.3 available)
[fund]:   decimal 2.1.0 (2.2.0 available)
[fund]:   injectable_generator 1.5.2 (1.5.3 available)
[fund]:   json_annotation 4.4.0 (4.5.0 available)
[fund]:   lottie 1.2.2 (1.3.0 available)
[fund]:   material_color_utilities 0.1.4 (0.1.5 available)
[fund]:   mobx 2.0.6+1 (2.0.7 available)
[fund]:   petitparser 4.4.0 (5.0.0 available)
[fund]:   pub_semver 2.1.0 (2.1.1 available)
[fund]:   rational 2.1.0 (2.2.0 available)
[fund]:   shared_preferences_ios 2.0.10 (2.1.0 available)
[fund]:   shelf 1.2.0 (1.3.0 available)
[fund]:   source_gen 1.2.1 (1.2.2 available)
[fund]:   source_span 1.8.2 (1.9.0 available)
[fund]:   url_launcher 6.0.20 (6.1.0 available)
[fund]:   url_launcher_android 6.0.15 (6.0.16 available)
[fund]:   url_launcher_web 2.0.6 (2.0.9 available)
[fund]:   web_socket_channel 2.1.0 (2.2.0 available)
[fund]:   win32 2.4.0 (2.5.2 available)
[fund]:   xml 5.3.1 (6.0.1 available)
[fund]: Downloading injectable_generator 1.5.2...
[fund]: Downloading build_runner 2.1.7...
[fund]: Downloading url_launcher 6.0.20...
[fund]: Downloading intl_generator 0.2.1...
[fund]: Downloading injectable 1.5.3...
[fund]: Downloading get_it 7.2.0...
[fund]: Downloading flutter_svg 1.0.3...
[fund]: Downloading system_info 1.0.1...
[fund]: Downloading sprung 3.0.0...
[fund]: Downloading modal_bottom_sheet 2.0.1...
[fund]: Downloading lottie 1.2.2...
[fund]: Downloading device_info 2.0.3...
[fund]: Downloading decimal 2.1.0...
[fund]: Downloading analyzer 2.8.0...
[fund]: Downloading shelf 1.2.0...
[fund]: Downloading build_resolvers 2.0.6...
[fund]: Downloading build_daemon 3.0.1...
[fund]: Downloading mad_pay_ios 2.2.7...
[fund]: Downloading mad_pay_android 2.2.9...
[fund]: Downloading mad_pay_platform_interface 2.2.4...
[fund]: Downloading grpc 3.0.2...
[fund]: Downloading protobuf 2.0.1...
[fund]: Downloading url_launcher_windows 3.0.0...
[fund]: Downloading url_launcher_web 2.0.6...
[fund]: Downloading url_launcher_platform_interface 2.0.5...
[fund]: Downloading url_launcher_macos 3.0.0...
[fund]: Downloading url_launcher_linux 3.0.0...
[fund]: Downloading url_launcher_ios 6.0.15...
[fund]: Downloading url_launcher_android 6.0.15...
[fund]: Downloading petitparser 4.4.0...
[fund]: Downloading xml 5.3.1...
[fund]: Downloading path_drawing 1.0.0...
[fund]: Downloading shared_preferences_ios 2.0.10...
[fund]: Downloading file_utils 1.0.1...
[fund]: Downloading archive 3.2.0...
[fund]: Downloading device_info_platform_interface 2.0.1...
[fund]: Downloading rational 2.1.0...
[fund]: Downloading _fe_analyzer_shared 31.0.0...
[fund]: Downloading http2 2.0.0...
[fund]: Downloading googleapis_auth 1.3.0...
[fund]: Downloading path_parsing 1.0.0...
[fund]: Downloading globbing 1.0.0...
[fund]: Downloading win32 2.4.0...
[fund]: Got dependencies!
[componentlib]: Resolving dependencies...
[componentlib]:   _fe_analyzer_shared 30.0.0 (39.0.0 available)
[componentlib]:   analyzer 2.7.0 (4.0.0 available)
[componentlib]:   archive 3.1.6 (3.3.0 available)
[componentlib]:   async 2.8.2 (2.9.0 available)
[componentlib]:   build 2.2.1 (2.3.0 available)
[componentlib]:   build_daemon 3.0.1 (3.1.0 available)
[componentlib]:   build_resolvers 2.0.6 (2.0.8 available)
[componentlib]:   build_runner 2.1.8 (2.1.10 available)
[componentlib]:   built_value 8.1.4 (8.2.3 available)
[componentlib]:   crypto 3.0.1 (3.0.2 available)
[componentlib]:   dart_style 2.2.0 (2.2.3 available)
[componentlib]:   decimal 2.1.0 (2.2.0 available)
[componentlib]:   device_info 2.0.3 (discontinued)
[componentlib]:   injectable_generator 1.5.2 (1.5.3 available)
[componentlib]:   json_annotation 4.4.0 (4.5.0 available)
[componentlib]:   lottie 1.2.2 (1.3.0 available)
[componentlib]:   material_color_utilities 0.1.4 (0.1.5 available)
[componentlib]:   mobx 2.0.6+1 (2.0.7 available)
[componentlib]:   petitparser 4.4.0 (5.0.0 available)
[componentlib]:   pub_semver 2.1.0 (2.1.1 available)
[componentlib]:   rational 2.1.0 (2.2.0 available)
[componentlib]:   shared_preferences_ios 2.0.10 (2.1.0 available)
[componentlib]:   source_gen 1.2.1 (1.2.2 available)
[componentlib]:   source_span 1.8.2 (1.9.0 available)
[componentlib]:   system_info 1.0.1 (discontinued)
[componentlib]:   web_socket_channel 2.1.0 (2.2.0 available)
[componentlib]:   win32 2.4.0 (2.5.2 available)
[componentlib]:   xml 5.3.1 (6.0.1 available)
[componentlib]: Downloading injectable_generator 1.5.2...
[componentlib]: Downloading build_runner 2.1.8...
[componentlib]: Downloading system_info 1.0.1...
[componentlib]: Downloading sprung 3.0.0...
[componentlib]: Downloading modal_bottom_sheet 2.0.1...
[componentlib]: Downloading lottie 1.2.2...
[componentlib]: Downloading intl_generator 0.2.1...
[componentlib]: Downloading get_it 7.2.0...
[componentlib]: Downloading flutter_svg 1.0.3...
[componentlib]: Downloading device_info 2.0.3...
[componentlib]: Downloading decimal 2.1.0...
[componentlib]: Downloading dart_style 2.2.0...
[componentlib]: Downloading analyzer 2.7.0...
[componentlib]: Downloading build_resolvers 2.0.6...
[componentlib]: Downloading build_daemon 3.0.1...
[componentlib]: Downloading mad_pay_ios 2.2.7...
[componentlib]: Downloading mad_pay_android 2.2.9...
[componentlib]: Downloading mad_pay_platform_interface 2.2.4...
[componentlib]: Downloading grpc 3.0.2...
[componentlib]: Downloading protobuf 2.0.1...
[componentlib]: Downloading file_utils 1.0.1...
[componentlib]: Downloading archive 3.1.6...
[componentlib]: Downloading petitparser 4.4.0...
[componentlib]: Downloading xml 5.3.1...
[componentlib]: Downloading path_drawing 1.0.0...
[componentlib]: Downloading device_info_platform_interface 2.0.1...
[componentlib]: Downloading rational 2.1.0...
[componentlib]: Downloading shared_preferences_ios 2.0.10...
[componentlib]: Downloading _fe_analyzer_shared 30.0.0...
[componentlib]: Downloading http2 2.0.0...
[componentlib]: Downloading googleapis_auth 1.3.0...
[componentlib]: Downloading globbing 1.0.0...
[componentlib]: Downloading path_parsing 1.0.0...
[componentlib]: Downloading win32 2.4.0...
[componentlib]: Got dependencies!
[app]: Running "flutter pub get" in app...                             
[app]: Warning: You are using these overridden dependencies:
[app]: ! cached_network_image 3.2.0 from git https://github.com/bottlepay/flutter_cached_network_image.git at 49530c in cached_network_image
[app]: ! common 0.0.1+1 from path ../common
[app]: ! componentlib 0.0.1+1 from path ../componentlib
[app]: ! confetti 0.6.0 from git https://github.com/bottlepay/flutter_confetti.git at 3ef399
[app]: ! fund 0.0.1+1 from path ../fund
[app]: ! google_geocoding 0.2.0 from git https://github.com/bottlepay/google_geocoding.git at 39c922
[app]: ! infinite_scroll_pagination 3.1.0 from git https://github.com/bottlepay/infinite_scroll_pagination.git at 89916e
[app]: ! nisaba 1.0.0 from path ../nisaba
[app]: ! oktoast 4.0.0 from git https://github.com/bottlepay/flutter_oktoast.git at 26c8ed
[app]: ! rate_my_app 1.1.2 from git https://github.com/bottlepay/RateMyApp.git at 37213d
[app]: ! scope 2.0.1 from path ../scope
[app]: ! sentry_flutter 6.5.0-beta.1 from git https://github.com/bottlepay/sentry-dart.git at 311468 in flutter
[app]: ! syncfusion_flutter_pdfviewer 20.1.47 from git https://github.com/bottlepay/flutter-widgets.git at 7bd233 in packages/syncfusion_flutter_pdfviewer
[app]: Running "flutter pub get" in app...                                36.4s
  ✓ _app
    └> packages/scope/example/modules/_app
  ✓ _root
    └> packages/scope/example/modules/_root
  ✓ app
    └> packages/app
  ✓ common
    └> packages/common
  ✓ componentlib
    └> packages/componentlib
  ✓ example
    └> packages/scope/example
  ✓ fund
    └> packages/fund
  ✓ nisaba
    └> packages/nisaba
  ✓ scope
    └> packages/scope

Linking workspace packages...
  > SUCCESS

Generating IntelliJ IDE files...
  > SUCCESS

 -> 11 plugins bootstrapped

@blaugold
Copy link
Collaborator

Yeah, and some tests are failing too. I'll take look what's going on here.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

I have one failed test which seems to be from flutter 2.13 where there are several breaking changes, in this case in HttpClient where they added new overrides.

00:00 +10 -1: loading test/package_test.dart [E]                                                                                                                                                                      
  Failed to load "test/package_test.dart":
  ../../../../.pub-cache/hosted/pub.dartlang.org/nock-1.2.0/lib/src/overrides.dart:20:7: Error: The non-abstract class 'MockClient' is missing implementations for these members:
   - HttpClient.connectionFactory=
   - HttpClient.keyLog=
  Try to either
   - provide an implementation,
   - inherit an implementation from a superclass or mixin,
   - mark the class as abstract, or
   - provide a 'noSuchMethod' implementation.
  
  class MockClient implements HttpClient {
        ^^^^^^^^^^
  ../../../flutter/bin/cache/dart-sdk/lib/_http/http.dart:1554:12: Context: 'HttpClient.connectionFactory=' is defined here.
    void set connectionFactory(
             ^^^^^^^^^^^^^^^^^
  ../../../flutter/bin/cache/dart-sdk/lib/_http/http.dart:1704:12: Context: 'HttpClient.keyLog=' is defined here.
    void set keyLog(Function(String line)? callback);
             ^^^^^^
00:01 +107 -2: test/commands/run_test.dart: script supports passing package filter options to "melos exec" scripts [E]                                                                                                
  ScriptException: The script test_script failed to execute
  package:melos/src/commands/run.dart 37:7  _RunMixin.run
  ===== asynchronous gap ===========================
  test/commands/run_test.dart 57:9          main.<fn>.<fn>
  
00:04 +142 ~4 -2: Some tests failed.                        

That package needs to release a new major version to support flutter 2.13 and onward.

Otherwise i fixed the other failing test with b006a68 and that resolved my issue.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Must be something that changed in master but even if i set runPubGetInParallel: false i get unexpected behavior that didn't happen before the force push, like having to run bootstrap twice for it to succeed now 🤔

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

This is the assumption breaking the flutter logic:

///  c) a lib/main.dart file exists in the package.

Not all flutter apps have to use main.dart and in fact it can be whatever the developer wants and can pass to the flutter command via the target parameter.
You could also have a module/package that depends on flutter but has no main.dart because it's intended to be a package and not ran directly.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

How strange, even though I set this in my melos.yaml:

runPubGetInParallel: false

I still had the race condition issues. The only way I could make it work again for me was to override it to return false from canRunPubGetConcurrently.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Doh! I did not have my setting yaml structured correctly. Turning off parallelism fixed mine, and ad3e80e fixes the flutter package identification issue.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Another edge case found: if the sdkPath is invalid in the melos yaml, but is valid via what's passed from the CLI command flag (--sdk-path), the workspace validate function causes the command to fail.
This would be common in CI where the melos workspace sdk path is valid on a dev machine but not in CI so the CI job specifies the right sdk path via the command flag.

acoutts added 2 commits April 29, 2022 10:53
…ation the global command sdk path and instead always tried to validate the workspace sdk first
@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Fixed that in 9635af4 including a test. I had to modify the workspace validate function to take into account the command sdk path as well because commandSdkPath (passed at command line) has precedence over the workspace sdk setting.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

There seems to be something wrong with how the global option for sdkPath was implemented, digging into it now.

@blaugold
Copy link
Collaborator

blaugold commented Apr 29, 2022

Not all flutter apps have to use main.dart and in fact it can be whatever the developer wants and can pass to the flutter command via the target parameter.
You could also have a module/package that depends on flutter but has no main.dart because it's intended to be a package and not ran directly.

The assumption that a Flutter app package always has a lib/main.dart is not completely sound, but it's true in most cases. If we don't check for that we get false positives for Flutter packages that are not plugins. You can always just add an empty lib/main.dart to your Flutter app. I'd rather not change that in this PR because it might break something in unexpected ways.

There seems to be something wrong with how the global option for sdkPath was implemented, digging into it now.

I changed the value that signals that the user wants to use the SDK on the PATH from the empty string to "auto". That's probably causing some confusion. Using the empty string to convey special meaning is an antipattern, IMHO. A meaningful value makes the command line invocation more readable. I don't think 9635af4 and b006a68 are necessary if you use melos <command> --sdk-path auto.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

The issue seems to be in how globalOptions works when we use the melos commands in the melos.yaml.

If I run this command:

flutter pub run melos --sdk-path '' exec dart analyze

Then _parseGlobalOptions runs once and sets sdk path accordingly:

[_parseGlobalOptions MelosCommand.sdkPath: ''

If I put the same dart analyze command above in melos.yaml as a command and run that, now the global options get computed twice - the second time it misses the sdk path setting.

flutter pub run melos analyze-ci --sdk-path ''        
[_parseGlobalOptions MelosCommand.sdkPath: ''
melos run analyze-ci
   └> flutter pub run melos exec -- "dart analyze"
       └> RUNNING

[_parseGlobalOptions MelosCommand.sdkPath: 'null'
melos.yaml: SDK path is not valid. Could not find dart tool at /Users/andrew.coutts/Projects/app/.fvm/flutter_sdk2/bin/dart

I ended up finding that i have to add the --sdk-path flag in my melos.yaml in the command itself there which calls melos exec, and that seems to work.
So i guess the global settings don't cascade down. Not a breaking issue now, just the nature of how dart arguments work.

@blaugold
Copy link
Collaborator

So i guess the global settings don't cascade down.

That's actually a good point. There is a good case for adding the MELOS_SDK_PATH environment variable. This might be useful in CI, where you can set it globally and then don't have to pass --sdk-path everywhere, and we could use it to cascade the SDK path in melos exec.

If an SDK path is specified, we should also add it to the path when running scripts, so that when dart or flutter is used in a script it uses the SDK.

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Got it now on the auto setting and confirmed it works. reverted 9635af4 and b006a68

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Good idea on the environment variable and i agree, sounds like a good way to make it cascade down.

@blaugold
Copy link
Collaborator

@acoutts I've added the MELOS_SDK_PATH env var and PATH is now updated if an SDK path has been provided. Tests are all passing now, too. Does the PR look good to go for you?

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

Looks awesome! Thanks a lot for your help.
Only comment I have is i did some testing and noticed the order of precedence is

  1. command sdk path (--sdk-path ..)
  2. MELOS_SDK_PATH
  3. melos.yaml sdkPath setting

Do you think it's alright that the command line will override the environment variable, or should it be the other way around? That way if people utilize the command line flag, they can set the flag in CI and they won't have to modify their commands.

To test this,

  • I set my melos.yaml sdkPath to an invalid path
  • I ran the bootstrap command without specifying sdkPath in the command, it fails (expected)
  • I ran the bootstrap command specifying sdkPath in the command, it succeeds (expected)
  • I set MELOS_SDK_PATH to an invalid path and set --sdk-path to a valid path and it succeeds (unexpected)

@blaugold
Copy link
Collaborator

My gut feeling is that command line arguments should generally have the highest precedence because they are the most granular way for specifying an option. It's what most tools do, so it's also what most devs expect. It might make sense in this case, but it's not obvious to me in what situation. Do you have a concrete use case?

@acoutts
Copy link
Contributor Author

acoutts commented Apr 29, 2022

No other concrete example. You’re right here as well, if someone wants to use the environment variable they could pass the command like argument and check the environment variable in a shell command.

@blaugold blaugold requested a review from Salakar April 30, 2022 09:18
@blaugold
Copy link
Collaborator

blaugold commented May 3, 2022

@Salakar This PR is ready to review, when you get a chance.

@Salakar
Copy link
Contributor

Salakar commented May 4, 2022

@Salakar This PR is ready to review, when you get a chance.

Sorry I'm a bit busy with Google I/O things at the moment 😫 I'm happy for this to go without my review unless you explicitly want me to review?

@blaugold
Copy link
Collaborator

blaugold commented May 4, 2022

👍 No problem. I'm fine with merging it without another review.

@blaugold blaugold merged commit 740050c into invertase:main May 9, 2022
@acoutts
Copy link
Contributor Author

acoutts commented May 12, 2022

@blaugold after bumping to melos 2.2.0, my commands that worked on my fork with --sdk-path auto do not work now. Is there a change in how the auto api works?

The commit that works for me:
https://github.com/bottlepay/melos.git
b006a68ceeb3a934380c0518471996d4fd28867f

To test:

  • Set sdkpath in melos.yaml to some invalid path
  • Run a melos command and pass --sdk-path auto, it should then use your path to find the flutter sdk and the command will work

If i try the exact same thing on 2.2.0/2.3.0, it instead tries to use the sdkPath setting in melos.yaml even though i passed "auto" to the command line.

@blaugold
Copy link
Collaborator

When I set an invalid sdkPath in melos.yaml, running melos bootstrap --sdk-path auto, for example, works as expected for me. What is the full command that is not working?

@acoutts
Copy link
Contributor Author

acoutts commented May 12, 2022

Aha you're right, so to further explain,

  • It works if i run a melos exec --sdk-path auto ... from the command line
  • It does not work if I define a command in melos.yaml and then try to run it.

melos.yaml command:

analyze-ci: flutter pub run melos exec --sdk-path auto -- "dart analyze"

Running this will fail:

fvm flutter pub run melos analyze-ci

But going into the terminal and pasting that command will work.

So the regression between my last commit on my fork and what got merged, is that it's broken for commands put in the melos.yaml file.

@blaugold
Copy link
Collaborator

blaugold commented May 12, 2022

You're right, there was a change in melos run from the commit you're referencing and the final merged commit. In the final commit, the sdk path is validated when melos starts up before any melos command is executed, which was not the case for melos run in the earlier commit. The idea is that Melos always has a valid sdk path and fails fast if that is not the case.

Do you have to work around a limitation where the MELOS_SDK_PATH environment variable cannot be used in CI?
Otherwise, you could just have one analyze script, which is also simpler:

analyze: melos exec -- "dart analyze"

Or run the script in CI like so:

melos run analyze --sdk-path auto

Edit: Actually, melos run analyze --sdk-path auto does not work, but MELOS_SDK_PATH=auto melos run analyze does. That is a bug we should fix.

@acoutts
Copy link
Contributor Author

acoutts commented May 13, 2022

I see - that's a good rationale there I agree.
I tried to pass an empty path to the environment variable but it seems to ignore it and still say it doesn't work.
If I try adding --sdk-path auto to the melos run command, it also still tries to use sdkPath in the melos yaml instead.

🤔

Hard coding the path via the environment variable does work, but now i wonder when does the auto setting work and when does it not?

@blaugold
Copy link
Collaborator

#296 fixes the issue with the --sdk-path command line option so that it works for all Melos commands.

@acoutts
Copy link
Contributor Author

acoutts commented May 17, 2022

works!! thank you so much for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support custom flutter sdk / dart sdk paths
3 participants