allow all command-line goals to be available to OptionsBootstrapper

[cosmicexplorer]

In #7205, we want to avoid using the daemon client depending on the command-line goals -- see this change in pants_runner.py from the branch linked in that diff. This doesn't currently work because the command-line goals aren't available to the OptionsBootstrapper. Specifically:

• OptionsBootstrapper.create() will call .parse_bootstrap_options()
• .parse_bootstrap_options() then calls Options.create()
• Options.create() will then invoke ArgSplitter#split_args().

The reason this doesn't get command-line goals for bootstrap options is this part in OptionsBootstrapper.create():

pants/src/python/pants/option/options_bootstrapper.py

Lines 127 to 130 in 434533a

	# Take just the bootstrap args, so we don't choke on other global-scope args on the cmd line.
	# Stop before '--' since args after that are pass-through and may have duplicate names to our
	# bootstrap options.
	bargs = tuple(filter(is_bootstrap_option, itertools.takewhile(lambda arg: arg != '--', args)))

We stop parsing the command line after finding all bootstrap options here (meaning we stop at any command-line goals), and bargs is what we then pass to the ArgSplitter. There's nothing wrong with this, but it means we don't know what the command-line goals are in the OptionsBootstrapper, which makes it difficult to implement #7205 unless we parse the arguments again, without stopping before the command-line goals.

One way to implement this would be to figure out some way to use ArgSplitter to split arguments only once, without knowing the scope infos in advance (this is a current limitation of ArgSplitter), then consume the result of that splitting (possibly some form of nested dict) in OptionsBootstrapper to extract bootstrap options, and then later use it again when extracting the ._full_options. This solution would likely fix #5282 as well.

[stuhood]

The challenge is that until you have all "known scope infos", you can't know (generically) which of the remaining tokens on the CLI represent goals.

Having said that, knowing "generically" probably isn't strictly necessary for #7205... some amount of special casing there would likely be fine.

[Eric-Arellano]

Closing as it seems we were able to land #7205 without this. Please reopen if we still have a concrete need for this mechanism.