Stardoc crashes when interacting with `apple_common` global #76

achew22 · 2020-10-22T07:33:54Z

When Stardoc encounters the following bit of code in rules_go,

_PLATFORMS = {
    "armv7-apple-ios": (apple_common.platform.ios_device, apple_common.platform_type.ios),
    "armv7-apple-tvos": (apple_common.platform.tvos_device, apple_common.platform_type.tvos),
    "armv7k-apple-watchos": (apple_common.platform.watchos_device, apple_common.platform_type.watchos),
    "arm64-apple-ios": (apple_common.platform.ios_device, apple_common.platform_type.ios),
    "arm64-apple-tvos": (apple_common.platform.tvos_device, apple_common.platform_type.tvos),
    "i386-apple-ios": (apple_common.platform.ios_simulator, apple_common.platform_type.ios),
    "i386-apple-tvos": (apple_common.platform.tvos_simulator, apple_common.platform_type.tvos),
    "i386-apple-watchos": (apple_common.platform.watchos_simulator, apple_common.platform_type.watchos),
    "x86_64-apple-ios": (apple_common.platform.ios_simulator, apple_common.platform_type.ios),
    "x86_64-apple-tvos": (apple_common.platform.ios_simulator, apple_common.platform_type.tvos),
    "x86_64-apple-watchos": (apple_common.platform.watchos_simulator, apple_common.platform_type.watchos),
}

it throws the following error:

SUBCOMMAND: # //go:def-docs [action 'Generating proto for Starlark doc for def-docs', configuration: faff19e6fd939f490ac11578d94024c6b7a032836cde039fd5edd28b838194e8, execution platform: @local_config_platform//:host]
(cd <snip> && \
  exec env - \
  bazel-out/host/bin/external/io_bazel_stardoc/stardoc/stardoc '--input=//go:def.bzl' '--workspace_name=io_bazel_rules_go' '--dep_roots=.' '--dep_roots=external/io_bazel_rules_go' '--output=bazel-out/k8-fastbuild/bin/go/def-docs.raw')
ERROR: <snip>/rules_go/go/BUILD.bazel:61:8: Generating proto for Starlark doc for def-docs failed (Exit 1) stardoc failed: error executing command bazel-out/host/bin/external/io_bazel_stardoc/stardoc/stardoc '--input=//go:def.bzl' '--workspace_name=io_bazel_rules_go' '--dep_roots=.' '--dep_roots=external/io_bazel_rules_go' ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
com.google.devtools.build.skydoc.SkydocMain$StarlarkEvaluationException: Starlark evaluation error
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:423)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:397)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:397)
        at com.google.devtools.build.skydoc.SkydocMain.eval(SkydocMain.java:255)
        at com.google.devtools.build.skydoc.SkydocMain.main(SkydocMain.java:158)
Caused by: Traceback (most recent call last):
        File "./go/platform/apple.bzl", line 16, column 46, in <toplevel>
                "armv7-apple-ios": (apple_common.platform.ios_device, apple_common.platform_type.ios),
Error:
        at net.starlark.java.eval.Starlark.errorf(Starlark.java:555)
        at net.starlark.java.eval.Starlark.getattr(Starlark.java:606)
        at net.starlark.java.eval.Eval.evalDot(Eval.java:530)
        at net.starlark.java.eval.Eval.eval(Eval.java:446)
        at net.starlark.java.eval.Eval.evalList(Eval.java:704)
        at net.starlark.java.eval.Eval.eval(Eval.java:466)
        at net.starlark.java.eval.Eval.evalDict(Eval.java:509)
        at net.starlark.java.eval.Eval.eval(Eval.java:444)
        at net.starlark.java.eval.Eval.execAssignment(Eval.java:108)
        at net.starlark.java.eval.Eval.exec(Eval.java:260)
        at net.starlark.java.eval.Eval.execStatements(Eval.java:80)
        at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:64)
        at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:155)
        at net.starlark.java.eval.Starlark.fastcall(Starlark.java:509)
        at net.starlark.java.eval.Starlark.execFileProgram(Starlark.java:779)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:420)
        ... 4 more

To view, clone, and test this change, please see my CL to be: https://github.com/achew22/rules_go/pull/new/stardoc
and bazel build //go:def-docs

The text was updated successfully, but these errors were encountered:

alandonovan · 2020-10-22T13:29:13Z

There are two bugs here. The first is that the error message is terrible: instead of showing you the names of various Java functions inside Skydoc, it should report the Starlark stack and (crucially) the error message. The second bug is that the failure, of the .ios_device operator, is spurious: it is caused by a mismatch between the fake environment used by skydoc and the reality of bazel. Compare:

https://github.com/bazelbuild/bazel/blob/750c938ad180d69cce95af704738360677269d85/src/main/java/com/google/devtools/build/skydoc/fakebuildapi/apple/FakeAppleCommon.java#L62
https://github.com/bazelbuild/bazel/blob/450c7ad65bfcffebf753159ee2b617e557ffb4fd/src/main/java/com/google/devtools/build/lib/rules/apple/ApplePlatform.java#L194

The solution to the first bug is to change the exception handling in skydoc. The solution to the second is to reimplement skydoc as a client of bazel query, which would load .bzl files and print information about them so that skydoc doesn't need all this fakery. As a short-term hack, one could modify the fake environment, but this is only postponing the inevitable.

c-parsons · 2020-10-22T15:03:13Z

Is the plan to reimplement stardoc as a client of bazel query something that's being actively worked on? Or, is it on a roadmap? If not, we may want to modify the fake environment regardless. It's a quick change.

achew22 · 2020-10-22T18:56:48Z

@c-parsons I'm not sure exactly how one would go about doing that. I presume this is the fake that would need to be modified. More specifically, I'm not sure why this line is insufficient?

achew22 · 2020-10-27T06:18:41Z

Friendly ping

achew22 · 2020-10-28T15:00:06Z

Even friendlier ping

alandonovan · 2020-10-28T16:05:11Z

Is the plan to reimplement stardoc as a client of bazel query something that's being actively worked on?

It's on the roadmap, but it's blocked behind several other changes, so it won't happen for a while.

Modifying the fake environment is the expedient move.

c-parsons · 2020-10-28T17:37:31Z

@c-parsons I'm not sure exactly how one would go about doing that. I presume this is the fake that would need to be modified.

Yes, particularly this demonstrates how one should return a non-empty struct for getPlatformStruct.

More specifically, I'm not sure why this line is insufficient?

Lets consider the evaluation of (apple_common.platform.ios_device, apple_common.platform_type.ios).
The latter is fine, because, in the fake environment, apple_common.platform_type does have a field "ios". However, apple_common.platform is equivalent to struct(), thus an error is thrown.

See bazelbuild/stardoc#76 for more information.

achew22 · 2020-10-29T04:36:44Z

@c-parsons is this what you were expecting bazelbuild/bazel#12383?

achew22 · 2020-11-08T05:36:12Z

Friendly ping @c-parsons

See bazelbuild/stardoc#76 for more information. Closes #12383. PiperOrigin-RevId: 341621899

c-parsons · 2021-01-06T16:23:58Z

Related issue: #83

Apologies that response has been so slow on this issue.
Maintenance of this repository is changing hands, and thus some issues have fallen through the cracks.

…lable * When available (i.e. in Bazel 7, or in current development Bazel at HEAD), try use the `starlark_doc_extract` native rule for doc extraction instead of the legacy pre-built extraction jar. This behavior can be disabled by passing `use_starlark_doc_extract = False` to the `stardoc` macro. * Add templates and markdown rendering functionality for repository rule and module extension info protos (which are output by `starlark_doc_extract`). * Temporary wart: for module extensions, by default we would want the summary blurb to look something like ``` my_ext = use_extension("@my_local_repo//some:file.bzl", "my_ext") my_ext.tag(foo, bar) ``` but alas, we don't have a good way to get the name of the local repo from Starlark when bzlmod is enabled. * ... and of course, update tests. Which means in some cases, we have to fork the golden files into current (i.e. `starlark_doc_extract`-enabled) and legacy flavors. Fixes #69 Fixes #76 Fixes #81 Fixes #123

achew22 mentioned this issue Oct 22, 2020

Add bzl_library targets to Gazelle bazel-contrib/bazel-gazelle#760

Merged

achew22 mentioned this issue Oct 26, 2020

Export .bzl files in @bazel_tools//tools/... bazelbuild/bazel#12258

Closed

achew22 added a commit to achew22/bazel that referenced this issue Oct 29, 2020

Improve FakeApple to include additional fake data

f5fe30d

See bazelbuild/stardoc#76 for more information.

achew22 added a commit to achew22/bazel that referenced this issue Oct 29, 2020

Improve FakeApple to include additional fake data

fec5ac3

See bazelbuild/stardoc#76 for more information.

achew22 mentioned this issue Oct 29, 2020

Improve FakeApple to include additional fake data bazelbuild/bazel#12383

Closed

bazel-io pushed a commit to bazelbuild/bazel that referenced this issue Nov 10, 2020

Improve FakeApple to include additional fake data

20d7a6d

See bazelbuild/stardoc#76 for more information. Closes #12383. PiperOrigin-RevId: 341621899

brandjon self-assigned this Jan 12, 2021

brandjon added P3 type: bug labels Jan 12, 2021

tetromino mentioned this issue Jul 19, 2023

Use native starlark_doc_extract rule for doc extraction if it is available #166

Merged

tetromino closed this as completed in #166 Jul 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stardoc crashes when interacting with `apple_common` global #76

Stardoc crashes when interacting with `apple_common` global #76

achew22 commented Oct 22, 2020

alandonovan commented Oct 22, 2020

c-parsons commented Oct 22, 2020 •

edited

Loading

achew22 commented Oct 22, 2020

achew22 commented Oct 27, 2020

achew22 commented Oct 28, 2020

alandonovan commented Oct 28, 2020

c-parsons commented Oct 28, 2020

achew22 commented Oct 29, 2020

achew22 commented Nov 8, 2020

c-parsons commented Jan 6, 2021

Stardoc crashes when interacting with apple_common global #76

Stardoc crashes when interacting with apple_common global #76

Comments

achew22 commented Oct 22, 2020

alandonovan commented Oct 22, 2020

c-parsons commented Oct 22, 2020 • edited Loading

achew22 commented Oct 22, 2020

achew22 commented Oct 27, 2020

achew22 commented Oct 28, 2020

alandonovan commented Oct 28, 2020

c-parsons commented Oct 28, 2020

achew22 commented Oct 29, 2020

achew22 commented Nov 8, 2020

c-parsons commented Jan 6, 2021

Stardoc crashes when interacting with `apple_common` global #76

Stardoc crashes when interacting with `apple_common` global #76

c-parsons commented Oct 22, 2020 •

edited

Loading