Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ICE when calling syn::parse_macro_input! in proc macro applied to a function with a doc comment in a declarative macro #81007

Closed
DimiDimit opened this issue Jan 14, 2021 · 0 comments · Fixed by #81177
Closed

ICE when calling syn::parse_macro_input! in proc macro applied to a function with a doc comment in a declarative macro #81007

DimiDimit opened this issue Jan 14, 2021 · 0 comments · Fixed by #81177
Assignees
@Aaron1011
Labels
A-proc-macros Area: Procedural macros C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@DimiDimit
Copy link

Code

macro_rules! test_fixtures {
    ($($item:item)*) => {
        $(
            #[async_std::test] // This serves the role of calling syn::parse_macro_input!
            $item
        )*
    }
}

test_fixtures! {
    // Normal comments work fine.
    /// Doc comments cause a panic.
    async fn test() {
        eprintln!("Hello world!");
    }
}

Meta

rustc --version --verbose:
To get this output I compiled commit rustc on commit c7b0ddb, because the binaries don't show anything useful.

rustc 1.51.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.51.0-dev

Error output

thread 'rustc' panicked at 'internal error: entered unreachable code', compiler/rustc_expand/src/proc_macro_server.rs:193:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.51.0-dev running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C linker=/usr/bin/clang -C incremental -C link-arg=-fuse-ld=lld --crate-type lib

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: custom attribute panicked
  --> src/tests.rs:4:13
   |
4  |               #[async_std::test] // This serves the role of calling syn::parse_macro_input!
   |               ^^^^^^^^^^^^^^^^^^
...
10 | / test_fixtures! {
11 | |     // Normal comments work fine.
12 | |     /// Doc comments cause a panic.
13 | |     async fn test() {
14 | |         eprintln!("Hello world!");
15 | |     }
16 | | }
   | |_- in this macro invocation
   |
   = help: message: internal error: entered unreachable code
   = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
Backtrace 

thread 'rustc' panicked at 'internal error: entered unreachable code', compiler/rustc_expand/src/proc_macro_server.rs:193:20
stack backtrace:
   0: rust_begin_unwind
             at /tmp/rust/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /tmp/rust/library/core/src/panicking.rs:92:14
   2: core::panicking::panic
             at /tmp/rust/library/core/src/panicking.rs:50:5
   3: <proc_macro::bridge::TokenTree<rustc_expand::proc_macro_server::Group,rustc_expand::proc_macro_server::Punct,rustc_expand::proc_macro_server::Ident,rustc_expand::proc_macro_server::Literal> as rustc_expand::proc_macro_server::FromInternal<((rustc_ast::tokenstream::TokenTree,rustc_ast::tokenstream::Spacing),&rustc_session::parse::ParseSess,&mut alloc::vec::Vec<proc_macro::bridge::TokenTree<rustc_expand::proc_macro_server::Group,rustc_expand::proc_macro_server::Punct,rustc_expand::proc_macro_server::Ident,rustc_expand::proc_macro_server::Literal>>)>>::from_internal
             at /tmp/rust/compiler/rustc_expand/src/proc_macro_server.rs:193:20
   4: <rustc_expand::proc_macro_server::Rustc as proc_macro::bridge::server::TokenStreamIter>::next::{{closure}}
             at /tmp/rust/compiler/rustc_expand/src/proc_macro_server.rs:449:22
   5: core::option::Option<T>::or_else
             at /tmp/rust/library/core/src/option.rs:784:21
   6: <rustc_expand::proc_macro_server::Rustc as proc_macro::bridge::server::TokenStreamIter>::next
             at /tmp/rust/compiler/rustc_expand/src/proc_macro_server.rs:447:24
   7: <proc_macro::bridge::server::MarkedTypes<S> as proc_macro::bridge::server::TokenStreamIter>::next
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:72:34
   8: <proc_macro::bridge::server::Dispatcher<proc_macro::bridge::server::MarkedTypes<S>> as proc_macro::bridge::server::DispatcherTrait>::dispatch::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:106:33
   9: core::ops::function::FnOnce::call_once
             at /tmp/rust/library/core/src/ops/function.rs:227:5
  10: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /tmp/rust/library/std/src/panic.rs:322:9
  11: std::panicking::try::do_call
             at /tmp/rust/library/std/src/panicking.rs:379:40
  12: std::panicking::try
             at /tmp/rust/library/std/src/panicking.rs:343:19
  13: std::panic::catch_unwind
             at /tmp/rust/library/std/src/panic.rs:396:14
  14: <proc_macro::bridge::server::Dispatcher<proc_macro::bridge::server::MarkedTypes<S>> as proc_macro::bridge::server::DispatcherTrait>::dispatch
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:115:33
  15: <proc_macro::bridge::server::SameThread as proc_macro::bridge::server::ExecutionStrategy>::run_bridge_and_client::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:153:32
  16: <proc_macro::bridge::closure::Closure<A,R> as core::convert::From<&mut F>>::from::call
             at /tmp/rust/library/proc_macro/src/bridge/closure.rs:19:13
  17: proc_macro::bridge::closure::Closure<A,R>::call
             at /tmp/rust/library/proc_macro/src/bridge/closure.rs:27:18
  18: proc_macro::bridge::client::TokenStreamIter::next::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:244:25
  19: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::with::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:336:47
  20: proc_macro::bridge::client::BridgeState::with::{{closure}}::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:293:17
  21: proc_macro::bridge::scoped_cell::ScopedCell<T>::replace
             at /tmp/rust/library/proc_macro/src/bridge/scoped_cell.rs:75:9
  22: proc_macro::bridge::client::BridgeState::with::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:291:13
  23: std::thread::local::LocalKey<T>::try_with
             at /tmp/rust/library/std/src/thread/local.rs:272:16
  24: std::thread::local::LocalKey<T>::with
             at /tmp/rust/library/std/src/thread/local.rs:248:9
  25: proc_macro::bridge::client::BridgeState::with
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:290:9
  26: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::with
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:329:9
  27: proc_macro::bridge::client::TokenStreamIter::next
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:237:17
  28: <proc_macro::token_stream::IntoIter as core::iter::traits::iterator::Iterator>::next
             at /tmp/rust/library/proc_macro/src/lib.rs:240:13
  29: <proc_macro2::imp::TokenTreeIter as core::iter::traits::iterator::Iterator>::next
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/proc-macro2-1.0.24/src/wrapper.rs:304:46
  30: <proc_macro2::token_stream::IntoIter as core::iter::traits::iterator::Iterator>::next
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/proc-macro2-1.0.24/src/lib.rs:1213:13
  31: syn::buffer::TokenBuffer::inner_new
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/syn-1.0.58/src/buffer.rs:53:19
  32: syn::buffer::TokenBuffer::new2
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/syn-1.0.58/src/buffer.rs:112:9
  33: <F as syn::parse::Parser>::parse2
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/syn-1.0.58/src/parse.rs:1207:19
  34: syn::parse::Parser::parse
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/syn-1.0.58/src/parse.rs:1161:9
  35: syn::parse_macro_input::parse
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/syn-1.0.58/src/parse_macro_input.rs:139:5
  36: async_attributes::test
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/async-attributes-1.1.1/src/lib.rs:99:17
  37: core::ops::function::FnOnce::call_once
             at /tmp/rust/library/core/src/ops/function.rs:227:5
  38: proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) .> proc_macro::TokenStream>::expand2::run::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:426:17
  39: proc_macro::bridge::client::run_client::{{closure}}::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:377:26
  40: proc_macro::bridge::scoped_cell::ScopedCell<T>::set::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/scoped_cell.rs:80:33
  41: proc_macro::bridge::scoped_cell::ScopedCell<T>::replace
             at /tmp/rust/library/proc_macro/src/bridge/scoped_cell.rs:75:9
  42: proc_macro::bridge::scoped_cell::ScopedCell<T>::set
             at /tmp/rust/library/proc_macro/src/bridge/scoped_cell.rs:80:9
  43: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:325:35
  44: std::thread::local::LocalKey<T>::try_with
             at /tmp/rust/library/std/src/thread/local.rs:272:16
  45: std::thread::local::LocalKey<T>::with
             at /tmp/rust/library/std/src/thread/local.rs:248:9
  46: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:325:9
  47: proc_macro::bridge::client::run_client::{{closure}}
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:370:9
  48: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /tmp/rust/library/std/src/panic.rs:322:9
  49: std::panicking::try::do_call
             at /tmp/rust/library/std/src/panicking.rs:379:40
  50: __rust_try
  51: std::panicking::try
             at /tmp/rust/library/std/src/panicking.rs:343:19
  52: std::panic::catch_unwind
             at /tmp/rust/library/std/src/panic.rs:396:14
  53: proc_macro::bridge::client::run_client
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:369:5
  54: proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) .> proc_macro::TokenStream>::expand2::run
             at /tmp/rust/library/proc_macro/src/bridge/client.rs:425:13
  55: <proc_macro::bridge::server::SameThread as proc_macro::bridge::server::ExecutionStrategy>::run_bridge_and_client
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:155:9
  56: proc_macro::bridge::server::run_server
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:291:9
  57: proc_macro::bridge::server::<impl proc_macro::bridge::client::Client<fn(proc_macro::TokenStream,proc_macro::TokenStream) .> proc_macro::TokenStream>>::run
             at /tmp/rust/library/proc_macro/src/bridge/server.rs:334:9
  58: <rustc_expand::proc_macro::AttrProcMacro as rustc_expand::base::AttrProcMacro>::expand
             at /tmp/rust/compiler/rustc_expand/src/proc_macro.rs:53:9
  59: rustc_expand::expand::MacroExpander::expand_invoc
             at /tmp/rust/compiler/rustc_expand/src/expand.rs:753:44
  60: rustc_expand::expand::MacroExpander::fully_expand_fragment
             at /tmp/rust/compiler/rustc_expand/src/expand.rs:497:53
  61: rustc_expand::expand::MacroExpander::expand_crate
             at /tmp/rust/compiler/rustc_expand/src/expand.rs:390:15
  62: rustc_interface::passes::configure_and_expand_inner::{{closure}}::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/passes.rs:302:50
  63: rustc_data_structures::profiling::VerboseTimingGuard::run
             at /tmp/rust/compiler/rustc_data_structures/src/profiling.rs:570:9
  64: rustc_session::utils::<impl rustc_session::session::Session>::time
             at /tmp/rust/compiler/rustc_session/src/utils.rs:9:9
  65: rustc_interface::passes::configure_and_expand_inner::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/passes.rs:302:21
  66: rustc_data_structures::profiling::VerboseTimingGuard::run
             at /tmp/rust/compiler/rustc_data_structures/src/profiling.rs:570:9
  67: rustc_session::utils::<impl rustc_session::session::Session>::time
             at /tmp/rust/compiler/rustc_session/src/utils.rs:9:9
  68: rustc_interface::passes::configure_and_expand_inner
             at /tmp/rust/compiler/rustc_interface/src/passes.rs:254:13
  69: rustc_interface::passes::configure_and_expand::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/passes.rs:118:19
  70: alloc::boxed::<impl core::ops::generator::Generator<R> for core::pin::Pin<alloc::boxed::Box<G,A>>>::resume
             at /tmp/rust/library/alloc/src/boxed.rs:1606:9
  71: rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new
             at /tmp/rust/compiler/rustc_data_structures/src/box_region.rs:44:26
  72: rustc_interface::passes::BoxedResolver::new
             at /tmp/rust/compiler/rustc_data_structures/src/box_region.rs:101:41
  73: rustc_interface::passes::configure_and_expand
             at /tmp/rust/compiler/rustc_interface/src/passes.rs:114:30
  74: rustc_interface::queries::Queries::expansion::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/queries.rs:180:13
  75: rustc_interface::queries::Query<T>::compute
             at /tmp/rust/compiler/rustc_interface/src/queries.rs:39:28
  76: rustc_interface::queries::Queries::expansion
             at /tmp/rust/compiler/rustc_interface/src/queries.rs:176:9
  77: rustc_driver::run_compiler::{{closure}}::{{closure}}
             at /tmp/rust/compiler/rustc_driver/src/lib.rs:393:13
  78: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
             at /tmp/rust/compiler/rustc_interface/src/queries.rs:418:19
  79: rustc_driver::run_compiler::{{closure}}
             at /tmp/rust/compiler/rustc_driver/src/lib.rs:341:22
  80: rustc_interface::interface::create_compiler_and_run::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/interface.rs:197:13
  81: rustc_span::with_source_map
             at /tmp/rust/compiler/rustc_span/src/lib.rs:787:5
  82: rustc_interface::interface::create_compiler_and_run
             at /tmp/rust/compiler/rustc_interface/src/interface.rs:191:5
  83: rustc_interface::interface::run_compiler::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/interface.rs:213:12
  84: rustc_interface::util::setup_callbacks_and_run_in_thread_pool_with_globals::{{closure}}::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/util.rs:152:13
  85: scoped_tls::ScopedKey<T>::set
             at /home/dimitar/.cargo/registry/src/jackfan.us.kg-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137:9
  86: rustc_span::with_session_globals
             at /tmp/rust/compiler/rustc_span/src/lib.rs:103:5
  87: rustc_interface::util::setup_callbacks_and_run_in_thread_pool_with_globals::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/util.rs:150:9
  88: rustc_interface::util::scoped_thread::{{closure}}
             at /tmp/rust/compiler/rustc_interface/src/util.rs:125:24
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Was rmanoka/async-scoped#6 but we narrowed it down to rustc.
This only occurs on any rustc version after commit 6184f23, which merges #80830.
@DimiDimit DimiDimit added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 14, 2021
@DimiDimit DimiDimit changed the title ICE when calling syn::parse_macro_input! in proc macro decorating a function with a doc comment in a declarative macro ICE when calling syn::parse_macro_input! in proc macro applied to a function with a doc comment in a declarative macro Jan 14, 2021
@DimiDimit DimiDimit mentioned this issue Jan 14, 2021
Closed
@Aaron1011 Aaron1011 added the A-proc-macros Area: Procedural macros label Jan 14, 2021
@Aaron1011 Aaron1011 self-assigned this Jan 14, 2021
@Aaron1011 Aaron1011 mentioned this issue Jan 15, 2021
Closed
Aaron1011 added a commit to Aaron1011/rust that referenced this issue Jan 15, 2021
@Aaron1011
Always collect tokens when we are already inside a collect_tokens call
372b567 
Fixes rust-lang#81007

This is needed to ensure that the `LazyTokenStream` we use does not
include attributes - the outer call may include the attributes, which we
don't want to include in the `LazyTokenSTream`.
taiki-e added a commit to taiki-e/auto_enums that referenced this issue Jan 16, 2021
@taiki-e
Temporarily disable test that ICE occurred
8b465b0 
Refs: rust-lang/rust#81007
@Aaron1011 Aaron1011 mentioned this issue Jan 18, 2021
Merged
@Aaron1011 Aaron1011 mentioned this issue Jan 18, 2021
Merged
Aaron1011 added a commit to Aaron1011/rust that referenced this issue Jan 22, 2021
@Aaron1011
Force token collection to run when parsing nonterminals
b2e55bf 
Fixes rust-lang#81007

Previously, we would fail to collect tokens in the proper place when
only builtin attributes were present. As a result, we would end up with
attribute tokens in the collected `TokenStream`, leading to duplication
when we attempted to prepend the attributes from the AST node.

We now explicitly track when token collection must be performed due to
nomterminal parsing.
bors added a commit to rust-lang-ci/rust that referenced this issue Jan 22, 2021
@bors
Auto merge of rust-lang#81177 - Aaron1011:fix/force-capture-tokens, r…
dc1eee2 
…=petrochenkov

Force token collection to run when parsing nonterminals

Fixes rust-lang#81007

Previously, we would fail to collect tokens in the proper place when
only builtin attributes were present. As a result, we would end up with
attribute tokens in the collected `TokenStream`, leading to duplication
when we attempted to prepend the attributes from the AST node.

We now explicitly track when token collection must be performed due to
nomterminal parsing.
@bors bors closed this as completed in 11b1e37 Jan 22, 2021
flip1995 pushed a commit to flip1995/rust that referenced this issue Jan 30, 2021
@Aaron1011
Force token collection to run when parsing nonterminals
2be935d 
Fixes rust-lang#81007

Previously, we would fail to collect tokens in the proper place when
only builtin attributes were present. As a result, we would end up with
attribute tokens in the collected `TokenStream`, leading to duplication
when we attempted to prepend the attributes from the AST node.

We now explicitly track when token collection must be performed due to
nomterminal parsing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Aaron1011 Aaron1011

Labels
A-proc-macros Area: Procedural macros C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
2 participants
@Aaron1011 @DimiDimit