Unexpected tail in unsized_info_ty with complex futures

[vorner]

When I do cargo test on this code: https://github.com/vorner/tokio-jsonrpc/tree/bug, I get an internal compiler error (the full error is below). It might be related to #40231, or #37096, but they feel a bit different, so I'm opening a new issue. I didn't attempt to minimize the example, but I have few observations:

• The issue happens on stable, beta and nightly, though the backtraces are bit different.
• It may be unrelated, but the compilation of the tests/endpoint.rs takes a very long time even when it is short. Maybe there's some kind of limit of something (type complexity?) and the compiler gives up?
• It comes from the seq test in tests/endpoint.rs. If I comment the line https://github.com/vorner/tokio-jsonrpc/blob/bug/tests/endpoint.rs#L312, the problem goes away.
• If I try to join the futures in different order (eg. the join3 called on a different future), I still get the same error.
• If I join just the big future with the client_endpoint_finished one, it depends on the order. If I call client_endpoint_finished.join(the_big_one), I get a compile error (which I think is also wrong, because I use such futures in the rest of the tests and they work and each one of the futures separately can be used). If I do it in reverse (just changing the .join3(...) line to join(client_endpoint_finished), I get the internal compiler error. The compile error is:

error[E0161]: cannot move a value of type futures::Join<std::boxed::Box<futures::Future<Error=std::io::Error, Item=()>>, futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>>: the size of futures::Join<std::boxed::Box<futures::Future<Error=std::io::Error, Item=()>>, futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>> cannot be statically determined
   --> endpoint.rs:312:9
    |
312 |         client_endpoint_finished.join(client_finished)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0161]: cannot move a value of type futures::Join<std::boxed::Box<futures::Future<Error=std::io::Error, Item=()>>, futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>>: the size of futures::Join<std::boxed::Box<futures::Future<Error=std::io::Error, Item=()>>, futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>> cannot be statically determined
   --> endpoint.rs:314:17
    |
314 |     reactor.run(all).unwrap();

The backtrace and versions of nightly are:

$ cargo test
   Compiling tokio-jsonrpc v0.1.0 (file:///home/vorner/prog/tokio-jsonrpc)
error: internal compiler error: /checkout/src/librustc_trans/type_of.rs:157: Unexpected tail in unsized_info_ty: futures::future::join::MaybeDone<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>> for ty=futures::future::join::MaybeDone<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, futures::Join<futures::Map<std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:306:30: 309:18 first_finished_cloned:std::rc::Rc<std::cell::Cell<bool>>]>, futures::Map<futures::AndThen<std::boxed::Box<futures::Future<Error=std::io::Error, Item=(tokio_jsonrpc::Client, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>)>>, std::boxed::Box<futures::Future<Error=std::io::Error, Item=std::option::Option<tokio_jsonrpc::message::Response>>>, [[email protected]:296:31: 296:61]>, [[email protected]:297:26: 305:22 first_finished:std::rc::Rc<std::cell::Cell<bool>>]>>, [[email protected]:292:23: 311:14]>>

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:417
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: rustc_errors::Handler::bug
   1: rustc::session::opt_span_bug_fmt::{{closure}}
   2: rustc::session::opt_span_bug_fmt
   3: rustc::session::bug_fmt
   4: rustc_trans::type_of::unsized_info_ty
   5: rustc_trans::type_of::sizing_type_of
   6: <core::iter::Map<I, F> as core::iter::iterator::Iterator>::next
   7: <collections::vec::Vec<T> as collections::vec::SpecExtend<T, I>>::from_iter
   8: rustc_trans::adt::struct_llfields
   9: rustc_trans::adt::generic_type_of
  10: rustc_trans::type_of::sizing_type_of
  11: rustc_trans::abi::FnType::unadjusted::{{closure}}
  12: rustc_trans::abi::FnType::unadjusted
  13: rustc_trans::declare::declare_fn
  14: rustc_trans::trans_item::TransItem::predefine
  15: rustc_trans::base::trans_crate
  16: rustc_driver::driver::phase_4_translate_to_llvm
  17: rustc_driver::driver::compile_input::{{closure}}
  18: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  19: rustc_driver::driver::phase_3_run_analysis_passes
  20: rustc_driver::driver::compile_input
  21: rustc_driver::run_compiler
  22: std::panicking::try::do_call
  23: __rust_maybe_catch_panic
             at /checkout/src/libpanic_unwind/lib.rs:98
  24: <F as alloc::boxed::FnBox<A>>::call_box
  25: std::sys::imp::thread::Thread::new::thread_start
             at /checkout/src/liballoc/boxed.rs:648
             at /checkout/src/libstd/sys_common/thread.rs:21
             at /checkout/src/libstd/sys/unix/thread.rs:84
  26: <unknown>
  27: clone

error: Could not compile `tokio-jsonrpc`.

To learn more, run the command again with --verbose.
  101 vorner@hydra ~/prog/tokio-jsonrpc/tests (bug 1) 82%
$ rustc --version --verbose
rustc 1.17.0-nightly (b1e31766d 2017-03-03)
binary: rustc
commit-hash: b1e31766da75b188062c59f38cd6e8544b902afd
commit-date: 2017-03-03
host: x86_64-unknown-linux-gnu
release: 1.17.0-nightly
LLVM version: 3.9

Backtraces of the other versions (stable, nightly) can be found here: https://travis-ci.org/vorner/tokio-jsonrpc/builds/207742553.

Is there any other info I might try providing?

[Victor-Savu]

Just wanted to say that I am having the same problem in my code as well. Any workarounds for it?

[pshc]

I got similar crashes ("Unexpected tail in unsized_info_ty") when I was using impl Trait.

Switching to boxed futures was the simplest workaround. Besides boxing, I seem to remember I also managed to avoid ICEs by adding type annotations to my callback closures. (Particularly return types|...| -> T {. My uninformed conjecture is that the "unexpected tail" is the absence of a closure's inferred return type.)

HTH!

[Victor-Savu]

@pshc Thanks for your answer! I fear the problem is even deeper. I went back to the code, added the type annotations and magic! it worked! But then I got curious and removed the type annotations. And... it still works! I feel like in a Lovecraft story.

[pshc]

@Victor-Savu I'm spooked! 👻

[steveklabnik]

I am not able to reproduce:

> cargo +nightly test
   Compiling tokio-jsonrpc v0.1.0 (C:\Users\Steve Klabnik\tmp\padgrid\tokio-jsonrpc)
    Finished dev [unoptimized + debuginfo] target(s) in 4.23s
     Running target\debug\deps\tokio_jsonrpc-80ee290b51107abb.exe

running 5 tests
test codec::tests::encode ... ok
test codec::tests::decode ... ok
test message::tests::broken ... ok
test message::tests::message_serde ... ok
test message::tests::constructors ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running target\debug\deps\endpoint-7bde32359485fe52.exe

running 8 tests
test notification ... ok
test rpc_answer ... ok
test wrong_method ... ok
test client_only ... ok
test parallel ... ok
test delayed ... ok
test seq ... FAILED
test timeout ... ok

failures:

---- seq stdout ----
Terminating
MSG: Ok(Request(Request { jsonrpc: Version, method: "timeout", params: Some(Array([Number(PosInt(0)), Number(PosInt(500000000))])), id: String("381c9496-8fdb-48fc-9a22-9ff13e40db5b") }))
MSG: Ok(Request(Request { jsonrpc: Version, method: "wrong", params: None, id: String("2ec80644-b94a-44c7-bdc2-9442a31fe0ca") }))
Terminating
Dropping
Cleanup
MSG: Ok(Response(Response { jsonrpc: Version, result: Ok(Bool(true)), id: String("381c9496-8fdb-48fc-9a22-9ff13e40db5b") }))
MSG: Ok(Response(Response { jsonrpc: Version, result: Err(RPCError { code: -32601, message: "Method not found", data: Some(String("wrong")) }), id: String("2ec80644-b94a-44c7-bdc2-9442a31fe0ca") }))
Answer receivedAnswer receivedDropping
thread 'seq' panicked at 'assertion failed: !first_finished.get()', tests\endpoint.rs:304:25
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    seq

test result: FAILED. 7 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

Is this still happening, @varkor ?

[vorner]

I assume this was meant to be for me?

No, I can't reproduce any more either, the problem (or, all the problems) seem to have gone away. Unless someone else has a still failing example, I guess this can be closed.

[steveklabnik]

Whoops, tab complete messed up, sorry. Yeah, let's close.