Ark: vector memory exhausted

[testlabauto]

System details:

Positron and OS details:

OSX
2024.08.0-88

Interpreter details:

R 4.3.3

Describe the issue:

[R] R thread backtrace:
[R]    0: std::backtrace_rs::backtrace::libunwind::trace
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/../../backtrace/src/backtrace/libunwind.rs:116:5
[R]    1: std::backtrace_rs::backtrace::trace_unsynchronized
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
[R]    2: std::backtrace::Backtrace::create
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/backtrace.rs:331:13
[R]    3: harp::exec::try_catch::handler::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:237:30
[R]    4: harp::exec::try_catch::handler
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:213:27
[R]    5: do_internal
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/names.c:1404:11
[R]    6: Rf_eval
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:1092:12
[R]    7: R_execClosure
[R]    8: Rf_applyClosure
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:2113:16
[R]    9: bcEval
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:7414:12
[R]   10: Rf_eval
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:1013:8
[R]   11: R_execClosure
[R]   12: Rf_applyClosure
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:2113:16
[R]   13: Rf_eval
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/eval.c:1140:12
[R]   14: vsignalError
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/errors.c:1903:3
[R]   15: Rf_errorcall
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/errors.c:912:5
[R]   16: mem_err_heap
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/memory.c:2143:5
[R]   17: R_gc_internal
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/memory.c:3130:8
[R]   18: Rf_allocVector3
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/memory.c:2808:2
[R]   19: Rf_allocVector
[R]              at /Volumes/Builds/R4/R-4.3.1/src/include/Rinlinedfuns.h:595:12
[R]   20: ExpandDeferredStringElt
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/altclasses.c:692:8
[R]   21: deferred_string_Elt
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/altclasses.c:779:13
[R]   22: ALTSTRING_ELT
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/altrep.c:529:11
[R]   23: libr::r::STRING_ELT
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   24: <harp::vector::character_vector::CharacterVector as harp::vector::Vector>::get_unchecked_elt
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/vector/character_vector.rs:77:18
[R]   25: harp::vector::Vector::get_unchecked
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/vector/mod.rs:64:17
[R]   26: harp::vector::Vector::format_elt_unchecked
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/vector/mod.rs:114:15
[R]   27: harp::vector::formatted_vector::FormattedVector::get_unchecked
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/vector/formatted_vector.rs:141:17
[R]   28: <harp::vector::formatted_vector::FormattedVectorIter as core::iter::traits::iterator::Iterator>::next
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/vector/formatted_vector.rs:192:28
[R]   29: ark::variables::variable::WorkspaceVariableDisplayValue::from_default
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:286:18
[R]   30: ark::variables::variable::WorkspaceVariableDisplayValue::from
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:83:18
[R]   31: ark::variables::variable::WorkspaceVariableDisplayValue::from_list
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:129:29
[R]   32: ark::variables::variable::WorkspaceVariableDisplayValue::from
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:75:56
[R]   33: ark::variables::variable::PositronVariable::from
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:487:13
[R]   34: ark::variables::variable::PositronVariable::new
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/variable.rs:475:17
[R]   35: ark::variables::r_variables::RVariables::update::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/variables/r_variables.rs:413:47
[R]   36: ark::r_task::r_task::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/r_task.rs:170:44
[R]   37: core::ops::function::FnOnce::call_once{{vtable.shim}}
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/ops/function.rs:250:5
[R]   38: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/alloc/src/boxed.rs:2063:9
[R]   39: harp::exec::try_catch::callback
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:200:31
[R]   40: R_withCallingErrorHandler
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/errors.c:2613:16
[R]   41: libr::r::R_withCallingErrorHandler
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   42: harp::exec::try_catch::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:272:9
[R]   43: harp::exec::top_level_exec::callback
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:342:31
[R]   44: R_ToplevelExec
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/context.c:799:2
[R]   45: libr::r::R_ToplevelExec
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   46: harp::exec::top_level_exec
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:345:14
[R]   47: harp::exec::try_catch
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:271:20
[R]   48: harp::exec::r_sandbox
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:388:5
[R]   49: ark::interface::RMain::handle_task::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:972:63
[R]   50: tracing::span::Span::in_scope
[R]              at /Users/dfalbel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/span.rs:1102:9
[R]   51: ark::interface::RMain::handle_task
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:972:30
[R]   52: ark::interface::RMain::handle_task_interrupt
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:944:34
[R]   53: ark::interface::RMain::read_console
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:731:21
[R]   54: ark::interface::r_read_console::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:1604:31
[R]   55: harp::exec::try_catch::callback
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:200:31
[R]   56: R_withCallingErrorHandler
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/errors.c:2613:16
[R]   57: libr::r::R_withCallingErrorHandler
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   58: harp::exec::try_catch::{{closure}}
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:272:9
[R]   59: harp::exec::top_level_exec::callback
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:342:31
[R]   60: R_ToplevelExec
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/context.c:799:2
[R]   61: libr::r::R_ToplevelExec
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   62: harp::exec::top_level_exec
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:345:14
[R]   63: harp::exec::try_catch
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:271:20
[R]   64: harp::exec::r_sandbox
[R]              at /Users/dfalbel/Documents/posit/ark/crates/harp/src/exec.rs:388:5
[R]   65: r_read_console
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:1604:18
[R]   66: Rf_ReplIteration
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/main.c:210:10
[R]   67: R_ReplConsole
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/main.c:314:11
[R]   68: run_Rmainloop
[R]              at /Volumes/Builds/R4/R-4.3.1/src/main/main.c:1200:5
[R]   69: libr::r::run_Rmainloop
[R]              at /Users/dfalbel/Documents/posit/ark/crates/libr/src/functions.rs:31:21
[R]   70: ark::sys::unix::interface::run_r
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/sys/unix/interface.rs:79:9
[R]   71: ark::interface::start_r
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/interface.rs:284:5
[R]   72: ark::start_kernel
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/main.rs:130:5
[R]   73: ark::parse_file
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/main.rs:216:13
[R]   74: ark::main
[R]              at /Users/dfalbel/Documents/posit/ark/crates/ark/src/main.rs:498:9
[R]   75: core::ops::function::FnOnce::call_once
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/ops/function.rs:250:5
[R]   76: std::sys_common::backtrace::__rust_begin_short_backtrace
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:155:18
[R]   77: std::rt::lang_start::{{closure}}
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:159:18
[R]   78: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/ops/function.rs:284:13
[R]   79: std::panicking::try::do_call
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:559:40
[R]   80: std::panicking::try
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:523:19
[R]   81: std::panic::catch_unwind
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panic.rs:149:14
[R]   82: std::rt::lang_start_internal::{{closure}}
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:141:48
[R]   83: std::panicking::try::do_call
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:559:40
[R]   84: std::panicking::try
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:523:19
[R]   85: std::panic::catch_unwind
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panic.rs:149:14
[R]   86: std::rt::lang_start_internal
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:141:20
[R]   87: std::rt::lang_start
[R]              at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:158:17
[R]   88: _main

The important part of the back trace is this, which shows it's another codepath that's materializing the large array

Steps to reproduce the issue:

Reprex

f <- function() {
  x <- 1:1e10
  names(x) <- x
  list(x = names(x))
}

x <- f()

This will crash the console:

thread 'ark-variables' panicked at crates/ark/src/r_task.rs:220:13:
While running task: vector memory exhausted (limit reached?)

R backtrace:
0. try_catch_handler(structure(list(message = "vector memory exhausted (limit reached?)", 
       call = NULL), class = c("simpleError", "error", "condition"
   )))
1. h(simpleError(msg, call))
2. .handleSimpleError(function (cond) 
   .Internal(C_tryCatchHelper(addr, 1L, cond)), "vector memory exhausted (limit reached?)", 
       base::quote(NULL))


R thread backtrace:
   0: std::backtrace::Backtrace::create
   1: harp::exec::try_catch::handler
   2: do_internal
             at /Volumes/Builds/R4/R-4.3.3/src/main/names.c:1404:11
   3: Rf_eval
             at /Volumes/Builds/R4/R-4.3.3/src/main/eval.c:1094:12
   4: R_execClosure
   5: Rf_applyClosure
             at /Volumes/Builds/R4/R-4.3.3/src/main/eval.c:2115:16
   6: bcEval
             at /Volumes/Builds/R4/R-4.3.3/src/main/eval.c:7416:12
   7: Rf_eval
             at /Volumes/Builds/R4/R-4.3.3/src/main/eval.c:1
R 4.3.3 exited unexpectedly: killed with signal 6 (SIGABRT)

Expected or desired behavior:

Variables pane doesn't crash R, and the user sees the error in the console when they try and print x

Were there any error messages in the UI, Output panel, or Developer Tools console?

INFO Starting session for language runtime e0f5a4e62ac703e2ea49c2bf1102f8b1 (language: R name: R 4.3.3 version: 4.3.3) (Source: The runtime exited unexpectedly and is being restarted automatically.)

[DavisVaughan]

Reprex runs without issue.

This would probably not be the actual ideal behavior. For me just running this in an R terminal does hit my memory limit, and I get an R error. It does not crash R, you just see the error. So I think the ideal behavior is probably "variables pane doesn't crash R, and the user sees the error in the console when they try and print x"

[dfalbel]

The variables pane seems to crash the session because it forces materialization of the vector. Ideally, the variables pane should not force full materialization of altrep vectors.