Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion error: linear solve did not converge in single precision #362

Closed
SVMendoza opened this issue Jun 20, 2022 · 8 comments
Closed

Comments

@SVMendoza
Copy link

Hello,
I am running circuitscape on Julia Version 1.6.4 on a linux server with 40 processors and 250 GB of RAM.

I'm running all-to-one scenarios. using the option parallel (max_parallel = 40)
and I have to make 12 runs (I have 12 friction value scenarios that I want to run), to automate it I call the files in a loop in julia. perform tests and all very well, but when I run the complete files it sends a error.
I thought it was a problem of memory allocation to the nodes, but it is not the case.
the error message is:

ERROR: LoadError: On worker 31:
AssertionError: norm(G * v .- curr, Inf) / norm(v, Inf) < 1.0e-5
Stacktrace:
[1] solve_linear_system
@ ~/.julia/packages/Circuitscape/GMT6U/src/core.jl:616
[2] macro expansion
@ ./timing.jl:287 [inlined]
[3] multiple_solve
@ ~/.julia/packages/Circuitscape/GMT6U/src/raster/advanced.jl:312
[4] multiple_solver
@ ~/.julia/packages/Circuitscape/GMT6U/src/raster/advanced.jl:292
[5] advanced_kernel
@ ~/.julia/packages/Circuitscape/GMT6U/src/raster/advanced.jl:205
[6] f
@ ~/.julia/packages/Circuitscape/GMT6U/src/raster/onetoall.jl:0
[7] #201
@ ~/.julia/packages/Circuitscape/GMT6U/src/raster/onetoall.jl:158
[8] #106
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278
[9] run_work_thunk
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:63
[10] macro expansion
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278 [inlined]
[11] #105
@ ./task.jl:411
Stacktrace:

@SVMendoza
Copy link
Author

SVMendoza commented Jun 20, 2022

I don't understand, it's like it does all the calculations and when it's cleaning memory it sends the message. It returns the curmap and voltmap (77 maps equal to the number of nodes). and of course after the error message does not continue with the next run.
I attach the first two scenarios. and and all the messages printed on the screen
Escen.zip
hardcopy.0.txt
.

@ranjanan
Copy link
Member

Which version of CS are you on? Can you make sure you're on the latest (5.11.2). Running using Pkg; Pkg.update() should update your packages.

@ranjanan
Copy link
Member

This error stack looks like it is from an older version.

@SVMendoza
Copy link
Author

Thanks Ranjan,
I'll check and let you know if I can fix it.
Sergio

@SVMendoza
Copy link
Author

the version is Circuitscape v5.10.1

@ranjanan
Copy link
Member

update it and check please

@SVMendoza
Copy link
Author

Dear Ranjan,

the problem persists.
I already did the update and the version of circuitscape was updated but it sends the same error message.

[2b7a1792] ‘ Circuitscape v5.10.1 à v5.11.2

20220629_error.txt

@ranjanan ranjanan changed the title LoadError: On worker Assertion error: linear system did not converge Dec 28, 2022
@ranjanan ranjanan changed the title Assertion error: linear system did not converge Assertion error: linear solve did not converge Dec 28, 2022
@ranjanan ranjanan changed the title Assertion error: linear solve did not converge Assertion error: linear solve did not converge in single precision Dec 28, 2022
@ranjanan
Copy link
Member

I am closing this issue for now, because this appears to run in double precision. Single precision support is currently experimental.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants