Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Cannot get cloud GPU offers #2208

Open
jvstme opened this issue Jan 21, 2025 · 0 comments · May be fixed by #2210
Open

[Bug]: Cannot get cloud GPU offers #2208

jvstme opened this issue Jan 21, 2025 · 0 comments · May be fixed by #2210
Assignees
Labels
bug Something isn't working major

Comments

@jvstme
Copy link
Collaborator

jvstme commented Jan 21, 2025

Steps to reproduce

Apply this configuration:

type: dev-environment
ide: vscode
resources:
  gpu: 1

Actual behaviour

No offers.

> dstack apply
 Project          ilya                                  
 User             admin                                 
 Configuration    .dstack.yml                           
 Type             dev-environment                       
 Resources        2..xCPU, 8GB.., 1xGPU, 100GB.. (disk) 
 Max price        -                                     
 Max duration     6h                                    
 Spot policy      on-demand                             
 Retry policy     -                                     
 Creation policy  reuse-or-create                       
 Idle duration    5m                                    
 Reservation      -                                     

Submit a new run? [y/n]

Expected behaviour

Cloud offers suggested.

dstack version

master

Server logs

ERROR    dstack._internal.server.services.backends:435 Got exception when requesting offers from backend BackendType.RUNPOD                        
                    Traceback (most recent call last):                                                                                                        
                      File "/home/jvstme/git/dstack/dstack/src/dstack/_internal/utils/common.py", line 20, in run_async                                       
                        return await asyncio.get_running_loop().run_in_executor(None, func_with_args)                                                         
                      File "/usr/lib64/python3.9/concurrent/futures/thread.py", line 58, in run                                                               
                        result = self.fn(*self.args, **self.kwargs)                                                                                           
                      File "/home/jvstme/git/dstack/dstack/venv/lib64/python3.9/site-packages/cachetools/__init__.py", line 832, in wrapper                   
                        k = key(self, *args, **kwargs)                                                                                                        
                      File "/home/jvstme/git/dstack/dstack/src/dstack/_internal/core/backends/base/compute.py", line 191, in _get_offers_cached_key           
                        return hash(json.dumps(requirements.dict(), sort_keys=True))                                                                          
                      File "/usr/lib64/python3.9/json/__init__.py", line 234, in dumps                                                                        
                        return cls(                                                                                                                           
                      File "/usr/lib64/python3.9/json/encoder.py", line 199, in encode                                                                        
                        chunks = self.iterencode(o, _one_shot=True)                                                                                           
                      File "/usr/lib64/python3.9/json/encoder.py", line 257, in iterencode                                                                    
                        return _iterencode(o, 0)                                                                                                              
                      File "/usr/lib64/python3.9/json/encoder.py", line 179, in default                                                                       
                        raise TypeError(f'Object of type {o.__class__.__name__} '                                                                             
                    TypeError: Object of type AcceleratorVendor is not JSON serializable

Additional information

No response

@jvstme jvstme added bug Something isn't working major labels Jan 21, 2025
@jvstme jvstme self-assigned this Jan 22, 2025
@jvstme jvstme linked a pull request Jan 22, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working major
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant