Incompatible with MacOS

[shyamalschandra]

Hi,

I just ran the code with torchrun after pip3 install -e . and this is what I got:

NOTE: Redirects are currently not supported in Windows or MacOs.
Traceback (most recent call last):
  File "/Users/dominodatasystems/code2notes/llama/example_text_completion.py", line 69, in <module>
    fire.Fire(main)
  File "/usr/local/lib/python3.11/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dominodatasystems/code2notes/llama/example_text_completion.py", line 32, in main
    generator = Llama.build(
                ^^^^^^^^^^^^
  File "/Users/dominodatasystems/code2notes/llama/llama/generation.py", line 84, in build
    torch.distributed.init_process_group("nccl")
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py", line 907, in init_process_group
    default_pg = _new_process_group_helper(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py", line 1013, in _new_process_group_helper
    raise RuntimeError("Distributed package doesn't have NCCL " "built in")
RuntimeError: Distributed package doesn't have NCCL built in
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 2320) of binary: /usr/local/opt/[email protected]/bin/python3.11
Traceback (most recent call last):
  File "/usr/local/bin/torchrun", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
example_text_completion.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-09-09_11:15:08
  host      : Shyamals-iMac.local
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 2320)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Can you fix this problem ASAP? Also, I don't have graphics card that is compatible with CUDA. Are you going to release one for OpenCL/Vulkan/Metal? Thanks again!

[artursapek]

I'm having the same problem and have no idea how to fix it!

[shyamalschandra]

@artursapek: Thanks for the help! I will need to spend more time on this but any help would be very cool from the FOSS community and FB! Thanks!

[subramen]

WIP PR for Apple support: #504 You can either wait for this to be merged or use the code from this PR. Note that you can't use MPS yet

[shyamalschandra]

@subramen: What about support for intel macs? Thanks!

[subramen]

@shyamalschandra yeah you should be able to run it on the CPU with that PR

[shyamalschandra]

@shyamalschandra yeah you should be able to run it on the CPU with that PR

Have you tried it yourself on an Intel MacOS machine?

[TimYao18]

I encounter similar error on Macbook Air M1.

Traceback (most recent call last):
  File "/Users/Me/survey/LLM/llama-main/example_text_completion.py", line 4, in <module>
    import fire
ModuleNotFoundError: No module named 'fire'

But I do have fire module:

$ pip show fire
Name: fire
Version: 0.5.0
Summary: A library for automatically generating command line interfaces.
Home-page: https://github.com/google/python-fire
Author: David Bieber
Author-email: [email protected]
License: Apache Software License
Location: /Users/Me/survey/LLM/LLM_env/lib/python3.11/site-packages
Requires: six, termcolor
Required-by: llama

So this might be related to the fire module not support macOS?

[jeffxtang]

You can use llama.cpp and lllama-cpp-python to run Llama on Mac - example.