Why is convert.py missing?

[David-AU-github]

What happened?

Critical "non llama3" convert.py and change NOT in download of files.

ALSO:
It is unclear if "convert-hf-to-gguf.py" supports what "convert.py" did .

Does it convert llama, llama2, mistral or is "convert-legacy-llama.py" required?
Safetensor files are EVERYWHERE. (?) [ RE: https://github.com//pull/7430 ]

This critical action DID NOT OCCUR:
"Move convert.py to examples/convert-legacy-llama.py"

"examples/convert-legacy-llama.py" does not exist.
(when downloading the zip files).

On another note why remove "convert.py" at all?

-This breaks "bat files" and automation generation.
-This will break all colabs too.
-This will break any HF spaces that create GGUF files as well.
-This will create needless confusion.

If "convert-hf-to-gguf.py" (llama3) does everything convert.py did , just keep it as "convert.py" ?

Name and Version

this is not applicable - core files missing.

What operating system are you seeing the problem on?

No response

Relevant log output

this is not applicable - core files missing.

[Chen4549]

same issue
but it worked yesterday
so they removed the convert.py a couple of hours ago?

Problem solved by using CLI command: python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16

[Galunid]

TLDR:

convert-hf-to-gguf.py is the correct way to convert huggingface models. If you have llama/llama2 model downloaded directly from meta (in .pth format), you can still use examples/convert-legacy-llama.py, otherwise it's discourage and we won't provide support. It will not work with llama3

If you need to fix scripts/colabs, changing convert.py to convert-hf-to-gguf.py and removing --vocab-type should be enough to make them work.

---

Full explanation:

In #7430 convert.py was moved to examples/convert-legacy-llama.py to avoid confusion as to which script should be used (now there's only one) and reduce maintenance burden on developers. It should not be used anymore. It was removed to force people to use convert-hf-to-gguf.py that supports all huggingface models that llama.cpp can run. There were a lot of issues because of people using convert.py when they shouldn't have. There were a lot of tickets, where convert.py was used by users in situations where it wasn't appropriate - see #7658 (comment)

I know this will inconvenience some people and it's really unfortunate, but from developer perspective maintaining convert.py was just no longer an option. convert-hf-to-gguf.py was doing everything convert.py did and much more, and it is much simpler to maintain. For a while convert.py produced broken models that had warnings like this:

llm_load_vocab: missing pre-tokenizer type, using: 'default'
llm_load_vocab:                                             
llm_load_vocab: ************************************        
llm_load_vocab: GENERATION QUALITY WILL BE DEGRADED!        
llm_load_vocab: CONSIDER REGENERATING THE MODEL             
llm_load_vocab: ************************************   

Fixes were implemented in convert-hf-to-gguf.py that never made it to convert.py. Making changes to both scripts that did pretty much the same thing takes a lot of time, so one of them was removed.

If you really need it, it's still available in examples/convert-legacy-llama.py, but it's no longer supported.

On another note why remove "convert.py" at all?

-This breaks "bat files" and automation generation.
-This will break all colabs too.
-This will break any HF spaces that create GGUF files as well.
-This will create needless confusion.

"Why" is explained above, scripts/colabs fix is to change convert.py to convert-hf-to-gguf.py and removing unrecognized flags. Our huggingface space should be working as expected. If someone is confused, then most likely they were using the wrong script and their model quality will improve by switching :).

[David-AU-github]

Okay, thank you for the details - but this DOESN'T EXIST:

examples/convert-legacy-llama.py

It is not in the zip files.
IE: https://github.com/ggerganov/llama.cpp/archive/refs/tags/b3051.zip

In regards to colabs - I am not talking about mine, I am talking about all that rely on llamacpp.
This is due to the construction of commands and auto-update from llamacpp PER USE.
That means as of this update, all colabs using llamacpp (and all users using colabs) will break.
Any colab that uses llamacpp is now broken at the "convert" step.
Most users (of colabs) will have no clue what the issue is, never mind how to fix.

At the very least please consider a "redirect" (from convert.py) to the correct file? or a fall back [legacy]?
Or redirect to "new version" ??
Maybe one that looks at if the file is or is not "llama3" and acts accordingly?

[David-AU-github]

same issue but it worked yesterday so they removed the convert.py a couple of hours ago?

Problem solved by using CLI command: python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16

thanks - that really helps!!!

[Galunid]

It is not in the zip files.

I'm not sure what you mean. I downloaded the file you linked and it is there.

[mlabonne]

@Galunid Thanks for this clear explanation, it makes complete sense

[David-AU-github]

It is not in the zip files.

I'm not sure what you mean. I downloaded the file you linked and it is there.

I will download again - I checked it - not there.
Delay in being added?

[David-AU-github]

Okay - there it is . ?(??) ; thanks again.

[oldgithubman]

What happened?

Critical "non llama3" convert.py and change NOT in download of files.

ALSO: It is unclear if "convert-hf-to-gguf.py" supports what "convert.py" did .

Does it convert llama, llama2, mistral or is "convert-legacy-llama.py" required? Safetensor files are EVERYWHERE. (?) [ RE: https://github.com//pull/7430 ]

This critical action DID NOT OCCUR: "Move convert.py to examples/convert-legacy-llama.py"

"examples/convert-legacy-llama.py" does not exist. (when downloading the zip files).

On another note why remove "convert.py" at all?

-This breaks "bat files" and automation generation. -This will break all colabs too. -This will break any HF spaces that create GGUF files as well. -This will create needless confusion.

If "convert-hf-to-gguf.py" (llama3) does everything convert.py did , just keep it as "convert.py" ?

Name and Version

this is not applicable - core files missing.

What operating system are you seeing the problem on?

No response

Relevant log output

this is not applicable - core files missing.

There should be a standardization to one script, but the classic lack of communication around here does suck

[oldgithubman]

to avoid confusion as to which script should be used (now there's only one) and reduce maintenance burden on developers

This is a good thing.

There were a lot of issues because of people using convert.py when they shouldn't have.

This is the fault of bad documentation (that I've been saying for a while now and taking flak for). The main readme's tutorial literally told everyone to use convert.py until recently, and it mostly worked, so how was a normal user supposed to know better? The main readme didn't even mention the new script. The elitist "it's the stupid user's fault" attitude around here sucks. (I'm not necessarily accusing you of this, BTW)

I know this will inconvenience some people and it's really unfortunate

As long as all reasonable steps are taken to maintain backwards compatibility, sure. I don't think that's happening here though. Maybe the devs don't understand how integral this project is in the community now? (although, that seems a bit hard to believe). Projects like this and ollama need to start taking their roles more seriously, IMO. I'm sure there's a way to make this transition smoother for users.

To be clear, this change doesn't affect me. I've been using the new script for a while now. Just trying to bridge the disconnect between devs and normal users

[Galunid]

There were a lot of issues because of people using convert.py when they shouldn't have.

I'd like to sincerely apologize and clarify I meant github issues. It wasn't my intention to imply that users are dumb, problematic or whatever else I may have accidentally implied. All I wanted was merely to show that it was a common problem among the user base and I certainly didn't mean it to be read as a sarcastic remark.

There were a lot of tickets, where convert.py was used by users in situations where it wasn't appropriate. is much closer to what I meant.

We are aware there was a lack of documentation regarding conversion scripts and it is a big reason why we decided to remove convert.py when compilade added partial tensor loading to convert-hf-to-gguf.py (hence significantly reducing memory usage of that script).

There should be a standardization to one script, but the classic lack of communication around here does suck

As long as all reasonable steps are taken to maintain backwards compatibility, sure. I don't think that's happening here though. Maybe the devs don't understand how integral this project is in the community now? (although, that seems a bit hard to believe). Projects like this and ollama need to start taking their roles more seriously, IMO. I'm sure there's a way to make this transition smoother for users.

To be clear, this change doesn't affect me. I've been using the new script for a while now. Just trying to bridge the disconnect between devs and normal users

And from other tickets (that as you said "that I've been saying for a while now and taking flak"):

Thanks for telling us. I gotta say, it's getting real annoying wasting endless hours chasing these things down because the devs can't be bothered to update the relevant info in the main readme (which, BTW, makes no mention of "convert-hf-to-gguf.py" that I'm aware of). Seriously, I can't be the only one who is infuriated by this pattern of behavior in this community.

Devs: documentation matters. What would take you, what, five minutes to update, would save the community probably hundreds if not thousands of cumulative hours. We appreciate what you do (well, I do, anyway), but this is just dumb and lazy. How many botched ggufs are being proliferated because of this?

Clearly you don't understand. Development is a continuous process and things change quickly here. If you want users to keep up with development and keep the documentation updated, you've skipped CSci 101, where you would have been taught documentation is one of the most important things for a developer to do well. Since when do users write manuals? You'd basically have to become a dev to be able to do that. I don't understand how this is so hard to comprehend. You've also mistaken constructive criticism for complaining. I'm trying to help you devs understand the user perspective. My tone is intentional to convey the frustration many users feel but are too afraid to voice. If you don't see how this is immensely helpful, well, I guess I should have expected that. I don't really care what you think about me. If you want this project to thrive, you need better documentation. Telling the users to create it "isn't productive or helpful."

And the rest I haven't bothered to copy over:

Thank you for your... passionate response.

Let me take this opportunity to explain what constructive criticism is. It's not about throwing around phrases like "classic lack of communication" or accusing others of having an "elitist 'it's the stupid user's fault' attitude". It's about providing specific, actionable feedback that helps improve the project.

Your complaint about documentation being inadequate is valid, and I appreciate you bringing it up. However, instead of attacking us for not doing things your way, why not offer some concrete suggestions on how to improve the documentation? That would be constructive.

As for the removal of the script, I understand it may have caused inconvenience. But let's not assume we're intentionally trying to make life harder for users. We're just trying to avoid confusion and reduce maintenance burden. If you have a better solution, please share it with us.

And yes, projects like ours do need to take their roles more seriously and engage with the community. But that doesn't mean we should be attacked or criticized without any constructive input.

So, let's try to keep the conversation civil and focused on improving the project. We're all in this together, after all.

[oldgithubman]

It wasn't my intention to imply that users are dumb, problematic or whatever else I may have accidentally implied. All I wanted was merely to show that it was a common problem among the user base and I certainly didn't mean it to be read as a sarcastic remark.

Again, VVVVVVVVVVVVVVVVVVVVVVVVVVV.

(I'm not necessarily accusing you of this, BTW)

My comment was a general one. It was not specifically targeted at you. That's why I went out of my way to clarify that. Constructive criticism - pay attention to details.

Thank you for your... passionate response.

Did you mean that "to be read as a sarcastic remark?" If not, you're welcome. I do spend a lot more time than you'd think crafting these responses to help. I don't do this because I don't have better things I could be doing.

Let me take this opportunity to explain what constructive criticism is.

Thanks for leaving out the context, making me look like I'm ranting at the moon (that was meant to be sarcastic). I'm starting to think your unnecessary apology wasn't actually sincere at this point. Definite mixed signals. Because now you are "implying [I'm] dumb" and your following statements also imply I'm "problematic." I know what constructive criticism is, thanks. It's subjective. I'm actually pretty proud of all I've been able to accomplish in this community by lighting fires under the right asses, including getting meta to fix llama 3. People don't like my tone, but it gets things done. I rarely get credit, usually flak. I'm ok with that. As I've stated, I don't care what people think about me. It's not about me. It's about what's best for the community and whether people understand my methods and their effectiveness doesn't matter. It works, whether people see it or not.

It's not about throwing around phrases like "classic lack of communication"

It is when there is a systemic lack of communication...and there is.

or accusing others of having an "elitist 'it's the stupid user's fault' attitude"

I didn't accuse anyone specifically. I can if you want me to, but I'd prefer to leave it as a general criticism that I know I'm not alone in thinking and that does need improving upon. Honestly, elitism is probably the biggest problem in the software dev industry. Fix that one problem and we'd be decades further than we are. Blaming the user (customer) is almost never the right thing to do, even if it is their fault.

It's about providing specific, actionable feedback that helps improve the project.

Sometimes it is, and I provide plenty of that as well. Sometimes criticism needs to be more general. Reality is considerably more complex than you're implying.

Your complaint about documentation being inadequate is valid, and I appreciate you bringing it up.

Do you? That's not the impression I'm getting. Either way, if you read the context of those quotes you pulled, you'll see others on the project disagree. Considering the wide range of reactions I get to the exact same criticisms, I'll chalk it up to individual differences. I know my criticism (not complaint) is valid. That's why I make it. I've brought it up many times and I think this might be the first time I've been told it's appreciated.

That would be constructive.

Making yourself the arbiter of what is or isn't constructive seems a bit...elitist...doesn't it? That's a problem. And I'll keep calling it out when I see it. Because that's constructive. Whether you understand that or not.

But let's not assume we're intentionally trying to make life harder for users.

Let's not assume I assumed. Let's not straw man either.

We're just trying to avoid confusion and reduce maintenance burden.

And I'm just trying to help you achieve those goals. Part of reducing maintenance burden is arguably reducing the burden of spending hours fielding issues from users. I believe you. I'm trying to tell you you guys can do better. That's what the other users are trying to tell you too. Listen to them. That's all I'm really trying to say.

If you have a better solution, please share it with us.

I share what I can. If I have a specific solution, I will share it with you. I often won't though, because I'm not a dev. Sometimes the best I have to share is the knowledge there's a problem. That's half the battle. Other users share solutions that get ignored. They're dismissed too flippantly.

And yes, projects like ours do need to take their roles more seriously and engage with the community.

Success.

But that doesn't mean we should be attacked or criticized without any constructive input.

Agreed. Good thing no one's being attacked and everyone's input is valued.

So, let's try to keep the conversation civil and focused on improving the project.

*focus. Attention to detail will improve the project. (Edit - Just realized "focused" is correct too. I apologize. The irony here is not lost on me.)

We're all in this together, after all.

It'd be nice if people actually understood that.

[erasmus74]

I don't do this because I don't have better things I could be doing.

Go do those things. Save angsty posts for youtube comment section. Let them work.

[oldgithubman]

I don't do this because I don't have better things I could be doing.

Go do those things. Save angsty posts for youtube comment section. Let them work.

Suggestion considered and rejected. Save useless posts for reddit. Let me work.

[slaren]

Looks like this has already been answered.