Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix MSVC build; and associated merge #4

Merged
merged 225 commits into from
Jan 22, 2025
Merged

Fix MSVC build; and associated merge #4

merged 225 commits into from
Jan 22, 2025

Conversation

Jeadie
Copy link

@Jeadie Jeadie commented Jan 22, 2025

🗣 Description

🔨 Related Issues

🤔 Concerns

lz1998 and others added 30 commits August 1, 2024 08:26
@lz1998 @LaurentMazare
bert attention mask (huggingface#1934)
4a52aeb 
* bert attention mask

* Allow for using None as a mask.

* Revert part of the changes so that the proper default mask applies.

* Cosmetic change.

* Another cosmetic tweak.

---------

Co-authored-by: Laurent <[email protected]>
@LaurentMazare
Enable the affine kernel for u8/u32. (huggingface#2376)
8696cf6
@ivarflakstad
Metal bgemm min changes (huggingface#2364)
fea46cb 
* Add updated mfa metallib

* Add bgemm and tests
@yunjhongwu
Fix log_sum_exp to handle large positive/negative inputs (huggingface…
bd80078 
…#2367)
@LaurentMazare
Use BF16 on metal when possible. (huggingface#2378)
1ba87a9
@spaghetti-source
Add get_ids to GradStore (huggingface#2379)
ce90287
@LaurentMazare
Enable BF16 on metal. (huggingface#2380)
957d604
@LaurentMazare
Add a minimal test for the metal bf16 matmul. (huggingface#2381)
d4b6f6e
@janimo
Add Hiera vision model. (huggingface#2382)
ac51f47
@JoanFM
Jina Bert Example fix and more configuration (huggingface#2191)
2e9c010 
* fix: fix jina bert example logic

* feat: enable jina embeddings de

* feat: allow more flexibility on Jina Bert
@LaurentMazare
Fix cargo fmt. (huggingface#2383)
9ca277a 
* Fix cargo fmt.

* Clippy fix.

* Cosmetic tweaks.
@singjc
update: LSTMState and GRUState fields to be public (huggingface#2384)
6991a37
@LaurentMazare
Revert the bf16 gemm metal changes for now. (huggingface#2386)
0fcb40b
@LaurentMazare
Add the flux model for image generation. (huggingface#2390)
19db6b9 
* Add the flux autoencoder.

* Add the encoder down-blocks.

* Upsampling in the decoder.

* Sketch the flow matching model.

* More flux model.

* Add some of the positional embeddings.

* Add the rope embeddings.

* Add the sampling functions.

* Add the flux example.

* Fix the T5 bits.

* Proper T5 tokenizer.

* Clip encoder path fix.

* Get the clip embeddings.

* No configurable weights in layer norm.

* More weights related fixes.

* Yet another shape fix.

* DType fix.

* Fix a couple more shape issues.

* DType fixes.

* Fix the latent dims.

* Fix more shape issues.

* Autoencoder fixes.

* Get some generations out.

* Bugfix.

* T5 padding.

* Clippy fix.

* Add the decode only mode.

* Fix.

* More fixes.

* Finally get some generations to work.

* Add readme.
@LaurentMazare
Simplify handling of flux modulations. (huggingface#2394)
aa7ac18
@MilkFather
optimize gradient for silu a bit (huggingface#2393)
c0a559d
@LaurentMazare
Support the flux-dev model too. (huggingface#2395)
89eae41
@LaurentMazare
Support for mistral-nemo. (huggingface#2396)
2be9bd2
@EricLBuehler
Add sdpa function with cublaslt
1a48767
@EricLBuehler
Update docs
7bbcf00
@EricLBuehler
Add matmul_bias_and_scale
1bf7101
@EricLBuehler
Rename
d6d3d18
@EricLBuehler
Add a simple test and fix for cpu
e20d85a
@EricLBuehler
Update sdpa function
8d2f32a
@EricLBuehler
Add matmul_alpha
9f144d6
@EricLBuehler
Use matmul_with_alpha in sdpa
c830f26
@EricLBuehler
Add it to mistral
86d0876
@EricLBuehler
Add it to q llama
8d8889c
@EricLBuehler
Add attention benches
d18eb13
@donjuanplatinum @LaurentMazare
add models support and example for THUDM/glm-4 (huggingface#2362)
500c9f2 
* add models support and example for THUDM/glm-4

* fix the ci report

* fmt

* fix

* Update README.org

* Update README.org

* fmt

* Update README.org

* README.md add codegeex4

* README.md add glm4

* Typo.

* change expect into ?

---------

Co-authored-by: Laurent Mazare <[email protected]>
EricLBuehler and others added 26 commits November 12, 2024 20:18
@EricLBuehler
Complete merge
be54a9a
@EricLBuehler
Complete merge
855fe38
@EricLBuehler
Attention-optimized softmax for prompts (huggingface#38)
77a6cc6 
* Add attn softmax

* Add some docs

* Add bf16

* Update kernels for f16

* All tests pass

* Fix cpu clippy

* Fix doc
@EricLBuehler
Metal qmatmul mat-mat product (huggingface#39)
6be03dd 
* Test passes

* All tests pass

* Now all the tests really pass

* Try out always using mm

* Mirror llama.cpp metric

* Mirror llama.cpp metric

* Update test
@EricLBuehler
Metal: Use mtl resource shared to avoid copy (huggingface#40)
cb8082b
@EricLBuehler
Dont always compile fp8, bf16 for CUDA (huggingface#42)
e97177b 
* Don't always compile fp8

* Correct includes
@EricLBuehler
F8E4M3 support on Metal (huggingface#43)
6b10eac 
* Add bf16, f32 conversions for f8e4m3 on metal

* Support storage from slice
@EricLBuehler
Integrate fast MLX kernel for SDPA with long seqlen (huggingface#45)
823a83a 
* Metal fast sdpa for long seqlen

* Add a test

* Add softcapping test
@EricLBuehler
General Metal bf16 support (huggingface#46)
8742354 
* General Metal bf16 support

* Fix compilation

* Fix compilation

* Math

* Math unary

* Math affine

* Maybe diambiguate

* Define hugevalbf

* Define matrix

* Pare back?

* Fix

* Format

* Clippy

* Clippy
@EricLBuehler
Experimental imatrix and I- quants support (huggingface#47)
e5685ce 
* Add q4k quantization with imatrix

* Sketch some imatrix generation

* Fixes

* Add quantize_imatrix_onto

* Support loading the imatrix file

* Fix load_imatrix

* Implement imatrix quantization for q2k

* Implement imatrix quantization for q3k

* Fix build on cuda

* Add imatrix q5k, q6k quants
@EricLBuehler
Fix q3k imatrix quantization (huggingface#48)
84c89a6
@EricLBuehler
Ensure support to cuda cc 53 (huggingface#49)
db0e646
@EricLBuehler
Fix Metal F8E4M3 impl (huggingface#50)
ce17ba6
@EricLBuehler
Fix duplicate cuda cast instantiations (huggingface#51)
a3814f5
@EricLBuehler
Add inplace softmax
df172aa
@EricLBuehler
Add varbuilder get_unchecked (huggingface#52)
2c7408b
@EricLBuehler
Merge branch 'main' into inplace_softmax
394ec76
@EricLBuehler
inplace_attn_softmax_last_dim
c7bd96d
@EricLBuehler
Fix cuda
e573895
@EricLBuehler
Format
0e8e8cb
@EricLBuehler
Merge pull request huggingface#53 from EricLBuehler/inplace_softmax
6800496 
* Add inplace softmax

* inplace_attn_softmax_last_dim

* Fix cuda

* Format
@EricLBuehler
Metal addmm support (huggingface#54)
c0c2b23 
* Metal addmm support

* Format
@EricLBuehler
Use cudarc fork to fix windows build (huggingface#58)
af655eb
@EricLBuehler
Use float8 mistralrs_cudarc_fork feature (huggingface#59)
f524bc6
@EricLBuehler
Begin to remove ug (huggingface#60)
f7d9f06 
* Begin to remove ug

* Begin to remove ug

* Begin to remove ug

* Begin to remove ug
@sgrebnov
Fix Windows (msvc) build
fd28f08
@Jeadie Jeadie self-assigned this Jan 22, 2025
@Jeadie Jeadie merged commit 296f14f into spiceai Jan 22, 2025
15 of 22 checks passed
@sgrebnov
Copy link

sgrebnov commented Jan 22, 2025 •
edited
Loading

@Jeadie - I think moving forward instead of merging upstream to spiceai we should create new spiceai-0.8.2 branch based on upstream version / tag and propagate our specific fixes - ideally this should be none (everything should be upstreamed and release as part of official candle library)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sgrebnov sgrebnov sgrebnov approved these changes

Assignees

@Jeadie Jeadie

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.