feat: lora fine-tuning in FHE + gpt2 use case example

[RomanBredehoft]

currently running the notebook with 100 epochs

refs https://github.com/zama-ai/concrete-ml-internal/issues/4522

[RomanBredehoft]

timings are from my computer, it hasn't been refreshed

[andrei-stoian-zama]

I haven't reviewed everything yet.

Main issue is :

• can we make it work for other models than GPT2 (for example the simple MLP)
• make the LoraTraining use self.training to determine if it's doing inference or training

[andrei-stoian-zama]

I would like to remove the transformer dependency:

• move the dep to the use case example, replace conv1d -> nn.Linear in the usecase
• parametrize the remote module finding function in way that avoids the dep

[andrei-stoian-zama]

adding the dependency to transformers .. we had removed it earlier

[jfrery]

alright done

[andrei-stoian-zama]

this is too specific for a library function. e.g. for the lm_head moniker.

I suggest you add two arguments remote_layer_types and layer_reject_filter.

You can then call with remote_layer_types = [nn.Linear, nn.Embedding, transformer.Conv1D]. Thus CML does not need to know TransformerConv1D. and layer_reject_filter = ['lm_head]`

You can thus remove the include_embedding_layers flag.

[andrei-stoian-zama]

but in the function we manually add CustomLinear if the user asks for nn.Linear.

[jfrery]

I am not sure here. This is very specific with the replace_layers_with_custom. I would not allow the user to select layers himself. For now we do linear layers in FHE and we have a workaround for lm_head and embedding as long as there are not fixed.

[github-actions]

⚠️ Known flaky tests have been rerun ⚠️

One or several tests initially failed but were identified as known flaky. tests. Therefore, they have been rerun and passed. See below for more details.

Failed tests details

Known flaky tests that initially failed:

• tests/torch/test_compile_torch.py::test_compile_torch_or_onnx_conv_networks[False-True-CNN-relu]

[github-actions]

Coverage failed ❌

Coverage details

---------- coverage: platform linux, python 3.8.18-final-0 -----------
Name                                 Stmts   Miss  Cover   Missing
------------------------------------------------------------------
src/concrete/ml/onnx/onnx_utils.py      59     10    83%   594-620
------------------------------------------------------------------
TOTAL                                 8276     10    99%

60 files skipped due to complete coverage.

[andrei-stoian-zama]

Excellent work here @RomanBredehoft and @jfrery !