"can't set attribute" on call to ai.train (GPT-Neo) in Colab notebook

[mesotron]

Hi - Nothing has changed in my code since last week, but all of a sudden my Colab notebook is crashing on the call to ai.train. I created a fresh copy of the colab notebook at https://colab.research.google.com/drive/15qBZx5y9rdaQSyWpsreMDnTiZ5IlN0zD?usp=sharing -- using the GPT-Neo model -- and the problem replicated. Here is the error message I get upon running the ai.train cell:

05/06/2021 22:12:58 — INFO — aitextgen — Loading text from input.txt with generation length of 2048.
100%
40000/40000 [00:00<00:00, 43736.33it/s]
05/06/2021 22:12:58 — INFO — aitextgen.TokenDataset — Encoding 40,000 sets of tokens from input.txt.

---

AttributeError Traceback (most recent call last)
in ()
8 learning_rate=1e-3,
9 fp16=False,
---> 10 batch_size=1,
11 )

2 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in setattr(self, name, value)
993 buffers[name] = value
994 else:
--> 995 object.setattr(self, name, value)
996
997 def delattr(self, name):

AttributeError: can't set attribute

[minimaxir]

PyTorch Lightning 1.3 released today and broke training (or more accurately, bugs in the initial training were exemplified by changes in 1.3)

Will pin an earlier version to the Colab for the time being. You can fix this for the time being by adding this line to the beginning of the setup cell:

!pip install pytorch-lightning==1.2.10

[minimaxir]

May have to cherrypick some of the changes in #105

[mesotron]

Great, thanks so much for the prompt workaround.

[minimaxir]

Merging #105 appears to have fixed this issue.