lora_trainer.py seems to be expecting images

[peteromallet]

I'm trying to train with the lora_trainer. I have folders set up in the correct format with mp4 videos inside them for training

However, when I try to run it, I'm getting the below error that seems to say it's expecting images.

Any idea what's going on here? Am I supposed to be feeding in individual frames rather than videos?

File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 396, in call
return self.format_row(pa_table)
File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 485, in format_row
formatted_batch = self.format_batch(pa_table)
File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 515, in format_batch
return self.transform(batch)
File "/workspace/AnimateDiff/train_lora.py", line 670, in preprocess_train
images = [image.convert("RGB") for image in examples[image_column]]
File "/workspace/AnimateDiff/train_lora.py", line 670, in
images = [image.convert("RGB") for image in examples[image_column]]
AttributeError: 'str' object has no attribute 'convert'
Steps: 0%| | 0/700 [00:00<?, ?it/s]

[tumurzakov]

Hi, train lora/dreambooth with animatediff seems to be wrong way. I got pure results with it. The idea was to train multiple frames simultanously. But it led to background overfit without any generalization of training object. Try to train lora traditional way. It will work with animatediff too.