Question about Text-to-Image Pretraining

[Shuaizhang7]

The paper states that the training phase is divided into Text-to-Image Pre-training and Text-to-Video Pre-training. I would like to know how to use image data to train 3D full attention, since image data has one less dimension than video data? Can the 3D Full Attention trained with image data continue to be trained with video data without modification?

[TikhonovDongqiudi]

Thanks for your question. Actually the Text-to-Image Pre-training phase uses the 2D attention and then it switches into 3D attention in the Text-to-Video Pre-training phase.