Skip to content

Latest commit

 

History

History
71 lines (55 loc) · 1.97 KB

README.md

File metadata and controls

71 lines (55 loc) · 1.97 KB
Raw

G-Transformer

This code is for ACL 2021 paper G-Transformer for Document-level Machine Translation.

Python Version: Python3.6

Package Requirements: torch==1.4.0 tensorboardX numpy==1.19.0

Framework: Our model and experiments are built upon fairseq. We use a snapshot version between 0.9.0 and 1.10.0 as our initial code.

Before running the scripts, please install fairseq dependencies by:

    pip install --editable .

Please also follow the readmes under folder raw_data and mbart.cc25 to download raw data and pretrained model. (Notes: Our models were trained on 4 GPUs. If you trained them on 2 GPUs, in theory you could double the number for argument --update-freq. However, we haven't tested such settings.)

Non-pretraining Settings

G-Transformer random initialized

  • Prepare data:
    mkdir exp_randinit
    bash exp_gtrans/run-all.sh prepare-randinit exp_randinit
  • Train model:
    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-randinit train exp_randinit
  • Evaluate model:
    bash exp_gtrans/run-all.sh run-randinit test exp_randinit

G-Transformer fine-tuned on sent Transformer

  • Prepare data:
    mkdir exp_finetune
    bash exp_gtrans/run-all.sh prepare-finetune exp_finetune
  • Train model:
    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-finetune train exp_finetune
  • Evaluate model:
    bash exp_gtrans/run-all.sh run-finetune test exp_finetune

Pretraining Settings

G-Transformer fine-tuned on mBART25

  • Prepare data:
    mkdir exp_mbart
    bash exp_gtrans/run-all.sh prepare-mbart exp_mbart
  • Train model:
    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-mbart train exp_mbart
  • Evaluate model:
    bash exp_gtrans/run-all.sh run-mbart test exp_mbart