Add config system to export_llama

[iseeyuan]

Summary

The configuration system in export_llama is limited

• only via cli command. There's no python API. We cannot export an arbitrary nn.Module
• the cli args are complicated. It's hard for users to get what can be configured

Add yaml file option for export_llama configuration.

• Move all default values from command line args to config/default.yaml
• Add --config option to load user defined configuration yaml file
• To keep it backward compatible, allow cli args override the configs in the yaml file

Test plan

Reference run:

python3 -m examples.models.llama.export_llama -c stories110M.pt -p params.json -d fp32 -n tinyllama_xnnpack+custom_fp32_main.pte -kv -X --xnnpack-extended-ops -qmode 8da4w -G 128 --use_sdpa_with_kv_cache --output-dir tmp

With config:

python3 -m examples.models.llama.export_llama --config examples/models/llama/config/examples/stories.yaml

Those two commands should provide the same .pte file and used_config.yaml

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9367

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 18 New Failures, 2 Cancelled Jobs, 3 Unrelated Failures

As of commit bd471d2 with merge base f789df2 ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner / linux-job (gh)
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 8f37e8b125b2bf3fcbab3277a6cb48ae3f52af0da5b19c1bc7aa37020cc7af93 /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 7f9da05e9cbf4f313e0a80668f42484793a76c9224f082824dc98122c1d21d4c /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 1182e5c8d2d5b2bbb54d3ed397a1f1dba7d5f04b7442e198ff48d8d364c2ce7a /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 34103b0b6436f107224c7ba558876774e916f091096722b05651ab2fbbef1ee6 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 7c9a960ed48c13d3bbeca0b774b296ecf1dc19274ec977ef885c9451c909231b /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  test_llava_export
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t ea414046adaef72f1333400600fa0380f4616e3077fc2141f3b506440e7bb275 /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 676c19d3087fba79d3996052b64e5f89812bd312bb0f6e89b07071a0b111de8e /exec failed with exit code 3
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5c06a67129675e4ef85b53d89e2dc89a5c6291a7abc86a8f955bb126d2b1166c /exec failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 4dd3b7500620754f07dd66ef3e24182f4a1eeeb8818dd55ecc6d9bcc81175868 /exec failed with exit code 1
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t c3224ca74595fc4f9706ccd0572b6c6f2b6de2254924178f0222ba84c3b55ccf /exec failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 38d90f1feb081b3c55145ad118774f49911424b6b4fcc5697ad67680ec6117e0 /exec failed with exit code 1
• trunk / unittest-release / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• trunk / test-apple-model / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-selective-build-macos (cmake) / macos-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• trunk / test-arm-reference-delegation / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-buck / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3
• trunk / test-arm-backend-delegation / linux-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[iseeyuan]

@pytorchbot label "topic: not user facing"