Added OLMo support to builder.py

[shobrienDMA]

No description provided.

[shobrienDMA]

@microsoft-github-policy-service agree company="AMD"

[BowenBao]

@kunal-vaishnavi ptal, thanks!

[kunal-vaishnavi]

Thanks for the contribution! Does OLMo run end-to-end with the ONNX Runtime GenAI tokenizer?

Can you also update the following places?

1. Add OLMo to the repo README and the model builder README to show that OLMo is now supported
2. Add OLMo to the CIs

onnxruntime-genai/test/python/_test_utils.py

Lines 55 to 77 in 0f59a90

	def get_model_paths():
	hf_paths = {
	"phi-2": "microsoft/phi-2",
	# "phi-3-mini": "microsoft/Phi-3-mini-128k-instruct",
	}
	ci_data_path = os.path.join("/", "data", "ortgenai_pytorch_models")
	if not os.path.exists(ci_data_path):
	return {}, hf_paths
	# Note: If a model has over 4B parameters, please add a quantized version
	# to `ci_paths` instead of `hf_paths` to reduce file size and testing time.
	ci_paths = {
	"llama-2": os.path.join(ci_data_path, "Llama-2-7B-Chat-GPTQ"),
	"llama-3": os.path.join(ci_data_path, "Meta-Llama-3-8B-AWQ"),
	"mistral-v0.2": os.path.join(ci_data_path, "Mistral-7B-Instruct-v0.2-GPTQ"),
	# "phi-2": os.path.join(ci_data_path, "phi2"),
	# "gemma-2b": os.path.join(ci_data_path, "gemma-1.1-2b-it"),
	"gemma-7b": os.path.join(ci_data_path, "gemma-7b-it-awq"),
	# "phi-3-mini": os.path.join(ci_data_path, "phi3-mini-128k-instruct"),
	}
	return ci_paths, hf_paths

The models in hf_paths are downloaded from Hugging Face, and the models in ci_paths are currently uploaded to /data/ortgenai_pytorch_models in the Linux CUDA CI VM.

onnxruntime-genai/.github/workflows/linux-gpu-x64-build.yml

Line 122 in 0f59a90

--volume /data/ortgenai_pytorch_models:/data/ortgenai_pytorch_models \

You can add it to hf_paths for now. If you can also add Qwen to the CIs, that would be helpful.

[shobrienDMA]

That is be updated as requested now. It runs end to end and I've also added Qwen to the CI list.

[kunal-vaishnavi]

Thank you for adding the changes. The end-to-end tests in the CIs appear to be failing due to the transformers version. Can you pin it to v4.44.2?

onnxruntime-genai/test/python/requirements.txt

Line 9 in 9055e68

transformers

[shobrienDMA]

This should be good to go!

[kunal-vaishnavi]

After some further investigation, it appears that the tokenizer CI failure is happening because the tokenizer for OLMo is not currently supported in ONNX Runtime Extensions. Once the support is added, the main branch of ONNX Runtime GenAI can be merged into this PR to integrate the changes.

[kunal-vaishnavi]

The ONNX Runtime Extensions PR has been merged now. You can update its commit ID in deps.txt to pull in those changes.

onnxruntime-genai/cmake/deps.txt

Line 17 in 41c2543

onnxruntime_extensions;https://github.com/microsoft/onnxruntime-extensions.git;4e10ee046a2f035351f3fe88740bd8215a18fdb9

[shobrienDMA]

Hi @kunal-vaishnavi thanks for letting me know, I've bumped that dependency.