add gptq and awq int4 support in intel platform

[sywangyi]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[sywangyi]

@Narsil @danieldk please help review.

[sywangyi]

@ErikKaum could you help review the PR?

[ErikKaum]

Hi @sywangyi 👋

Yes, let me run the tests in a separate branch so that we don't get the permission errors 👍 I should have time to do it today or tomorrow 👍

[sywangyi]

@ErikKaum @Narsil upload fix for ci, please rerun the ci

[ErikKaum]

@sywangyi there seems still to be an error in the dockerfile:

Dockerfile_intel:154
--------------------
 152 |     RUN git clone https://github.com/intel/intel-extension-for-pytorch && cd intel-extension-for-pytorch && git checkout f86e93e4890dc2c989024d148d415c9aa8a1649f
 153 |     RUN git clone https://github.com/intel/torch-ccl.git && cd torch-ccl && git checkout v2.4.0+cpu+rc0
 154 | >>> RUN cd intel-extension-for-pytorch && git submodule sync && git submodule update --init --recursive && python setup.py install
 155 |     RUN cd torch-ccl && git submodule sync && git submodule update --init --recursive && pip install .
 156 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cd intel-extension-for-pytorch && git submodule sync && git submodule update --init --recursive && python setup.py install" did not complete successfully: exit code: 1

[sywangyi]

@ErikKaum Could you help to retrigger the CI build/for intel-cpu, since we did not see the build error in previous ci and I have not made any change to the Dockerfile_intel in the new commits

[sywangyi]

will rework it after #2517 is merged. since python is upgraded from 3.10 to 2.11

[sywangyi]

@ErikKaum rebase done, please retrigger the CI, review and merge it.

[sywangyi]

seems the failure is not related with the PR
ERROR integration-tests/models/test_flash_medusa.py::test_flash_medusa_simple - RuntimeError: Launcher crashed
ERROR integration-tests/models/test_flash_medusa.py::test_flash_medusa_all_params - RuntimeError: Launcher crashed
ERROR integration-tests/models/test_flash_medusa.py::test_flash_medusa_load - RuntimeError: Launcher crashed

[sywangyi]

@ErikKaum could you help retrigger it?

[sywangyi]

this PR is also needed to make mllama output correct in ipex-cpu. since it will upgrade ipex. could anyone help merge it?

[sywangyi]

@ErikKaum @Narsil ,please help. @yao-matrix

[Narsil]

Can you explain why this is needed exactly? Probably as comment in code too.

Your code seems correct because this should be about sharding alignment.
However since desc_act should be checked before the fact that this pathway is failing seems to indicate that something is wrong with the target model possibly.

[sywangyi]

If desc_act is False, exllama should be used. But in sharding case. if g_idx is not in ascend order, use_exllama is set to False.
which means for model like https://huggingface.co/TheBloke/Llama-2-7B-Chat-GPTQ, it will not use exllama in TP case in previous logic. now ipex also implements function like exllama. but exllama kernel does not support intel cpu/xpu.

[Narsil]

We should keep use_exllama here.

Exllama is a really specific kernel. Purely about semantics it's easier for us to know what this is exllama specific.

[sywangyi]

use_exllama is False since exllama only support cuda. but ipex quantization runtime kernel also implement similar logic like exllama. so we need such sharded logic for qzeros/scale/g_idx if desc_act is False as well.

[Narsil]

Isn't keeping use_exllama and simply fixing the TP (with - g_idx[0]) in the conditional to fix the issues on IPEX ?

[sywangyi]

https://github.com/huggingface/text-generation-inference/blob/main/server/text_generation_server/layers/gptq/__init__.py#L134,this line set the use_exllama to false, since in intel platform, exllama is not installed.

[Narsil]

IT's merged from an updated PR I prepared for CI (#2665) (only minor fixes have been updated in the control flow and adding a few comments).