Windows wheel build error

[divi0001]

I am getting errors while building deepspeed wheel, i set a whole bunch of options to 0 in cmd before since they were also throwing errors it seems, listing them: DS_BUILD_GDS, DS_BUILD_FP_QUANTIZER, DS_BUILD_EVOFORMER_ATTN, DS_BUILD_AIO
Installing dskernels via pip resulted in
pip install dskernels ERROR: Could not find a version that satisfies the requirement dskernels (from versions: none) ERROR: No matching distribution found for dskernels

The previous python setup.py bdist_wheel resulted in:
[2024-08-30 13:42:30,907] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-08-30 13:42:31,331] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) test.c LINK : fatal error LNK1181: cannot open input file 'aio.lib' test.c LINK : fatal error LNK1181: cannot open input file 'cufile.lib' ...\DeepSpeed\deepspeed\runtime\zero\linear.py:49: FutureWarning: torch.cuda.amp.custom_fwd(args...)is deprecated. Please usetorch.amp.custom_fwd(args..., device_type='cuda')instead. def forward(ctx, input, weight, bias=None): ...\DeepSpeed\deepspeed\runtime\zero\linear.py:67: FutureWarning:torch.cuda.amp.custom_bwd(args...)is deprecated. Please usetorch.amp.custom_bwd(args..., device_type='cuda') instead. def backward(ctx, grad_output): W0830 13:42:33.709243 10348 torch\distributed\elastic\multiprocessing\redirects.py:28] NOTE: Redirects are currently not supported in Windows or MacOs. DS_BUILD_OPS=1 test.c LINK : fatal error LNK1181: cannot open input file 'aio.lib' test.c LINK : fatal error LNK1181: cannot open input file 'cufile.lib' [WARNING] Filtered compute capabilities ['6.0', '6.1', '7.0'] Traceback (most recent call last): File "...\DeepSpeed\setup.py", line 197, in <module> ext_modules.append(builder.builder()) File "...\DeepSpeed\op_builder\builder.py", line 719, in builder extra_link_args=self.strip_empty_entries(self.extra_ldflags())) File "...\DeepSpeed\op_builder\inference_cutlass_builder.py", line 74, in extra_ldflags import dskernels ModuleNotFoundError: No module named 'dskernels'

For my specs/installed requirements, i have a relatively new intel CPU, RTX 4070, CUDA v.11.1 and VS22 C++ Desktop installed

[loadams]

Hi @divi0001 - it looks like you are using Windows, is that correct? If so, we recommend using the pre-built wheel that we provide on PyPI, so you can install via just pip install deepspeed. If you prefer to build from source, the directions are listed here. Additionally, dskernels is linux only because the cutlass kernels it depends on are also linux only.

[divi0001]

Yes, it is windows 10.

When i use pip install deepseed, it's basically the same:

pip install deepspeed
Collecting deepspeed
Using cached deepspeed-0.15.0.tar.gz (1.4 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [23 lines of output]
[2024-08-30 17:54:09,407] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-08-30 17:54:10,484] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
W0830 17:54:14.593891 3928 torch\distributed\elastic\multiprocessing\redirects.py:28] NOTE: Redirects are currently not supported in Windows or MacOs.
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "...\AppData\Local\Temp\pip-install-xbe_71oj\deepspeed_69f87a5d35374266b64c4a041ab849a8\setup.py", line 197, in
ext_modules.append(builder.builder())
File "...\AppData\Local\Temp\pip-install-xbe_71oj\deepspeed_69f87a5d35374266b64c4a041ab849a8\op_builder\builder.py", line 719, in builder
extra_link_args=self.strip_empty_entries(self.extra_ldflags()))
File "...\AppData\Local\Temp\pip-install-xbe_71oj\deepspeed_69f87a5d35374266b64c4a041ab849a8\op_builder\inference_cutlass_builder.py", line 74, in extra_ldflags
import dskernels
ModuleNotFoundError: No module named 'dskernels'
DS_BUILD_OPS=1
[WARNING] Filtered compute capabilities ['6.0', '6.1', '7.0']
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[loadams]

@divi0001 - it looks like that is selecting the linux wheel, you need to ensure your system is pulling in the windows whl (linked here. You can download and install that way if you need as well.

[loadams]

And confirming if you build from source that you set all of the following to 0?

set DS_BUILD_AIO=0
set DS_BUILD_CUTLASS_OPS=0
set DS_BUILD_EVOFORMER_ATTN=0
set DS_BUILD_FP_QUANTIZER=0
set DS_BUILD_RAGGED_DEVICE_OPS=0
set DS_BUILD_SPARSE_ATTN=0

[divi0001]

DS_BUILD_GDS,
DS_BUILD_FP_QUANTIZER,
DS_BUILD_EVOFORMER_ATTN,
DS_BUILD_AIO

those are set to 0, but only via cmd with set DS_NAME_GOES_HERE=0
(the cmd is still open, i read that this is non permanent)

[loadams]

Yes, that is correct, but you can have it all run in a single file if you run it with build_win.bat.

[divi0001]

alright. I'm going to look for prebuilt wheel then, the one you linked didn't work (wrong platform). Thanks for the help :)

[loadams]

What platform are you using? I believe it should just be the python version? Pip should pick it up if you are using python 3.11

[divi0001]

python 3.10.0
nvidia

[loadams]

Can you try with python 3.11 since that's what the whl is built wtih?

[divi0001]

worked with 3.11 thank you! Any chance 3.10 will be supported with this architecture in the future?

[loadams]

Possibly, we would need to determine how many different configurations we will support, but we haven't not decided on that yet.

[Kas1o]

I am using python 3.11.9 on windows 10, pip install deepspeed still downloads a linux wheel. cause same problem above.

[loadams]

@Kas1o - what cuda version? Can you also confirm the DeepSpeed version you are downloading/specifying? Please try 0.15.0, or download the whl listed here and pip install it?

https://pypi.org/project/deepspeed/0.15.0/#files

[erew123]

Hi @loadams

Re DeepSpeed Windows wheel file.

I'm on:

Python Version : 3.11.10
PyTorch Version : 2.2.1
CUDA Version : 12.1

• If you pip install deepspeed on Windows at a command prompt, I am guessing it downloads the linux build? So maybe it needs a slightly different name to be published under or some information in the way its published to pip? (Ive never done that before so dont know how platform identification works with pip requests). Either way, that build results in:

Collecting deepspeed
  Downloading deepspeed-0.15.1.tar.gz (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 8.9 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      [2024-10-04 00:41:54,029] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
      [2024-10-04 00:41:57,385] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
      test.c
      LINK : fatal error LNK1181: cannot open input file 'aio.lib'
      test.c
      LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
      [2024-10-04 00:42:04,070] torch.distributed.elastic.multiprocessing.redirects: [WARNING] NOTE: Redirects are currently not supported in Windows or MacOs.
      test.c
      LINK : fatal error LNK1181: cannot open input file 'aio.lib'
      test.c
      LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\useraccount\AppData\Local\Temp\pip-install-d_bzkejb\deepspeed_0b3a659d00484122b71ee8d88dadd606\setup.py", line 198, in <module>
          ext_modules.append(builder.builder())
                             ^^^^^^^^^^^^^^^^^
        File "C:\Users\useraccount\AppData\Local\Temp\pip-install-d_bzkejb\deepspeed_0b3a659d00484122b71ee8d88dadd606\op_builder\builder.py", line 720, in builder
          extra_link_args=self.strip_empty_entries(self.extra_ldflags()))
                                                   ^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\useraccount\AppData\Local\Temp\pip-install-d_bzkejb\deepspeed_0b3a659d00484122b71ee8d88dadd606\op_builder\inference_cutlass_builder.py", line 74, in extra_ldflags
          import dskernels
      ModuleNotFoundError: No module named 'dskernels'
      DS_BUILD_OPS=1
       [WARNING]  Skip pre-compile of incompatible async_io; One can disable async_io with DS_BUILD_AIO=0
       [WARNING]  Skip pre-compile of incompatible evoformer_attn; One can disable evoformer_attn with DS_BUILD_EVOFORMER_ATTN=0
       [WARNING]  Skip pre-compile of incompatible fp_quantizer; One can disable fp_quantizer with DS_BUILD_FP_QUANTIZER=0
       [WARNING]  Skip pre-compile of incompatible gds; One can disable gds with DS_BUILD_GDS=0
       [WARNING]  Filtered compute capabilities ['6.0', '6.1', '7.0']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

When you manually download the version you provided from your link and install that with pip install deepspeed-0.15.0-cp311-cp311-win_amd64.whl, the installation does go through ok:

However, when a python script imports deepspeed, you do get the 2x following errors:

test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'

Just so you have a clear understanding of my OS:

OPERATING SYSTEM:
OS Version: Windows 10.0.22631 (so Windows 11)
CUDA:
CUDA Working: Success - CUDA is available and working.
CUDA Device : NVIDIA GeForce RTX 4070
CUDA Memory : 11.99 GB
CUDA Version: 12.1
Windows C++ Build tools & Windows SDK:
Windows Version: Windows 11
Visual C++ Build Tools and/or Visual Studio found:
Visual Studio 2019 Build Tools (Path: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools)
Windows SDK(s) found:
SDK Version: 10.0.22000.0
Path: C:\Program Files (x86)\Windows Kits\10
Python setuptools version: 75.1.0
PYTHON & PYTORCH:
Torch Version: 2.2.1
Python Version: 3.11.10
Python Version Info: sys.version_info(major=3, minor=11, micro=10, releaselevel='final', serial=0)
CUBLAS & CUDNN: (Basically 11.8 of NVidia Toolkit installed as Python Packages)
nvidia-cublas-cu11: 11.11.3.6
nvidia-cuda-nvrtc-cu11: 11.8.89
nvidia-cudnn-cu11: 9.4.0.58

So not sure if these errors are problems in the v15 pip package? Also it may be handy to have the link to the updated whl files and manual install instructions within the readme for Windows DeepSpeed https://github.com/microsoft/DeepSpeed/blob/master/blogs/windows/08-2024/README.md

Hope that helps you internally figure what needs changing or feed back here/on the readme about what we may need to do (or if we should be ignoring those errors).

Thanks

[erew123]

@loadams

In addition to my above post, I assume the whl file is built only for Pytorch 2.3 as despite installing fine, when I use it, I do get the following error:

RuntimeError: PyTorch version mismatch! DeepSpeed ops were compiled and installed with a different version than what is being used at runtime. Please re-install DeepSpeed or switch torch versions. Install torch version=2.3, Runtime torch version=2.2

Though the whl being built only for PyTorch 2.3 isn't clearly stated in the readme https://github.com/microsoft/DeepSpeed/blob/master/blogs/windows/08-2024/README.md

Also I it may be handy to have to front page Github instructions for Windows link to that readme. https://github.com/microsoft/DeepSpeed?tab=readme-ov-file#windows

Thanks

[loadams]

Hi @erew123 - yes that is correct, I can update the readme with that clearer information. We also need to publish the 0.15.1 whl for Windows since that may be part of the issue as well.

To make sure I understand the state of your install, you're able to install the specific whl file now, but you're getting issues with the RuntimeError on the python version, and you're having issues with those .lib files?

Are you able to share the output of ds_report from that install?

[erew123]

Hi @loadams

Thanks for the reply! Ive managed to get the ds_report remotely from my machine, though I am now away travelling for a while, so will have limited access to my machine and to respond, but I will respond when I can.

Just to remind, my Torch on this machine Windows 11/python environment is 2.2.1 and the wheel (And ds_report at the bottom) does say the wheel is compiled for Torch 2.3 and CUDA 12.1. I know from the amount of DeepSpeed wheels Ive compiled over the last year you have to compile for the specific verison of Torch and CUDA... but I dont know if the

test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'

mesages are becuase of it being a different torch version. My full machine specs are on the above post and my ds_report is below.

ds_report
[2024-10-04 22:20:58,550] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
[2024-10-04 22:21:04,069] torch.distributed.elastic.multiprocessing.redirects: [WARNING] NOTE: Redirects are currently not supported in Windows or MacOs.
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
fused_adam ............. [YES] ...... [OKAY]
cpu_adam ............... [YES] ...... [OKAY]
cpu_adagrad ............ [YES] ...... [OKAY]
cpu_lion ............... [YES] ...... [OKAY]
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
evoformer_attn ......... [NO] ....... [NO]
 [WARNING]  please install triton==2.3.0 or 2.3.1 if you want to use the FP Quantizer Kernels
fp_quantizer ........... [NO] ....... [NO]
fused_lamb ............. [YES] ...... [OKAY]
fused_lion ............. [YES] ...... [OKAY]
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
gds .................... [NO] ....... [NO]
inference_core_ops ..... [YES] ...... [OKAY]
cutlass_ops ............ [NO] ....... [OKAY]
transformer_inference .. [YES] ...... [OKAY]
quantizer .............. [YES] ...... [OKAY]
ragged_device_ops ...... [NO] ....... [OKAY]
ragged_ops ............. [YES] ...... [OKAY]
random_ltd ............. [YES] ...... [OKAY]
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.2
 [WARNING]  please install triton==1.0.0 if you want to use sparse attention
sparse_attn ............ [NO] ....... [NO]
spatial_inference ...... [YES] ...... [OKAY]
transformer ............ [YES] ...... [OKAY]
stochastic_transformer . [YES] ...... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['E:\\installtest\\alltalk_tts\\alltalk_environment\\env\\Lib\\site-packages\\torch']
torch version .................... 2.2.1
deepspeed install path ........... ['E:\\installtest\\alltalk_tts\\alltalk_environment\\env\\Lib\\site-packages\\deepspeed']
deepspeed info ................... 0.15.0, 55b4cae8, HEAD
torch cuda version ............... 12.1
torch hip version ................ None
nvcc version ..................... 12.1
deepspeed wheel compiled w. ...... torch 2.3, cuda 12.1
shared memory (/dev/shm) size .... UNKNOWN

Thanks

[FurkanGozukara]

@Kas1o - what cuda version? Can you also confirm the DeepSpeed version you are downloading/specifying? Please try 0.15.0, or download the whl listed here and pip install it?

https://pypi.org/project/deepspeed/0.15.0/#files

are there any wheels for 3.10?

i have everything torch cuda c++ anything but still fails

[notice] A new release of pip is available: 23.0.1 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
(venv) G:\EasyAnimate_v1\EasyAnimate\venv\Scripts>pip install deepspeed
Collecting deepspeed
  Using cached deepspeed-0.15.4.tar.gz (1.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      [2024-11-12 22:32:27,535] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)
      [2024-11-12 22:32:28,458] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)
      test.c
      LINK : fatal error LNK1181: cannot open input file 'aio.lib'
      test.c
      LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\Furkan\AppData\Local\Temp\pip-install-7vc2npw3\deepspeed_93d6e2948b094fefb9d3b284ff03a34f\setup.py", line 200, in <module>
          ext_modules.append(builder.builder())
        File "C:\Users\Furkan\AppData\Local\Temp\pip-install-7vc2npw3\deepspeed_93d6e2948b094fefb9d3b284ff03a34f\op_builder\builder.py", line 720, in builder
          extra_link_args=self.strip_empty_entries(self.extra_ldflags()))
        File "C:\Users\Furkan\AppData\Local\Temp\pip-install-7vc2npw3\deepspeed_93d6e2948b094fefb9d3b284ff03a34f\op_builder\inference_cutlass_builder.py", line 74, in extra_ldflags
          import dskernels
      ModuleNotFoundError: No module named 'dskernels'
      DS_BUILD_OPS=1
       [WARNING]  Skip pre-compile of incompatible async_io; One can disable async_io with DS_BUILD_AIO=0
       [WARNING]  Skip pre-compile of incompatible evoformer_attn; One can disable evoformer_attn with DS_BUILD_EVOFORMER_ATTN=0
       [WARNING]  Skip pre-compile of incompatible fp_quantizer; One can disable fp_quantizer with DS_BUILD_FP_QUANTIZER=0
       [WARNING]  Skip pre-compile of incompatible gds; One can disable gds with DS_BUILD_GDS=0
       [WARNING]  Filtered compute capabilities ['6.0', '6.1', '7.0']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0.1 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip

(venv) G:\EasyAnimate_v1\EasyAnimate\venv\Scripts>

[loadams]

Hi @FurkanGozukara - there are no wheels published for 3.10 yet, the latest whl we have published is 3.11.

But your installation is still picking the linux whl, you would need python 3.11 and to specify this DeepSpeed version to get the whl here: https://pypi.org/project/deepspeed/0.15.0/#files

[FurkanGozukara]

Hi @FurkanGozukara - there are no wheels published for 3.10 yet, the latest whl we have published is 3.11.

But your installation is still picking the linux whl, you would need python 3.11 and to specify this DeepSpeed version to get the whl here: https://pypi.org/project/deepspeed/0.15.0/#files

how do i make it get windows file? can you elaborate more a pip install command perhaps?

[FurkanGozukara]

@loadams i need to install on python 3.10 on windows any help appreciated thank you

can't you release a whl for python 3.10? for windows?

[loadams]

To use the whl file I linked you would need python 3.11 - but if your system is Windows and has python 3.11, it should just be:

pip install deepspeed==0.15.0

This should look at what your system supports and grab that whl. If you want to try building from source, from that commit, but with python 3.10, that should work as well with the build_win.bat command.

[FurkanGozukara]

@loadams can you please publish python 3.10 wheel i dont know how to make

[erew123]

@FurkanGozukara These are manual build instructions here https://github.com/erew123/deepspeedpatcher?tab=readme-ov-file#manual-builds-of-deepspeed-0150-and-later These are written by myself, not Microsoft. I provide no support/assistance on this or the process and I doubt Microsoft will provide you support on my instructions. However, that will guide you though a manual build, or you can try the tool there.

[loadams]

@FurkanGozukara - we need to make some changes to our publishing pipeline to support this, we hope to do that soon but have to balance all the work items. We are aware of this and working on having that published.

Thanks @erew123!

[FurkanGozukara]

@loadams we are looking forward to python 3.10

[FurkanGozukara]

Yes, that is correct, but you can have it all run in a single file if you run it with build_win.bat.

Microsoft Windows [Version 10.0.22631.4541]
(c) Microsoft Corporation. All rights reserved.

C:\DeepSpeed>build_win.bat
DS_BUILD_OPS=1
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
test.c
LINK : fatal error LNK1181: cannot open input file 'cufile.lib'
[WARNING] Skip pre-compile of incompatible gds; One can disable gds with DS_BUILD_GDS=0
Install Ops={'async_io': False, 'fused_adam': 1, 'cpu_adam': 1, 'cpu_adagrad': 1, 'cpu_lion': 1, 'evoformer_attn': False, 'fp_quantizer': False, 'fused_lamb': 1, 'fused_lion': 1, 'gds': False, 'transformer_inference': 1, 'inference_core_ops': 1, 'cutlass_ops': False, 'quantizer': 1, 'ragged_device_ops': False, 'ragged_ops': 1, 'random_ltd': 1, 'sparse_attn': False, 'spatial_inference': 1, 'transformer': 1, 'stochastic_transformer': 1}
Traceback (most recent call last):
File "C:\DeepSpeed\setup.py", line 209, in
result = subprocess.check_output(git_hash_cmd)
File "C:\Python310\lib\subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "C:\Python310\lib\subprocess.py", line 503, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Python310\lib\subprocess.py", line 971, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Python310\lib\subprocess.py", line 1456, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
C:\DeepSpeed>

[FurkanGozukara]

@FurkanGozukara These are manual build instructions here https://github.com/erew123/deepspeedpatcher?tab=readme-ov-file#manual-builds-of-deepspeed-0150-and-later These are written by myself, not Microsoft. I provide no support/assistance on this or the process and I doubt Microsoft will provide you support on my instructions. However, that will guide you though a manual build, or you can try the tool there.

I did huge testing but not working yet :(

erew123/deepspeedpatcher#3

[erew123]

@FurkanGozukara As per my manual build instructions, I state you need to build from the Visual Studio Developer console. Looking above you have just run a standard command prompt. It should look like this before you start a manual build.

[FurkanGozukara]

@erew123 thank you so much i just did as you instructed

running all commands below in single administrator cmd session

here my pip freeze - i have cuda paths set

Microsoft Windows [Version 10.0.22631.4541]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Furkan>pşğ
'pşğ' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Furkan>pip freeze
ae-ffmpeg==1.2.0
aiofiles==24.1.0
aiohappyeyeballs==2.4.3
aiohttp==3.11.7
aiosignal==1.3.1
anyio==4.6.2.post1
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asttokens==2.4.1
async-lru==2.0.4
async-timeout==5.0.1
attrs==24.2.0
auto-editor==25.3.0
babel==2.16.0
beautifulsoup4==4.12.3
bleach==6.1.0
Brotli==1.1.0
bs4==0.0.2
cachetools==5.5.0
certifi==2024.8.30
cffi==1.17.1
charset-normalizer==3.3.2
colorama==0.4.6
comm==0.2.2
cryptography==44.0.0
debugpy==1.8.7
decorator==4.4.2
defusedxml==0.7.1
discord==2.3.2
discord.py==2.4.0
exceptiongroup==1.2.2
executing==2.1.0
fastjsonschema==2.20.0
ffmpeg==1.4
ffmpeg-python==0.2.0
filelock==3.13.1
fire==0.7.0
fqdn==1.5.1
frozenlist==1.5.0
fsspec==2024.2.0
future==1.0.0
GPUtil==1.4.0
h11==0.14.0
hf_transfer==0.1.8
httpcore==1.0.6
httpx==0.27.2
huggingface-hub==0.26.2
idna==3.10
imageio==2.36.1
imageio-ffmpeg==0.5.1
imutils==0.5.4
inputimeout==1.0.4
ipykernel==6.29.5
ipython==8.28.0
ipywidgets==8.1.5
iso8601==2.1.0
isoduration==20.11.0
jedi==0.19.1
Jinja2==3.1.3
json5==0.9.25
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jupyter-events==0.10.0
jupyter-lsp==2.2.5
jupyter_client==8.6.3
jupyter_core==5.7.2
jupyter_server==2.14.2
jupyter_server_terminals==0.5.3
jupyterlab==4.2.5
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab_widgets==3.0.13
llvmlite==0.43.0
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.7
mdurl==0.1.2
mistune==3.0.2
more-itertools==10.5.0
moviepy==1.0.3
mpmath==1.3.0
multidict==6.1.0
mutagen==1.47.0
natsort==8.4.0
nbclient==0.10.0
nbconvert==7.16.4
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==3.2.1
ninja==1.11.1.2
notebook_shim==0.2.4
numba==0.60.0
numpy==1.26.3
nvidia-ml-py==12.535.161
nvitop==1.3.2
openai-whisper @ git+https://github.com/openai/whisper.git@90db0de1896c23cbfaf0c58bc2d30665f709f170
opencv-contrib-python==4.10.0.84
opencv-python==4.10.0.84
overrides==7.7.0
packaging==24.1
pandocfilters==1.5.1
parso==0.8.4
pillow==10.2.0
pillow_heif==0.20.0
platformdirs==4.3.6
proglog==0.1.10
prometheus_client==0.21.0
prompt_toolkit==3.0.48
propcache==0.2.0
psutil==6.0.0
pure_eval==0.2.3
py-cpuinfo==9.0.0
pyav==13.0.0
pycparser==2.22
pycryptodomex==3.21.0
Pygments==2.18.0
PyNaCl==1.5.0
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-json-logger==2.0.7
pywin32==308
pywinpty==2.0.14
PyYAML==6.0.2
pyzmq==26.2.0
referencing==0.35.1
regex==2024.9.11
requests==2.32.3
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.2
rpds-py==0.20.0
safetensors==0.4.5
Send2Trash==1.8.3
sentencepiece==0.2.0
six==1.16.0
sniffio==1.3.1
soupsieve==2.6
stack-data==0.6.3
sympy==1.13.1
termcolor==2.4.0
terminado==0.18.1
tiktoken==0.7.0
tinycss2==1.3.0
tokenizers==0.20.0
tomli==2.0.2
torch==2.5.0+cu124
torchao==0.7.0
torchaudio==2.5.0+cu124
torchvision==0.20.0+cu124
tornado==6.4.1
tqdm==4.66.5
traitlets==5.14.3
transformers==4.45.1
triton @ https://github.com/woct0rdho/triton-windows/releases/download/v3.1.0-windows.post5/triton-3.1.0-cp310-cp310-win_amd64.whl
twitchio==2.10.0
types-python-dateutil==2.9.0.20241003
typing_extensions==4.9.0
uri-template==1.3.0
urllib3==2.2.3
Wand==0.6.13
wcwidth==0.2.13
webcolors==24.8.0
webencodings==0.5.1
webp==0.4.0
websocket-client==1.8.0
websockets==13.1
widgetsnbextension==4.0.13
windows-curses==2.3.3
WMI==1.5.1
xformers==0.0.28.post2
yarl==1.18.0
yt-dlp==2024.10.7

C:\Users\Furkan>

[FurkanGozukara]

after i fixed the setup py - which was written only for Linux by Microsoft :) i got a compile

fix is like below

# Write out version/git info.
git_hash_cmd = ["git", "rev-parse", "--short", "HEAD"]
git_branch_cmd = ["git", "rev-parse", "--abbrev-ref", "HEAD"]
if command_exists('git') and not is_env_set('DS_BUILD_STRING'):
    try:
        result = subprocess.check_output(git_hash_cmd, shell=True)
        git_hash = result.decode('utf-8').strip()
        result = subprocess.check_output(git_branch_cmd, shell=True)
        git_branch = result.decode('utf-8').strip()
    except subprocess.CalledProcessError:
        git_hash = "unknown"
        git_branch = "unknown"
else:
    git_hash = "unknown"
    git_branch = "unknown"

still compiling i hope it works. how can i verify it is working?

[FurkanGozukara]

it failed after working huge time

i would break my teeth if it was successful :/

cl : Command line warning D9025 : overriding '/D__CUDA_NO_HALF2_OPERATORS__' with '/U__CUDA_NO_HALF2_OPERATORS__'
slice_attn_masks.cu
tmpxft_0000b2e4_00000000-7_slice_attn_masks.compute_90.cudafe1.cpp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\include\cuda_fp16.hpp(992): fatal error C1001: Internal compiler error.
(compiler file 'D:\a\_work\1\s\src\vctools\Compiler\CxxFE\sl\p1\c\token_io.cpp', line 2687)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.4\\bin\\nvcc.exe' failed with exit code 4294967295

entire compile logs 3200 lines

entire logs.txt

[erew123]

@FurkanGozukara That may be an issue of build-win.bat changing these settings:

set DS_BUILD_AIO=0
set DS_BUILD_CUTLASS_OPS=0
set DS_BUILD_EVOFORMER_ATTN=0
set DS_BUILD_FP_QUANTIZER=0
set DS_BUILD_RAGGED_DEVICE_OPS=0
set DS_BUILD_SPARSE_ATTN=0

So you may want to make sure those are set in the build-win.bat.

Beyond that, I may be something with PyTorch being 2.5 or CUDA being 12.4.... That I really dont know and MS would have to provide you assistance on that.

[FurkanGozukara]

@erew123 ty for reply again

bat file is like this do you recommend anything else?

if i compile it on older pytorch like 2.4 do you think it would work on 2.5?

i mean the wheel they provide for python 11 doesnt have certain pytorch version or cuda version :

and this is working with pytorch 2.5 and cuda 12.4 i tested - with python 3.11

[erew123]

@FurkanGozukara Im honestly not a DeepSpeed expert by any means... but I did get it working/compiling on Windows a year or so ago and dug through so many unclear, misunderstood, incorrect instructions, that when I finally got it working, I decide to share my knowledge, which you will find on this post here #4729

I have since had a need to build more versions and I have distilled my knowledge here https://github.com/erew123/deepspeedpatcher for everything I believe/think I know to be correct, but I don't classify myself to be an expert.

I do believe something has changed in the 0.15.x onwards builds of DeepSpeed, in its build routine, but I have not delved into what those changes are. I don't have time to do that. Please see a post here on another project of mine that will give you an insight as to why I dont have time to investigate things like this erew123/alltalk_tts#377. I will say I believe that version 0.15.x onwards you MUST have the Nvidia Toolkit installed as it appears to require access to nvcc from the Toolkit, no matter what. Previous builds didnt appear to require this e.g. DeepSpeed 0.14.x or less

As for your question re can you build for one version and use it on another version, the answer is no you cannot. Why it is built this way I dont know, but you have to precisely match all requirements of the compiled wheel with the Python/PyTorch/CUDA environment it was built on. I explain this here https://github.com/erew123/deepspeedpatcher?tab=readme-ov-file#important-version-compatibility-information

[FurkanGozukara]

@erew123 thanks again

it is so ironic that Microsoft not giving support to Microsoft Windows but giving support to Linux

Any info on this @loadams ?