prefect deploy exits with error 'charmap' codec can't encode character '...' in position 0: chacarter maps to <undefined>

[khsater]

Bug summary

Background:

The following error appears when running prefect deploy in a relatively fresh installation on Windows Server 2019 (actual character may vary according to search hits on this):
'charmap' codec can't encode character '\u280b' in position 0: character maps to <undefined>

I installed prefect via a PowerShell script on Windows Server 2019. During the installation process, I occasionally ran across character encoding issues, e.g. when writing .bat files for launch scripts. This suggests that the error may have occurred due to installing via PowerShell. However, I have not been able to identify the offending file.

Investigation:

Running Procmon against python and prefect when this error occurs shows that the last files to be accessed include:

• prefect.db-shm
• .\Lib\site-packages\anyio_backends_pycache__init_.cpython-312.pyc
• .\Lib\site-packages\anyio_backends_pycache__asyncio.cpython-312.pyc
• .\Lib\site-packages\rich_pycache__windows_renderer.cpython-312.pyc

The latter appears to best correspond to the timing of the appearance of the error and exit from prefect deploy.
I found this linen thread which lead to the below workaround
https://linen.prefect.io/t/23209832/hi-i-have-just-starting-using-prefect-on-windows-was-initail

Workaround

The solution referred to above was setting the environment variable:
PYTHONIOENCODING=UTF-8

This is a bit inconvenient since it may not be desirable to set this at the OS or even user level. Preferably, this would be handled as a bug fix or self-configuration by prefect.

For anyone encountering this, you can use the following to temporarily set this in cmd prior to running prefect deploy or any other affected command.

set PYTHONIOENCODING=UTF-8

Version info

Version:             3.1.15
API version:         0.8.4
Python version:      3.12.4
Git commit:          3ac3d548
Built:               Thu, Jan 30, 2025 11:31 AM
OS/Arch:             win32/AMD64
Profile:             dev
Server type:         server
Pydantic version:    2.10.6
Integrations:
  prefect-github:    0.3.1

Additional context

No response

[zzstoatzz]

hi @khsater - thanks for the detailed write up and for including a workaround for other users ❤

do you have a reproduction of this error? i.e. what's the relevant portion of your prefect.yaml look like?

[khsater]

The prefect.yaml is fresh. Considering that it might be these files, I deleted prefect.yaml and .prefectignore and ran:

prefect init
prefect deploy

I also confirmed that neither of the files contained that character at position 0 and both were encoded as UTF-8.
Are there any other files created by init? It's possible I ran that previously in PowerShell and there is a file I haven't checked.

So, the deployments section looks like:

# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: null
  version: null
  tags: []
  description: null
  schedule: {}
  flow_name: null
  entrypoint: null
  parameters: {}
  work_pool:
    name: null
    work_queue_name: null
    job_variables: {}

[khsater]

I forgot to mention - prefect is running in an activated pyenv virtual environment.

[GalLadislav]

Hi have you tried PYTHONUTF8=1 ? We had to set this on our Windows platforms to deal with this (probably the same) error.