Remote file storage setup with DockerFlowRunner

[tanguymartinez]

Hello,

I have been trying to set up a remote File Storage so that I can test docker deployments.
For testing, I have set up a local sftp server. I can connect using a specific username/password combination.
I have run the command prefect storage create and chosen the option 1).
The base_path I am using is: sftp://username:password@myipaddress/. I don't get any error during the setup.
The problem arises when I create a deployment:

Loading deployment specifications from python script at 'harp/flows.py'...
/usr/lib/python3/dist-packages/paramiko/transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
Packaging flow for deployment 'harp/harp-deployment'...
Traceback (most recent call last):
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/cli/deployment.py", line 244, in create
    await _create(spec)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/client.py", line 105, in with_injected_client
    return await fn(*args, **kwargs)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/cli/deployment.py", line 267, in _create
    deployment_create = await spec._packager.package(spec)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/client.py", line 105, in with_injected_client
    return await fn(*args, **kwargs)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/packaging/script.py", line 119, in package
    storage_token = await self.storage.write(flow_bytes)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/blocks/storage.py", line 148, in write
    await run_sync_in_worker_thread(self._write_sync, ff, data)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/utilities/asyncio.py", line 53, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(call, cancellable=True)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/anyio/to_thread.py", line 28, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread
    return await future
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 754, in run
    result = context.run(func, *args)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/prefect/blocks/storage.py", line 156, in _write_sync
    with ff as io:
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/fsspec/core.py", line 103, in __enter__
    f = self.fs.open(self.path, mode=mode)
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/fsspec/spec.py", line 1009, in open
    f = self._open(
  File "/homel/tmartinez/.local/lib/python3.8/site-packages/fsspec/implementations/sftp.py", line 147, in _open
    f = self.ftp.open(path, mode, bufsize=block_size if block_size else -1)
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 372, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 813, in _request
    return self._read_response(num)
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 865, in _read_response
    self._convert_status(msg)
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 896, in _convert_status
    raise IOError(errno.EACCES, text)
PermissionError: [Errno 13] Permission denied

Failed to create deployment!
Failed to create 1 out of 1 deployments.

I understand that the File Storage uses the fsspec package.
Could you tell me how I can debug the storage created using the prefect cli?
From the docs, it is not clear how one is supposed to pass optional parameters (such as credentials).
The useful bit reads:

Credentials for external services will be retrieved.

but it does not state how.
I stumbled upon this page: https://orion-docs.prefect.io/api-ref/prefect/blocks/storage/#prefect.blocks.storage.FileStorageBlock.
More specifically, in the source code of the FileStorageBlock class, there is an attribute called options which seems to be what I am interested in.
I saw that fsspec accepts kwargs parameters for connections (link):

 ssh_kwargs: dict
            Parameters passed on to connection. See details in
            http://docs.paramiko.org/en/2.4/api/client.html#paramiko.client.SSHClient.connect
            May include port, username, password...
        """

Therefore, is there any way I can pass a dictionary during the creation of new storage ?

I might be going in the wrong direction.

Thank you very much!