Spyder freezes when generating a lot of small files

[tyNeanderthal]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Mgiht not be a bug but certainly an issue for me. I tried upgrade conda and Spyder (to 3.3.6), and the issue persists. I attached the sample code that combined with steps mentioned below should reproduce the same issue (on Windows), which is that the Spyder slows down and freeze. Somehow I have to rename it to sample.log, but it is really a .py file.

sample.log

What steps reproduce the problem?

1. Create a new project (say, E:/test)
2. Create a new folder in the empty project (E:/test/temp)
3. Create an empty file in the folder (say, E:/test/temp/tmp)
4. Make sure you have double-clicked and opened the folder, and then closed it
5. Copy sample.py to the root of the project (E:/test)
6. Run sample.py from within Spyder

Note that step 4 is somewhat important. If you have never opened the folder, Spyder might not freeze. If the folder is open while runing sample.py, then it should freeze. But even if you closed it after opening, it should still freeze. I think step 4 somehow results in Spyder starting to track files in the folder and then becomes slow when you add a lot of files into it. I wasn't able to confirms this thought (or find a way to instruct Spyder to do differently)

What is the expected output? What do you see instead?

The actual workload shoud take very short time (which can be confimed by runing it through "python sample.py" in conda prompt), but Spyder freezes instead (sometime it recovers after a few minutes, but the task really just take about ten seconds).

Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

• Spyder version: 3.3.6
• Python version: 3.7
• Qt version:
• PyQt version:
• Operating System name/version: Windows 7 Professional (service pack 1)

Dependencies

IPython >=4.0     :  7.10.1 (OK)
cython >=0.21     :  0.29.14 (OK)
jedi >=0.9.0      :  0.14.1 (OK)
matplotlib >=2.0.0:  2.2.2 (OK)
nbconvert >=4.0   :  5.6.1 (OK)
numpy >=1.7       :  1.17.4 (OK)
pandas >=0.13.1   :  0.25.3 (OK)
pycodestyle >=2.3 :  2.5.0 (OK)
pyflakes >=0.6.0  :  2.1.1 (OK)
pygments >=2.0    :  2.5.2 (OK)
pylint >=0.25     :  2.4.4 (OK)
qtconsole >=4.2.0 :  4.6.0 (OK)
rope >=0.9.4      :  0.14.0 (OK)
sphinx >=0.6.6    :  2.2.2 (OK)
sympy >=0.7.3     :  1.4 (OK)

[tyNeanderthal]

I wasn't able to attach the sample.py file so I renamed it to sample.log. Alternatively, I pasted the code of sample.py here: (the indentation seems to fail tho)

import numpy as np
import pickle
import os


def test(directory, filename):
    tmp = np.zeros(1000)
    with open(directory+filename, 'wb') as f:
        pickle.dump(tmp, f)


def main():
    directory = 'temp/'  # Windows
    N = 20000
    for i in range(N):
        test(directory, str(i))
    for i in range(N):
        os.unlink(directory + str(i))


if __name__ == '__main__':
    main()

[ccordoba12]

This is a know issue and we're still unsure about how to solve it.

[ccordoba12]

Duplicate of #10081