Why does reticulate aggressively push miniconda over the system python?

[cboettig]

I have a nicely set up python environment with venv which reticulate easily discovers, as shown below. However, before even reporting this environment, py_discover_config() stops to tell me "No non-system installation of Python could be found, would you like to download and install Miniconda?". Why does reticulate throw this message and prompt? Is there a strong reason to only use python & python libraries that are installed in a user's home directory?

reticulate::py_discover_config()
No non-system installation of Python could be found.
Would you like to download and install Miniconda?
Miniconda is an open source environment management system for Python.
See https://docs.conda.io/en/latest/miniconda.html for more details.

Would you like to install Miniconda? [Y/n]: n
Installation aborted.
python:         /opt/venv/reticulate/bin/python
libpython:      /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6.so
pythonhome:     //usr://usr
version:        3.6.9 (default, Nov  7 2019, 10:44:02)  [GCC 8.3.0]
numpy:          /opt/venv/reticulate/lib/python3.6/site-packages/numpy
numpy_version:  1.18.2

python versions found: 
 /usr/bin/python3
 /usr/bin/python
 /opt/venv/reticulate/bin/python

[kevinushey]

This seems like a bug -- we shouldn't be nagging about installing Miniconda here.

How have you configured reticulate to find your venv environment?

[kevinushey]

For context: the primary goal is to avoid users who inadvertently bind to whatever the "default" version of Python is, which is often an older or crippled version that comes pre-installed with the system.

[cboettig]

I've simply the env variable WORKON_HOME to /opt/venv. /opt/venv/reticulate is the first (alphabetically) venv subdir in /opt/venv/reticulate.

(For context: I'm building a docker container for ML use cases and want the user to be able to have a valid python environment "out-of-the-box". I'm aiming for this to work without forcing/assuming the container will only be ever run using the "default user", since our RStudio-server containers are often used in multi-user mode, or run as root, etc, hence my messing with WORKON_HOME. I'm more than happy to hear suggestions about better ways to go around this too)

[cboettig]

p.s. if you have docker, you should be able to reproduce the above warning with our dev container:

docker run --rm -ti rockerdev/ml:3.6.3-cuda-10.0 R

[skeydan]

I can confirm that in a situation where you have virtual environments (in .virtualenvs, in my case), reticulate will discover the alphabetically first one but still ask if it should install miniconda.

However, you can prevent this by setting

export RETICULATE_AUTOCONFIGURE=0

in your .bashrc.

(It also wouldn't happen if you issued use_virtualenv() right when starting a new session, but I tend to forget that sometimes so the env variable is nice.)

[eddelbuettel]

I think I also fell victim to this and ended up with several hundred mb of miniconda because of reticulate (which, adding insult to injury, was only called as a suggested package to test a package). I eventually found that adding RETICULATE_PYTHON=/usr/bin/python3 to .Renviron sorted things out but I was a little surprised too. Didn't have time to dig at the time but still strikes me as overly aggressive reach for miniconda on a system where Python is a first class citizen (Ubuntu, in this case, and it relies fairly heavily internally on Python and python3).

[cboettig]

Thanks all. I didn't know about RETICULATE_AUTOCONFIGURE, but in this case I do like that reticulate is finding my python environment out of the box, and don't mind that it defaults to the alphabetical listing. It gives me a predictable way to have a default without hard-coding it. I'd rather not set RETICULATE_PYTHON or RETICULATE_PYTHON_ENV, since then a user is unable to switch to a different virtualenv without first overriding these environmental variables, and that seems a bit heavy handed in my case (i.e. having (a) zero-config python env working out of the box, (b) while still allowing users to set an alternative environment without needing to know about and to override a setting I have hardwired).

But would still prefer we could avoid the miniconda prompt in this case (or at least on Linux systems like Dirk says!)

[skeydan]

@cboettig just to clarify, in case what I wrote gave rise to a misunderstanding...

RETICULATE_AUTOCONFIGURE just switches off that "install miniconda" prompt, it does not mean you have to use any specific virtualenv.

You can still pick the one you want by issuing use_virtualenv("someenv") in R, or as I usually do it, doing . /path/to/env/bin/activate on the command line before starting RStudio.

[cboettig]

@skeydan thanks much for the clarification, I had totally misunderstood that! Yes, that does just what I want it to do.

I'm happy to leave this open if you still want to change this behaviour (I'm still a bit unclear what specifically triggers the miniconda prompt), but for me just turning off the miniconda prompt with the env var does what I need. thanks again.

[cgeger]

Just a note to say that this same Miniconda prompt also occurs in RStudio Cloud when opening a new project, even though there are at least two installs of Python already available (Py 2.7 runs in the terminal by default on startup and Py 3.5 is available if Miniconda install prompt response is ignored by typing "n", as above). I haven't tried setting up a venv on RStudio Cloud yet though... kind of assumed I was already working in one, but perhaps not?

[kevinushey]

@cgeger it might be worth voicing these concerns on the RStudio community forums: https://community.rstudio.com/c/rstudio-cloud/14

[flying-sheep]

It is pretty insulting to give me that prompt and interpret a “no, I don’t want miniconda” as “cancel installation”.

How about reticulate checks if the python it finds is actually old or crippled instead of assuming that?

[eddelbuettel]

This (documented) variable helps:

edd@rob:~$ grep -i reticulate ~/.Renviron
## reticulate, please use my system python
RETICULATE_PYTHON=/usr/bin/python3
edd@rob:~$ Rscript -e 'library(reticulate); np <- import("numpy"); np$reshape(np$arange(1, 9), c(2L, 2L, 2L))'
, , 1

     [,1] [,2]
[1,]    1    3
[2,]    5    7

, , 2

     [,1] [,2]
[1,]    2    4
[2,]    6    8

edd@rob:~$ 

[flying-sheep]

The way it looked to me:

1. Install something like knitr that pulls in reticulate as dependency / “Suggests”
2. It offers to install something I deliberately do not have (miniconda) and if I wanted it, would install using my system package manager, so I decline
3. Instead of just continuing, it fails its build
4. I now need to look up how to ask some dependency I’m not interested in at the moment to “pretty please” let me install it

Don’t get me wrong, I’m sure you have reasons for that prompt.

But as a library developer myself I make very sure to just confront my users with disruption like this when absolutely necessary.

So I repeat. To be nice to users, either

• Saying no in the prompt should mean “I don‘t care, use my system python anyway”, or
• The prompt should only appear when you’re 100% sure that the Python installation is outdated or broken

[eddelbuettel]

For contexct: I don't disagree personally. I also happen to like having a working Linux distribution system and strongly prefer all 'other' components to come from it. But last time I brought this up (years ago, I think) @kevinushey pointed me to this variable and it seems to work for me in the (relatively light regular) usage I make of reticulate, incl use via Suggests or Imports from different packages. I also happen to dislike *conda given that I am on working Linux setup, but I understand the appeal of *conda to the developers as it covers other OSs for which there is no consistent alternative. It is a really hard problem to tackle.

[kevinushey]

As always, a reproducible example would be helpful. It's not clear to me when or why @flying-sheep is being prompted for Miniconda. I understand it's frustrating (and it seems like the behavior here is probably unintentional), but that doesn't help me solve your problem.

[salim-b]

A few more observations from a Linux system (Ubuntu 20.04):

• Setting RETICULATE_AUTOCONFIGURE=0 (or =FALSE) does not disable reticulate's miniconda installation prompt.

• Setting RETICULATE_PYTHON=/usr/bin/python3 (or to whatever you prefer) does disable reticulate's miniconda installation prompt.

• A "no" choice in reticulate's miniconda installation prompt is stored permanently via creation of the file ~/.local/share/r-reticulate/miniconda.json with the following content:

  {
    "DisableInstallationPrompt": true
  }

  Unless that file is deleted again, reticulate doesn't prompt for miniconda installation anymore.

[t-kalinowski]

Reticulate no longer suggests or encourages usage of miniconda