Add a configuration option to disable dotenv-expand if desired

[benpsnyder]

Description

Please add a flag in nx.json for disabling the use of dotenv-expand for more advanced use cases. Example:

{
  "env": {
    "expand": "false"
  }
}

or we can simply set NX_ENV_EXPAND=false

What we're solving for

We don't want $ signs to get expanded in the root .env sometimes

Motivation

Many of our repos are multi-language, using the @nx-dotnet/core package for .NET 8 projects for example. Sometimes, we might have a little Rust in there too :)

All of our projects use a Makefile with documented commands. Here is a common setup:

(notice this line specifically bunx --bun dotenv-cli -e .config/environment/._shared_pre.env -e .config/environment/.$(environment).env -e .config/environment/._shared_post.env -- bun --bun ./tmp/dotenv-expand-transpiler.ts $(DOTENV_EXPAND_PREFIXES) ./ .env false >&2)

#!make
MAKEFLAGS += --silent --job=10

################# SETUP START ##################

REPO_ROOT := $(CURDIR)
SHELL := /bin/bash

nprocs ?= $(shell node -e 'console.log(require("os").cpus().length - 2)')
environment ?= development
DOTENV_EXPAND_PREFIXES ?= SNYDER_,NX_,ASPNETCORE_,DOTNET_,NODE_,PRODUCTION,WORKING_ENV
ifneq ($(wildcard pnpm-lock.yaml),)
_ := $(shell bunx --bun download-cli --out ./tmp https://raw.githubusercontent.com/snydertechnologies/bun-utils/main/nx/dotenv-expand-transpiler.ts)
_ := $(shell bunx --bun rimraf .env package-lock.json yarn.lock >&2)
_ := $(shell bunx --bun dotenv-cli -e .config/environment/._shared_pre.env -e .config/environment/.$(environment).env -e .config/environment/._shared_post.env -- bun --bun ./tmp/dotenv-expand-transpiler.ts $(DOTENV_EXPAND_PREFIXES) ./ .env false >&2)
else
$(error Please 'pnpm install'...)
endif
include .env
export $(shell sed 's/=.*//' .env)

# trick variables
noop =
comma := ,
space = $(noop) $(noop)

copy.env.dist:
	@if [ "$(environment)" != "development" ]; then \
		find ./dist/ -type d -name "$(DotnetVersion)" -exec cp ./.env {} \; ;\
		find ./dist/ -type d -name "publish" -exec cp ./.env {} \; ;\
		find ./dist/ -type f -name "main.js" -exec sh -c 'cp ./.env $$(dirname {})' \; ;\
	fi

build:
#nx run-many --all --target=build --configuration=$(environment) --parallel=$(nprocs)
	bun --bun run oxide.lint
	$(MAKE) build.styleguide
	$(MAKE) build.docs
	$(MAKE) build.storybook
	nx run-many --all --target=build --configuration=$(environment) --parallel=$(nprocs)
	$(MAKE) copy.env.dist

publish:
	@if [ "$(environment)" = "development" ]; then \
		echo "Aborting: environment is set to development"; \
		exit 1; \
	fi
	$(MAKE) build.styleguide
	$(MAKE) build.docs
	$(MAKE) build.storybook
	nx run-many --all --target=build --configuration=$(environment) --parallel=$(nprocs) --skip-nx-cache
	nx run-many --all --target=publish --configuration=$(environment) --parallel=$(nprocs) --skip-nx-cache
	$(MAKE) copy.env.dist

dapr.start:
	bunx rimraf .dapr/logs
	dapr run -f ./.config/dapr/dapr.$(environment).yaml &

dapr.stop:
	dapr stop -f ./.config/dapr/dapr.$(environment).yaml

### ........ and so on ... not all commands are shown

We like to have ._shared_pre.env and ._shared_post.env and .<environment>.env files to transform the variables ahead of time and drop it to the root .env.
bunx --bun dotenv-cli -e .config/environment/._shared_pre.env -e .config/environment/.$(environment).env -e .config/environment/._shared_post.env -- bun --bun ./tmp/dotenv-expand-transpiler.ts $(DOTENV_EXPAND_PREFIXES) ./ .env false >&2

[jaysoo]

You can workaround expansion by using the \$ escape key. We'd rather not fork expand vs not expand unless it is absolutely blocking for someone. It would bloat the implementation, and we'd have to pull in two different packages.

[benpsnyder]

@jaysoo I will have to revisit code and get you specific examples but we have several projects where we have to generate our ENV VARS and output the root .env rather than using the built-in Nx inheritance/override system.

At the end of our scripts, adding in the escapes is NOT a viable solution because utilities like docker-compose break.

For advanced repositories that need to custom pre-rendering of the environment variables, its helpful when Nx's expansion system can be disabled.

[jaysoo]

I converted this to a discussion, if enough people need the feature and can list out use cases where it is absolutely needed, then we can take a look.

[sir-captainmorgan21]

This is problematic for us. Our env vars come from Vault, a hosted service. The expand is only an issue in local development when we use NX to serve the app.

It is not an issue in production where we only use NX to build the app, but we just run node index.js.... to run the app. Escaping the environment variable in Vault would solve for local development, but break production deployment. Its essentially the docker-compose issue @benpsnyder referenced. I think a simple flag to disable expand across the board seems reasonable.