How I can publish a library with all dependencies

[dangviettuan]

I got an error while trying to publish my library. Here is example source code: https://github.com/dangviettuan/nx-publishable
I have:

• capitalize util
• button lib
• button showcase lib (publishable)
  When I build the button showcase lib an error occurs like the picture below
  

Originally posted by @dangviettuan in #3602 (comment)

[dangviettuan]

After a week try many solutions I found a way to publish a library with all dependencies.

1. All dependencies must be buildable projects. We can convert non-buildable projects by update workspace.json

   "build": {
         "builder": "@nrwl/web:package",
         "outputs": ["{options.outputPath}"],
         "options": {
           "outputPath": "dist/libs/<YOUR LIB IMPORT PATH>",
           "tsConfig": "libs/<YOUR PACKAGE DIRECTORY>/tsconfig.lib.json",
           "project": "libs/<YOUR PACKAGE DIRECTORY>/package.json",
           "entryFile": "libs/<YOUR PACKAGE DIRECTORY>/src/index.ts",
           "external": [<IF YOU HAVE ANY EXTERAL LIBS WANT TO IGNORE FROM BUNDLE (react, react-dom)>],
           "babelConfig": "@nrwl/react/plugins/bundle-babel", (Optional: If you lib is React Lib)
           "rollupConfig": "@nrwl/react/plugins/bundle-rollup", (Optional: If you lib is React Lib)
           "assets": [
             {
               "glob": "README.md",
               "input": ".",
               "output": "."
             }
           ]
         }
       },```

2. Update paths in tsconfig.base.json file same with . Example: YOUR LIB IMPORT PATH = @myorg/my-awesome-lib

   "paths": {
     "@myorg/my-awesome-lib": ["libs/ui/awsome-button/src/index.ts"],
     .....
   },

3. Build your library with --with-deps flag.
4. Rebuild you library with custom rollup config

   'use strict';
   Object.defineProperty(exports, '__esModule', { value: true });
   const builtins = require('rollup-plugin-node-builtins');
   const resolve = require('@rollup/plugin-node-resolve');
   const fileExtensions = ['.js', '.jsx', '.ts', '.tsx'];
   
   function getRollupOptions(options) {
     const extraGlobals = {
       // <YOUR EXTRAGLOBALS NAMES> 
       // Example:
       'react-dom': 'ReactDOM',
       'styled-components': 'styled',
       '@emotion/core': 'emotionCore',
     };
     if (Array.isArray(options.output)) {
       options.output.forEach((o) => {
         o.globals = Object.assign(Object.assign({}, o.globals), extraGlobals);
       });
     } else {
       options.output.globals = Object.assign(
         Object.assign({}, options.output.globals),
         extraGlobals
       );
     }
     options.plugins.push(
       resolve.nodeResolve({
         preferBuiltins: true,
         extensions: fileExtensions,
         moduleDirectories: ['dist/libs', 'node_modules'], // IMPORTANT
       })
     );
     options.plugins.push(builtins());
     options.external = [
       // YOUR EXTERNAL LIBS (Libs you don't want to bundle)
       // Example:
       'react',
       'react-is'
     ];
     return options;
   }
   module.exports = getRollupOptions;

5. NX will automatically add dependencies and buildable libraries to dependencies or peerDependencies. You can select by add flag buildableProjectDepsInPackageJsonType= dependencies or peerDependencies to buid command.
6. Go to the library and manually remove your buildable libraries from package.json because after publishing to NPM you can not download them.
7. Publish your Package to NPM
   PS: You should test your package before publishing to NPM to make sure it works properly. I faced 3 problems.

• CSS not extract. To fix it just add the flag --no-extractCss to the build command. I don't know why my buildable library exported to 2 CSS files but when bundle my publishable library CSS files not included. So embed CSS inside JS is my choice.
• NX missing some dependencies in package.json. After build my publish library I found that some dependencies is missing and I have to manually add them to package.json before publish to NPM.
• Typescript module augmentation: In my project, I use module augmentation to override a third-party interface. After the bundle, an error occurred because of undefined property. To fix it I remove third-party library from the external list to bundle it with my library.

[nemonemi]

I find this to be a workaround for something that I would expect to exist out of the box.
@dangviettuan why have you closed this issue?

[dangviettuan]

@nemanjamilosavljevic-newtron Exactly, That is my workaround solution.

I read all issues relate to publishing a lib in repository issues and in the Slack channel but there is no answer so I decided to read, understand Nx source code, and finished with the custom Rollup build script.

I think Nx have done their job. Because we are building publishable libs and used it locally. The import statement can be treated as an external package.

Although it still has some issues with this approach I think it relates to different issues like export package.json dependencies.

Do you have any suggestion? Should we re-open this issue?

[nemonemi]

I also think that the framework is super nice!

From the slack channel I've learned that all libraries in question should be publishable, however, I've tried that approach, and it hasn't worked.
What I would love is to hear the official solution to this problem.

[dangviettuan]

Ya. Because I just want to publish only one package instead of many packages and take time to manage and deploy them so I decide to use custom Rollup Script. What is your current problem with publishable libs?

Btw. I will reopen this issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[dangviettuan]

Comment to keep this issue open

[effinrich]

@dangviettuan @nemonemi Commenting to keep this open. definitely doesn't feel like a feature we need to hack together to work as expected.

[iamvinitk]

Hi @vsavkin
Would this issue be fixed in the upcoming versions?

[Fasani]

I struggled a lot with this over the last few days trying many many things, I have some core helpers and types which I wanted to include in a publishable UMD app. In the end, for me, it was very simple to bypass the 'rootDir' is expected to contain all source files. error.

Maybe this helps some of you, I added the helper and the types like normal.

import { colorSchemesFindHexById } from '@org-name/shared/helpers/color';
import {
    WidgetInitOptions,
    WidgetCommunicationData,
    InitProps,
} from '@org-name/shared';

Then I have added the exact locations of where these things can be found to the tsconfig.json in the app:

"compilerOptions": {
    ...
    "rootDirs": [
        "../../libs/shared/src/helpers/color",
        "../../libs/shared/src/models"
    ]
},

[Bielik20]

Hmm, I feel like this is a recurring issue. There was a long thread where people asked about ability to bundle multiple workspace libs (not event buildable ones) into one package. It was closed with a decision that NX does not support it, and you shouldn't do it. That being said I very much disagree with that decision, and existence of this new issue suggests that I am not alone in that.

I do agree that for 80% of cases making every lib publishable is the way to go. But there is that remaining 20% where that doesn't make sense. Where I work we use monorepo for my team, not entire organisation. We are expected to deliver a package that other team consumes. They only need that one package, yet we end up building 10 of them because of that restrictions. No one cares or needs remaining 9. We use them to structure our code and create boundaries but we are forced to publish them. This also makes the build and publish steps extremely longer, as I am not building and publishing 1 but 10 packages.

I know that NX Cloud could make that a little faster but:

• This is a paid solution created by nrwl to solve the problem manufactured by nrwl.
• It should be used to improve dev experience, not create a workaround for inadequate dev ops configuration.

Don't get me wrong, this is a great tool. I use it in my private repository, but to introduce something like that to the work environment is a lot harder. There is pricing, but also security implications, not to be done overnight.

[maxisam]

@Bielik20 Maybe you shouldn't break them into 10 libraries. You can have multiple modules in one package. I don't see much benefit to break it up unless it helps speeding up the build process because of the incremental build. But in your case, it doesn't. Maybe you shouldn't break it up. As for NX cloud, I think it makes it much faster. And I think it worth the money to pay NX. We want open source projects can have a way to live and grow.

[Bielik20]

@maxisam Yeah, that 10 libs and 1 was a simplification, sorry I didn't want to dive too much into details. It is a couple of libs that hold some "shared" functionalities and then couple that "really" get published. Still two libs that "really" get published are never included in the same project, so it would still be ok to bundle all the code inside. Having it split that way allows us to only run tests and publish those that changed using NX affected.

My point is that, there are use cases where you would want to bundle multiple libs together.

Another issue is that the way I publish those libs is suboptimal, but I don't know how to do it in NX, as there is no mechanism to track version of individual libs like lerna does. I think there is somewhere here issue for that as well. So when I detect that lib changed I need to republish all its dependencies with a new version (so no NX cache for me, because I need to bump version) even though they have not changed. Now version is not committed, but kept as a github tag (using semantic release), and I really do not want to commit that version to repo. We are getting off topic, but I am trying to illustrate here is that there are many use cases and sometimes (not most of the time, but still) bundling everything into one lib is actually a way to go.

[maxisam]

@Bielik20 Well, like you said you want to avoid unnecessary test and build, so you split it. Therefore, it is unfair to say "This is a paid solution created by nrwl to solve the problem manufactured by nrw". You had a problem already and NX is just trying to fix you problem. :-)

Just a thought, have you thought about using dependency in package.json in the library?
https://github.com/ng-packagr/ng-packagr/blob/master/docs/dependencies.md#allowing-in-the-dependencies-section

Btw, why do you need to republish all its dependencies with a new version? for maintenance purpose? I think your second question is more about DevOps.

[Bielik20]

@maxisam I don't want to spam here, I am happy to discuss details with you though. Since there is no messaging on GitHub, if you want to talk in more details, please create an issue in here ;)

Here I just want to clarify. I think NX is a great tool and I love using it. NX Cloud is also a great idea. What I tried to say was that given NX is essentially a DevOps tool, it lacks in flexibility when it comes down to building publishable libs.

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[dearlordylord]

please unstale

[FrozenPandaz]

This has been addressed with our new Nx release command: https://nx.dev/features/manage-releases

Please check that out and let us know what you think!

[ajwootto]

@FrozenPandaz can you provide an example of how the release command addresses the concern here? It doesn't seem to have anything to do with bundling libraries inside of other libraries

[dearlordylord]

The last time I was told to use the nx release command, it was doing something entirely different. It's either bugged out or has completely separate scope since, at the least, January 8th: https://discord.com/channels/1143497901675401286/1193795596235063356/1193795596235063356

[FrozenPandaz]

Ah, sorry. I've reopened the issue. 👍

[billdinger]

Thanks to everyone who has commented. As someone who just spent ~13 hours going slowly mad thinking he had forgotten or was crazy or didn't survive his last plane trip because this scenario just wasn't working...

This sure would be nice to have! It's not obvious/spelled out at all that you can't have a normal nx library that is imported by other published libraries. And attempting to do so causes the 'rootDir' is expected to contain all source files. errors which don't make a lot of sense.

I can confirm that external: "none" works although if you want to limit the scope of what is inlined in your package to just the specific shared library you are using in your project.json

  "targets": {
    "build": {
      "executor": "@nx/js:tsc",
      "outputs": ["{options.outputPath}"],
      "options": {
        "external": ["@my-project/lib/shared"],

where "@my-project/lib/shared" is whatever shared library or libraries you need. That will inline them into the package and cause the error to go away. Unfortuantely this feature is marked deprecated and scheduled for removal in nx 20 which means this solution, while it works, means you can never upgrade your project. Not great!

Anway +1 and thanks to all the folks who have commented here and helped.

[indra4you]

First most, I am not that experienced with monorepo & nx. I am getting similar error and best to my knowledge I tried all possible options shared on this thread as well as whatever answer I got on google, but none of them worked for me. May I am doing it completely wrong or my expectation itself is wrong!

What I am expecting is, during the build

• first-package package will be published as separate package, so it will be referenced in the second-package - this works fine
• utility package code should become part of the second-package so it can be published as a single package with name second-package

I am yet to figure out on how to change symbolic link of first-package in second-package package.json file with exact version during publishing.

Here is my repo and attached screenshot of the error I am getting.

[sonnd08]

Also, seem like external: "none" only bundle packages inside nx workspace, it does not bundle outside package like dayjs, moment, ...
I would like to create a package that not dependent to any other packages instead.

[rolandas-valantinas]

There is very basic example where this is required for us and I'm surprised that no one mentioned it.
We have about 10 logically distinct libraries which we use to create very thin lambdas. Whole NX affected approach is very beneficial as only some lambdas are packaged and deployed when changes in libraries occur - no problem at this level.

However, we also have utility methods across those libraries which we want to share with other teams.
One could say just pull all of the utility methods into single library, but then affected becomes way less useful - affected doesn't operate at method level, so updating single utility method will cause all lambdas to rebuild though only couple are using updated method.

external: "none" sort of works, however it is not compatible with @nx/dependency-checks eslint plugin which is shouting about missing local libraries, even though everything works fine both locally and in published package. The only way to stop this is to list all of libraries used within in ignoredDependencies in .eslintrc.json and it doesn't seem like there is way to use regex or specify just scope.

And like someone else said external: "none" is getting removed in nx 20 🤷‍♂️

[dearlordylord]

And like someone else said external: "none" is getting removed in nx 20 🤷‍♂️

I see it's possible that the decision makers in the nx team are confused by the name of the feature publish mentioned here.

It's either misnamed or not documented enough, as a result there's evidence (I encountered this more than 2 times, including in this thread) that the nx team thinks that library publishing is sorted out despite it's not.

So far, the nx publish feature seems to create more confusion than benefits.

I don't exclude that they are removing external: "none" because they think that publishing is possible with nx publish command which actually does something tangential to actual publishing.

The usual conversation about the publish feature could easily be "do we have a way to publish?" - "yes we have nx publish" command - "ok case closed, delete external: "none""

I'd hate this to be the case. NX team, please let us publish packages.

[ajwootto]

external: "none" sort of works, however it is not compatible with @nx/dependency-checks eslint plugin which is shouting about missing local libraries, even though everything works fine both locally and in published package. The only way to stop this is to list all of libraries used within in ignoredDependencies in .eslintrc.json and it doesn't seem like there is way to use regex or specify just scope.

This is the key issue for us as well when trying to use the "external" feature, nothing else in Nx seems to be designed around the idea that "externals" exists. Apparently it's supposed to be understood as a "build tool specific feature" (like specific to tsc, rollup etc.) and not an "Nx project graph" concept, which doesn't make any sense to me. It seems like Nx is occupying a confusing half way point between actually understanding the full scope of how your libraries depend on each other vs. just making it your problem to keep track of.

We've tried using the dependency checks rule in our repo but due to the amount of "ignore" statements we have to include in it it basically isn't useful or trustworthy for us, we are pretty much back to just keeping track of dependencies manually.

[Bielik20]

Came here from notification. Can someone point me to blog post / info about deprecating external? Are they deprecating external as a whole or just external: "none"? What's the rationale?

It is a feature we are really relying on. Removing that will make our work way harder than it needs to be :/

[rolandas-valantinas]

I'm only aware about this https://nx.dev/nx-api/js/executors/swc#external

[leosvelperez]

Hi all!

First of all, I'd like to apologize for the lack of feedback here.

Bundling non-publishable libraries is a recurring issue in the repo, which might highlight some gaps in our docs and potentially the need for some extra diagnostics in our executors to provide guidance. We'll be taking a look at improving in that area.

I'd like to make a key distinction in this matter: building and bundling are two different things. A lot of the confusion comes from trying to have compilers like tsc or swc bundle dependencies. Those tools don't have bundling capabilities. We experimentally tried to provide some inlining features (external option) on top of those tools but we bumped into a lot of issues.

Ultimately, we came to the conclusion that trying to add those features on top of tools that don't support it was not the right approach. There are other tools available to build and bundle code (e.g. esbuild, rollup, vite, etc.). We're also committed to reducing our abstractions on top of tools and not deviating from how they fundamentally work.

This is why the external option is deprecated for the @nx/js:swc and @nx/js:tsc executors. It was an experiment that failed. Experimental features are never 100% guaranteed to become stable. Luckily, there are other tools to achieve the same thing. I'll improve the deprecation message in those executors so the reason is clearer.

So, to summarize, if you're using one of the @nx/js:swc or @nx/js:tsc executors and have issues bundling non-publishable libraries into publishable libraries, switch to a different executor that supports bundling. For example:

• @nx/rollup:rollup
• @nx/esbuild:esbuild
• @nx/vite:build

[tylerprice1]

@leosvelperez, what would you suggest for the case where my publishable library is using @nx/rollup:rollup, and my non-publishable dependency library is using @nx/js:tsc?

I have an alias defined in tsconfig.base.json for the dependency library, but if I just import it, it gets treated as an external dependency (regardless of whether I specify it in external in my build options for the Rollup executor), but since the dependency is not published, the output bundle fails to run.

My directory structure is:

• apps
  • ...
    • test app (Angular)
• libs
  • shared
    • dependency library (TypeScript)
    • ui
      • publishable library (React)

Everything works fine locally when serving the Angular app, but the React library needs to include the dependency in its bundle so that it can run when published and used outside of this repository.

For a bit more context, the Angular app is the only app and is only used to test the React library. It is not actually used in any production capacity. I feel like this is somewhat of a unique use-case for Nx, but I'd still love to get everything working correctly.

Thanks in advance!