How I can publish a library with all dependencies

[dangviettuan]

I got an error while trying to publish my library. Here is example source code: https://github.com/dangviettuan/nx-publishable I have:

• capitalize util
• button lib
• button showcase lib (publishable) When I build the button showcase lib an error occurs like the picture below

Originally posted by @dangviettuan in https://github.com/nrwl/nx/issues/3602#issuecomment-768173108

[dangviettuan]

After a week try many solutions I found a way to publish a library with all dependencies.

1. All dependencies must be buildable projects. We can convert non-buildable projects by update workspace.json

   "build": {
         "builder": "@nrwl/web:package",
         "outputs": ["{options.outputPath}"],
         "options": {
           "outputPath": "dist/libs/<YOUR LIB IMPORT PATH>",
           "tsConfig": "libs/<YOUR PACKAGE DIRECTORY>/tsconfig.lib.json",
           "project": "libs/<YOUR PACKAGE DIRECTORY>/package.json",
           "entryFile": "libs/<YOUR PACKAGE DIRECTORY>/src/index.ts",
           "external": [<IF YOU HAVE ANY EXTERAL LIBS WANT TO IGNORE FROM BUNDLE (react, react-dom)>],
           "babelConfig": "@nrwl/react/plugins/bundle-babel", (Optional: If you lib is React Lib)
           "rollupConfig": "@nrwl/react/plugins/bundle-rollup", (Optional: If you lib is React Lib)
           "assets": [
             {
               "glob": "README.md",
               "input": ".",
               "output": "."
             }
           ]
         }
       },```
   

2. Update paths in tsconfig.base.json file same with <YOUR LIB IMPORT PATH>. Example: YOUR LIB IMPORT PATH = @myorg/my-awesome-lib

   "paths": {
     "@myorg/my-awesome-lib": ["libs/ui/awsome-button/src/index.ts"],
     .....
   },
   

3. Build your library with --with-deps flag.
4. Rebuild you library with custom rollup config

   'use strict';
   Object.defineProperty(exports, '__esModule', { value: true });
   const builtins = require('rollup-plugin-node-builtins');
   const resolve = require('@rollup/plugin-node-resolve');
   const fileExtensions = ['.js', '.jsx', '.ts', '.tsx'];
   
   function getRollupOptions(options) {
     const extraGlobals = {
       // <YOUR EXTRAGLOBALS NAMES> 
       // Example:
       'react-dom': 'ReactDOM',
       'styled-components': 'styled',
       '@emotion/core': 'emotionCore',
     };
     if (Array.isArray(options.output)) {
       options.output.forEach((o) => {
         o.globals = Object.assign(Object.assign({}, o.globals), extraGlobals);
       });
     } else {
       options.output.globals = Object.assign(
         Object.assign({}, options.output.globals),
         extraGlobals
       );
     }
     options.plugins.push(
       resolve.nodeResolve({
         preferBuiltins: true,
         extensions: fileExtensions,
         moduleDirectories: ['dist/libs', 'node_modules'], // IMPORTANT
       })
     );
     options.plugins.push(builtins());
     options.external = [
       // YOUR EXTERNAL LIBS (Libs you don't want to bundle)
       // Example:
       'react',
       'react-is'
     ];
     return options;
   }
   module.exports = getRollupOptions;
   

5. NX will automatically add dependencies and buildable libraries to dependencies or peerDependencies. You can select by add flag buildableProjectDepsInPackageJsonType= dependencies or peerDependencies to buid command.
6. Go to the library and manually remove your buildable libraries from package.json because after publishing to NPM you can not download them.
7. Publish your Package to NPM PS: You should test your package before publishing to NPM to make sure it works properly. I faced 3 problems.

• CSS not extract. To fix it just add the flag --no-extractCss to the build command. I don't know why my buildable library exported to 2 CSS files but when bundle my publishable library CSS files not included. So embed CSS inside JS is my choice.
• NX missing some dependencies in package.json. After build my publish library I found that some dependencies is missing and I have to manually add them to package.json before publish to NPM.
• Typescript module augmentation: In my project, I use module augmentation to override a third-party interface. After the bundle, an error occurred because of undefined property. To fix it I remove third-party library from the external list to bundle it with my library.

[nemonemi]

I find this to be a workaround for something that I would expect to exist out of the box. @dangviettuan why have you closed this issue?

[dangviettuan]

@nemanjamilosavljevic-newtron Exactly, That is my workaround solution.

I read all issues relate to publishing a lib in repository issues and in the Slack channel but there is no answer so I decided to read, understand Nx source code, and finished with the custom Rollup build script.

I think Nx have done their job. Because we are building publishable libs and used it locally. The import statement can be treated as an external package.

Although it still has some issues with this approach I think it relates to different issues like export package.json dependencies.

Do you have any suggestion? Should we re-open this issue?

[nemonemi]

I also think that the framework is super nice!

From the slack channel I've learned that all libraries in question should be publishable, however, I've tried that approach, and it hasn't worked. What I would love is to hear the official solution to this problem.

[dangviettuan]

Ya. Because I just want to publish only one package instead of many packages and take time to manage and deploy them so I decide to use custom Rollup Script. What is your current problem with publishable libs?

Btw. I will reopen this issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

[dangviettuan]

Comment to keep this issue open

[effinrich]

@dangviettuan @nemonemi Commenting to keep this open. definitely doesn't feel like a feature we need to hack together to work as expected.

[iamvinitk]

Hi @vsavkin Would this issue be fixed in the upcoming versions?

[Fasani]

I struggled a lot with this over the last few days trying many many things, I have some core helpers and types which I wanted to include in a publishable UMD app. In the end, for me, it was very simple to bypass the 'rootDir' is expected to contain all source files. error.

Maybe this helps some of you, I added the helper and the types like normal.

import { colorSchemesFindHexById } from '@org-name/shared/helpers/color';
import {
    WidgetInitOptions,
    WidgetCommunicationData,
    InitProps,
} from '@org-name/shared';

Then I have added the exact locations of where these things can be found to the tsconfig.json in the app:

"compilerOptions": {
    ...
    "rootDirs": [
        "../../libs/shared/src/helpers/color",
        "../../libs/shared/src/models"
    ]
},

[Bielik20]

Hmm, I feel like this is a recurring issue. There was a long thread where people asked about ability to bundle multiple workspace libs (not event buildable ones) into one package. It was closed with a decision that NX does not support it, and you shouldn't do it. That being said I very much disagree with that decision, and existence of this new issue suggests that I am not alone in that.

I do agree that for 80% of cases making every lib publishable is the way to go. But there is that remaining 20% where that doesn't make sense. Where I work we use monorepo for my team, not entire organisation. We are expected to deliver a package that other team consumes. They only need that one package, yet we end up building 10 of them because of that restrictions. No one cares or needs remaining 9. We use them to structure our code and create boundaries but we are forced to publish them. This also makes the build and publish steps extremely longer, as I am not building and publishing 1 but 10 packages.

I know that NX Cloud could make that a little faster but:

• This is a paid solution created by nrwl to solve the problem manufactured by nrwl.
• It should be used to improve dev experience, not create a workaround for inadequate dev ops configuration.

Don't get me wrong, this is a great tool. I use it in my private repository, but to introduce something like that to the work environment is a lot harder. There is pricing, but also security implications, not to be done overnight.

[maxisam]

@Bielik20 Maybe you shouldn't break them into 10 libraries. You can have multiple modules in one package. I don't see much benefit to break it up unless it helps speeding up the build process because of the incremental build. But in your case, it doesn't. Maybe you shouldn't break it up. As for NX cloud, I think it makes it much faster. And I think it worth the money to pay NX. We want open source projects can have a way to live and grow.

[Bielik20]

@maxisam Yeah, that 10 libs and 1 was a simplification, sorry I didn't want to dive too much into details. It is a couple of libs that hold some "shared" functionalities and then couple that "really" get published. Still two libs that "really" get published are never included in the same project, so it would still be ok to bundle all the code inside. Having it split that way allows us to only run tests and publish those that changed using NX affected.

My point is that, there are use cases where you would want to bundle multiple libs together.

Another issue is that the way I publish those libs is suboptimal, but I don't know how to do it in NX, as there is no mechanism to track version of individual libs like lerna does. I think there is somewhere here issue for that as well. So when I detect that lib changed I need to republish all its dependencies with a new version (so no NX cache for me, because I need to bump version) even though they have not changed. Now version is not committed, but kept as a github tag (using semantic release), and I really do not want to commit that version to repo. We are getting off topic, but I am trying to illustrate here is that there are many use cases and sometimes (not most of the time, but still) bundling everything into one lib is actually a way to go.

[maxisam]

@Bielik20 Well, like you said you want to avoid unnecessary test and build, so you split it. Therefore, it is unfair to say "This is a paid solution created by nrwl to solve the problem manufactured by nrw". You had a problem already and NX is just trying to fix you problem. :-)

Just a thought, have you thought about using dependency in package.json in the library? https://github.com/ng-packagr/ng-packagr/blob/master/docs/dependencies.md#allowing-in-the-dependencies-section

Btw, why do you need to republish all its dependencies with a new version? for maintenance purpose? I think your second question is more about DevOps.

[Bielik20]

@maxisam I don't want to spam here, I am happy to discuss details with you though. Since there is no messaging on GitHub, if you want to talk in more details, please create an issue in here ;)

Here I just want to clarify. I think NX is a great tool and I love using it. NX Cloud is also a great idea. What I tried to say was that given NX is essentially a DevOps tool, it lacks in flexibility when it comes down to building publishable libs.

[alexn-s]

i would love the feature to bundle a library which includes all dependencies and not having to find a "hack" which makes it kinda work. i dont want to publish every dependency in order to use them in the library that i actuall want to publish.

[dangviettuan]

i would love the feature to bundle a library which includes all dependencies and not having to find a "hack" which makes it kinda work. i dont want to publish every dependency in order to use them in the library that i actuall want to publish.

Ya. I have tons of libraries in my workspace and I want to expose only 1 lib which built by combining some other libs and publish it to npm for my customer to download and install. Because I have a security rule that prevents publishing too much source code to npm. And I also don't want to manage the npm version of many libs.

[shubhamdeodia]

Do we have any solution for this ?

[cpfouche]

i would love the feature to bundle a library which includes all dependencies and not having to find a "hack" which makes it kinda work. i dont want to publish every dependency in order to use them in the library that i actuall want to publish.

Ya. I have tons of libraries in my workspace and I want to expose only 1 lib which built by combining some other libs and publish it to npm for my customer to download and install. Because I have a security rule that prevents publishing too much source code to npm. And I also don't want to manage the npm version of many libs.

I also need this.

[ngfelixl]

I would also be interested in a solution here. The builders for the apps also bundling the internal lib dependencies. So it should definitely be possible to do this. What would be awesome is to be able to configure which internal dependencies (maybe also external dependencies) to bundle.

[legovadim]

@dangviettuan

I cannot make it work for the case when I import only types from another library.

example: import { InterfaceName } from "@org/library"

it just stays like that. Is it possible to somehow import types from another library?

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

[keverw]

I noticed it doesn't add the NPM modules used to the package.json in the dist build output to publish as I expected. My thought is to just use the command to grab the NX deps and crawl it, then read the rootpackage.json file and just sync it all up is my thought on how I'd work around this issue unless it's going to be tackled more in the future.

Edit: I checked today and noticed it's working as expected now... Not sure what I could have changed, maybe since I added @nrwl/web since was getting an error about something missing when doing Jest match inline snapshots? I kinda think it'd be neat to mark if a peerDependencies but I think for that just create a blank placeholder and write a little post processing script. Then maybe I'll write a custom "build and publish all" script if last and current version doesn't match, but I'm starting to like this setup!

[npwork]

Really useful feature. Struggle myself. +1 please add it to nx

[pamatulli81]

Same problem here we have a list of libs which needs to be published and to deploy them manually is an absolute overkill. Would love to see something from NX here which helps us in this process.

[Falven]

It's been a whole year since this issue was opened, could we at least get a comment from an official NX dev?

[vincent-thomas]

Nx dev team Look at this issue, theres alot of people who needs this feature discussed in this issue!

[dynamikus]

It will be nice to have this feature.

[dzmitry-penda-arabesque]

@vsavkin can we please get some attention of NX team please on this issue? It could be a huge thing if we could use NX for developing packages that are split in several libraries and then exported in one npm package

[ghost]

Any update on this issue so far, it seems sharing every internal lib that doesn't need to be known to consumer of package is additional work.