Document how to not exclude node_modules

[alexanderkjeldaas]

Current Behavior

The readme mentiones a few limitations, among them that files under node_modules are not transpiled.

Expected Behavior

README.md includes either pointer or description of how to get haul to work, including how to ensure either node_modules are transpiled or how to configure webpack to do so.

[junibrosas]

you can include the node_modules when you use a loader.

 include: [
    path.resolve(__dirname, "src"),
    path.resolve(__dirname, "node_modules/ui") // include specific package from node_modules
  ]

 include: [
    path.resolve(__dirname, "src"),
    path.resolve(__dirname, "node_modules") // include the whole node_modules folder
  ]

[Traviskn]

This is actually a bit more complex than it seems because the default haul config uses happypack (or the thread-loader in the most recent version) to transform js files in parallel with babel. We'd need to remove and replace the existing js loader entirely. If we don't want to lose out on the speed gain of happypack/thread-loader we'd also need to recreate that config. It would be really helpful if we could somehow parameterize ignoring/including node_modules, it is (unfortunately) common for react native libraries to publish un-transformed code due to the default behavior of packager transforming node_modules

[Traviskn]

Actually now that I'm looking at beta 7, it looks like there is no more exclude config? Is this no longer ignoring node_modules?

[Traviskn]

Newer releases are ignoring node modules now - This is a fairly hacky way to override the exclude config by reaching into the default rules and modifying them:

// webpack.haul.js
module.exports = function getHaulWebpackConfig(options, defaults) {
  defaults.module.rules[1].exclude = undefined;

  return {
    ...defaults,
    entry: './index.js',
    module: {
      ...defaults.module,
      rules: [
        ...defaults.module.rules,
      ],
    },
  };
};

[maxammann]

Currently having problems with using rn-fetch-blob. There is no easy solution to transpile specific modules currently.

[chiubaka]

Any new information on this? The above discussion about internals that make it difficult to not exclude specific node_modules is concerning.

Currently having trouble with what I think is a transpilation issue with aws-amplify-react-native.

[chiubaka]

Mostly I'm wondering if the above is still relevant, given that much has probably happened since Nov 10, 2017.

[chiubaka]

Cool, I figured this out by poking around the configs returned in haul.config.js. @Traviskn's comment is still relevant, so the approach should likely be to modify the default js processing rule. I modified it in two ways:

1. Change the test from /\.js$/ to /\.jsx$/, since I think many cases where one might want to transpile a node_module will involve React components shipped by that module.
2. Updated the exclude regex to include the name of the package I want to include.

For reference, this is what the default rule looked like originally:

{
  test: /\.js$/,
  exclude: /node_modules(?!.*[\/\\](react|@react-navigation|@expo|pretty-format|haul|metro))/,
  use: [
    {
      loader: '/Users/dchiu/Developer/serenity-web/mobile/node_modules/cache-loader/dist/cjs.js',
      options: {
        cacheDirectory: '/Users/dchiu/Developer/serenity-web/mobile/node_modules/.cache/cache-loader'
      }
    },
    {
      loader: '/Users/dchiu/Developer/serenity-web/mobile/node_modules/thread-loader/dist/cjs.js',
      options: {
        workers: 7
      }
    },
    { 
      loader: '/Users/dchiu/Developer/serenity-web/mobile/node_modules/babel-loader/lib/index.js',
      options: {
        extends: '/Users/dchiu/Developer/serenity-web/mobile/.babelrc',
        plugins: [
          '/Users/dchiu/Developer/serenity-web/mobile/node_modules/haul/src/utils/fixRequireIssues.js',
          '/Users/dchiu/Developer/serenity-web/mobile/node_modules/react-hot-loader/babel.js',
          '/Users/dchiu/Developer/serenity-web/mobile/node_modules/haul/src/hot/babelPlugin.js'
        ],
        cacheDirectory: true
      }
    }
  ]
}

Here's my complete config (I'm also using TypeScript):

import { createWebpackConfig } from "haul";

const INCLUDE_PACKAGES = [
  "react",
  "@react-navigation",
  "@expo",
  "pretty-format",
  "haul",
  "metro",
  "aws-amplify-react-native",
];

function findDefaultJsRule(rules) {
  return rules.findIndex((rule) => {
    return String(rule.test) === String(/\.js$/);
  });
}

function modifyDefaultJsRule(rules) {
  const index = findDefaultJsRule(rules);
  const defaultJsRule = rules[index];

  const exclude = new RegExp("node_modules(?!.*[\\/\\\\](" + INCLUDE_PACKAGES.join("|") + "))");

  const newJsRule = {
    // Include .jsx files, not just .js files
    test: /\.jsx?$/,
    exclude,
    use: defaultJsRule.use,
  }

  const newRules = [...rules];
  newRules[index] = newJsRule;
  return newRules;
}

export default {
  webpack: env => {
    const config = createWebpackConfig({
      entry: `./src/index.js`,
    })(env);

    config.module.rules = modifyDefaultJsRule(config.module.rules);

    config.module.rules = [
      {
        test: /\.tsx?$/,
        use: [
          {
            loader: 'babel-loader'
          },
          {
            loader: 'ts-loader'
          }
        ]
      },
      ...config.module.rules,
    ];

    config.resolve.extensions = [
      '.ts',
      '.tsx',
      `.${env.platform}.ts`,
      '.native.ts',
      `.${env.platform}.tsx`,
      '.native.tsx',
      ...config.resolve.extensions,
    ]

    return config;
  }
};

For broader context in case this comment is discovered by anyone experiencing similar issues, I was getting a dict is not defined error when including the withAuthenticator HOC from aws-amplify-react-native. I'm new to React Native and don't usually spend this much time with my build tools, but I pieced together that the issue arose because aws-amplify-react-native distributes its code as ES6/ES2015 (this is obvious from looking at it--it doesn't look transpiled at all). The default React Native Packager seems to take care of this just fine by transpiling the code in the module, but haul doesn't do this by default. If that code doesn't get transpiled, the runtime may not understand some of its syntax (it's expecting something else), hence the weird syntax-error-like message on what should otherwise be functional code. Successfully modifying my haul.config.js to also process the .js and .jsx files distributed in aws-amplify-react-native solved the issue.

I'd love for this to be easier to do out-of-the-box, perhaps as an option passed to createWebpackConfig? I'd also propose modifying the test for the default rule to encompass .jsx regardless. I'd be happy to take a crack at a PR for this if it would be welcome.

[thymikee]

Happy to accept any improvements to this in a PR! :) jsx should be resolved by default