Skip to content

Latest commit

 

History

History
331 lines (251 loc) · 21 KB

REFERENCES.md

File metadata and controls

331 lines (251 loc) · 21 KB
Raw

Using TypeScript Project References with ts-loader and webpack

Project References were added to TypeScript in 3.0. The benefits of using project references include:

  • Better code organisation
  • Logical separation between components
  • Faster build times

If you are using TypeScript in your web project you can also use project references to improve your code and build workflow. This article describes some of the ways to set up your project to use references. I am using ts-loader to transpile the TypeScript code to JavaScript and webpack to bundle code.

An example repo using the configuration above is available at the link below:

https://github.com/appzuka/project-references-example

There are 2 stages to using project references in your project:

  1. Configure and build the project references
  2. Setup your codebase to consume the compiled projects

To gain an understanding of how project references work, for the first part of this guide we will use tsc to build the project references. Later on, we will configure ts-loader to do this automatically.

Configure and build the project references

This stage just involves following the directions from the TypeScript documentation: https://www.typescriptlang.org/docs/handbook/project-references.html

There are a few points to note:

  1. Referenced projects must have the new composite setting enabled.

  2. Each referenced project has its own tsconfig.json

  3. There will be a root level tsconfig.json which includes the lower level projects as references. Building this will build all subprojects.

  4. You should be using configuration file inheritance ({ “extends”: …}) to avoid duplication in your config.

  5. You need to use tsc --build to compile the project.

  6. When you compile the project tsc --build will create a file called tsconfig.tsbuildinfo that contains the signatures and timestamps of all files required to build the project. On subsequent builds TypeScript will use that information to detect the least costly way to type-check and emit changes to your project.

  7. There is no need to use the incremental compiler option. tsc --build will generate and use tsconfig.tsbuildinfo anyway.

  8. If you delete your compiled code and re-run tsc --build the code will not be rebuilt unless you also delete the tsconfig.tsbuildinfo file. Use the tsc --build --clean command to do this for you.

  9. If you set the declaration and declarationMap settings in tsconfig.json the outDir folder will contain .d.ts and .d.ts.map files alongside the transpiled JavaScript. When you consume the compiled project you should consume the outDir folder, not the src. Even though your root project is in TypeScript it can use full syntax checking without the subproject’s TypeScript source because the outDir folder contains the definitions in the .d.ts file. Vscode (and many other code editors and IDEs) will be able to find the definitions and perform syntax checking in the editor just as if you were not using project references and importing the TypeScript source directly.

Project Structure

The TypeScript implementation of project references allows you to structure the project in almost any way you wish. Just configure the input and output folders in tsconfig.json to your needs and TypeScript will build it for you.

For a web project you might like a structure similar to the one below. You could put all your project references in a packages folder with the top-level project code in src:

    tsconfig.json
    tsconfig-base.json
    src
     - (source code for the main project)
    dist
     - main.js (final bundle produced by webpack)
    packages
     - reference1
       - tsconfig.json (inherits from tsconfig-base.json)
       - src
       - lib
     - reference2
       - tsconfig.json (inherits from tsconfig-base.json)
       - src
       - lib

Each project reference has its own tsconfig.json with the source code for each package in a src subfolder. When the project is built the compiled JavaScript for each project will be in its lib subfolder.

The source code for your main project is in a top-level src folder and the final bundle will be in a top-level dist folder. The top-level src folder is not a referenced project — it is normal TypeScript source that webpack will bundle. It imports from the lib folders of the referenced projects built by tsc.

This structure works well because:

  • Having packages grouped together under a packages folder organises your codebase nicely.
  • Other tools such as yarn workspaces and lerna use and understand this organisation.
  • Each package is fully self-contained in its own folder. It contains the source, compiled code, tsconfig.json and (optionally) its own package.json which describes how the package is used.
  • You can drop the package into another project, import it with a simple statement and everything will be linked up.

This is just one way to structure your project. Some other options include:

  • Not putting the projects references in a packages folder. They could all be at the top level, or a different folder, or nested folders.
  • The output folder of each project does not have to be in a lib folder of that project. You could have a top-level lib folder which contains the output of all projects.

Almost any structure is compatible with project references. You have freedom to specify the paths of the referenced projects and their outputs in the tsconfig.json files. You will import the compiled JavaScript files into the main project and some structures make this easier than others, but you have the freedom to choose what works for you.

Test Build your Projects

You should now check that the building of the projects is successful and produces the code you expect.

In each project reference folder execute tsc --build, check there are no errors and the output is as you expect. Use tsc --build --clean to remove the output and repeat. You can use tsc --build --verbose to see what tsc is doing.

If you have a top-level tsconfig.json similar to:

    {
      "files": ["src/index.ts"],
      "references": [
        { "path": "./reference1" },
        { "path": "./reference2" }
      ]
    }

Then executing tsc --build in the top-level will compile all of your subprojects with one command. The build process is smart and can manage dependencies between subprojects.

In the final step of this guide we will get ts-loader to do the build automatically when called from webpack, but for now, just make sure that the build process works when using tsc --build manually.

Setup your codebase to consume the project

Now your subprojects are built you can use them in your root project. Let’s say your reference1 project exports a number:

    // packages/reference1/src/index.ts

    export const Meaning = 42;

After building the reference with tsc --build the compiled JavaScript will be found in packages/reference1/lib/index.js. In your root project you need to import this. There are several ways you can do this. Let’s start with a naive approach that will work but has severe downsides:

    // src/index.ts

    // Don't do this!
    import { Meaning } from '../packages/reference1/lib';

This will work because TypeScript and webpack will both find the file. The downsides are:

  • The organisation of your root project and components are now intertwined. If you change the internal structure of your subproject you will need to update every import statement in the entire project.

  • The import location will depend on the location on the source file. For example, if you want to do the same import from a subfolder in your root project you will need to replace ../packages/reference1/lib with ../../packages/reference1/lib. If you re-organise your project structure you will need to fix every import.

The solution to this is module resolution — how TypeScript and webpack resolve the targets of import statements. You can read about this at the links below:

Module resolution is nothing new and it is not part of project references, but understanding it will be a huge help getting everything working. Some points to note:

  • TypeScript and webpack can use different methods to resolve modules. It will help if you can set them up so they are using the same method. (See the example below using alias in webpack and/or tsconfig-paths-webpack-plugin.)
  • Resolution works differently for relative (./reference1) and absolute (reference1) imports.
  • TypeScript has 2 strategies for module resolution: classic and node. You probably want to use node.
  • You can use a webpack plugin tsconfig-paths-webpack-plugin so that you just need to define paths in your tsconfig.json and then don’t need to repeat these in your webpack config.

Using the example above, we would like to just import from packages/reference and have TypeScript and webpack both know that this refers to the actual location.

    // src/index.ts

    // Better!
    import { Meaning } from 'packages/reference1/lib';

We can achieve this using the paths configuration in tsconfig.json (or better, in tsconfig-base.json so the settings are made once and inherited by all projects):

    {
      "compilerOptions": {
        "baseUrl": ".", // This must be specified if "paths" is.
        "paths": {
          "packages/*": ["packages/*"]
        }
      }
    }

Now TypeScript understands that when it sees packages/reference1 in an import statement, it should look in ./packages/reference1. The path is relative to the root tsconfig.json so it does not matter where the source file which imports this is located.

Unless you are using tsconfig-paths-webpack-plugin you may need to include a corresponding resolve-alias setting in your webpack.config.js:

    const path = require('path');

    module.exports = {
      modules: [
        "node_modules",
        path.resolve(__dirname)
      ],
      resolve: {
        alias: {
          packages: path.resolve(__dirname, 'packages/'),
        }
      }
    };

(In this case the path.resolve(__dirname) in the modules section accomplishes the same thing, but depending on your project structure you may need an alias.)

If you are getting module not found errors when you build, knowing whether these are coming from TypeScript or webpack will help you to resolve the issue.

Errors which come from TypeScript when you build the project look similar to the following:

    ERROR in ...project-references-demo/src/index.tsx
    ./src/index.tsx
    [tsl] ERROR in ...project-references-demo/src/index.tsx(1,27)
          TS2307: Cannot find module 'mypackages/zoo' or its corresponding type declarations.

Note the [tsl] in the message and also the TypeScript error code TS2307. This indicates that the error was passed to webpack by ts-loader when it tried to transpile the file. You can also check whether errors are coming from TypeScript by building your project manually with tsc and checking whether it reports errors.

Errors from webpack look similar to the following:

    ERROR in ./src/index.tsx
    Module not found: Error: Can't resolve 'mypackages/zoo' in '...project-references-demo/src'
     @ ./src/index.tsx 6:12-37

If you just get these errors it indicates that tsconfig.json is correctly configured and TypeScript is able to resolve your modules, but webpack is not. Look into the resolve section of webpack.config.js and check whether you need to add an alias.

You can use module resolution to make your project work with project references even if your structure is very different from that outlined here. As long as webpack and TypeScript can find the built code it will work.

Can you import the TypeScript Source instead of the JavaScript?

You can import the TypeScript source from your projects, but you probably should not. If you do set up your project to import the TypeScript, webpack will bundle your project just fine, but then you are not using project references. You have succeeded in organising your codebase but you are not getting the advantage of reducing build time by using the compiled files in lib. In fact, you are slowing down your build by requiring tsc or ts-loader to build the reference and then not using it.

If your project is large you could see a significant benefit from pre-building large sections of code. If your project is not so large you may prefer to just structure your codebase and skip project references.

Using ts-loader to build project references

Up to this point, we ran tsc --build on its own and then used webpack and ts-loader to build the whole project, importing the references. You can configure ts-loader to build the references for you, which simplifies the build process.

The top-level project in src is TypeScript code, so you will already be using ts-loader to load the TypeScript source into webpack. Just add projectReferences: true to the ts-loader configuration and you no longer need to run tsc in a separate process:

    // webpack.config.js

    "module": {
      "rules": [
        {
          "test": /\.tsx?$/,
          "exclude": /node_modules/,
          "use": {
            "loader": "ts-loader",
            "options": {
              "projectReferences": true
            }
          }
        }
      ]
    }

When webpack uses ts-loader to process a TypeScript file ts-loader will now check whether any of your project references need rebuilding and rebuild them before webpack proceeds if necessary. This includes when webpack is in watch mode as used by webpack-dev-server.

Setting projectReferences: true in ts-loader alone will not magically convert your code to use project references. All it does is to run tsc --build as part of the build process. You need to configure project references and structure your project to use them as described here.

If you have come this far congratulations — you are now using TypeScript project references in your web project. You can stop here, but in the next section of this guide there are some tips to clean up the project further and create a library of reusable, version-controlled components.

Using package.json

We can clean this up further by including a package.json in the project reference subfolder. If this contains the following:

    //packages/reference1/package.json

    {
      "name": "reference1",
      "version": "1.0.0",
      "description": "Project Reference1",
      "main": "lib/index.js",
      "directories": {
        "lib": "lib"
      },
      "license": "ISC"
    }

then you can just import as follows:

    // src/index.ts

    import { Meaning } from 'packages/reference1';

The module setting in package.json tells the bundler to import from lib/index.js when it sees the import statement above.

Using node_modules

In the above approach we need to add paths to tsconfig.json so that the module resolution knows where to find our package. But the module resolution system automatically looks in node_modules, so if we link our reference in node_modules we won’t need the paths and aliases:

    ln -s ../packages/reference1 node_modules/reference1
    node_modules/reference1 -> packages/reference1

It probably makes sense to use npm scopes:

    ln -f ../../packages/reference1 node_modules/@myscope/reference1
    node_modules/@myscope/reference1 -> packages/reference1

then you can consume the code with:

    // src/index.ts

    import { Meaning } from '@myscope/reference1';

So you benefit from not having to configure paths and aliases, but you need to create the links in node_modules after cloning the project, unless you use Yarn workspaces.

Using Yarn Workspaces

If you use yarn workspaces the node_modules links will automatically be created for you when you execute yarn install. Simply include the following in your root level package.json: { "private": true, "workspaces": ["packages/*"] } In the subproject’s package.json you should use the name of the package you want to be linked in node_modules:

    //packages/reference1/package.json

    {
      "name": "@myscope/reference1",
      "version": "1.0.0",
      "module": "lib/index.js
    }

When you run yarn install the links in node_modules will be created for you.

You can now use your project references anywhere in your codebase with a simple import statement, exactly like you import npm modules. If you have a more complex application, for example with client and server applications, you can share modules easily.

Building a Component Library

A common problem in code organisation is how to re-use code in multiple projects. Project references help toward this goal by providing a logical separation between components. This will mean you can drop a component into another project and use it. But there is still the matter of how you do this:

  • You could copy the project reference folder into all top-level projects you want to use it in. This has the disadvantage that you end up with multiple copies of code. If you patch or enhance a component you need to copy the patch to all the other projects, rebuild them and test.
  • Another approach would be to symlink the component into each top-level project. The downside of this is that once you amend the component you could break all of the projects which depend on it.

A smarter solution is to publish the components as npm packages. You can use semantic versioning each time you publish using a version in the format major.minor.patch. You then add the components to other projects using yarn add @myscope/reference1.

Versioning works exactly the same as any other npm package. You specify in the consuming project’s tsconfig.json what version changes are acceptable:

    "@myscope/reference1": "1.0.1",   // Only version 1.0.1 can be used
    "@myscope/reference1": "~1.0.1",  // Patch updates are acceptable
    "@myscope/reference1": "^1.0.1",  // Minor version changes are OK

You can then update and publish new versions of the component with new version numbers. The other project will not be broken as it will continue to use the version specified in its package.json. When you are ready to update you can use the same yarn tools you would use to update any package (yarn outdated / upgrade / upgrade-interactive or the npm equivalents).

If you want to keep your packages private you can set up your own private npm repository with Verdaccio or you can use Github Packages

Lerna

If your project references are complex and have their own scripts for testing and building you could use Lerna. This works well with yarn workspaces and the project structure outlined above. If you have a test script in reference1 you could use the following command to execute it:

    lerna run --scope=reference1 test

The same command without the --scope argument would execute the test scripts in all subprojects.

Yarn workspaces and Lerna introduce more power but also more complexity in the workflow. They are not required to use project references so it is up to you whether the extra learning curve they introduce is worthwhile.

Build Times in Development

Using ts-loader and webpack-dev-server, when you change a file in one of the project references ts-loader will automatically rebuild the reference and include the change in the new bundle. Rebuilding the reference may take a few seconds. By comparison, when you change a file in the root source (non-reference) webpack will get ts-loader to rebuild just that file and create a new bundle very quickly, typically less than 1 second.

So if you are developing code in a reference and find the few seconds it takes to rebuild the reference too much, you could benefit from importing from the TypeScript source directly. This will be at the expense of longer warm start times as you will not be using the pre-built code for that referenced project.