Skip to content

yarn

Hermeto's Yarn support scope

Supported Yarn versions

Hermeto currently supports Yarn versions 1, 3 and 4. Version 1 is referred to as "Yarn Classic" and is covered in Hermeto's Yarn Classic doc. This document describes Yarn v3 and v4 support.

Supported Yarn protocols/locators

Hermeto currently supports all standard Yarn protocols except for

Due to the nature of how the two protocols above work, mainly related to potentially executing arbitrary code, adding support for them with future releases of Hermeto is unlikely. For further details on Yarn protocols and their practical package.json examples, please head to the official Yarn documentation on protocols linked earlier in this section.

Dealing with .yarnrc.yml

Hermeto parses the project's .yarnrc.yml file and analyzes configuration settings. Before hermeto proceeds with the actual dependency fetching, it verifies whether all configuration settings that set a path to a resource don't point outside of the source repository, so in order to avoid any issues reported by Hermeto in this regard make sure all your project resource references are bound by the repository. Part of the analysis of the repository's .yarnrc.yml file is detection of plugin usage which is further explained in Dealing with plugins.

Dealing with Yarn Zero-Installs

Yarn's PnP Zero-Installs are unsupported due to the potentially unplugged dependencies checked into the repository which simply make it impossible for the Yarn cache to be checked for integrity using Yarn's standard tooling (i.e. yarn install --check-cache).

NOTE

The same applies to dealing with the node_modules top level directory which, if checked into the repository, can also serve the Zero-Install purpose. If you need further information on which dependency linking mode is used, have a look at the nodeLinker and on the PnP approach in general.

Also note that we may reconsider our initial decision when it comes to Zero-Installs provided the input repository doesn't rely on any dependencies which may include install scripts leading to their unpacking in a form of .yarn/unplugged entries.

Dealing with plugins

Due to the nature of plugins (which can potentially execute arbitrary code, by e.g. adding new protocol resolvers), all plugins except for the official ones (see "Default Plugins" in the Yarn API docs) are disabled during the dependency prefetch stage to ensure no other changes apart from downloading dependencies took action.

For Yarn v3, even the official plugins are disabled, with the exception of exec.

NOTE

hermeto doesn't taint your project files, so any plugins you set will be enabled normally in your build environment, the only problem that can arise is if any of your specified plugins adds a new protocol which hermeto doesn't know about in which case the dependency pre-fetch stage will fail with an error.

Specifying packages to process

A package is a file or directory that is described by a package.json file (also called a manifest).

Hermeto can be run as follows

hermeto fetch-deps \
  --source ./my-repo \
  --output ./hermeto-output \
  '<JSON input>'

where 'JSON input' is

{
  // "yarn" tells Hermeto to process Yarn packages
  "type": "yarn",
  // path to the package (relative to the --source directory)
  // defaults to "."
  "path": ".",
}

or more simply by just invoking hermeto fetch-deps yarn.

For complete example of how to pre-fetch dependencies, see Example: Pre-fetch dependencies.

Controlling Yarn's behavior

Hermeto instructs Yarn to download dependencies explicitly declared in package.json. The dependencies are then further managed in a yarn.lock file that Yarn CLI manages automatically and creates it if missing. However, Hermeto will refuse to process your repository if the file is missing, so be sure to check that file into the repository. Also make sure that the file is up to date for which you can use yarn install.

Downloading dependencies

If Yarn is configured to operate in the PnP mode (the default in Yarn v3 or v4) Yarn will store all dependencies as ZIP archives.

Once the source repository analysis and verification described in the earlier sections of this document has been completed, then it's essentially just a matter of hermeto internally invoking yarn install --mode=skip-build to fetch all dependencies (including transitive dependencies).

Known pitfalls

If your repository isn't in a pristine state (i.e. you tried to run yarn install previously on your own without Hermeto) what may happen is that Hermeto will assume the repository makes use of Zero-Installs. The workaround here is simple, just run yarn cache clean and hermeto will then process your repository as normal.

Using fetched dependencies

See the Example for a complete walkthrough of Hermeto usage.

Hermeto downloads the Yarn dependencies into the deps/yarn/ subpath of the output directory (see the snippet below).

hermeto-output/deps/yarn
└── cache
    ├── abbrev-npm-1.1.1-3659247eab-8.zip
    ├── agent-base-npm-6.0.2-428f325a93-8.zip
    ├── agentkeepalive-npm-4.3.0-ac3d8e6807-8.zip
    ├── aggregate-error-npm-3.1.0-415a406f4e-8.zip
    ├── ansi-regex-npm-3.0.1-01f44078a3-8.zip
...

Building your project using the pre-fetched Yarn dependency cache

In order to use the hermeto pre-fetched Yarn dependency cache obtained from the previous step several environment variables need to be set in your build environment. See Example: Generate environment variables for more details on how these can be generated by hermeto automatically in a form of a environment file that can sourced as part of your container build recipe. Here's a snippet of the most important variables hermeto needs to be set in the build environment along with explanation

# Point Yarn to our pre-populated global cache
YARN_GLOBAL_FOLDER=<hermeto_output_dir>/deps/yarn

# Yarn must not rely solely on the global cache (the pre-fetched one) because
# it'll likely only be available (i.e. mounted) during the (container) build
# time, but not runtime. We specifically want Yarn to copy those dependencies
# from the global cache to the project's local cache
YARN_ENABLE_GLOBAL_CACHE=false

# Must be set to true, otherwise Yarn will not make use of the pre-populated
# global cache we're pointing it at with YARN_GLOBAL_FOLDER at build time.
YARN_ENABLE_MIRROR=true

# Must be false otherwise 'yarn install' will fail to populate the project's
# local cache (pointed to by the 'cacheFolder' setting) from the global cache
# (the pre-fetched one).
YARN_ENABLE_IMMUTABLE_CACHE=false

Example

For the Yarn example let's use the same sample Node.js project, but this time modified to use Yarn as the package manager. Get the repo if you want to try for yourself

git clone -b yarn https://github.com/cachito-testing/sample-nodejs-app.git

Pre-fetch dependencies

The steps for pre-fetching the dependencies are very similar to the previous examples, this time using the Yarn package manager. Like with the previous examples the default path for the package we assume is ..

See the Yarn documentation for more details about running Hermeto for pre-fetching yarn dependencies.

hermeto fetch-deps --source ./sample-yarn-app --output ./hermeto-output '{"type": "yarn"}'

OR more simply (without the need of a JSON formatted argument) just

hermeto fetch-deps --source ./sample-yarn-app --output ./hermeto-output yarn

Generate environment variables

There are a few environment variables we'll have to set for Yarn during the hermetic build, so we need to generate an environment file.

$ hermeto generate-env ./hermeto-output -o ./hermeto.env --for-output-dir /tmp/hermeto-output
$ cat ./hermeto.env
export YARN_ENABLE_GLOBAL_CACHE=false
export YARN_ENABLE_IMMUTABLE_CACHE=false
export YARN_ENABLE_MIRROR=true
export YARN_GLOBAL_FOLDER=/tmp/hermeto-output/deps/yarn

Inject project files

Like the gomod package manager Yarn does not currently need to modify any content in the source directory for the cached dependencies to be used in a hermetic build, however that might change in the future.

Build the application image

Yarn is installed using a Node.js tool called Corepack which has been shipped by Node.js by default since v16.9.0 and v14.19.0. Therefore, we'll use the node:18 base image in our example which definitely has Corepack and we can start using Yarn right away.

FROM node:18

COPY sample-yarn-app/ /src/sample-yarn-app
WORKDIR /src/sample-yarn-app

# Run yarn install command and list installed packages
RUN . /tmp/hermeto.env && yarn install

EXPOSE 9000

CMD ["yarn", "run", "start"]

We can then build the image as before while mounting the required Hermeto data!

podman build . \
  --volume "$(realpath ./hermeto-output)":/tmp/hermeto-output:Z \
  --volume "$(realpath ./hermeto.env)":/tmp/hermeto.env:Z \
  --network none \
  --tag sample-nodejs-app