Generic fetcher¶

Specifying artifacts to fetch
Using fetched dependencies
Full example walkthrough

Support scope¶

Generic fetcher is made specifically for use cases where hermeto will not implement a full package manager support, or for ecosystems where no such package manager exists. It is highly discouraged for this feature to be used for anything already supported by hermeto in other ways (such as e.g. pip packages), because the produced SBOM component will not be accurate.

Specifying artifacts to fetch¶

The generic fetcher requires a lockfile artifacts.lock.yaml that specifies which files to download. This file is expected to be in the source repository. Alternatively, it can be supplied as an absolute path via the lockfile key in the JSON input to hermeto.

Below are sections for each type of supported artifact. Several artifacts of different types can be specified in a single lockfile.

The lockfile must always contain a metadata header and a list of artifacts. Currently, the only supported version is 1.0:

---
metadata:
  version: "1.0"
artifacts: []

Hermeto can be run as follows

hermeto fetch-deps \
  --source ./my-repo \
  --output ./hermeto-output \
  '<JSON input>'

where 'JSON input' is

{
  "type": "generic",
  // path to the package (relative to the --source directory)
  // defaults to "."
  "path": ".",
  // option to specify lockfile path, must be an absolute path if specified
  // defaults to "artifacts.lock.yaml", relative to path
  "lockfile": "artifacts.lock.yaml",
}

Arbitrary files¶

This artifact type is intended for whatever files are needed at build time that do not fit neatly into other package managers.

---
metadata:
  version: "1.0"
artifacts:
  - download_url: "https://example.com/file.zip"
    checksum: "algorithm:hash"
    filename: "optional-custom-name.zip"  # optional

Each artifact requires:

download_url The URL to download the file from
checksum In format "algorithm:hash" (e.g., "sha256:123...")
filename Optional custom filename for the downloaded file. If not present, it will be derived from the url

Arbitrary SBOM component¶

Since there can't be any assumptions about these files beyond checking their identity against a checksum, these files will be reported with pkg:generic purl in the output SBOM.

Maven artifacts¶

This type is for downloading maven repository artifacts. These are specified using GAV coordinates that are enumerated in the artifact's attributes in the lockfile. The download URL will be assembled using this information.

---
metadata:
    version: "1.0"
artifacts:
    - type: "maven"
      filename: "ant.jar"
      attributes:
          repository_url: "https://repo1.maven.org/maven2"
          group_id: "org.apache.ant"
          artifact_id: "ant"
          version: "1.10.14"
          type: "jar"
      checksum: "sha256:4cbbd9243de4c1042d61d9a15db4c43c90ff93b16d78b39481da1c956c8e9671"

Each artifact requires

type type of the artifact (always maven)
filename Optional custom filename for the downloaded file. If not present, it will be derived from the url
attributes Maven-specific attributes
repository_url URL of the Maven repository (required)
group_id Maven group ID (required)
artifact_id Maven artifact ID (required)
version Version of the artifact (required)
type Type of the artifact ("jar" by default)
classifier Maven classifier (optional)
checksum In format "algorithm:hash" (e.g., "sha256:123...")

Maven SBOM component¶

These files will be reported with pkg:maven purl in the output SBOM, because the URL is fully assembled from the provided attributes and therefore the file can be assumed to be a maven artifact.

Using fetched dependencies¶

Hermeto downloads the files into the deps/generic/ subpath of the output directory. Files are named according to the filename field if specified, otherwise derived from the URL. During your build, you would typically mount hermeto's output directory into your container image and reference the individual files.

See the Example below for a complete walkthrough of Hermeto usage.

Example¶

Generic fetcher is a package manager that can fetch arbitrary files. Let's build a sample container image that would be inconvenient to build hermetically otherwise. This image will provide OWASP Dependency check tool, which is available to install from GitHub releases page. Get the repo if you want to try for yourself

git clone -b sample-app https://github.com/cachito-testing/cachi2-generic.git

Pre-fetch dependencies¶

In order to retrieve the archive with the tool, either a artifacts.lock.yaml needs to be in the repository, or an absolute path needs to be supplied in the JSON input, pointing to a lockfile. You can find a sample lockfile below. It is identical to the one found in the sample repository. A lockfile for the generic fetcher must contain a metadata header and a list of artifacts, where each artifact is represented as a pair of URL and a checksum string in the format of "algorithm:checksum". Optionally, you can also specify an output filename for the artifact. If not specified, it will be derived from the url.

---
metadata:
  version: "1.0"
artifacts:
  - download_url: "https://github.com/jeremylong/DependencyCheck/releases/download/v11.1.0/dependency-check-11.1.0-release.zip"
    checksum: "sha256:c5b5b9e592682b700e17c28f489fe50644ef54370edeb2c53d18b70824de1e22"
    filename: "dependency-check.zip"

As with other examples, the command to fetch dependencies is very similar. The default path is assumed to be ..

hermeto fetch-deps --source ./hermeto-generic --output ./hermeto-output generic

Build the application image¶

We'll use the ibmjava:11-jdk as base image because it already has java pre-installed. During the build, the downloaded release will be extracted and modified to have execute rights.

FROM ibmjava:11-jdk

WORKDIR /tmp


# use jar to unzip file in order to avoid having to install more dependencies
RUN jar -xvf hermeto-output/deps/generic/dependency-check.zip

RUN chmod +x dependency-check/bin/dependency-check.sh

ENTRYPOINT ["/tmp/dependency-check/bin/dependency-check.sh", "--version"]

We can then build the image as before while mounting the required Hermeto data.

podman build . \
  --volume "$(realpath ./hermeto-output)":/tmp/hermeto-output:Z \
  --network none \
  --tag sample-generic-app