Scan projects

• 1: Scan for open source risks
• 1.1: Java
• 1.2: Kotlin
• 1.3: Go
• 1.4: Python
• 1.5: JavaScript/TypeScript
• 1.6: Ruby
• 1.7: .NET
• 1.8: Bazel
• 1.9: Swift/Objective-C
• 1.10: Scala
• 1.11: PHP
• 1.12: Rust
• 2: Scanning strategies
• 3: Perform incremental scans
• 4: Scoping scans
• 5: Working with monorepos
• 6: Repository Security Posture Management
• 7: Rescan projects
• 8: Manage build tools (Beta)
• 8.1: Configure scan profile through Endor Labs UI
• 8.2: Configure scan profile through Endor Labs API
• 8.3: Configure scan profile through scanprofile.yaml
• 8.4: Enable auto detection
• 9: Scan artifacts and binaries
• 10: Scan containers (Beta)

1 - Scan for open source risks

Endor Labs supports the following major capabilities to help teams reduce the risk and expense of software dependency management across the lifecycle of software reuse.

• SCA - Software composition analysis is the identification of the bill of materials for first-party software packages and the mapping of vulnerabilities to these software component versions. SCA helps teams to maintain compliance and get visibility into the risks of their software inventory.
• Endor Scores - Endor Labs provides a holistic risk score that includes the security, quality, popularity and activity of a package. Risk scores help in identifying leading indicators of risk in addition to if a software component is outdated, or unmaintained. Risk analysis helps teams to go beyond vulnerabilities and approach the risk of their software holistically.
• Reachability Analysis - Reachability analysis is Endor Labs’ capability to perform static analysis on your software packages to give context to how each vulnerability may be reached in the context of your code. This includes mapping vulnerabilities back to vulnerable functions so that deep static analysis can target vulnerabilities with higher levels of granularity as well as the identification of unused software dependencies.
• Upgrade Impact Analysis - Upgrade impact analysis allows security teams to set better expectations with their development teams by identifying breaking changes associated with an update of a direct dependency.

The resource requirements, both minimum and recommended, for build runners or workers executing scans using endorctl are listed here.

Note: Large applications may require additional resources to complete or enhance the scan performance.

Minimum Resources

Recommended Resources

Supported languages

The following table shows Endor Labs language coverage:

Complete support matrix

The following comprehensive matrix lists the supported languages, build tools, manifest files, and supported requirements.

See the detailed procedure for all supported languages:

1.1 - Java

Java is a high-level, object-oriented programming language widely used by developers. Endor Labs supports scanning and monitoring of Java projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

Endor Labs requires the following prerequisites in place for successful scans.

• Install JDK versions between 11 and 22
  • For JDK 8, see Scan the projects on JDK version 8
• Make sure your repository includes one or more files with .java extension.
• Install any version of the Maven Package Manager if your project uses Maven.
• Install Gradle build system version 6.0.0 and higher, if your project uses Gradle. To support lower versions of Gradle, see Scan projects on Gradle versions between 4.7 and 6.0.0.
• For projects not using Maven or Gradle, make sure that your project is set up properly to scan without the pom.xml file. See Scan projects without pom.xml for more information.

Build Java projects

You must build your Java projects before running a scan. Additionally, ensure that the packages are downloaded into the local package caches and that the build artifacts are present in the standard locations.

Use Gradle

To analyze your software built with Gradle, Endor Labs requires that the software be able to be successfully built. To perform a quick scan, dependencies must be located in the local package manager cache. The standard $GRADLE_USER_HOME/caches or /User/<username>/.gradle/caches must exist and contain successfully downloaded dependencies. To perform a deep scan the target artifact must be generated on the file system as well.

To build your project with Gradle, use the following procedure:

1. If you would like to run a scan against a custom configuration, specify the Gradle configuration by setting an environment variable.

      export endorGradleJavaConfiguration="<configuration>"
   

   The default configuration is runtimeClasspath if none is provided.

2. Ensure that you can resolve the dependencies for your project without errors by running the following command:

   For Gradle wrapper:

      ./gradlew dependencies
   

   For Gradle:

      gradle dependencies
   

3. Run ./gradlew assemble or gradle assemble to resolve dependencies and to create an artifact that may be used for deep analysis.

Override subproject level configuration

In a multi-build project, if you set the environment variable endorGradleJavaConfiguration=[GlobalConfiguration], the specified configuration is used for dependency resolution across all projects and subprojects in the hierarchy below.

\--- Project ':samples'
     +--- Project ':samples:compare'
     +--- Project ':samples:crawler'
     +--- Project ':samples:guide'
     +--- Project ':samples:simple-client'
     +--- Project ':samples:slack'
     +--- Project ':samples:static-server'
     +--- Project ':samples:tlssurvey'
     \--- Project ':samples:unixdomainsockets'

To override the configuration only for the :samples:crawler and :samples:guide subprojects, follow these steps:

1. Navigate to the root workspace, where you execute endorctl scan, and run ./gradlew projects to list all projects and their names.

2. Run the following command at the root of the workspace:

   echo ":samples:crawler=testRuntimeClasspath,:samples:guide=macroBenchMarkClasspath" >> .endorproperties
   

   This creates a new file named .endorproperties in your root directory. This enables different configurations for the specified subprojects in the file.

3. Run endorctl scan as usual.

At this point, all other projects will adhere to the GlobalConfiguration. However, the :samples:crawler subproject will use the testRuntimeClasspath configuration, and the :samples:guide subproject will use the macroBenchMarkClasspath configuration.

Use Maven

To analyze your software built with Maven, Endor Labs requires that the software be able to be successfully built. To perform a quick scan, dependencies must be located in the local package manager cache. The standard .m2 cache must exist and contain successfully downloaded dependencies. To perform a deep scan the target artifact must be generated on the file system as well.

To build your project with Maven, use the following procedure:

1. Ensure that you can resolve the dependencies for your project without error by running the following command.

   mvn dependency:tree
   

2. Run mvn install and make sure the build is successful.

   info

   If you want to skip the execution of tests during the build, you can use -DskipTests to quickly build and install your projects.

   mvn install -DskipTests
   

3. If you have multiple Java modules not referenced in the root pom.xml file, make sure to run mvn install separately in all the directories.

Configure Maven private registries

If you have a private registry and internal dependencies on other projects, you must configure private registries. In addition to scanning public Java projects and repositories, Endor Labs provides the support to integrate with private registries.

1. Sign in to Endor Labs and select Integrations under Manage from the left sidebar.
2. From Package Managers, select Maven and click Manage.
3. Click Add Package Manager.
4. Enter a package manager URL.
5. To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and enter the username and password of your private package manager repository.
6. Click Add Package Manager to save your configuration.

Run a scan

Use the following options to scan your repositories. Perform a scan after building the projects.

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

endorctl scan --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --quick-scan -o json | tee /path/to/results.json

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

endorctl scan

Use the following flags to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

When deep analysis is performed all private software dependencies are completely analyzed by default if they have not been previously scanned. This is a one-time operation and will slow down initial scans, but won’t impact subsequent scans.

Organizations might not own some parts of the software internally and the related findings are not actionable by them. They can choose to disable this analysis using the flag disable-private-package-analysis. By disabling private package analysis, teams can enhance scan performance but may lose insights into how applications interact with first-party libraries.

Use the following command flag to disable private package analysis:

endorctl scan --disable-private-package-analysis

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Scan projects without pom.xml (Beta)

Endor Labs supports projects that do not use Maven or Gradle, and have no pom.xml in the following cases.

Uber jars

If there is an uber jar (fat jar) that contains all application classes and dependency jars of the project, you can set the environment variable, ENDOR_JVM_USE_ARTIFACT_SCAN as true and run the scan.

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
endorctl scan --package --path=<jar/ear/war location> --project-name=<project name>

For example:

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
endorctl scan --package --path=/Users/johndoe/projects/project21.jar --project-name=Project21

Application dependencies in classpath

If you do not have an uber jar with dependencies, but only have application dependency files (like jar, war, or ear), you can set the path to these files in the environment variable, ENDOR_JVM_USE_ARTIFACT_SCAN_CLASSPATH and run the scan.

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
export ENDOR_JVM_USE_ARTIFACT_SCAN_CLASSPATH=<path that contains application depedencies>
endorctl scan --package --path=<jar/ear/war location> --project-name=<project name>

For example:

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
export ENDOR_JVM_USE_ARTIFACT_SCAN_CLASSPATH=/Users/johndoe/caches/modules/files-2.1
endorctl scan --package --path=/Users/johndoe/projects/project21.jar --project-name=Project21

Application classes and dependencies extracted as first-party class files

If application class files and dependency jar files are extracted as first-party class files, you can provide the first-party files in an environment variable, ENDOR_JVM_FIRST_PARTY_PACKAGE.

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
export ENDOR_JVM_FIRST_PARTY_PACKAGE="<dependency/application 1>,<dependency/application 2>,...,<dependency/application N>"
endorctl scan --package --path=<jar/ear/war location> --project-name=<project name>

For example:

Your project jar has the following structure where com.org.doe and com.org.deer are application class files and dependencies.

fawn.jar
├── com.org.doe
│   ├── A.class
│   └── B.class
├── com.org.deer
│   ├── Util.class
│   └── Utilities.class
├── com.org.dep1
│   ├── Dep1.class
│   └── Dep2.class
└── com.org.dep2
    ├── 2Dep1.class
    └── 2Dep2.class

export ENDOR_JVM_USE_ARTIFACT_SCAN=true
export ENDOR_JVM_FIRST_PARTY_PACKAGE="com.org.doe,com.org.deer"
endorctl scan --package --path=/Users/johndoe/projects/fawn.jar --project-name=Fawn

Scan the projects on JDK version 8

Endor Labs supports JDK versions between 11-22, however, you can scan projects on JDK 8 using the following procedure:

1. Build your Java project on JDK 8.

2. After building, switch your Java home to JDK 11 or higher versions.

   export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home
   

3. Run a scan

Scan projects on Gradle versions between 4.7 and 6.0.0

To scan Java projects on Gradle versions between 4.7 and 6.0.0, make sure to

1. Check the version of your project using:

   ./gradlew --version
   

2. The project must have a Gradle wrapper. You can generate the Gradle wrapper using:

   --gradle-version <your required version>.
   

   Endor Labs prioritizes Gradle wrapper over Gradle and it is a recommended best practice to use Gradle Wrapper.

3. Before executing the endorctl scan, ensure the project can be built in your required version.

   Execute ./gradlew assemble.
   

4. Use --bypass-host-check during endorctl scan to execute scans on projects that have Gradle versions lower than 6.0.0.

Understand the scan process

Endor Labs analyzes your Java code and dependencies to detect known security issues, including open-source vulnerabilities and generates call graphs.

How Endor Labs resolves dependencies for package versions

Endor Labs resolves the dependencies for Java packages based on the following factors:

• For packages built using Maven, it leverages the Maven cache in the .m2 directory of your file system to resolve the package’s dependencies and mirrors Maven’s build process for the most accurate results.
• For Maven, Endor Labs respects the configuration settings present in the settings.xml file. If this file is included in your repository, you need not provide any additional configuration.
• For packages built using Gradle, it uses Gradle and Gradle wrapper files to build packages and resolve dependencies.
• Endor Labs supports EAR, JAR, RAR, and WAR files.

How Endor Labs performs static analysis on the code

Endor Labs performs static analysis on the Java code based on the following factors:

• Call graphs are created for your package. These are then combined with the call graphs of the dependencies in your dependency tree to form a comprehensive call graph for the entire project.
• Endor Labs performs an inside-out analysis of the software to determine the reachability of dependencies in your project.
• The static analysis time may vary depending on the number of dependencies in the package and the number of packages in the project.

Known limitations

• If a package can not be successfully built in the source control repository, static analysis will fail.
• Spring dependencies are analyzed based on spring public entry points to reduce the impact of Inversion of Control (IOC) frameworks. Dependencies and functions are identified as reachable and unreachable in the context of a spring version and its entry points.
• Annotation processing is limited only to the usage of the code they annotate.
• Static analysis of reflection and callbacks are not supported.
• Endor Labs requires JDK 11 to generate call graphs for Java projects. Gradle versions lacking JDK 11 support are not compatible.

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Host system check failure errors:

  • Java is not installed or not present in the PATH environment variable. Install Java and try again. See Java documentation for more information.
  • The installed version of Java is lower than the required version. Install JDK versions between 11-22 and try again.
  • Java is installed but Maven or Gradle is not installed. In such cases, the dependency resolution may not be complete.

• Unresolved dependency errors: Maven is not installed properly or the system is unable to build root pom.xml. Run mvn dependency:tree in the root of the project and try again. In such cases, the dependency resolution may not be complete.

• Resolved dependency errors: A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.

• Gradle variant incompatibility message: Gradle performs JVM toolchain checks for subprojects or dependencies and may raise errors indicating a Java version mismatch between dependencies declared in Gradle manifest and Java home setup.

  Example error message:

  Incompatible because this component declares a component for use during compile-time, compatible with Java version 21 and the consumer needed a component for use during runtime, compatible with Java version 17 - To resolve this and taking advantage of Java’s backward compatibility, instruct Gradle to use the higher version of JDK detected in the error message. For example, for the message above, specify org.gradle.java.home=<path of java> in .gradle/gradle.properties. The path needs to be to the root of the directory bin/java. For example, if your Java is at /Users/Downloads/jdk-21/Contents/Home/bin/java, specify org.gradle.java.home=/Users/Downloads/jdk-21/Contents/Home. - If you are scanning a purely Java 8 Gradle project and if you encounter the above error, set org.gradle.java.home to point to Java 8 home, before you execute the endorctl scan. - A general guideline for determining which Java version to use, is to match the Java version specified in .gradle/gradle.properties with the one used for building your Gradle project.

• Call graph errors: - The project can not be built because a dependency cannot be located in the repository. - Sometimes, the project is not built, if a Java version discrepancy exists between the required repository version and the version on the system running the scan. For example, the Java required version is 1.8 but the system has 12 installed. Install the required version and try again.

• If you have a private registry and internal dependencies on other projects, you must configure the credentials of the registry. See Configure Maven private registries.

• If you have a large repository or if the scan fails with out-of-memory issues, you may need to increase the JVM heap size before you can successfully scan. Export the ENDOR_SCAN_JVM_PARAMETERS environment variable with additional JVM parameters before performing the scan as shown below:

  export ENDOR_SCAN_JVM_PARAMETERS="-Xmx32G"
  

• If you use a remote repository configured to authenticate with a client-side certificate, you must add the certificate through an endorctl parameter. Export the ENDOR_SCAN_JVM_PARAMETERS parameter before performing a scan. See Maven documentation for details.

  export ENDOR_SCAN_JVM_PARAMETERS="-Xmx16G,-Djavax.net.ssl.keyStorePassword=changeit,
  -Djavax.net.ssl.keyStoreType=pkcs12,
  -Djavax.net.ssl.keyStore=/Users/myuser/Documents/nexustls/client-cert1.p12"
  

1.2 - Kotlin

Kotlin is a statically typed programming language that runs on the Java Virtual Machine (JVM), known for its concise syntax, null safety, and seamless integration with Java. Endor Labs supports scanning and monitoring of Kotlin projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

To ensure a successful scan using Endor Labs, it is essential to have the following prerequisite software installed:

• Maven Package Manager: Version 3.6.1 and higher

• Gradle Build System: Version 6.0.0 and higher

• Java Runtime Environment: JDK versions between 11 and 19

• Kotlin: version 1.4 through 2.1

  For users on JDK 8, specific instructions can be found in the section Scan the Projects on JDK Version 8.

• Make sure your repository includes one or more files with .kt extension.

Build Kotlin projects

Before initiating a scan with Endor Labs, ensure that your Kotlin projects are built successfully. Additionally, ensure that the packages are downloaded into local package caches and build artifacts are present in their standard locations. Follow the guidelines to use Gradle and Maven:

Use Gradle

To analyze your software built with Gradle, Endor Labs requires:

• The software must be successfully built with Gradle.
• For quick scans, dependencies must be located in the local package manager cache. The standard $GRADLE_USER_HOME/caches or /User/<<username>>/.gradle/caches cache must exist.
• For deep scans, the target artifact must be generated on the filesystem.

To build your project with Gradle, run the following commands:

1. Specify the Gradle configuration by setting an environment variable.

export endorGradleKotlinConfiguration="compileClasspath"

If you need to override the default configuration, use the command:

export endorGradleKotlinConfiguration="<configuration>"

The default configuration is `runtimeClasspath` if none is provided.

For Android projects, you can set the configuration using:

export endorGradleAndroidConfiguration="<configuration>"

The default configuration for an Android application or library uses the same structure established by Android Studio. For applications, all possible combinations of application variants are examined. For libraries, all possible combinations of library variants are examined. The first variant from the alphabetically sorted list is then selected for runtimeClasspath.

If neither approach provides a value, the default configuration is set to releaseRuntimeClasspath.

2. Confirm an error-free dependency resolution for your project.

gradle dependencies

or, with a Gradle wrapper.

./gradlew dependencies

3. Generate the artifact for deep analysis.

gradle assemble

or, with a Gradle wrapper.

./gradlew assemble

Override sub project level configuration

In a multi-build project, if you set the environment variable endorGradleKotlinConfiguration=[GlobalConfiguration] and/or endorGradleAndroidConfiguration=[GlobalConfiguration], the specified configuration is used for dependency resolution across all projects and sub-projects in the hierarchy below.

    \--- Project ':samples'
         +--- Project ':samples:compare'
        +--- Project ':samples:crawler'
        +--- Project ':samples:guide'
        +--- Project ':samples:simple-client'
        +--- Project ':samples:slack'
        +--- Project ':samples:static-server'
        +--- Project ':samples:tlssurvey'
        \--- Project ':samples:unixdomainsockets'

To override the configuration only for the :samples:crawler and :samples:guide sub-projects, follow these steps:

1. Navigate to the root workspace, where you execute endorctl scan, and run ./gradlew projects to list all projects and their names.

2. Run the following command at the root of the workspace:

echo ":samples:crawler=testRuntimeClasspath,:samples:guide=macroBenchMarkClasspath" >> .endorproperties

This creates a new file named .endorproperties in your root directory. This enables different configurations for the specified sub-projects in the file.

3. Run endorctl scan as usual.

At this point, all other projects will adhere to the GlobalConfiguration. However, the :samples:crawler sub-project will use the testRuntimeClasspath configuration, and the :samples:guide sub-project will use the macroBenchMarkClasspath configuration.

Use Maven

To analyze your software built with Maven, Endor Labs requires:

• The software must be successfully built with Maven.
• For quick scans, dependencies must be located in the local package manager cache. The standard .m2 cache must exist.
• For deep scans, the target artifact must be generated on the filesystem.

To build your project with Maven, run the following commands:

1. Confirm an error-free dependency resolution for your project.

mvn dependency:tree

2. Run mvn install and ensure the build is successful.

   info

   If you want to skip the execution of tests during the build, you can use -DskipTests to quickly build and install your projects.

mvn install -DskipTests

3. If you have multiple Kotlin modules not referenced in the root pom.xml file, ensure to run mvn install separately in each directory.

Configure Maven private registries

If you have a private registry with internal dependencies on other projects, you must configure private registries. Endor Labs not only supports scanning public Kotlin projects and repositories but also allows integration with private registries.

Follow these steps to configure Maven private registries:

1. Sign in to Endor Labs and navigate to Integrations under Manage from the left sidebar.
2. In the Package Managers > Maven section, click Manage.
3. On the Maven integrations page, click Add Package Manager.
4. Specify a package manager URL.
5. (Optional) To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and specify the username and password of your private package manager repository.
6. Click Add Package Manager to save your configuration.

Run a scan

To scan your repositories with Endor Labs, you can use the following options after building your Kotlin projects.

Option 1 - Quick scan

To quickly gain insight into your software composition, initiate a quick scan using the following command:

endorctl scan --quick-scan

This scan offers a quick overview without performing reachability analysis, helping you prioritize vulnerabilities.

Save local results

To scan a Git project repository from the root directory and save the results locally in the results.json file, use the following command:

endorctl scan --quick-scan -o json | tee /path/to/results.json

This generates comprehensive results and analysis information, accessible from the Endor Labs user interface.

Access results

To access and review detailed results, sign in to the Endor Labs user interface. Navigate to Projects on the left sidebar, and locate your project for a thorough examination of the scan results.

Option 2 - Deep scan

To perform dependency resolution and reachability analysis, use deep scan with Endor Labs. This option is recommended only after successfully completion of quick scan.

endorctl scan

Save local scan results

To save the local results to a results.json file, use the following flag.

endorctl scan -o json | tee /path/to/results.json

This generates comprehensive results and analysis information, accessible from the Endor Labs user interface.

Analyze private packages

During deep analysis, Endor Labs thoroughly analyzes all private software dependencies that have not been previously scanned. While this initial operation may slow down scans, subsequent scans remain unaffected.

If your organization does not own specific software parts and related findings are non-actionable, you can choose to disable this analysis using the disable-private-package-analysis flag. Disabling private package analysis enhances scan performance but may result in a loss of insights into how applications interact with first-party libraries.

To disable private package analysis, use the following command flag:

endorctl scan --disable-private-package-analysis

Access scan results

To access and review detailed results, sign in to the Endor Labs user interface. Navigate to Projects on the left sidebar, and locate your project for a thorough examination of the scan results.

Scan the projects on JDK version 8

While Endor Labs primarily supports JDK versions between 11-22, you can still scan projects on JDK 8 by following these steps:

1. Build your Java project on JDK 8.
2. After successful build, switch your Java home to JDK 11 or higher versions.

export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home

3. Run a scan.

Understand the scan process

Endor Labs analyzes your Kotlin code and dependencies to identify known security issues, including open-source vulnerabilities.

How Endor Labs resolves dependencies for package versions

Endor Labs resolves Kotlin package dependencies by considering the following factors:

• For packages built with Maven, it leverages the Maven cache in the .m2 directory of your file system. This mirrors Maven’s build process for precise results.
• For packages built with Maven, it respects the configuration settings present in the settings.xml file. If the file is included in your repository, any additional configuration is not necessary.
• For packages built with Gradle, it leverages Gradle and Gradle wrapper files to build and resolve dependencies.
• Endor Labs supports AAR, EAR, JAR, RAR, and WAR files.

How Endor Labs performs static analysis on the code

Endor Labs performs static analysis on the code based on the following factors:

• Call graphs are created for your package. These are then combined with the call graphs of the dependencies in your dependency tree to form a comprehensive call graph for the entire project.
• Endor Labs performs an inside-out analysis of the software to determine the reachability of dependencies in your project.
• The static analysis time may vary depending on the number of dependencies in the package and the number of packages in the project.

Known limitations

• If a package can not be successfully built in the source control repository, static analysis will fail.
• Spring dependencies are analyzed based on spring public entry points to reduce the impact of Inversion of Control (IOC) frameworks. Dependencies and functions are identified as reachable and unreachable in the context of a spring version and its entry points.
• Annotation processing is limited only to the usage of the code they annotate.
• Static analysis of reflection and callbacks are not supported.
• If Endor Labs fails to resolve dependencies using default Kotlin configurations, the Kotlin configuration must be specified.
• Static analysis for Kotlin projects using Gradle is only supported when the Kotlin plugin for Gradle versions 1.5.30 to 1.9.x

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Host system check failure errors:
  • Java is not installed or not present in the PATH environment variable. Install Java and try again. See Java documentation for more information.
  • For android applications, $ANDROID_HOME must be specified as an environment variable.
  • The installed version of Java is lower than the required version. Install JDK versions between 11-22 and try again.
  • Java is installed but Maven or Gradle is not installed. In such cases, the dependency resolution may not be complete.
• Unresolved dependency errors: Maven is not installed properly or the system is unable to build root pom.xml. Run mvn dependency:tree in the root of the project and try again. In such cases, the dependency resolution may not be complete.
• Resolved dependency errors: A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.
• Call graph errors:
  • If the project is not compiled, call graphs are not generated. Run gradlew compileKotlin or gradlew compileReleaseKotlin for android based projects before running the scan.
  • Sometimes, the project is not compiled, if a Kotlin version discrepancy exists between the required repository version and the version on the system running the scan. For example, the Kotlin required version is 1.4 but the system has lower version installed. Install the required version and try again.
• If you have a private registry and internal dependencies on other projects, you must configure the credentials of the registry. See Configure Maven private registries.
• If you use a remote repository configured to authenticate with a client-side certificate, you must add the certificate through an endorctl parameter. Export the ENDOR_SCAN_JVM_PARAMETERS parameter before performing a scan. See Maven documentation for details.

export ENDOR_SCAN_JVM_PARAMETERS="-Xmx16G,-Djavax.net.ssl.keyStorePassword=changeit,
-Djavax.net.ssl.keyStoreType=pkcs12,
-Djavax.net.ssl.keyStore=/Users/myuser/Documents/nexustls/client-cert1.p12"

1.3 - Go

Go or Golang is a software development programming language widely used by developers. Endor Labs supports scanning and monitoring of Go projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

• Make sure that you have Go 1.12 or higher versions.
• Make sure your repository includes one or more files with .go extension.

Build Go projects

You must build your Go projects before running the scan. Additionally, ensure that the packages are downloaded into the local package caches and that go.mod file well formed and is available in the standard location.

To ensure that your go.mod file is well formed, run the following command:

go mod tidy

go get ./

This removes any dependencies that are not required by your project and ensures to resolve the dependencies without errors.

Run a scan

Use the following options to scan your repositories. Perform the endorctl scan after building the projects.

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

endorctl scan --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --quick-scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

endorctl scan

Use the following flags to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs resolves your Golang-based dependencies by leveraging built-in Go commands to replicate the way a package manager would install your dependencies.

To discover package names for Go packages Endor Labs uses the command:

GOMOD=off go list -e -mod readonly -json -m

To analyze the dependency graph of your package Endor Labs uses the command:

GOMOD=off go list -e -deps -json -mod readonly all

To assess external dependencies, specifically third-party packages or libraries that your Go project relies on, Endor Labs uses the command:

GOMOD=off go list -e -deps -json -mod vendor all

These commands allow us to assess packages’ unresolved dependencies, analyze the dependency tree, and resolve dependencies for your Go projects.

Known Limitations

Endor Labs creates go.mod files for you when projects do not have a go.mod file. This can lead to inconsistencies with the actual package created over time and across versions of the dependencies.

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Host system check failure errors:

  • Go is not installed or not present in the PATH environment variable. Install Go and try again.
  • The installed version of Go is lower than 1.12. Install Go version 1.12 or higher and try again.

• Resolved dependency errors:

  • A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.
  • If the go.mod file is not well-formed then dependency resolution may return errors. Run go mod tidy and try again.

• Call graph errors:

  These errors often mean the project won’t build. Please ensure any generated code is in place and verify that go build ./... runs successfully.

1.4 - Python

Python is a high-level, interpreted programming language widely used by developers. Endor Labs supports the scanning and monitoring of Python projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

Ensure that the following prerequisites are complete:

• Install Python 3.6 or higher versions. See the Python documentation on how to install Python.
• Ensure that the package manager pip, Poetry, or PDM is used by your projects to build your software packages. From Python 3.12, install setuptools if you use pip.
• Set up any build, code generation, or other dependencies that are required to install your project’s packages.
• Organize the project as one or more packages using setup.py, setup.cfg, pyproject.toml, or requirements.txt package manifest files.
• Make sure your repository includes one or more files with .py extension or pass either one of requirements.txt, setup.py, setup.cfg or pyproject.toml using the --include-path flag. See Scoping scans.

Build Python projects

You must create a virtual environment and build your Python projects before running the endorctl scan. Additionally, ensure that the packages are downloaded into the local package caches and that the build artifacts are present in the standard locations.

1. Configure any private repositories
   • If you use dependencies from a PyPI compatible repository other than pypi.org, configure it in the Integrations section of the Endor Labs web application. See Configure Python private repositories for more details.
2. Clone the repository and create a virtual environment inside it

   1. Clone the repository using git clone or an equivalent workflow.

   2. Enter the working copy root directory that’s created.

   3. Create a virtual environment based on your package manager:

      For pip or setuptools

      • Use python3 -m venv venv. Set up the virtual environment in the root folder that you want to scan and name it venv or .venv.
      • Install your project’s dependencies using venv/bin/python -m pip install -r requirements.txt or venv/bin/python -m pip install.
      • If the virtual environment is created outside the project, use one of the ways defined in Virtual environment support to specify the path of the Python virtual environment to Endor Labs.

      For Poetry projects

      • Install your project’s dependencies using poetry install.

      For PDM projects

      • Install your project’s dependencies using pdm sync.

Virtual environment support

For Poetry and PDM, virtual environments endorctl automatically picks the virtual environments.

For pip, you need to use one of the following ways to specify the virtual environment details of your Python projects for both quick and deep scans.

• Set up the virtual environment in the root folder that you want to scan and name it venv or .venv, it is automatically picked up by the Endor Labs application.

export PYTHONPATH=/usr/tmp/venv:/usr/tmp/another-venv

• Set the environment variable ENDOR_SCAN_PYTHON_VIRTUAL_ENV to the path of the virtual environment of your Python project.

export ENDOR_SCAN_PYTHON_VIRTUAL_ENV=/usr/tmp/venv

• Set the environment variable ENDOR_SCAN_PYTHON_GLOBAL_SITE_PACKAGES to true to indicate that a virtual environment is not present and Endor Labs can use the system-wide Python installation packages and modules.

export ENDOR_SCAN_PYTHON_GLOBAL_SITE_PACKAGES=true

If you do not set up the virtual environment, Endor Labs attempts to set it up with all the code dependencies, however, we recommend that you install all dependencies in a virtual environment for the most accurate results.

If you are using custom scripts without manifest files to assemble your dependencies, make sure to set up the virtual environment and install the dependencies.

Configure private Python repositories

In addition to scanning public Python projects, Endor Labs provides support to fetch and scan private Python repositories. Endor Labs will fetch the resources from the authenticated endpoints and perform the scan, and you can view the dependencies and findings.

1. Sign in to Endor Labs and select Integrations under Manage from the left sidebar.
2. From Package Managers, select PyPI and click Manage.
3. Click Add Package Manager.
4. Enter a package manager URL.
5. To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and enter the username and password of your private package manager repository.
6. Click Add Package Manager to save your configuration.

Run a scan

Use the following options to scan your repositories. Perform the endorctl scan after building the projects.

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition and perform dependency resolution. It discovers dependencies that the package has explicitly declared. If the package’s build file is incomplete then the dependency list will also be incomplete. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

endorctl scan --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --quick-scan -o json | tee /path/to/results.json

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully. The deep scan performs the following operations for the Python projects.

• Discovers explicitly declared dependencies,
• Discovers project dependent OSS packages present in the venv/global and scope/python.
• Performs reachability analysis and generates call graphs.
• Detects dependencies used in source code but not declared in the package’s manifest files called phantom dependencies.

endorctl scan

Use the following flags to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

When a deep scan is performed all private software dependencies are completely analyzed by default if they have not been previously scanned. This is a one-time operation and will slow down initial scans, but won’t impact subsequent scans.

Organizations might not own some parts of the software internally and the related findings are not actionable by them. They can choose to disable this analysis using the flag disable-private-package-analysis. By disabling private package analysis, teams can enhance scan performance but may lose insights into how applications interact with first-party libraries.

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs uses the following two methods to analyze your Python code.

• Dependency resolution using manifest files
• Dependency resolution using static analysis

Endor Labs uses the results from both these methods to perform superior dependency resolution, identify security issues, detect open-source vulnerabilities, and generate call graphs.

Dependency resolution using manifest files

In this method, Endor Labs analyzes the manifest files present in a project to detect and resolve dependencies. The manifest files are analyzed in the following priority.

For Poetry and PDM, if both lock and toml files are available, both are analyzed based to detect and resolve dependencies.

For pip, the dependency resolution is as follows, where the first available file in the priority list is analyzed to detect and resolve dependencies, and others are ignored.

On initialization of a scan, Endor Labs identifies the package manager by inspecting files such as the pyproject.toml, poetry.lock, pdm.lock, setup.py, and requirements.txt. When the files, poetry.lock or pyproject.tomlfiles are discovered, Endor Labs will use the Poetry package manager to build the project. When the files, pdm.lock or pyproject.toml files are discovered, Endor Labs will use the PDM package manager. Otherwise, it will use pip3.

Example

This is an example that demonstrates scanning a Python repository from GitHub on your local system using the endorctl scan. Here we are assuming that you are running the scan on a Linux or Mac operating system environment and that you have the following Endor Labs API key and secret stored in the environment variables. See endorctl flags and variables.

• ENDOR_API_CREDENTIALS_KEY set to the API key
• ENDOR_API_CREDENTIALS_SECRET set to the API secret
• ENDOR_NAMESPACE set to your namespace (you can find this when logged into Endor Labs by looking at your URL: https://app.endorlabs.com/t/NAMESPACE/...; it is typically a form of your organization’s name)

pip

git clone https://github.com/HybirdCorp/creme_crm.git
cd creme_crm
python3 -m venv venv
source venv/bin/activate
venv/bin/python3 -m pip install
endorctl scan

Poetry

git clone https://github.com/HybirdCorp/creme_crm.git
cd creme_crm
poetry lock
endorctl scan

PDM

git clone https://github.com/HybirdCorp/creme_crm.git
cd creme_crm
pdm sync
endorctl scan

The scan for this repository is expected to be completed in a few minutes depending on the size of the project. You can now visit app.endorlabs.com, navigate to Projects, and choose the helloflas/flask-examples project to see your scan results.

Dependency resolution using static analysis

All Python projects do not always include manifest files. A project can be a series of install statements that are assembled by custom scripts. Even when manifest files are present, the dependency information and version declared in the manifest file may be drastically different from what is used in a project.

To solve this problem, Endor Labs has developed a unique method for dependency resolution by performing a static analysis on the code, giving you complete visibility of what is used in your code.

• Endor Labs enumerates all Python packages and recognizes the import statements within the project. An import statement is a Python code statement that is used to bring external modules or libraries into your Python script.
• It performs a static analysis of the code to match the import statements with the pre-installed packages and recursively traverses all files to create a dependency tree with the actual versions that are installed in the virtual environment.
• It detects the dependencies at the system level to identify which ones are resolved and retrieves the precise name and version information from the library currently in use.
• Also, it gives you accurate visibility into your project components and helps you understand how the components depend on one another.

Through this approach, Endor Labs conducts comprehensive dependency management, assesses reachability, and generates integrated call graphs.

Known Limitations

• Endor Labs specifically looks for the requirements.txt file for a Python project using pip. If you use a different file name, it won’t be automatically discovered.
• Python versions older than 3.7 are not supported but may work as expected.
• If a virtual environment is not provided, Python version constraints are not assumed based on the runtime environment of CI. Dependencies are shown for all possible versions of Python at runtime. If a virtual environment is provided, Endor Labs respects what is installed in the virtual environment.
• Symbolic links into manifest files may result in the same package being duplicated in the project.
• If a dependency is not available in the PyPI repository or in a configured private package repository, Endor Labs will be unable to build the software and scans may fail without first building the package in the local environment successfully.

Call Graph Limitations

• Function calls using dispatch table calls might not be included in the call graph.
• Function calls using unresolved variables might not be included in the call graph.
• Dynamically modified or extended function calls used to declare methods or attributes at run time might not be included in the call graph.
• Functions called indirectly through a function pointer and not by their direct name, might not be included in the call graph.
• Type stubs that provide hints or type annotations for functions, methods, and variables in your Python modules or libraries have to be installed manually before performing a scan.
• If your project has a pyproject.toml file that includes tools.pyright section, it overrides Endor Labs settings for Pyright and may result in incorrect call graph results. You will need to remove the tools.pyright section from the pyproject.toml file.

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Virtual environment errors: You can identify the errors that may occur during virtual environment installation by looking for the following message in the error logs; failed to create virtual environment or failed to install dependencies.
• Missing environment dependency: If your code depends on packages such as psycopg2, environment dependencies such as PostgreSQL are also required. The endorctl scan may fail if the environment where it is running does not have PostgreSQL installed.
• Incompatible Python version: The default Python version in the environment where the endorctl scan is running is incompatible with one or more of the dependencies that are needed by the code.
• Incompatible architecture: One or more dependencies are not compatible with the operating system architecture of the local system on which you are running the endorctl scan. For example, projects with dependency on PyObjC can be run on Mac-based systems, but not Linux systems. A few Python libraries are incompatible with x32 architectures and can only be run on x64 architectures.
• Resolved dependency errors: A version of a dependency does not exist, or it cannot be found. It may have been removed from the repository.
• Call graph errors: These errors come if pip or Poetry are unable to build the project because a required dependency cannot be located.

1.5 - JavaScript/TypeScript

JavaScript is a high-level, interpreted programming language primarily used for creating interactive and dynamic web content widely used by developers. Endor Labs supports the scanning and monitoring of JavaScript projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

• Endor Labs requires the following pre-requisite software to be installed to successfully perform a scan:
  • Yarn: Any version
  • npm: 6.14.18 or higher versions
  • pnpm: 3.0.0 or higher versions
• Make sure your repository includes one or more files with .js or .ts extension.

Build JavaScript projects

You can choose to build your JavaScript projects before running a scan. This will ensure that either a package-lock.json, yarn.lock, or pnpm-lock.yaml file is created enhancing the scan speed.

Ensure your repository has package.json and run the following command making sure it builds the project successfully.

For npm:

npm install

For Yarn:

yarn install

For pnpm:

pnpm install

If the project is not built, endorctl builds the project during the scan and generate either package-lock.json, yarn.lock, or pnpm-lock.yaml file. Make sure that either npm, Yarn, or pnpm is installed on your system. If your repository includes a lock file, endorctl uses the existing file for dependency resolution and does not create it again.

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies.

endorctl scan

Detect dependencies with pnpm

If you are using pnpm, set the environment variable ENDOR_PNPM_ENABLED to true and then run the scan.

export ENDOR_PNPM_ENABLED=true

endorctl scan

Enable dependency reachability

To enable dependency reachability that detects dependencies used in source code but not declared in the package’s manifest files, set the flag --call-graph-languages with javascript,typescript.

endorctl scan --call-graph-languages=javascript,typescript --build

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Enable call graphs (Beta)

To enable call graphs for JavaScript and TypeScript projects use the following process.

Prerequisites:

• Ensure endorctl version is 1.6.594 or higher.
• Ensure Node.js version 4.2.6 or higher is installed to support TypeScript version 4.9.
• Ensure TypeScript version 4.7 or higher is installed.

1. Install tsserver. tsserver is included with TypeScript, so installing the appropriate TypeScript version automatically installs tsserver. Follow these steps based on your Node.js version:

   • For Node.js versions lower than 12.2, install TypeScript version 4.9:

     npm install -g typescript@4.9
     

   • For Node.js versions between 12.2 and 14.17, install TypeScript version 5.0:

     npm install -g typescript@5.0
     

   • For Node.js version higher than or equal to 14.17, install the latest TypeScript version:

     npm install -g typescript
     

   • Check the tssserver installation.

     # Run 'which tsserver' to confirm installation
     which tsserver
     /opt/homebrew/bin/tsserver
     

   If you are running the endorctl scan with --install-build-tools, you need not install tssserver. See Manage build tools for more information.

2. Run the endorctl scan by setting the environment variable ENDOR_JS_ENABLE_TSSERVER=true and the flag --call-graph-languages=javascript,typescript followed by --build.

ENDOR_JS_ENABLE_TSSERVER=true endorctl scan --call-graph-languages=javascript,typescript --build

Understand the scan process

Dependency analysis tools analyze the lock file of an npm, yarn, or pnpm based package and attempt to resolve dependencies. To resolve dependencies from private repositories, the settings of the .npmrc file in the repository is considered.

Endor Labs surpasses mere manifest file analysis by expertly resolving JavaScript dependencies and identifies:

• Dependencies listed in the manifest file but not used by the application
• Dependencies used by the application but not listed in the manifest file
• Dependencies listed in the manifest as transitive but used directly by the application
• Dependencies categorized as test in the manifest, but used directly by the application

Developers can eliminate the false positives, false negatives, and easily identify test dependencies with this analysis. The dependencies used in source code but not declared in the package’s manifest files are tagged as Phantom.

Endor Labs also supports npm, Yarn, and pnpm workspaces out-of-the-box. If your JavaScript frameworks and packages use workspaces, Endor Labs will automatically take the dependencies from the workspace to ensure that the package successfully builds.

Scan speed is enhanced if the lock file exists in the repository. endorctl does not perform a build and uses the existing files in the repository for analysis.

Known Limitations

• Endor Labs doesn’t currently support local package references
• If a dependency can not be resolved in the lock file, building that specific package may be unsuccessful. This package may have been removed from npm or the .npmrc file is not properly configured. Other packages in the workspace are scanned as usual.

Call graph limitations

• Functions that are passed in as arguments to call expressions might not be included in the call graph.
• Functions that are returned and then called might not be included in the call graph.
• Functions that are assigned to a variable based on a runtime value might not be included in the call graph.
• Functions that are assigned to an array element might not be included in the call graph.

Troubleshoot errors

• Unresolved dependency errors: The manifest file package.json is not buildable. Try running npm install, yarn install, or pnpm install in the root project to debug this error.
• Resolved dependency errors: A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.

1.6 - Ruby

Ruby is a widely used open-source programming language. Endor Labs supports scanning and monitoring of Ruby projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

Software prerequisites

The following prerequisites must be fulfilled:

• All applications monitored by Endor Labs must be on Ruby versions 2.6 or higher.
• A Gemfile or a *.gemspec file must be present in your Ruby project.
• Make sure your repository includes one or more files with .rb extension.

Build Ruby projects

You can choose to build your Ruby projects before running a scan. This will ensure thatgemfile.lock is created.

Ensure your repository has Gemfile and run the following command making sure it builds the project successfully.

bundler install

If the project is not built, endorctl will build the project during the scan and generate Gemfile.lock. If the repository includes a Gemfile.lock, endorctl uses this file for dependency resolution and does not create it again.

Configure Ruby private Gem sources

If you have a private registry and internal dependencies on other projects, you must configure private registries. Endor Labs can fetch the gems that are hosted in a private gem repository which are not available publicly after authenticating the endpoints.

1. Sign in to Endor Labs and select Integrations under Manage from the left sidebar.
2. From Package Managers, select Ruby Gems and click Manage.
3. Click Add Package Manager.
4. Enter a package manager URL.
5. To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and enter the username and password of your private package manager repository.
6. Click Add Package Manager to save your configuration.

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies.

endorctl scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs looks for Gemfile, *.gemspec, and Gemfile.lock files to find and monitor the dependency activity.

• A Gemfile is a configuration file used in Ruby projects to specify the required RubyGems (libraries or packages) for the project’s dependencies.
• A *.gemspec file is a RubyGems specification file used to define the metadata and dependencies for a RubyGem.
• The Gemfile.lock file is automatically generated by Bundler and is used to lock down the specific versions of gems and their dependencies installed in your project’s environment.

If the Gemfile.lock is not present in your project, Endor Labs generates this file and stores it in a temp directory. The file is deleted after extracting dependency information.

Endor Labs’ dependency resolution mechanism assesses multiple factors, including compatibility, stability, and availability, to determine the most suitable version for usage. The resolved dependency version is used during the build or execution of your Ruby project. By utilizing the dependency graph, you can access significant information regarding the dependencies. This includes determining whether a dependency is direct or transitive, checking its reachability, verifying source availability, and more. The dependency graph provides a visual representation that allows you to examine the graphical details of these dependencies.

Known limitations

• Call graphs are not supported for Ruby projects.
• If a dependency can not be resolved in the Gemfile, building that specific package may not be successful. This package may have been removed from the Gem package manager. Other packages in the workspace are scanned.

Troubleshoot errors

• Unresolved dependency errors: The Gemfile is not buildable. Try running bundler install in the root project to debug this error.
• Resolved dependency errors: A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.

1.7 - .NET

.NET is a free, cross-platform, open-source developer platform for building different types of applications. Endor Labs supports the scanning and monitoring of projects built on the .NET platform.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

The following prerequisites must be fulfilled:

• Make sure your repository includes one or more files with .cs extension.
• One or more *.csproj files must be present in your repository.
• The .NET command or Nuget command must be installed and available on the host system.
• At least one .NET SDK installed on the system must be compatible with the project’s global.json file settings.

Call graph requirements

• You must install .NET 7.0.1 (SDK 7.0.101) or later on the host system.
• Endor Labs’ call graph support for .NET is based on Microsoft’s Common Intermediate Language (CIL). Artifacts such as .exe or .dll files must be available in the project’s standard workspace through a build and restore or a restored cache.

Run a scan

Use the following options to scan your repositories. Perform a scan after building the projects.

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

You must restore your .NET projects before running a quick scan. Additionally, ensure that the packages are downloaded into the local package caches and that the build artifacts are present in the standard locations.

1. Run the following commands to resolve dependencies and create the necessary files to scan your .NET project.

To ensure that the build artifacts project.assets.json file is generated and dependencies are resolved run:

dotnet restore

If you use Nuget instead run:

nuget restore

To create a packages.lock.json file if your project uses a lock file run:

dotnet restore --use-lock-file

If project.assets.json or packages.lock.json are not present and if the project is buildable, endorctl will restore the project and create a project.assets.json or a packages.lock.json file to resolve dependencies. 2. You can run a quick scan with the following commands:

endorctl scan --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --quick-scan -o json | tee /path/to/results.json

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

You must restore and build your .NET projects before running a deep scan. Additionally, ensure that the packages are downloaded into the local package caches and that the build artifacts are present in the standard locations.

1. Run the following commands to restore and build your project. This may vary depending on your project’s configuration.

dotnet restore
dotnet build

2. You can run a deep scan with the following commands:

endorctl scan

Use the following flags to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

When a deep scan is performed all private software dependencies are completely analyzed by default if they have not been previously scanned. This is a one-time operation and will slow down initial scans, but won’t impact subsequent scans.

Organizations might not own some parts of the software internally and findings are actionable by another team. These organizations can choose to disable this analysis using the flag disable-private-package-analysis. By disabling private package analysis, teams can enhance scan performance but may lose insights into how applications interact with first-party libraries.

Use the following command flag to disable private package analysis:

endorctl scan --disable-private-package-analysis

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Configure Nuget private repositories

You can configure Endor Labs to fetch dependencies from private NuGet feeds after authenticating the endpoints.

1. Sign in to Endor Labs and select Integrations under Manage from the left sidebar.
2. From Package Managers, select Nuget and click Manage.
3. Click Add Package Manager.
4. Enter a package manager URL.
5. To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and enter the username and password of your private package manager repository.
6. Click Add Package Manager to save your configuration.

Understand the scan process

A *.csproj file is an XML-based C# project file that contains information about the project, such as its source code files, references, build settings, and other configuration details. The dependencies and findings are listed individually for every csproj file. The scan discovers all *.csproj** files and uses these files to resolve the appropriate dependency graph of your project.

(Beta) Endor Labs scans the .NET projects that are using the Central Package Management feature of Nuget for the packages declared as:

• Package references in Directory.Build.props or Directory.Packages.props files.
• Package references in any *.props file and the prop file is imported in the csproj file.
• Package references in *.Targets file.

Endor Labs enriches your dependency graph to help you understand if your dependencies are secure, sustainable, and trustworthy. This includes Endor Labs risk analysis and scores, if a dependency is direct or transitive, and if the source code of the dependency is publicly auditable.

Software composition analysis for .NET is performed in the following ways:

• Using Project.assests.json
• Using package.lock.json

How does dependency resolution happen using Project.assets.json

The Project.assets.json file is used in .NET projects to store metadata and information about the project’s dependencies and assets.

Endor Labs fetches resolved package versions, paths to the dependencies’ assets, such as assemblies and resources, and other related information from this file. If a project does not include a project.assets.json file, it is generated through the dotnet restore or the nuget restore command. This command uses all the configured sources to restore dependencies as well as project-specific tools that are specified in the project file.

How does dependency resolution happen using package.lock.json

The package.lock.json file is used in .NET projects to lock dependencies and their specific versions. It is a snapshot of the exact versions of packages installed in a project, including their dependencies and sub-dependencies, requested versions, resolved versions, and contenthash. The lock file provides a more dependable, uniform, and accurate representation of the dependency graph.

In Endor Labs’ dependency management, the resolution of dependencies is primarily based on package.lock.json, which takes precedence over projects.assets.json to resolve dependencies.

Endor Labs fetches the dependency information from package.lock.json and creates a comprehensive dependency graph. The vulnerabilities associated with the dependencies are listed on the Endor Labs’ user interface.

If the package.lock.json file is not present in the repository, Endor Labs triggers the restore process to generate the package.lock.json file and uses it to perform the dependency scans.

How Endor Labs performs static analysis on the code (Beta)

Endor Labs performs static analysis on the C# code based on the following factors:

• Call graphs are created for your package. These are then combined with the call graphs of the dependencies in your dependency tree to form a comprehensive call graph for the entire project.
• Endor Labs looks for the project’s .dll files typically located within the bin directory.
• Endor Labs performs an inside-out analysis of the software to determine the reachability of dependencies in your project.
• The static analysis time may vary depending on the number of dependencies in the package and the number of packages in the project.

Known Limitations

• The following .NET programming languages are not currently supported for dependency resolution or call graph generation.
  • Projects written in F#
  • Projects written in Visual Basic
• When using the GitHub app, either resolve all the private and internal dependencies, or Configure Nuget private repositories before running a scan.
• Call graph binary discovery may fail if you use environment variables to define the name of the package in your *.csproj file.

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Host system check failure errors: .NET or Nuget is not installed or not present in the PATH environment variable. Install Nuget and try again.
• Unresolved dependency errors: This error occurs when the .csproj file can not be parsed or if it has syntax errors.

1.8 - Bazel

Bazel is an open-source build and test tool, which is commonly used in monorepos. Bazel is used to quickly build software across multiple languages.

Using Endor Labs, organizations relying on Bazel can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for deep scan

Software prerequisites

Ensure that the following prerequisites are in place for a successful scan.

• One or more WORKSPACE files must be present in your repository.
• The bazel command must be installed and available on the host system. To install Bazel, see the Bazel documentation.
• Bazel versions 5.x.x and 6.x.x are supported.

Select and build your Bazel targets

Endor Labs supports scanning targets using the following language-specific Bazel rules:

• Bazel Rules for Java support included for 4.1 or higher versions.

  • java_library

• Bazel Rules for Python support included for 0.9.0 or higher versions.

  • py_binary
  • py_library

• Bazel Rules for Golang support included for 0.40.1 or higher versions with Bazel versions 5.x.x, 6.x.x. 0.42.0 or higher versions with Bazel versions 7.x.x.

  • go_binary
  • go_library
  • go_image
  • Bazel with Gazelle in vendored mode

• Bazel Rules for Scala support is available for versions 5.0.0 to 6.6.0, including 5.0.0 and 6.6.0.

  • scala_binary
  • scala_library

• To scan with Endor Labs you’ll need to specify a list of targets to scan or a Bazel query for which all targets matching your query criteria are scanned.

• To select your Bazel targets you can search using the Bazel query language for specific targets to scan, or scan all targets that match a given query.

Here are some examples.

• To get all targets created with a java_binary build rule you can use the query:

bazel query 'kind(java_binary, //...)'

• To get all targets created with a py_binary build rule you can use the query:

bazel query 'kind(py_binary, //...)'

• To get all targets created with a go_binary build rule under the golang directory you can use the query:

bazel query 'kind(go_binary, //...)'

• To get all targets created with a scala_binary build rule under the scala directory you can use the query:

bazel query 'kind(scala_binary, //...)'

In general, refine your query to select the most important targets in your monorepo and align with your existing build workflows.

You can choose to build the targets before running the scan. Use the bazel build commands to do this by passing a comma-separated list of targets. For example, //:test and //:test2 run bazel build //:test,//:test2. endorctl will attempt a scan if the targets are not built. endorctl uses bazel build //:test and bazel query 'deps( //:test)' --output graph to build your targets.

Run a scan

Use the following options to scan your repositories. Perform a scan after building the projects. See

Once you’ve selected your targets you can define the targets for scanning using one of three command line arguments:

To include or exclude specific targets, you can pass a comma-separated list of targets to --bazel-exclude-targets or --bazel-include-targets.

To scan a specific list of targets, use the command:

endorctl scan --use-bazel --bazel-include-targets=//your-target-name

To scan a list of targets using the Bazel query language, use the following command and replace the following query with your own.

endorctl scan --use-bazel --bazel-targets-query='kind(java_binary, //...)

If your workspace file is not located at the root of the repository you must define the WORKSPACE file location for the targets you would like to scan.

For example:

endorctl scan --use-bazel --bazel-targets-query='kind(java_binary, //...) --bazel-workspace-path=./src/java

For Go projects, if you use Bazel with Gazelle in vendored mode, you need to provide the path of your go.mod file in the command option, --bazel-vendor-manifest-path.

For example:

endorctl scan --use-bazel --bazel-include-targets=//your-target-name --bazel-vendor-manifest-path=<path to go.mod>

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

endorctl scan --use-bazel --bazel-include-targets=//your-target-name --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --use-bazel --bazel-include-targets=//your-target-name --quick-scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

You can run a deep scan with the following commands:

endorctl scan --use-bazel --bazel-include-targets=//your-target-name

Use the following flags to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --use-bazel --bazel-include-targets=//your-target-name -o json | tee /path/to/results.json

When a deep scan is performed all private software dependencies are completely analyzed by default if they have not been previously scanned. This is a one-time operation and will slow down initial scans, but won’t impact subsequent scans.

Organizations might not own some parts of the software internally and findings are actionable by another team. These organizations can choose to disable this analysis using the flag disable-private-package-analysis. By disabling private package analysis, teams can enhance scan performance but may lose insights into how applications interact with first-party libraries.

Use the following command flag to disable private package analysis:

endorctl scan --use-bazel --bazel-include-targets=//your-target-name --disable-private-package-analysis

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

To understand the scan process for Java projects, see Endor Labs for Java.

To understand the scan process for Python projects, see Endor Labs for Python.

To understand the scan process for Go projects, see Endor Labs for GO.

Known limitations

• Scanning Java binaries using Bazel is not supported.

1.9 - Swift/Objective-C

CocoaPods is a popular package manager for Swift and Objective-C. It simplifies the integration of Swift and Objective-C dependencies through Podfile declaration and automated installation.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

Software prerequisites

The following prerequisites must be fulfilled:

• All applications monitored by Endor Labs must be on CocoaPods versions 0.9.0 or higher.
• A Podfile and a Podfile.lock must be present in your CocoaPods project.
• Make sure your repository includes one or more files with .swift, .h, or .m extension.

Build CocoaPods projects

If the Podfile.lock is not present in your repository, run the following command to create the Podfile.lock for your Podfile.

pod install

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies. Use the flag --languages=swift along with the endorctl scan to discover Swift and Objective-C dependencies.

endorctl scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs looks for the Podfile and Podfile.lock files to discover the dependencies used by an application.

• A Podfile is a configuration file used in CocoaPods projects to specify the required libraries or packages for the project’s dependencies.
• A Podfile.lock file is a CocoaPods specification file used to define the metadata and dependencies.

To successfully discover Swift and Objective-C dependencies, both Podfile and Podfile.lock files must be present in your project for each Podfile.

Known limitations

• Call graphs are not supported for the CocoaPods projects.
• If a Podfile.lock file is not present, Endor Labs will skip analyzing the project and present a warning that the package was skipped.

1.10 - Scala

Scala is a general-purpose and scalable programming language widely used by developers. Endor Labs supports the scanning and monitoring of Scala projects managed by the interactive build tool sbt.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for scan

Make sure that your system has a minimum 8-core processor with 32 GB RAM to successfully scan Scala projects.

Software prerequisites

The following prerequisites must be fulfilled:

• A manifest file for the Scala build tool, build.sbt must be present in your repository.
• Make sure your repository includes one or more files with .sc or .scala extension.
• If your sbt version is lower than 1.4, you must install the sbt-dependency-graph plugin. The sbt-dependency-graph plug-in is by default integrated into the sbt versions 1.4 and higher.
• Make sure that the project/build.properties file has the required sbt version.

Option 1 - Quick scan

You must be able to build your Scala projects before running a scan. The standard .sbt cache must exist and contain successfully downloaded dependencies.

1. Ensure your repository has a build.sbt file and run the following commands making sure it builds the project successfully.

   sbt compile
   

   sbt projects
   

2. Make sure sbt dependencyTree runs successfully inside the project directory.

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies.

endorctl scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

You must be able to build your Scala projects before running a scan. The build artifact as well as the standard .sbt cache must exist and contain successfully downloaded dependencies.

1. Ensure your repository has a build.sbt file and run the following commands making sure it builds the project successfully.

   sbt package
   

   sbt projects
   

2. Make sure sbt dependencyTree runs successfully inside the project directory.

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies.

endorctl scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs scans Scala projects by executing sbt plugins and inspecting the build.sbt file to retrieve information about direct and transitive dependencies.

• The build.sbt file is a configuration file used in Scala projects with sbt to define project settings, dependencies, and build tasks. This file provides the necessary configuration and instructions to sbt on resolving and managing project dependencies.

• The sbt dependency graph plugin visualizes the dependencies between modules in a Scala project.

Endor Labs analyzes information from both these methods to determine different components, binary files, manifest files, images, and more in the Scala codebase and presents finding policy violations, identifying, and resolving dependencies.

Using Endor Labs users can gain significant insights into the structure and relationships of their Scala project’s dependencies, aiding in managing dependencies effectively, identifying potential issues, and ensuring a well-organized and maintainable codebase.

How Endor Labs performs static analysis on the code

Endor Labs performs static analysis based on the following factors:

• Call graphs are created for your package. These are then combined with the call graphs of the dependencies in your dependency tree to form a comprehensive call graph for the entire project.
• Endor Labs performs an inside-out analysis of the software to determine the reachability of dependencies in your project.
• The static analysis time may vary depending on the number of dependencies in the package and the number of packages in the project.

Known Limitations

• Software composition analysis for Scala on Microsoft Windows operating systems is currently unsupported.
• Scala packages built with Gradle are not currently supported.

Troubleshoot errors

Here are a few error scenarios that you can check for and attempt to resolve them.

• Host system check failure errors: These errors occur if:
  • If sbt is not installed or present in the path variable. Install sbt 1.4 or higher versions and try again.
  • If the sbt version mentioned in the project or the build.properties file is lower than 1.4 and the sbt-dependency-graph plug-in is not installed. Install the sbt-dependency-graph plug-in and try again.
• Dependency graph errors - Scala by default imports MiniDependencyTreePlugin which is a mini version of the sbt-dependency-graph plugin and it supports only dependencyTree command. To get complete features of the sbt-dependency-graph plugin, add DependencyTreePlugin to your project/plugins.sbt file and run the scan again. See Scala documentation for details.

1.11 - PHP

PHP is a popular server-side scripting language primarily used for web development. Endor Labs supports the scanning and monitoring of PHP projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

Software prerequisites

• One of the following prerequisites must be fulfilled:
  • The PHP project must contain a composer.json file. If the project includes the composer.lock file it is beneficial, but this is not a mandatory requirement.
  • If the composer.lock file is not present in the repository, it is necessary to have PHP and Composer installed before running a scan on your local system.
• Make sure your repository includes one or more files with .php extension.
• The following versions are supported for PHP and Composer:
  • PHP 5.3.2 and higher versions
  • Composer 2.2.0 and higher versions

Build PHP projects

You can choose to build your PHP projects before running a scan. This will ensure that composer.lock is created.

Ensure your repository has composer.json and run the following command making sure it builds the project successfully.

composer install

If the project is not built, endorctl will build the project during the scan and generate composer.lock. If the repository includes a composer.lock, endorctl uses this file for dependency resolution and does not create it again.

Configure private Composer package repositories

If you have a private registry and internal dependencies on other projects, you must configure private registries.

To configure private Composer package repositories:

1. Sign in to Endor Labs and select Integrations under Manage from the left sidebar.
2. From Package Managers, select Packagist and click Manage.
3. Click Add Package Manager.
4. In PACKAGE MANAGER HOST, enter the host domain of the package manager.
5. From the available options, choose the authentication method for private repositories.
6. To enable Endor Labs to authenticate to your registry, select Authenticate to this registry and enter the required details of your private package manager repository.
7. Click Add Package Manager to save your configuration.

Run a scan

Perform a scan to get visibility into your software composition and resolve dependencies.

endorctl scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs discovers all composer.json files in your PHP project and uses these files to resolve the dependencies of your packages. Composer is a PHP dependency management tool that enables you to specify the libraries your project relies on and manages the process of installing or updating them. The dependencies and findings are listed in the Endor Labs application individually for every composer.json file.

In Endor Labs’ dependency management, the resolution of dependencies is based on both composer.json and composer.lock files. The composer.lock file is generated by Composer and includes information such as resolved versions, package information, transitive dependencies, and other details. Using the composer.lock file ensures deterministic dependency installation by recording the exact versions of installed dependencies and their transitive dependencies. If the composer.lock file is not present in the repository, Endor Labs generates the composer.lock file, and uses it to analyze the operational and security risks associated with your package’s dependencies. Endor Labs fetches the dependency information and creates a comprehensive dependency graph.

Known Limitations

Call graphs are not supported for PHP projects.

Troubleshoot errors

• Unresolved dependency errors: The composer.json is not buildable. Try running composer install in the root project to debug this error.
• Resolved dependency errors: A version of a dependency does not exist or it cannot be found. It may have been removed from the repository.

1.12 - Rust

Rust is a software programming language widely used by developers. Endor Labs supports scanning and monitoring of Rust projects.

Using Endor Labs, developers can:

• Test their software for potential issues and violations of organizational policy
• Prioritize vulnerabilities in the context of their applications
• Understand the relationships between software components in their applications

System specifications for scan

Make sure that you have a minimum system requirement specification of an 8-core processor with 32 GB RAM.

Use a system equipped with either Mac OS X or Linux operating systems to perform the scans.

Software prerequisites

• Make sure the following prerequisites are installed: - Package Manager Cargo - Any version - Rust - Any version,
• Make sure your repository includes one or more files with .rs extension.
• Install Rust using the latest rustup tool.

Build Rust projects

Ensure your repository has Cargo.toml file and run the following command making sure it builds the project successfully.

cargo build

If the project is not built, endorctl will build the project during the scan and generate the Cargo.lock file. If the repository includes a Cargo.lock file, endorctl uses this file for dependency resolution and does not create it again.

Run a scan

Use the following options to scan your repositories. Perform the endorctl scan after building the projects.

Option 1 - Quick scan

Perform a quick scan to get quick visibility into your software composition and perform dependency resolution. It discovers dependencies that the package has explicitly declared. If the package’s build file is incomplete then the dependency list will also be incomplete. This scan won’t perform the reachability analysis to help you prioritize vulnerabilities.

endorctl scan --quick-scan

You can perform the scan from within the root directory of the Git project repository, and save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

endorctl scan --quick-scan -o json | tee /path/to/results.json

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

endorctl comes bundled as a Docker image for portable usage and integration into continuous integration (CI) pipelines. To configure the endorctl Docker image, you must perform the following steps:

1. Install Docker
2. Get Endor Labs API credentials
3. Export your environment variables
4. Scan Rust projects

Export your environment variables

endorctl requires four environment variables for your source control system and your Endor Labs tenant.

These environment variables are:

• ENDOR_API_CREDENTIALS_KEY - The API key used to authenticate against the Endor Labs API.
• ENDOR_API_CREDENTIALS_SECRET - The API key secret used to authenticate against the Endor Labs API.
• ENDOR_NAMESPACE - The Endor Labs namespace you want to scan against. Locate the namespace from the top left-hand corner, under the Endor Labs logo on the Endor Labs application,.
• SOURCE_PATH - The path to your source code that may be mounted to the Docker container.

To export your environment variables run the following commands and insert the environment variable values in each command.

export ENDOR_NAMESPACE=name-of-your-namespace
export ENDOR_API_CREDENTIALS_KEY=endorlabs-api-key
export ENDOR_API_CREDENTIALS_SECRET=endorlabs-api-secret
export SOURCE_PATH=/path/to/your/source/code

Scan Rust projects

To scan your rust project run the following command:

docker run -it --rm \
 -e ENDOR_NAMESPACE=$ENDOR_NAMESPACE \
 -e ENDOR_API_CREDENTIALS_KEY=$ENDOR_API_CREDENTIALS_KEY \
 -e ENDOR_API_CREDENTIALS_SECRET=$ENDOR_API_CREDENTIALS_SECRET \
 -v $SOURCE_PATH:/root/endorlabs \
 endorcipublic.azurecr.io/endorctl:latest \
 scan --path=/root/endorlabs

Use the following command to save the local results to a results.json file. The results and related analysis information are available on the Endor Labs user interface.

docker run -it --rm \
 -e ENDOR_NAMESPACE=$ENDOR_NAMESPACE \
 -e ENDOR_API_CREDENTIALS_KEY=$ENDOR_API_CREDENTIALS_KEY \
 -e ENDOR_API_CREDENTIALS_SECRET=$ENDOR_API_CREDENTIALS_SECRET \
 -v $SOURCE_PATH:/root/endorlabs \
 endorcipublic.azurecr.io/endorctl:latest \
 scan --path=/root/endorlabs -o json | tee /path/to/results.json

You can sign in to the Endor Labs user interface, click the Projects on the left sidebar, and find your project to review its results.

Understand the scan process

Endor Labs performs the following steps when it scans Rust projects:

• Resolves dependencies for the package version
• Performs static analysis on your Rust code

Resolving Dependencies & Static Analysis for Rust

Endor Labs leverages the Cargo.toml file in Rust and uses this file to build the package version using cargo. Endor Labs uses the output from cargo metadata to resolve dependencies specified in Cargo.toml files and construct the dependency graph.

Perform Static Analysis on Your Rust Code

• After building the package, call graphs are created for your package. These are then combined with the call graphs of the dependencies in your dependency tree to form a comprehensive call graph for the entire project. Use the call graphs to understand if vulnerabilities in your Rust code are reachable through a function associated with the known vulnerability.
• Endor Labs performs an inside-out analysis of the software to determine the reachability of dependencies in your project.
• The static analysis time may vary depending on the number of dependencies in the package and the number of packages in the project.

Known Limitations

• To perform static analysis on Rust packages, the package version must be able to be successfully built.
• Performing Endor Labs scans on the Microsoft Windows operating system is currently unsupported.

Troubleshoot errors

• Host system check failure errors: These errors occur when Rust is not installed or not present in the path variable. Install Rust and try again.
• Call graph errors: These errors occur when the project is not buildable and the required dependencies cannot be located.

2 - Scanning strategies

As you deploy Endor Labs in your environment, it’s important for your team to understand key scanning strategies.

Set a default branch

The findings, metrics, and data shown on the dashboard and the project listing page are based on scanning the default branch, which is also known as the main context.

endorctl scan --as-default-branch

If you do not set the flag as-default-branch, the first branch you scan is automatically considered as the default branch. After a scan, if you switch the default branch to another using --as-default-branch, scans from the previous branches are erased, and their findings will no longer be available.

You do not need to set a default branch if you are using the Endor Labs GitHub App or not scanning multiple branches.

Testing and monitoring different versions of your code

Across the software engineering lifecycle its important that continuous testing is separated from what is monitored and reported on regularly. Often, engineering organizations want to test each and every change that enters a code base, but if security teams reported on each test they would quickly find themselves overwhelmed with noise. Endor Labs enables teams to separate what should be reported on relative to what should be tested but not reported on. Endor Labs allows teams to select reporting strategies for their software applications when integrated into CI/CD pipelines.

Here are the primary scanning and reporting strategies:

• Reporting on the default branch - All pull request commits are tested and all pushes or merges to the default branch are reported on and monitored by security and management teams.
• Reporting on the latest release - All reporting and monitoring is performed against tagged release versions. This requires each team have a mature release tagging strategy.

How to deploy a strategy for reporting

The endorctl scan command by default will continuously monitor a version of your code for new findings such as unmaintained, outdated or vulnerable dependencies in the bill of materials for a package. To test a version of your code without monitoring and reporting on it, use the flag --pr or environment variable ENDOR_SCAN_PR as part of your scan.

When adopting a strategy such as reporting on the default branch, you will want to run any push or merge event to the default branch without the --pr flag and run any pull_request or merged_request event with the --pr flag. This allows you to test changes before they have been approved and report what has been merged to the default branch as your closest proxy to what is in production.

Lets use the following GitHub actions workflow as an example! In this workflow any push event will be scanned without the --pr flag but any pull_request event is scanned as a point in time test of that specific version of your code.

name: Endor Labs Scan
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  scan:
    permissions:
      security-events: write # Used to upload sarif artifact to GitHub
      contents: read # Used to check out a private repository but actions/checkout.
      actions: read # Required for private repositories to upload sarif files. GitHub Advanced Security licenses are required.
      id-token: write # Used for keyless authentication to Endor Labs
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3
      - name: Setup Java
        uses: actions/setup-java@v3
        with:
          distribution: 'microsoft'
          java-version: '17'
      - name: Build Package
        run: mvn clean install
      - name: Endor Labs Scan Pull Request
        if: github.event_name == 'pull_request'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: true
          sarif_file: 'findings.sarif'
          pr_baseline: $GITHUB_BASE_REF
      - name: Endor Labs Reporting Scan
        if: github.event_name == 'push'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: false
          sarif_file: 'findings.sarif'
      - name: Endor Labs Testing Scan
        if: github.event_name == 'pull_request'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: true
          sarif_file: 'findings.sarif'
      - name: Upload findings to github
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: 'findings.sarif'

Scanning detached refs

In some CI/CD based environments, each time code is pushed to the default branch the exact commit SHA is checked out as a detached Git Reference. This is notably the case with Jenkins, CircleCI and GitLab Pipelines.

In these scenarios, on push or merge events Endor Labs must be told that the reference should be monitored as the default branch. You can do this with the --detached-ref-name flag or ENDOR_SCAN_DETACHED_REF_NAME environment variable. You should also couple this flag with the --as-default-branch flag or ENDOR_SCAN_AS_DEFAULT_BRANCH environment variable. This allows you to set this version of code as a version that should be monitored as well as define the name associated with the branch.

This strategy may be used for both a strategy reporting on the default branch on push events and a strategy reporting on tag creation event for that version of code.

You can see in the below GitLab Pipelines example defining the logic to manage a detached reference on GitLab.

    - if [ "$CI_COMMIT_REF_NAME" == "$CI_DEFAULT_BRANCH" ]; then
        export ENDOR_SCAN_AS_DEFAULT_BRANCH=true;
        export ENDOR_SCAN_DETACHED_REF_NAME="$CI_COMMIT_REF_NAME";
      else
        export ENDOR_SCAN_PR=true;
      fi

You can find the full GitLab pipelines reference below:

Endor Labs Dependency Scan:
  stage: Scan
  image: node # Modify this image to align with the build tools nessesary to build your software packages
  dependencies: []
  variables:
    ENDOR_ENABLED: "true"
    ENDOR_ALLOW_FAILURE: "true"
    ENDOR_NAMESPACE: "demo"
    ENDOR_PROJECT_DIR: "."
    ENDOR_ARGS: |
      --path=${ENDOR_PROJECT_DIR}
      --show-progress=false
      --detached-ref-name=$CI_COMMIT_REF_NAME
      --output-type=summary
      --exit-on-policy-warning
      --dependencies --secrets --git-logs      
  before_script:
    - npm install yarn
  script:
    - curl https://api.endorlabs.com/download/latest/endorctl_linux_amd64 -o endorctl;
    - echo "$(curl -s https://api.endorlabs.com/sha/latest/endorctl_linux_amd64)  endorctl" | sha256sum -c;
      if [ $? -ne 0 ]; then
       echo "Integrity check failed";
       exit 1;
      fi
    - chmod +x ./endorctl
    - if [ "$DEBUG" == "true" ]; then
        export ENDOR_LOG_VERBOSE=true;
        export ENDOR_LOG_LEVEL=debug;
      fi
    - if [ "$CI_COMMIT_REF_NAME" == "$CI_DEFAULT_BRANCH" ]; then
        export ENDOR_SCAN_AS_DEFAULT_BRANCH=true;
        export ENDOR_SCAN_DETACHED_REF_NAME="$CI_COMMIT_REF_NAME";
      else
        export ENDOR_SCAN_PR=true;
      fi
    - ./endorctl scan ${ENDOR_ARGS}
  rules:
  - if: $ENDOR_ENABLED != "true"
    when: never
  - if: $CI_COMMIT_TAG
    when: never
  - if: $CI_COMMIT_REF_NAME != $CI_DEFAULT_BRANCH && $ENDOR_FEATURE_BRANCH_ENABLED != "true"
    when: never
  - if: $ENDOR_ALLOW_FAILURE == "true"
    allow_failure: true
  - if: $ENDOR_ALLOW_FAILURE != "true"
    allow_failure: false

Implementing baseline scans

One of the common concerns software development teams have when adopting preventative controls is ownership of issues. Often, software has accrued significant technical debt, or new vulnerabilities arise that don’t directly impact their changes. Security teams want to have all known issues addressed while the development teams are focused on fixing issues or delivering core business value. They can’t be hindered each time a new issue impacts their entire code base.

To prevent new issues from entering the environment, security teams sometimes set policies that may break the build or return a non-zero exit code that can fail automated tests. This creates friction as there is no context around what changes a developer is responsible for versus what technical debt exists in a codebase on that day.

Establishing a baseline of what issues already exist in a software project and what issues may occur because of new updates is crucial to enabling preventative control adoption.

Accelerating preventative control adoption with CI baselines

The high-level steps to establish and measure policies against a baseline scan are as follows:

1. Establish a baseline scan of your default branch or any other branch that undergoes regular testing
2. Integrate baseline scans into your automated workflows
3. Evaluate policy violations within the context of the branches to which you routinely merge

Implementing baseline scan into your program

Development teams often have different delivery strategies. Some merge changes to a default branch. Others merge to a release branch that is then released to their environment. While these strategies differ across organizations, a baseline scan must exist to measure against attribute ownership.

To establish a baseline scan, your team must perform regular scans on the branch to which you merge. This often means that you scan each push of your default branch to monitor your environment and you test each pull request using the --pr and --pr-baseline flags.

The --pr flag is a user’s declaration that they are testing their code as they would in a CI pipeline. The --pr-baseline flag tells Endor Labs which Git reference to measure any changes.

For this example, we will use the default branch as a merging strategy. In this strategy, you’ll want to scan the default branch on each push event to re-establish your baseline. You’ll also want to establish your CI baseline as the default branch.

The following GitHub workflow illustrates this strategy:

• See the GitHub Actions documentation for GitHub default variables.
• See the GitLab CI/CD documentation for the GitLab default variables.

name: Endor Labs Scan
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
jobs:
  scan:
    permissions:
      security-events: write # Used to upload sarif artifact to GitHub
      contents: read # Used to check out a private repository but actions/checkout.
      actions: read # Required for private repositories to upload sarif files. GitHub Advanced Security licenses are required.
      id-token: write # Used for keyless authentication to Endor Labs
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3
      - name: Setup Java
        uses: actions/setup-java@v3
        with:
          distribution: 'microsoft'
          java-version: '17'
      - name: Build Package
        run: mvn clean install
      - name: Endor Labs Scan Pull Request
        if: github.event_name == 'pull_request'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: true
          sarif_file: 'findings.sarif'
          pr_baseline: $GITHUB_BASE_REF
      - name: Endor Labs Reporting Scan
        if: github.event_name == 'push'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: false
          sarif_file: 'findings.sarif'
      - name: Endor Labs Testing Scan
        if: github.event_name == 'pull_request'
        uses: endorlabs/github-action@v1.1.1
        with:
          namespace: 'example'
          pr: true
          sarif_file: 'findings.sarif'
      - name: Upload findings to github
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: 'findings.sarif'

3 - Perform incremental scans

Perform incremental scans to scan the parts of a codebase that have changed since the last complete scan, rather than scanning the entire codebase every time. This process is efficient and faster, especially for large projects, because it focuses on new or modified code that might introduce new vulnerabilities or issues.

How does incremental scan work

To initiate incremental scans, first perform a complete scan to set a baseline. After establishing the baseline, you can start incremental scans to compare against it. The first scanned branch is considered the baseline, and this branch is continuously monitored. See set a default branch.

Ensure the baseline scan is successful and error-free. A successful complete scan will resolve dependencies, run analytics, and generate call graphs for supported languages.

During an incremental scan, Endor Labs first identifies the packages and their dependencies within a project. If a package and its dependencies remain unchanged, the rest of the scanning process is skipped, and a message No changes found in package is displayed along with the package name. If any changes are detected in the packages or dependencies, the scan proceeds with only the modified packages, and prints their results.

Incremental scanning is not performed if errors are found while resolving dependencies, if the project’s path has changed, or if there are failures within the project’s packages. In these cases, the system will automatically perform a complete scan.

Run an incremental scan

1. First, perform a complete scan.

endorctl scan

2. Use the --pr-incremental argument to perform an incremental scan, detecting and scanning dependencies that have changed since the baseline scan. This scan will generate call graphs only for the modified packages. See endorctl scan command arguments.

endorctl scan --pr --pr-baseline=main --pr-incremental

4 - Scoping scans

Exclude and include filters help your team to focus their attention on the open source packages that matter most and to improve scan performance. Use inclusion patterns when you have many packages that you want to scan separately and exclusion patterns when you want to filter out packages that are not important to you.

You can include or exclude packages using the following standard patterns:

1. Include or exclude specific packages.
2. Include or exclude specific directories.
3. Include or exclude with a Glob style expressions.
4. Use include and exclude patterns together to exclude specific directories such as a test directory from a scan.
5. Use multiple include and exclude patterns together to exclude or include specific directories or file paths.

To include or exclude a package based on its file name:

• Include path
• Exclude path

endorctl scan --include-path="path/to/your/manifest/file/package.json"

endorctl scan --exclude-path="path/to/your/manifest/file/package.json"

To include or exclude a package based on its directory

• Include Directory
• Exclude Directory

endorctl scan --include-path="directory/path/**"

endorctl scan --include-path="src/java/**"

endorctl scan --exclude-path="path/to/your/directory/**"

endorctl scan --exclude-path="src/ruby/**"

Examples

• Use --exclude-path="src/java/**" to exclude all files under src/java, including all its sub-directories.
• Use --exclude-path=src/java/* to only exclude the files under src/java, but not its sub-directories.
• Use --include-path and --exclude-path together to exclude specific directories such as test directories. For example,

endorctl scan --include-path="src/java/**" --exclude-path="src/java/test/**"

• Use multiple inclusion patterns together. For example,

endorctl scan --quick-scan --include-path="src/java/**" --include-path="src/dotnet/**"

• Use multiple exclusion patterns together. For example,

endorctl scan --include-path="src/java/**" --exclude-path="src/java/gradle/**" --exclude-path="src/java/maven/**"

Best practices

Here are a few best practices:

• Ensure that you enclose your exclude pattern in double quotes to avoid shell expansion issues. For example, do not use --exclude-path=src/test/**, instead, use --exclude-path="src/test/**".
• Inclusion patterns are not designed for documentation or example directories. You can not explicitly include documentation or example directories:
  • docs/
  • documentation/
  • groovydoc/
  • javadoc
  • man/
  • examples/
  • demos/
  • inst/doc/
  • samples/
• The specified paths must be relative to the root of the directory.
• If you are using Javascript workspaces, take special consideration when including and excluding the root package:
  • When using include or exclude patterns, it’s crucial to make sure you never exclude and always include the parent workspace package. Otherwise, all child packages won’t build properly.
  • You can always exclude child packages in the workspace if the root is included.
  • There is only one lock file for the workspace that exists in the workspace root directory. Make sure to include the lock file to perform a successful scan.

5 - Working with monorepos

Large monorepos are a reality for many organizations. Since monorepos can have anywhere from tens to even hundreds of packages scanning all packages in a monorepo can take significant periods of time. While the time requirements may vary based on your development team and pipeline times, in general, development teams need quick testing times to improve their productivity while security teams need full visibility across a monorepo. These two needs can conflict without performance engineering or an asynchronous scanning strategy. This documentation outlines some performance engineering and scanning strategies for large monorepos.

Asynchronous scanning strategies

When scanning a large monorepo, a common approach taken by security teams is to run an asynchronous cron job outside a CI/CD-based environment. This is often the point of least friction but is prohibitive. With this approach, inline blocking of critical issues is not generally possible. We would be remiss not to mention this as a scanning strategy for monorepos but this approach is NOT recommended beyond a step to get initial visibility into a large monorepo.

Performance Enhancements for inline scanning strategies

The following performance enhancements may be used with Endor Labs to enable the scanning of large monorepos:

Scoping scans based on changed files

For many CI/CD systems path filters are readily available. For example, with GitHub actions, dorny path filters is a readily accessible way to establish a set of filters by a path. This is generally the most effective path to handle monorepo deployments but does require the highest level of investment in terms of human time. The human time investment is made up for by the time saved by reducing the need to scan everything on each change.

Based on the paths that change you can scope scans based on the files that have actually changed. For example, you can scan only the packages in a monorepo that are housed under the ui/ directory when this path has changed by running a scan such as endorctl scan --include-path=ui/ when this path has been modified.

Using a path filtering approach each team working in a monorepo would need to be responsible for the packages that they maintain, but generally, each team may be associated with one to several pre-defined directory paths.

Parallelizing scans for many packages

When scanning a large monorepo organizations can choose to regularly scan the whole monorepo based on the packages or directories they’d like to scan. Different jobs may be created that scan each directory simultaneously.

Parallelizing with scoped scans

Using scoped scans for monorepos with multiple parallel include patterns is a common performance optimization for monorepos.

Below is an example parallel GitHub action scan that can be used as a reference. In this example, the directories ui/ and backend/ are both scanned simultaneously and the results are aggregated by Endor Labs.

This approach can improve the overall scan performance across a monorepo where each directory can be scanned independently.

name: Parallel Actions
on:
  push:
    branches: [main]
jobs:
  scan-ui:
    runs-on: ubuntu-latest
    steps:
      - name: UI Endor Labs Scan
        run: endorctl scan --include-path=ui/
  scan-backend:
    runs-on: ubuntu-latest
    steps:
      - name: Backend Endor Labs Scan
        run: endorctl scan --include-path=backend/

To include or exclude a package based on its directory.

endorctl scan --include-path="directory/path/"

See scoping scans for more information on approaches to scoping scans.

Parallelizing across languages

For teams that work out of smaller monorepos, it is often most reasonable to parallelize scanning based on the language that is being scanned and performance optimize for individual languages based on need.

Below is an example parallel GitHub action scan that can be used as a reference. In this example, JavaScript and Java are scanned at the same time and aggregated together by Endor Labs. This approach can improve the overall scan performance across a monorepo with multiple languages.

name: Parallel Actions
on:
  push:
    branches: [main]
jobs:
  scan-java:
    runs-on: ubuntu-latest
    steps:
      - name: Java Endor Labs Scan
        run: endorctl scan --languages=java
  scan-javascript:
    runs-on: ubuntu-latest
    steps:
      - name: Javascript Endor Labs Scan
        run: endorctl scan --languages=javascript,typescript

To scan a project for only packages written in typescript or javascript use the command:

endorctl scan --languages=javascript,typescript

To scan a project for only packages used for packages written in java use the command:

endorctl scan --languages=java

6 - Repository Security Posture Management

Secure critical components of your software supply chain, including code, open source libraries to ensure the security posture of your software development code and its repositories.

• Endor Labs comes with out-of-the box finding policies that help you detect misconfigurations, enforcing coding best practices, and staying compliant with industry standards such as CIS benchmarks for GitHub and more.

• Review the available finding RSPM policy templates.

• Endor Labs regularly updates its existing policies and also includes several new policies. Configure policy settings to ensure that you benefit from these regular updates.

• The policies provide up-to-date insights into critical risks, so you can manage security threats before your projects even start. They also include remediation advice that can help you fix and mitigate issues.

7 - Rescan projects

Endor Labs enables you to rescan your GitHub projects. When you make a code change or upgrade a dependency, rescanning your GitHub projects ensures the integrity and security of your software.

To enable periodic scanning of your GitHub projects, install the GitHub App from Endor Labs. For more information, see Install the GitHub App.

Endor Labs automatically triggers a rescan of your GitHub projects every 24 hours. However, you can manually trigger a rescan. Follow these steps:

1. Sign in to Endor Labs and click Manage Projects.

2. Select a project configured for automated scanning using the GitHub App.

3. Click Rescan Project to start rescanning.

   

8 - Manage build tools (Beta)

Endor Labs often requires pre-built or installed software to scan your application. Building software allows Endor Labs to ensure that your software bill of materials is accurate, especially in software languages such as Python, Java, or .NET where lock files are significantly less common.

Since software frequently relies on specific versions of a runtime or package manager, Endor Labs references the tools used in your software build process. This ensures that your software bill of materials and all associated risk information are accurate. Endor Labs provides ways to define the tools necessary for building your software for repeatable patterns used in CI or when build tools are not installed.

Endor Labs will automatically install build tools in a sandbox to ensure you can run highly accurate scans. The build tools are not installed on your host but are installed in an isolated sandbox. The feature is currently supported for Linux and macOS operating systems.

Scan Profile is used to configure build tool chain and scan parameters that are passed to the Github App. A Project should be associated to one Scan Profile so that the scans for that project uses the configuration in the Scan Profile.

You need to install and initialize endorctl CLI, before configuring the build toolchains in a scan profile.

The following pages describe the various methods in which you can create a scan profile.

• Configure scan profile through the Endor Labs UI
• Configure scan profile through the Endor Labs API
• Configure scan profile through scanprofile.yaml file
• Automatically detect tool chains
• Uses system defaults

Configure build tools for Endor Labs GitHub App

Endor Labs GitHub App continuously monitors your projects for security and operational risks. The app monitors all the projects included in your GitHub workspace and scans run once every 24 hours. For performing scans, the GitHub App checks the toolchain specifications in the following order:

1. Toolchain configuration specified through endorctl API.
2. Toolchain configuration specified in scanprofile.yaml file.
3. Enable auto detection and automatically detect the toolchains from your manifest files.
4. Uses the system defaults.

Configure build tools for repeatable CI patterns

After installing and initializing the endorctl CLI, run the endorctl scan using the --install-build-tools command to dynamically download and install the required build tools.

endorctl scan --install-build-tools

Run the endorctl scan

Here is the recommended flow for performing the endorctl scan.

1. For the first time, run the endorctl scan to create a project with Endor Labs.

endorctl scan

2. To automatically download and install build tools as part of your scan, run the endorctl scan using the --install-build-tools command.

endorctl scan --install-build-tools

3. The system checks for the required toolchain specifications in the following order before installing them in the sandbox.

   • Configure toolchain profile through endorctl API.
   • Configure toolchain profile in the profile.yaml file.
   • Automatically detect toolchain profiles
   • Uses the system defaults.

System default toolchain versions

If you do not provide a tool profile, the default toolchains are installed in the sandbox while performing the endorctl scan with the install-build-tools flag. See Toolchain reference for details on default versions.

Toolchain support matrix

The following table outlines the toolchain profile support details across different languages and platforms.

Reference toolchain specification

The following reference toolchain specification has examples for defining toolchains in linux | amd64 architecture.

kind: ToolchainProfile
spec:
  os:
    linux:
      arch:
        amd64:
          java_tool_chain:
            version:
              name: "1.8.412"
              urls:
                - "https://builds.openlogic.com/downloadJDK/openlogic-openjdk/8u412-b08/openlogic-openjdk-8u412-b08-linux-x64.tar.gz"
              relative_tool_chain_path: "openlogic-openjdk-8u412-b08-linux-x64/"
              sha256_sum: "eb06c9d62e031e3290f499a828cae66d4fadbf62eb8f490c63c8406b1a80172e"
            maven_version:
              name: "3.9.4"
              urls:
                - "https://archive.apache.org/dist/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz"
              relative_tool_chain_path: "apache-maven-3.9.4"
              sha256_sum: "ff66b70c830a38d331d44f6c25a37b582471def9a161c93902bac7bea3098319"
            gradle_version:
              name: "8.4"
              urls:
                - "https://services.gradle.org/distributions/gradle-8.4-bin.zip"
              relative_tool_chain_path: "gradle-8.4/"
              sha256_sum: "3e1af3ae886920c3ac87f7a91f816c0c7c436f276a6eefdb3da152100fef72ae"
          python_tool_chain:
            version:
              name: "3.10"
              urls:
                - "https://github.com/indygreg/python-build-standalone/releases/download/20240415/cpython-3.10.14+20240415-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst"
              relative_tool_chain_path: "python/"
              sha256_sum: "add8cc6cbb4f2a3f8af2272e62b7604f7529a8c357c0af0f8a9f7d3dd444ef1e"
          java_script_tool_chain:
            nodejs_version:
              name: "20.10.0"
              urls:
                - "https://nodejs.org/dist/v20.10.0/node-v20.10.0-linux-x64.tar.gz"
              relative_tool_chain_path: "node-v20.10.0-linux-x64/"
              sha256_sum: "d3f0908a9d9190a8525c5b9a716ed91bb57e908555841b0c47f75b2a001ff91b"
            yarn_version:
              name: "1.22.19"
              urls:
                - "https://github.com/yarnpkg/yarn/releases/download/v1.22.19/yarn-v1.22.19.tar.gz"
              relative_tool_chain_path: "yarn-v1.22.19/"
              sha256_sum: "732620bac8b1690d507274f025f3c6cfdc3627a84d9642e38a07452cc00e0f2e"
          dotnet_tool_chain:
              version:
                name: "8.0.303"
                urls:
                  - "https://download.visualstudio.microsoft.com/download/pr/60218cc4-13eb-41d5-aa0b-5fd5a3fb03b8/6c42bee7c3651b1317b709a27a741362/dotnet-sdk-8.0.303-linux-x64.tar.gz"
                sha256_sum: "214ee467f75c42f1512748fe7ca8dd82da2af29cdf54be614a8997f0466ef070"
    darwin:
      arch:
        arm64:
          java_tool_chain:
            version:
              name: "1.8.412"
              urls:
                - "https://builds.openlogic.com/downloadJDK/openlogic-openjdk/8u412-b08/openlogic-openjdk-8u412-b08-mac-x64.zip"
              relative_tool_chain_path: "openlogic-openjdk-8u412-b08-mac-x64/jdk1.8.0_412.jdk/Contents/Home"
              sha256_sum: "a16d297418f6800dfc5abfd4dfd8a16c0504d7e1f3b6fc9051cf2460f14a955e"
            maven_version:
              name: "3.9.4"
              urls:
                - "https://archive.apache.org/dist/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz"
              relative_tool_chain_path: "apache-maven-3.9.4"
              sha256_sum: "ff66b70c830a38d331d44f6c25a37b582471def9a161c93902bac7bea3098319"
          dotnet_tool_chain:
            version:
              name: "8.0.303"
              urls:
                - "https://download.visualstudio.microsoft.com/download/pr/d81d84cf-4bb8-4371-a4d2-88699a38a83b/9bddfe1952bedc37e4130ff12abc698d/dotnet-sdk-8.0.303-osx-arm64.tar.gz"
              relative_tool_chain_path: "dotnet-sdk-8.0.303-osx-arm64"
              sha256_sum: "c6f4150833e51d55cc4c4a435d7cc53269f3d6db308b34f4e001900c6fdf8149"
---
kind: AutomatedScanParameters
spec:
  languages:
    - java
    - python
  additional_environment_variables:
    - ENDOR_LOG_VERBOSE=false
    - ENDOR_LOG_LEVEL=info

8.1 - Configure scan profile through Endor Labs UI

While scanning projects using the GitHub App, you can configure a scan profile and assign it to your projects directly from the Endor Labs user interface.

Create a new scan profile

Create and customize a new scan profile to define scan parameters, toolchains, and projects.

1. Sign in to Endor Labs and click Settings under Manage in the left sidebar.
2. Select SCAN PROFILES and click New Scan Profile.
3. Enter a name for the scan profile and click Create Scan Profile.
4. Configure various settings like automated scan parameters and paths. See Configure General scan profile settings for more information.
5. Select TOOLCHAINS and configure the toolchains. See Configure toolchains for more information.
6. Select PROJECTS to associate the scan profile with projects. See Associate projects with a scan profile.

Configure general scan profile settings

Configure the necessary scan settings to tailor scans for your projects.

1. Select the features that you want to enable for the scan profile.

   • Enable pull request comments
   • Enable remediation action

   The selected features are automatically used when you run a scan on a project that uses the scan profile.

2. Select the languages to scan and the languages for which you need to generate call graphs. If you don’t select any language, all the languages detected in the repository will automatically be selected for the scan.

3. Enter the paths to include or exclude in the scan.

4. Enter any additional environment variables, if required. Only the environment variables starting with ENDOR_ are passed to the scan, all others are ignored.

5. Configure Bazel settings, if required.

6. Click Save Scan Profile to save the toolchain configuration.

Configure toolchains

Create and save a scan profile.

1. Select the operating system for the scan profile.

2. Select the architecture.

3. Select the toolchain available for the operating system-architecture combination.

4. Select the tool associated with the toolchain. For package managers like Python (pip), JavaScript (npm), and Android, you can configure a list of packages to install before the scan.

5. Select the version of the tool (or enter the package name if you chose a package in the previous step) and click Add to Profile.

   You can only assign one version of the tool for a scan profile for a particular operating system-architecture combination.

   You can also click Custom and define the custom version of the tool. See Configure custom versions for more information.

   The following image shows the creation of a scan profile for Go and JavaScript scans.

6. Click Save Scan Profile to save the toolchain configuration.

Configure a custom version for a tool

When you assign a version of the tool, you can choose to apply a custom version that is not provided by Endor Labs.

You must provide the following information.

• Version name
• The URL to download the archive package
• SHA256 checksum of the package
• The relative toolchain path, if required. The toolchain is extracted to the specified relative toolchain path if provided.

The following image shows a custom configuration for the Golang toolchain with Go 1.22.7 instead of the bundled 1.22.6.

Associate projects with a scan profile

Assign projects to your scan profile.

1. Select Actions > Add Projects.

   

2. Search the project and click Add to Scan Profile. You can associate multiple projects with a scan profile, but you cannot apply multiple scan profiles to a single project.

8.2 - Configure scan profile through Endor Labs API

You can use the endorctl api command to configure the toolchains for your project.

1. Run the endorctl scan to create a project.

endorctl scan

2. Fetch the UUID of the project, for example, to fetch the UUID of the app-java-demo project, you can use:

UUID=$(endorctl api list -r Project --filter="meta.name matches https://github.com/endorlabs/app-java-demo" --field-mask=uuid | jq -r '.list.objects[].uuid')

3. Create a ScanProfile object using the following command. Set the environment variable using set EDITOR=vim before executing the following command.

endorctl api create -i -r ScanProfile

Here is an example that you can use to create a ScanProfile object for installing .NET 8.0.303. After executing this command, you can fetch the UUID of the ScanProfile object. See Reference toolchain specification for a complete description of supported toolchains.

spec:
  toolchain_profile:
    os:
      linux:
        arch:
          amd64:
            dotnet_tool_chain:
              version:
                name: "8.0.303"
                urls:
                  - "https://download.visualstudio.microsoft.com/download/pr/60218cc4-13eb-41d5-aa0b-5fd5a3fb03b8/6c42bee7c3651b1317b709a27a741362/dotnet-sdk-8.0.303-linux-x64.tar.gz"
                sha256_sum: "214ee467f75c42f1512748fe7ca8dd82da2af29cdf54be614a8997f0466ef070"

4. Associate the scan_profile_uuid to your project UUID <project-uuid, using the following command.

endorctl api update -r Project --uuid=<project-uuid> -d '{"spec":{"scan_profile_uuid":"<scanprofile-uuid>"}}' --field-mask 'spec.scan_profile_uuid'

You have now created a toolchain for installing .NET 8.0.303.

8.3 - Configure scan profile through scanprofile.yaml

You can create a build tool profile for your Endor Labs scans in each repository to specify the build tools to automatically download for each scan.

Create a new file .endorctl/scanprofile.yaml file in the root directory of your repository and specify the required versions of the tools. You can specify the Operating system, architecture, language, tool, and install information in the scanprofile.yaml file:

The overall structure of a scanprofile.yaml file should look like this:

kind: ToolchainProfile
spec:
  os:
    <linux|darwin>:
      arch:
        <amd64|arm64>:
          python_tool_chain:

A profile that downloads Python 3.10 and the corresponding PyPI packages to run in a CI environment can look like this:

kind: ToolchainProfile
spec:
  os:
    linux:
      arch:
        amd64:
          python_tool_chain:
            version:
              name: "3.10"
              urls:
                - "https://python310.tar.gz"
              sha256_sum: "python310shasum"
            pip_packages:
              wheel:
                name: "1.2.3"
                urls:
                  - "https://wheel123.tar.gz"
                  - "https://wheel123_backup.targz"
                sha256_sum: "wheelshasum"

8.4 - Enable auto detection

The system can automatically detect toolchains required for your projects based on the manifest files present in your repository. Auto detection is supported for Java, Python, Golang and .NET(C#) projects. Only the Long Term Support (LTS) versions of the toolchains are supported in auto detection. See the Toolchain support matrix for a complete list of supported toolchain versions for auto detection.

Enable auto detection in CLI

To enable auto detection from the CLI,

endorctl scan --install-build-tools --enable-build-tools-version-detection

Enable auto detection in GitHub App

When using the GitHub App, you can enable auto detection either by a project or enable it for all projects in a tenant.

• To enable the auto detection by a project, update the project’s meta.annotations with "ENDOR_SCAN_ENABLE_BUILD_TOOLS_VERSION_DETECTION":"true".

meta:
  annotations: {"ENDOR_SCAN_ENABLE_BUILD_TOOLS_VERSION_DETECTION":"true"}

  endorctl api update -r Project --uuid=<project-uuid> -i

• To enable auto detection across all projects in a tenant, update the system config’s meta.annotations with "ENDOR_SCAN_ENABLE_BUILD_TOOLS_VERSION_DETECTION":"true".

 meta:
   annotations: {"ENDOR_SCAN_ENABLE_BUILD_TOOLS_VERSION_DETECTION":"true"}

 endorctl api update -r SystemConfig --uuid=<system-config-uuid> -i

The updates are applied during the next scheduled scan or whenever you perform a manual re-scan.

9 - Scan artifacts and binaries

You can now perform endorctl scan on your binaries and artifacts without requiring access to source code or build systems. Scan Java and Python packages that are pre-built, bundled, or downloaded into your local system by specifying a file path to your artifact or binary package.

Endor Labs scans the specified package, producing vital scan artifacts such as details about resolved dependencies and transitive dependencies, along with comprehensive call graphs. It enables you to acquire valuable insights and improve the security and reliability of the software components.

You can scan JAR, WAR, and EAR package file formats built using Maven or Gradle with a pom.xml configuration file. To scan packages without a pom.xml configuration, see Scan Java packages without pom.xml.

You can scan EGG(tar.gz) and Wheel(.whl) package file formats.

System specifications for deep scan

Software prerequisites

If you have a private registry and internal dependencies on other projects, you must configure private registries for the Python and Java projects. See Configure package manager integrations for more information.

Understand the scan arguments

Use --package as an argument to scan artifacts or binaries. You must provide the path of your file using --path and specify a name for your project using --project-name.

endorctl scan --package --path --project-name

Run the scan

Use the following options to scan your repositories.

Option 1 - Quick scan

Perform a quick scan of the local packages to get quick visibility into your software composition. This scan won’t perform reachability analysis to help you prioritize vulnerabilities.

Syntax:

endorctl scan --quick-scan --package --path=<<specify-the-path-of-your-file>> --project-name=<<specify-a-name-for-the-project>>

Example:

endorctl scan --quick-scan --package --path=/Users/username/packages/logback-classic-1.4.10.jar --project-name=package-scan-for-java

Option 2 - Deep scan

Use the deep scan to perform dependency resolution, reachability analysis, and generate call graphs. You can do this after you complete the quick scan successfully.

Syntax:

endorctl scan --package --path=<<specify-the-path-of-your-file>> --project-name=<<specify-a-name-for-the-project>>

Example:

endorctl scan --package --path=/Users/username/packages/logback-classic-1.4.10.jar --project-name=java-package-scan

View results

You can sign into the Endor Labs user interface, click the Projects on the left sidebar, and find your project using the name you entered to review its results.

You can view the list of projects created for scanning packages using the parameter Project Platform Source matches PLATFORM_SOURCE_BINARY to search on Projects.

10 - Scan containers (Beta)

Containers help developers create, test, and deploy applications in a consistent environment. Container images include standalone or executable files encompassing files, libraries, and dependencies needed to run a container. They include many open-source software, making them vulnerable to open-source risks.

Gaining visibility into container images is essential to identify and prioritize risks or maintain compliance obligations.

Endor Labs container scan detects and reports known vulnerabilities and other risks in:

• Operating system packages: Identifies packages installed through the container’s base operating system package manager.
• Programming language packages: Identifies packages installed through language-specific package managers.
• Libraries and dependencies: Identifies static and dynamic libraries, and runtime dependencies required by the application.

Additionally, it generates an SBOM (Software Bill of Materials) that details all components, their versions, and associated metadata, providing a complete inventory of the container’s contents.

Verify access to container registries

If the container image is in a private Docker registry, you must authenticate the container client before the scan.

Here are a few commands to authenticate the container client.

• Docker
• Podman
• containerd

docker login <host> -u <user_name> -p <password>

For more information, see [Docker documentation](https://docs.docker.com/reference/cli/docker/login/).

podman login -u myusername -p mypassword myregistry.example.com

For more information, see [Scanning podman built container images](../../troubleshooting/podman/) and [Podman documentation](https://docs.podman.io/en/v4.4/markdown/podman-login.1.html).

You must configure the containerd config file to authenticate with the container registry. For more information, see the [containerd documentation](https://github.com/containerd/containerd/blob/main/docs/cri/registry.md).

Perform the endorctl scan

Use the following options to scan containers using endorctl CLI.

1. To scan a container image from the registry, run the following command specifying the project’s name in the –project-name argument, and the container image name and tag in the –container argument.

endorctl scan  --project-name=<endor_project_name>  --container=<image_name:tag>

2. To scan a container image in a particular path, run the following command specifying the project’s path in the –path argument and the container image name in the –container argument. This will associate the container with the Git repository and branch of the project.

endorctl scan  --path=users/janedoe/endorlabs/npm/exampleproject --container=<image_name:tag>

3. To keep multiple versions of a container image in a project, use –container-as-ref

endorctl scan  --path=users/janedoe/enodorlabs/npm/exampleproject --container=<image_name:tag>  --container-as-ref

Perform container scan in CI pipelines

You can integrate container scanning into CI pipelines to automatically detect vulnerabilities and ensure the security of container images during the build and deployment process.

To perform container scanning in CI pipelines using GitHub Actions include the following command in the GitHub Actions script. You must also set the scan_container parameter to true.

endorctl scan  --project-name=<endor_project_name>  --container=<image_name:tag>

or

endorctl scan  --path=users/janedoe/endorlabs/npm/exampleproject --container=<image_name:tag>

See Performing scans in CI/CD pipelines for more information.

Understand container scan

Endor Labs fetches the container image from a container registry or loads it from a local file to scan containers. It then proceeds to extract the layers of the container image. It traverses the filesystem of each layer to identify files and directories. It looks for known package manager and metadata files to gather information about installed packages and their versions. It identifies various components and dependencies within the image and presents the findings in CLI and the Endor Labs user interface.

Discover base images of containers

A container image is often built upon a base image that is a foundational layer including an operating system and other essential components. It’s crucial to understand what’s in the base image for a thorough security assessment.

You can distinguish the base image related vulnerabilities from the application layer using any of the following methods:

• Scan Sequence - First, scan the base image. Then, scan any subsequent images built on that base image to distinguish vulnerabilities specific to the base image from those introduced by the other layers.
• Docker file label - Set the label directly in your Dockerfile with a command such as LABEL org.opencontainers.image.base.name="openjdk:17-slim".
• Build time label - Include the base image label during the build process with the --label flag, specifying both the base image and, optionally, its exact version via SHA256 hash. For example:

   docker build -t tictactoe:latest --label "org.opencontainers.image.base.name=openjdk@sha256:eddacbc7e24bf8799a4ed3cdcfa50d4b88a323695ad80f317b6629883b2c2a78" .

Create finding policies for containers

Container base images from untrusted sources may lack proper security audits or fail to comply with organizational standards, increasing the risk of vulnerabilities being exploited. To address this, you can configure a finding policy to detect unauthorised base images and raise a critical finding.

For example, to allow only base images that start with gcp or ghcr, use the Container policy template and Specify Base Image Name Regex as ^gcp, ^ghcr.

See also Create a finding policy from template.

Supported languages and package managers

The dependencies associated with the following list of components are identified in the endorctl scan:

View findings

To view findings from the scan,

1. Select Projects from the left sidebar.
2. Locate your project and select to view its findings.

4. To view and filter dependencies based on the container images, click Container Layers and select to view All Layers, Base Image Layers, or Application Layers only.

Limitations

• Scanning Windows containers is not supported.
• Docker file scans are not currently supported.
• Container registry direct integrations are not currently supported.
• Scanning binary files inside a container is not currently supported.
• Scanning container images packaged in the tarball format is not currently supported.
• Endor scores are not calculated for findings reported in the container scan.