Monitoring or supervisory scans

• 1: Deploy Endor Labs GitHub App
• 1.1: Scanning capabilities
• 1.2: Rescan projects
• 1.3: Technical limitations of the Endor Labs GitHub App
• 1.4: Deploy Endor Labs GitHub App (Pro)
• 2: Deploy Endor Labs Azure DevOps App
• 3: Deploy Endor Labs GitLab App
• 4: Deploy Endor Labs Bitbucket App in Bitbucket Data Center
• 5: Deploy Endor Labs Bitbucket App in Bitbucket Cloud
• 6: Outpost: The on-prem scheduler for monitoring scans
• 6.1: Outpost requirements
• 6.2: Outpost authentication
• 6.3: Outpost configuration
• 7: Set up Jenkins pipeline for supervisory scans

graph TD
    A(["Endor Labs App"]) -->|<span style='font-size: 12px'>Continuous monitoring</span>| B["Customer Repositories"]
    A -->|<span style='font-size: 12px'>Initiate scans every 24h or on-demand</span>| C["Endor Labs Cloud
    <span style='font-size: 12px'>Customer data destroyed after scans</span>"]
    B -->|<span style='font-size: 12px'>Clones and scans repositories</span>| C
    C -->|<span style='font-size: 12px'>Pass scan data</span>| D(["Endor Labs Platform
    <span style='font-size: 12px'>Generate findings from scan results</span>"])
    subgraph "<span style='font-size: 18px'>Supervisory scan workflow<span>"
    A
    B
    C
    D
    end

1 - Deploy Endor Labs GitHub App

Endor Labs provides a GitHub App that continuously monitors users’ projects for security and operational risk. You can use the GitHub App to selectively scan your repositories for SCA, secrets, RSPM, or CI/CD tools. GitHub App scans also establish baselines that are subsequently used during CI scans.

Endor Labs GitHub App scans your repositories every 24 hours and reports any new findings or changes to release versions of your code.

If you want to use PR remediations as part of your monitoring scan, you need to use GitHub App (Pro).

Prerequisites for GitHub App

Before installing and scanning projects with Endor Labs GitHub App, make sure you have:

• A GitHub cloud account and organization. If you don’t have one, create one at GitHub.
• Administrative permissions to your GitHub organization. Installing the Endor Labs GitHub App in your organization requires approval or permissions from your GitHub organizational administrator. If you don’t have the permissions, use the command line utility, endorctl, while you wait for the approval.
• Endor Labs GitHub App requires:
  • Read permissions to Dependabot alerts, actions, administration, code, commit statuses, issues, metadata, packages, repository hooks, and security events.
  • Write permissions to checks and pull requests to check the pull requests automatically and surface policy violations to developers as pull request comments.
  • Subscribe to check run, check suite, and pull request events.

Install the GitHub App

To automatically scan repositories using the GitHub App:

1. Sign in to Endor Labs.

2. Choose Projects and click Add Project.

3. From GITHUB, choose GitHub App

4. Click Install GitHub App.

   You will be redirected to GitHub to install the GitHub App.

5. Click Install.

6. Select a user to authorize the app.

7. Select the organization in which you want to install the app.

8. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

   

9. Review the permissions required for Endor Labs and click Install and Authorize.

   If the button to install says Install and Request instead of Install and Authorize, you don’t have permission to install the GitHub App. Use the endorctl command line interface or select Install and Request to notify your organizational administrator of your request to install.

10. Choose a namespace and click Next.

    

11. Based on your license, select and enable the scanners.

    

    • SCA: Perform software composition analysis.
    • RSPM: Scan the repository for misconfigurations.
    • Secret: Scan the repository for exposed secrets.
    • CI/CD: Scan the repository and identify all the CI/CD tools used in the repository.
    • SAST: Scan your source code for weakness and generate SAST findings.
    • AI Models: Scan your repository and discover AI models in your source code.

12. Select Include Archived Repositories to scan your archived repositories. By default, the GitHub archived repositories aren’t scanned.

13. Select PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

    

    • Select Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.

    • In Define Scanning Preferences, select either:

      • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

      • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

      See GitHub scan options for more information on the scans that you can do with the GitHub App.

14. Click Continue.

You have successfully installed the GitHub App.

Manage GitHub Apps on Endor Labs

You can edit or delete the GitHub App integrations.

1. Sign in to Endor Labs.
2. Select Manage > Integrations from the left sidebar.
3. Click Manage next to GitHub under Source Control Managers.
4. Click the ellipsis on the right side, and select Edit Integration.
5. Based on your license, select and enable from the available list of SCANNERS.
6. Choose PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

   • Select Enable Automatic Pull Request Scanning to automatically scan the PRs submitted by users.

   • Select Enable Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.

   • Set the Scanning Preferences to:

     • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

     • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges. The changes are applicable from the next scanning cycle.

7. Use Reset to clear your selection.
8. To delete a GitHub App integration, click the ellipsis on the right side, and select Delete Integration.
9. To manually trigger a scan, click Rescan Org. Endor Labs GitHub App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.
10. Click Scan More Repositories to go to Projects page, from which you can add more repositories to scan through the GitHub App.

Set up package repositories

You can improve your experience with the GitHub App by setting up package repositories. This will help you create a complete bill of materials and perform static analysis. Without setting package repositories, you may not be able to get an accurate bill of materials. See Set up package manager integration for more information.

1.1 - Scanning capabilities

With the Endor Labs GitHub App, you can enhance the security of your repository through the following types of scans.

Scan complete repository

The Endor Labs GitHub App automatically scans your repositories every 24 hours for potential security issues and operational risks, providing up-to-date information about your projects’ security posture.

• You can use the GitHub App to selectively scan your repositories for Software Composition Analysis (SCA), secrets, Repository Security Posture Management (RSPM), or CI/CD tools.
• While the automated scan happens every 24 hours, you can manually trigger a rescan outside this schedule from the Endor Labs user interface. See Rescan projects.
• After each scan, the GitHub App reports any new findings or changes to release versions of your code. Review the scan results from the Endor Labs user interface.

Scan PRs

After scanning the complete repository, it’s important to address the pull requests submitted by users. Administrators can enable a fully automated scanning process for all pull requests and merges initiated into the main branch.

To automatically scan the PRs, set the pull request preferences during the installing of the GitHub App or edit the integration preferences afterwards.

Whenever a PR is created against a repository, the Endor Labs GitHub App performs an incremental scan to detect any changes in resolved dependencies that may introduce new vulnerabilities. These incremental scans are CI runs and are not monitored. You can see the results of the scan on GitHub.

Based on your prefrences, it performs a quick scan or a full scan before merging the PRs into the main branch.

• Quick Scan performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.
• Full Scan performs dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues related to dependencies and call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

1.2 - Rescan projects

Endor Labs enables you to rescan your GitHub projects. When you make a code change or upgrade a dependency, rescanning your GitHub projects ensures the integrity and security of your software.

To enable periodic scanning of your GitHub projects, install the GitHub App from Endor Labs. For more information, see Install the GitHub App.

Endor Labs automatically triggers a rescan of your GitHub projects every 24 hours. However, you can manually trigger a rescan. Follow these steps:

1. Sign in to Endor Labs and select Projects from the left sidebar.
2. Select a project configured for automated scanning using the GitHub App.
3. Click Rescan Project to start rescanning.

1.3 - Technical limitations of the Endor Labs GitHub App

The Endor Labs GitHub App provides visibility across a GitHub organization, but it has technical limitations that do not account for the unique requirements of your application.

Here are some of these limitations.

Custom package build steps

Endor Labs requires executing custom build steps outside of standard package manager commands to build software packages and get an accurate bill of materials and perform static analysis. Sometimes, a complete bill of materials may not be generated, or static analysis may not be performed if custom steps are required for your software to build. Applications that require custom build steps may need to be implemented in a CI environment to successfully get an accurate bill of materials.

Custom resource profiles

Large applications may require significant memory allocations to perform static analysis on a package. The services scanning the GitHub App use 16 GB of memory by default. Applications that require more memory may not obtain vulnerability prioritization information using the GitHub App. Scan large applications in a CI environment using a runner with sufficient resource allocations.

Authentication for private software components

Private software components hosted in an internal package repository may require authentication credentials to create a complete bill of materials or perform static analysis.

If your authentication information to your private package repository is hosted outside the repository, you will need to configure a package manager integration. See Set up package manager integration for more details. If your package repository is inaccessible from the public internet, you can work with Endor Labs to evaluate options.

1.4 - Deploy Endor Labs GitHub App (Pro)

Endor Labs GitHub App (Pro) is an enhanced version of the Endor Labs GitHub App that supports PR remediation to fix vulnerabilities. See PR remediation for more information.

Prerequisites for GitHub App (Pro)

Before installing and scanning projects with Endor Labs GitHub App (Pro), make sure you have:

• Administrative permissions to your GitHub organization. Installing the Endor Labs GitHub App (Pro) in your organization requires approval or permissions from your GitHub organizational administrator. If you don’t have the permissions, use the command line utility, endorctl, while you wait for the approval.
• Endor Labs GitHub App (Pro) requires the following permissions:
  • Read access to Dependabot alerts, actions, administration, checks, code, commit statuses, issues, metadata, packages, pull requests, repository hooks, and security events.
  • Read and write access to checks, contents, and pull requests.

Install GitHub App (Pro)

To automatically scan repositories using the GitHub App and create automatic PRs to fix vulnerabilities:

1. Sign in to Endor Labs.

2. From the left sidebar, choose Projects and click Add Project.

3. From GITHUB, choose GitHub App

4. Select Enable Automated Pull Requests.

   

5. Click Install GitHub App (Pro).

   You will be redirected to GitHub to install the Endor Labs App (Pro).

6. Click Install.

7. Select a user to authorize the app.

8. Select the organization in which you want to install the app.

9. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

10. Review the permissions required for Endor Labs and click Install and Authorize.

    If the button to install says Install and Request instead of Install and Authorize, you don’t have permission to install the GitHub App. Use the endorctl command line interface or select Install and Request to notify your organizational administrator of your request to install.

11. Choose a namespace and click Next.

    

12. Based on your license, select and enable the scanners.

    

    The following scanners are available:

    • SCA: Perform software composition analysis.
    • RSPM: Scan the repository for misconfigurations.
    • Secret: Scan the repository for exposed secrets.
    • CI/CD: Scan the repository and identify all the CI/CD tools used in the repository.
    • SAST: Scan your source code for weakness and generate SAST findings.
    • AI Models: Scan your repository and discover AI models in your source code.

13. Select PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

    

    • Select Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.

    • Select Include Archived Repositories to scan your archived repositories. By default, the GitHub archived repositories aren’t scanned.

    • In Define Scanning Preferences, select either:

      • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

      • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

      See GitHub scan options for more information on the scans that you can do with the GitHub App.

14. Click Continue. You have successfully installed the GitHub App (Pro).

Endor Labs GitHub App (Pro) scans your repositories every 24 hours and reports any new findings or changes to release versions of your code. It can also raise a PR with a fix based on your remediation policy. Ensure that you configure automated PR scans in your environment. See Automated PR scans for more information.

Manage GitHub Apps on Endor Labs

You can edit or delete the GitHub App integrations.

Edit GitHub App (Pro)

To edit the GitHub App integration:

1. Sign in to Endor Labs.
2. Select Manage > Integrations from the left sidebar.
3. Click Manage next to GitHub under Source Control Managers.
4. Click the three verticals dots on the right side of the GitHub App (Pro) that you want to edit, and select Edit Integration.
5. Based on your license, select and enable from the available list of scanners. You can also choose to update the pull request scan options.
6. Click Save. The changes are applicable from the next scanning cycle.
7. Use Reset to clear your selection.

Migrate GitHub App

You can migrate your GitHub App (Pro) to standard GitHub App (or from standard to Pro).

1. Sign in to Endor Labs.

2. Select Manage > Integrations from the left sidebar.

3. Click Manage next to GitHub under Source Control Managers.

4. Click the three vertical dots on the right side of the GitHub App (Pro) that you want to edit, and select Migrate To Standard App.

5. Click Migrate.

   You will be redirected to GitHub.

6. Click Configure.

7. Select a user to authorize the app.

8. Select Configure in the organization in which you want to migrate the app.

9. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

   Warning

   When you migrate from one app to the other, select the same set of repositories as before to preserve the currently scanned projects and findings after the migration.

10. Choose the namespace and click Next.

    Warning

    You must choose the same namespace as your existing GitHub App installation.

11. Select and enable the scanners you require.

12. Select the preferences for scanning pull requests, if required.

13. Click Continue.

Delete GitHub App (Pro)

To delete a GitHub App integration, click the three vertical dots on the right side, and select Delete Integration.

You are to taken to the GitHub App page, where you can uninstall the app from your GitHub organization.

Manually trigger scans

To manually trigger a scan, click Rescan Org. Endor Labs GitHub App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Add more GitHub repositories

Click Scan More Repositories to go to Projects page, from which you can add more repositories to scan through the GitHub App.

Set up package repositories

You can improve your experience with the GitHub App by setting up package repositories. This will help you create a complete bill of materials and perform static analysis. Without setting package repositories, you may not be able to get an accurate bill of materials. See Set up package manager integration for more information.

Technical limitations of the GitHub App (Pro)

The Endor Labs GitHub App (Pro) has the same limitations as the GitHub App. See Limitations for more information.

2 - Deploy Endor Labs Azure DevOps App

Endor Labs provides an Azure DevOps App that continuously scans Azure repos in your projects for security risks. You can selectively scan your repositories for SCA, secrets, SAST, or CI/CD tools.

You can choose to configure the Azure DevOps App at the organization level or the project level. When you configure the Azure DevOps App at the organization level, Endor Labs adds all the projects under the organization and scans all the repos in the projects. When you add an Azure DevOps project, Endor Labs scans all repos within that project.

Managed namespaces for Azure DevOps

You need to add an Azure organization or a project to an Endor Labs namespace. Organizations and projects in Azure DevOps are mapped as managed namespaces in Endor Labs.

Managed namespaces have the following restrictions:

• You cannot delete managed namespaces.
• You cannot delete repos present within managed namespaces.
• You cannot add projects or create namespaces within managed namespaces.
• You cannot create any new Endor Labs App installation within the managed namespaces.

Namespace structure when you add an Azure organization

When you add an Azure organization to an Endor Labs namespace, Endor Labs creates a child namespace for the organization and creates child namespaces for each project in the organization under the organization namespace. The organization namespace and project namespaces are managed namespaces. You can add multiple projects to the same Endor Labs namespace. Each project will have its own managed namespace.

If your organization name is deerinc and you have three projects, buck, doe, and fawn, Endor Labs creates four managed namespaces: deerinc, buck, doe, and fawn. The namespaces buck, doe, and fawn are child namespaces of the deerinc namespace.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-azure]

      %% Azure projects
      O1[deerinc]
      P1[buck]
      P2[doe]
      P3[fawn]


      %% connections
      EN --> O1
      O1 --> P1
      O1 --> P2
      O1 --> P3

      class EN,EN2 endor
      class O1,P1,P2,P3 managed
      classDef managed fill:#3FE1F3

Namespace structure when you add an Azure DevOps project

When you add an Azure DevOps project to an Endor Labs namespace, Endor Labs creates a child namespace for the Azure DevOps project and maps all repositories in that project to this namespace. The child namespace that maps to the Azure DevOps project is a managed namespace. The managed namespace has the name, <organization name>-<project name>. For example, if your organization name is deerinc and project name is doe, the managed namespace will have the name, deerinc-doe.

You can add multiple projects to the same Endor Labs namespace. Each project will have its own managed namespace. For example, your organization name is deerinc, which has three projects, buck,doe, andfawn. You add each project to the Endor Labs namespace, endor-azure.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-azure]

      %% Azure projects
      A1[deerinc-buck]
      A2[deerinc-doe]
      A3[deerinc-fawn]


      %% connections
      EN --> A1
      EN --> A2
      EN --> A3

      class EN,EN2 endor
      class A1,A2,A3 managed
      classDef managed fill:#3FE1F3

Prerequisites for Azure DevOps App

Ensure the following prerequisites are in place before you install the Endor Labs Azure DevOps App.

• An Azure DevOps cloud account and organization. If you don’t have one, create one at Azure DevOps.
• Endor Labs Azure DevOps App requires read permissions to in your projects. You can grant these permissions by providing read access to the Code category when you create an Azure DevOps personal access token for Endor Labs.

Install the Azure DevOps App

To automatically scan repositories using the Azure DevOps App:

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From AZURE, select Azure DevOps App.

   

4. Enter the host URL of your Azure project.

   The URL must be in the format, https://dev.azure.com/<ORG_NAME>/ when you add an Azure organization. When you add an Azure DevOps project, the URL must be in the format, https://dev.azure.com/<ORG_NAME>/<PROJECT_NAME>.

5. Enter your personal access token from Azure.

   You must have at least read permissions in the Code category for your Azure DevOps personal access token.

6. Click Scanners and select the scan types to enable.

   • SCA: Perform software composition analysis.
   • Secret: Scan Azure repos for exposed secrets.
   • CI/CD: Scan Azure repos and identify all the CI/CD tools used.
   • SAST: Scan your source code for weakness and generate SAST findings.
   • AI models: Scan source code to detect AI models and assess associated risks.

   The available scan types depend upon your license.

7. Select Include Disabled Repositories to scan your archived repositories. By default, the Azure archived repositories aren’t scanned.

8. Click Create.

Endor Labs Azure DevOps App scans your Azure repos every 24 hours and reports any new findings or changes to release versions of your code.

Manage Azure DevOps Apps on Endor Labs

You can edit or delete the Azure DevOps App integrations.

To edit the Azure DevOps App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left sidebar.
2. Click Manage next to Azure under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token.
4. Click SCANNERS and based on your license, select and enable from the available list of scanners.
5. Click Save. The changes are applicable from the next scanning cycle.

To delete an Azure DevOps App integration, click on the three vertical dots next to the integration, and select Delete Integration.

To manually trigger a scan, click Rescan Org. Azure DevOps App scans your repositories every 24 hours. Use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more organizations or projects to scan through the Azure DevOps App.

3 - Deploy Endor Labs GitLab App

Endor Labs provides a GitLab App that continuously monitors users’ projects for security and operational risk. You can use the GitLab App to selectively scan your repositories for SCA, secrets, SAST, and CI/CD tools.

When you use Endor Labs GitLab App, Endor Labs creates namespaces based on your organization hierarchy in GitLab.

The namespaces created by the Endor Labs GitLab App are not like regular namespaces and are called managed namespaces. These namespaces are named after subgroup slugs in GitLab.

Limitations of GitLab groups in Endor Labs namespace

Ensure that you consider the following limitations when you use the GitLab monitoring scan.

• GitLab supports up to 20 levels of subgroup nesting, while Endor Labs currently supports a maximum of 10 levels, assuming the installation is created at the tenant level. If a GitLab installation is created within a nested namespace, such as tenant.namespace1.namespace2, the available nesting depth for subgroups in GitLab is reduced. In this case, Endor Labs can only support up to eight levels of nested subgroups.
• Endor Labs supports GitLab groups with a maximum of 64 characters.

Managed namespaces for GitLab

Managed namespaces are always reflective in terms of structure and content in GitLab.

Managed namespaces have the following restrictions:

• You cannot delete managed namespaces.

• You cannot delete projects present within managed namespaces.

• You cannot add projects or create namespaces within managed namespaces.

• You cannot create any new Endor Labs App installation within the managed namespaces.

  For example, you cannot create an Endor Labs GitHub App installation within a namespace that was created by the Endor Labs GitLab App.

Any modifications to the namespaces have to be in GitLab. The changes that you make to the namespaces and projects are reflected in Endor Labs after a rescan.

If your organization has the following hierarchy in GitLab:

graph TD
    GL((GitLab))
    HC[HappyCorp]

    %% Main divisions
    Web[Web]
    Mobile[Mobile]
    Desktop[Desktop]

    %% Web subgroups
    WA[Alpha]
    WB[Beta]
    WG[Gamma]

    %% Mobile subgroups
    MD[Delta]
    ME[Epsilon]
    MZ[Zeta]

    %% Desktop subgroups
    DP[Pi]
    DR[Rho]
    DS[Sigma]

    %% Main connections
    GL --> HC
    HC --> Web
    HC --> Mobile
    HC --> Desktop

    %% Web connections
    Web --> WA
    Web --> WB
    Web --> WG

    %% Mobile connections
    Mobile --> MD
    Mobile --> ME
    Mobile --> MZ

    %% Desktop connections
    Desktop --> DP
    Desktop --> DR
    Desktop --> DS

    class HC main
    class Web,Mobile,Desktop division
    classDef default fill:#D3D3D3
    classDef circle fill:white
    class GL circle

Endor Labs creates managed namespaces that mirror your GitLab groups under an Endor Labs namespace (for example, happyendor). Endor Labs creates happycorp as the parent namespace with web, mobile, and desktop as the child namespaces. The namespace happycorp will be under the Endor Labs namespace.

Each of these child namespaces have further child namespaces as follows:

• web: alpha, beta, gamma
• mobile: delta, epsilon, zeta
• desktop: pi, rho, sigma

The following diagram shows the organization of namespaces in Endor Labs.

graph TD
    EN[happyendor]
    HC[happycorp]

    %% Main divisions
    Web[web]
    Mobile[mobile]
    Desktop[desktop]

    %% Web subgroups
    WA[alpha]
    WB[beta]
    WG[gamma]

    %% Mobile subgroups
    MD[delta]
    ME[epsilon]
    MZ[zeta]

    %% Desktop subgroups
    DP[pi]
    DR[rho]
    DS[sigma]

    %% Main connections
    EN --> HC
    HC --> Web
    HC --> Mobile
    HC --> Desktop

    %% Web connections
    Web --> WA
    Web --> WB
    Web --> WG

    %% Mobile connections
    Mobile --> MD
    Mobile --> ME
    Mobile --> MZ

    %% Desktop connections
    Desktop --> DP
    Desktop --> DR
    Desktop --> DS

    class HC main
    class EN endor
    class Web,Mobile,Desktop division
    class WA,WB,WG,MD,ME,MZ,DP,DR,DS group
    classDef main fill:#3FE1F3
    classDef division fill:#3FE1F3
    classDef group fill:#3FE1F3

Manage multiple installations of GitLab App

You cannot create multiple GitLab installations with the same root group in the host URL within the same Endor Labs namespace.

For example, if a GitLab installation exists with a host URL like gitlab.com/group1/sg1, you cannot create another installation with a host URL like gitlab.com/group1/sg2 within the same Endor namespace. Instead, you must create the installation with a different root group in the host URL, such as gitlab.com/group2/sg2.

graph TD

      %% Endor Labs namespace
      EN[happyendor]

      %% GitLab groups
      G1[group1]
      G2[group2]
      SG1[sg1]
      SG2[sg2]

      %% connections
      EN --> G1
      EN --> G2
      G1 --> SG1
      G2 --> SG2

      class EN endor
      class G1,G2,SG1,SG2 managed
      classDef managed fill:#3FE1F3

If you wish to create an installation with a host URL like gitlab.com/group1/sg2, it should be inside a different Endor Labs namespace.

graph TD

      %% Endor Labs namespace
      EN[happyendor]
      EN2[happyendor2]

      %% GitLab groups
      G1[group1]
      G2[group1]
      SG1[sg1]
      SG2[sg2]

      %% connections
      EN --> G1
      EN2 --> G2
      G1 --> SG1
      G2 --> SG2

      class EN,EN2 endor
      class G1,G2,SG1,SG2 managed
      classDef managed fill:#3FE1F3

Prerequisites for GitLab App

Before installing and scanning projects with Endor Labs GitLab App, make sure you have:

• A GitLab cloud account and organization. If you don’t have one, create one at GitLab.
• Endor Labs GitLab App requires a GitLab personal access token with at least read_api permission.

Install the GitLab App

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From GITLAB, select GitLab App.

4. Enter the GitLab organization URL in the format: https://gitlab.com/{group}/{subgroup1}/....

   You need to enter at least the root group. For example, https://gitlab.com/group1.

   You can provide the host URL up to any subgroup level. For example, https://gitlab.com/group1/subgroup1/subgroup2/subgroup3.

   Endor Labs creates namespaces for groups and subgroups and maps projects to these namespaces.

   If the GitLab installation is created at the tenant level, Endor Labs supports up to 10 levels of GitLab group nesting. If the installation is created within a nested namespace under the tenant, the supported nesting depth decreases by one level for each additional level of nesting.

5. Enter the GitLab personal access token.

   The personal access token must have at least the read_api permission.

6. Select the scan types to enable.

   • SCA- Perform software composition analysis.
   • Secret - Scan GitLab projects for exposed secrets.
   • CI/CD - Scan GitLab projects and identify all the CI/CD tools used.
   • SAST - Scan GitLab projects to generate SAST findings.
   • AI models - Scan source code to detect AI models and assess associated risks.

   The available scan types depend upon your license.

7. Select Include Archived Repositories to scan your archived repositories. By default, the GitLab archived repositories aren’t scanned.

8. Click Create.

Endor Labs GitLab App scans your GitLab projects every 24 hours and reports any new findings or changes to release versions of your code.

Manage GitLab App on Endor Labs

You can edit or delete the GitLab App integrations.

To edit the GitLab App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left sidebar.
2. Click Manage next to GitLab under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token and choose the scanners.
4. Click Save. The changes are applicable from the next scanning cycle.

To delete a GitLab App integration, click on the three vertical dots next to the integration, and select Delete Integration. When you delete the integration, it will also delete all child namespaces, projects and references associated with the auto generated root group namespace. It also deletes any manually created namespaces and projects under auto generated namespace.

To manually trigger a scan, click Rescan Org. GitLab App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more projects to scan through the GitLab App.

4 - Deploy Endor Labs Bitbucket App in Bitbucket Data Center

Endor Labs provides a Bitbucket App that continuously monitors users’ projects for security and operational risks in Bitbucket Data Center. You can use the Bitbucket App to selectively scan your repositories for SCA, secrets, SAST, and CI/CD tools.

When you use the Endor Labs Bitbucket App, it creates namespaces based on your projects in Bitbucket Data Center. The namespaces created by the Endor Labs Bitbucket App are not like regular namespaces and are called managed namespaces. You can either configure the URL to Bitbucket Data Center to import all the projects or configure the project key to import a specific project in Endor Labs.

Managed namespaces for Bitbucket Data Center

You need to add the Bitbucket Data Center host or a project to an Endor Labs namespace. Bitbucket host and projects are mapped as managed namespaces in Endor Labs.

Managed namespaces have the following restrictions:

• You cannot delete managed namespaces.
• You cannot delete repositories within managed namespaces.
• You cannot add projects or create namespaces within managed namespaces.
• You cannot create new Endor Labs App installations within managed namespaces.

Namespace structure when you add a Bitbucket Data Center host

When you add a Bitbucket Data Center host to an Endor Labs namespace, Endor Labs creates a child namespace for the Bitbucket host and creates child namespaces for each project in the host under the host namespace. The namespaces of the host and projects are managed namespaces. If there are periods (.) in the Bitbucket datacenter hostname, it is converted to a hyphen (-). You can add multiple Bitbucket Data Center hosts to the same Endor Labs namespace. Each host will have its own managed namespace.

If your host name is bitbucket.deerinc.com and you have three projects, buck, doe, and fawn, Endor Labs creates four managed namespaces: bitbucket-deerinc-com, buck, doe, and fawn. The namespaces buck, doe, and fawn are child namespaces of the bitbucket-deerinc-com namespace.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-bitbucket]

      %% Bitbucket projects
      O1[bitbucket-deerinc-com]
      P1[buck]
      P2[doe]
      P3[fawn]


      %% connections
      EN --> O1
      O1 --> P1
      O1 --> P2
      O1 --> P3

      class EN,EN2 endor
      class O1,P1,P2,P3 managed
      classDef managed fill:#3FE1F3

Namespace structure when you add a Bitbucket Data Center project

When you add a Bitbucket Data Center project to an Endor Labs namespace, Endor Labs creates a child namespace for the Bitbucket Data Center project and maps all repositories in that project to this namespace. The child namespace that maps to the Bitbucket Data Center project is a managed namespace. The managed namespace has the name, <host name>_<project name>. For example, if your Bitbucket hostname is bitbucket.deerinc.com and project name is doe, the managed namespace will have the name, bitbucket-deerinc-com_doe.

You can add multiple projects to the same Endor Labs namespace. Each project will have its own managed namespace. For example, your hostname is bitbucket.deerinc.com, which has three projects, buck,doe, andfawn. You add each project to the Endor Labs namespace, endor-azure.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-bitbucket]

      %% Bitbucket projects
      A1[bitbucket-deerinc-com_buck]
      A2[bitbucket-deerinc-com_doe]
      A3[bitbucket-deerinc-com_fawn]


      %% connections
      EN --> A1
      EN --> A2
      EN --> A3

      class EN,EN2 endor
      class A1,A2,A3 managed
      classDef managed fill:#3FE1F3

Prerequisites for Bitbucket App for Bitbucket Data Center

Ensure the following prerequisites are in place before you install the Endor Labs Bitbucket App.

• A publicly accessible Bitbucket Data Center host set up with HTTPS.
• A Bitbucket HTTP access token with at least Project read permission at the project level.

Install the Bitbucket App

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From BITBUCKET, select Bitbucket App, and select Data Center to import projects from Bitbucket Data Center.

   

4. Enter the Bitbucket hostname URL in the format: https://<host name> to import all the projects.

   Endor Labs creates namespaces for all projects that are available in the Bitbucket Data Center host.

   You can also provide the URL up to a project level to import a specific project: https://<host name>/projects/<project-key>. For example, https://endor-bitbucket.com/projects/LAB.

5. Enter the Bitbucket Data Center HTTP access token.

   The HTTP access token must have at least the Project read permission at the project level.

6. Select the scan types to enable.

   • SCA- Perform software composition analysis.
   • Secret - Scan Bitbucket projects for exposed secrets.
   • CI/CD - Scan Bitbucket projects and identify all the CI/CD tools used.
   • SAST - Scan Bitbucket projects to generate SAST findings.
   • AI models - Scan source code to detect AI models and assess associated risks.

   The available scan types depend upon your license.

7. Select Include Archived Repositories to scan your archived repositories.

   By default, the Bitbucket archived repositories aren’t scanned.

8. Click Create.

Endor Labs Bitbucket App scans your Bitbucket projects every 24 hours and reports any new findings or changes to release versions of your code.

Manage Bitbucket App on Endor Labs

You can edit or delete the Bitbucket App integrations.

To edit the Bitbucket App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left sidebar.
2. Click Manage next to Bitbucket under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token and choose the scanners.
4. Click Save. The changes are applicable from the next scanning cycle.

To delete a Bitbucket App integration, click on the three vertical dots next to the integration, and select Delete Integration. When you delete the integration, it will also delete all child namespaces, projects and references associated with the auto generated root group namespace. It also deletes any manually created namespaces and projects under auto generated namespace.

To manually trigger a scan, click Rescan Org. Bitbucket App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more projects to scan through the Bitbucket App.

5 - Deploy Endor Labs Bitbucket App in Bitbucket Cloud

Endor Labs provides a Bitbucket App that continuously monitors users’ projects for security and operational risks in Bitbucket Cloud. You can use the Bitbucket App to selectively scan your repositories for SCA, secrets, SAST, and CI/CD tools.

When you use the Endor Labs Bitbucket App, it creates namespaces based on your workspace and projects in Bitbucket Cloud. The namespaces created by the Endor Labs Bitbucket App are not like regular namespaces and are called managed namespaces. You can either configure the URL to Bitbucket Cloud to import all the projects or configure the project key to import a specific project in Endor Labs.

Managed namespaces for Bitbucket Cloud

You need to add the Bitbucket Cloud workspace or a project to an Endor Labs namespace. Bitbucket Cloud workspace and projects are mapped as managed namespaces in Endor Labs.

Managed namespaces have the following restrictions:

• You cannot delete managed namespaces.
• You cannot delete repositories within managed namespaces.
• You cannot add projects or create namespaces within managed namespaces.
• You cannot create new Endor Labs App installations within managed namespaces.

Namespace structure when you add a Bitbucket Cloud workspace

When you add a Bitbucket Cloud workspace to an Endor Labs namespace, Endor Labs creates a child namespace for the Bitbucket Cloud workspace and creates child namespaces for each project in the workspace under the workspace namespace. The namespaces of the workspace and projects are managed namespaces. You can add multiple Bitbucket Cloud workspaces to the same Endor Labs namespace. Each workspace will have its own managed namespace.

If your workspace name is deerinc and you have three projects, buck, doe, and fawn, Endor Labs creates four managed namespaces: deerinc, buck, doe, and fawn. The namespaces buck, doe, and fawn are child namespaces of the deerinc namespace.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-bitbucket]

      %% Bitbucket projects
      O1[deerinc]
      P1[buck]
      P2[doe]
      P3[fawn]


      %% connections
      EN --> O1
      O1 --> P1
      O1 --> P2
      O1 --> P3

      class EN,EN2 endor
      class O1,P1,P2,P3 managed
      classDef managed fill:#3FE1F3

Namespace structure when you add a Bitbucket Cloud project

When you add a Bitbucket Cloud project to an Endor Labs namespace, Endor Labs creates a child namespace for the Bitbucket Cloud project and maps all repositories in that project to this namespace. The child namespace that maps to the Bitbucket Cloud project is a managed namespace. The managed namespace has the name, <workspace name>_<project name>. For example, if your Bitbucket Cloud workspace name is deerinc and project name is doe, the managed namespace will have the name, deerinc_doe.

You can add multiple projects to the same Endor Labs namespace. Each project will have its own managed namespace. For example, your workspace name is deerinc, which has three projects, buck,doe, andfawn. You add each project to the Endor Labs namespace, endor-bitbucket.

The following image shows the namespace structure in Endor Labs.

graph TD

      %% Endor Labs namespace
      EN[endor-bitbucket]

      %% Bitbucket projects
      A1[deerinc_buck]
      A2[deerinc_doe]
      A3[deerinc_fawn]


      %% connections
      EN --> A1
      EN --> A2
      EN --> A3

      class EN,EN2 endor
      class A1,A2,A3 managed
      classDef managed fill:#3FE1F3

Prerequisites for Bitbucket App for Bitbucket Cloud

Ensure the following prerequisites are in place before you install the Endor Labs Bitbucket App.

• Bitbucket Cloud instance with workspace and projects
• A Bitbucket access token either at the workspace level to import a workspace, or the project level to import a project. The token must have at least Project read permission.

Install the Bitbucket App

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From BITBUCKET, select Bitbucket App, and ensure that Cloud is selected.

   

4. Enter the Bitbucket Cloud workspace URL in the format: https://bitbucket.org/{workspace} to import all the projects in the workspace.

   Endor Labs creates namespaces for all projects that are available in the Bitbucket Cloud workspace.

   You can also provide the URL up to a project level to import a specific project: https://bitbucket.org/{workspace}/{project-key}. For example, https://bitbucket.org/endor-labs/lab.

5. Enter the Bitbucket access token.

   The access token must have at least the Project read permission.

6. Select the scan types to enable.

   • SCA- Perform software composition analysis.
   • Secret - Scan Bitbucket projects for exposed secrets.
   • CI/CD - Scan Bitbucket projects and identify all the CI/CD tools used.
   • SAST - Scan Bitbucket projects to generate SAST findings.
   • AI models - Scan source code to detect AI models and assess associated risks.

   The available scan types depend upon your license.

7. Select Include Archived Repositories to scan your archived repositories.

   By default, the Bitbucket archived repositories aren’t scanned.

8. Click Create.

Endor Labs Bitbucket App scans your Bitbucket projects every 24 hours and reports any new findings or changes to release versions of your code.

Manage Bitbucket App on Endor Labs

You can edit or delete the Bitbucket App integrations.

To edit the Bitbucket App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left sidebar.
2. Click Manage next to Bitbucket under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token and choose the scanners.
4. Click Save. The changes are applicable from the next scanning cycle.

To delete a Bitbucket App integration, click on the three vertical dots next to the integration, and select Delete Integration. When you delete the integration, it will also delete all child namespaces, projects and references associated with the auto generated root group namespace. It also deletes any manually created namespaces and projects under auto generated namespace.

To manually trigger a scan, click Rescan Org. Bitbucket App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more projects to scan through the Bitbucket App.

6 - Outpost: The on-prem scheduler for monitoring scans

Endor Labs monitoring scan regularly scans your source code to discover vulnerabilities. The Endor Labs Apps clone and analyze all repositories every 24 hours, ensuring continuous monitoring for open source vulnerabilities and code weaknesses. See Monitoring scans for more information.

Endor Labs uses a Kubernetes cluster where your source code repositories and cloned, and the scans are conducted. Your policies may require that the source code repositories are not exposed to the public cloud. In such situations, you can use Outpost, which is a deployable on-prem instance of the Endor Labs scheduler that runs on your private Kubernetes cluster. After you deploy Outpost, Endor Labs uses your Kubernetes cluster to run the monitoring scans.

The scheduling of the scans and the scans themselves are run within your firewall. After the scans are completed, only the scan results are sent to Endor Labs.

Outpost provides full feature parity with regular cloud-based Endor Labs scans.

The following diagram shows how monitoring scans work when you configure Outpost.

graph TD
    A(["Endor Labs App"]) -->|<span style='font-size: 12px'>Continuous monitoring</span>| B["Source Code Repositories"]
    B <--> F["Private Artifact Registry"]
    A -->|<span style='font-size: 12px'>Initiates scans</span>| C["Private Kubernetes cluster
    <span style='font-size: 12px'>Runs Outpost and the scans inside your firewall</span>"]
    B -->|<span style='font-size: 12px'>Clones repositories</span>| C
    C -->|<span style='font-size: 12px'>Pass scan data</span>| D(["Endor Labs Platform
    <span style='font-size: 12px'>Generate findings from scan results</span>"])

    subgraph E["Firewall"]
    A
    B
    C
    F
    end

    class A,D endor
    class B,C,F customer
    class E firewall
    classDef customer fill:#3FE1F3
    classDef endor fill:#5BF385
    classDef firewall fill:transparent,stroke-dasharray: 5 5,stroke:#FF4500,color:#FF4500

Outpost helps you in the following instances:

• Security and compliance requirements prevent you from exposing source code repositories to external services
• Firewall restrictions that prevent direct integration with external scan services and requires complex VPN configurations
• Integrating endorctl into multiple CI/CD pipelines can be complex and costly, and relying on manual scanning processes increases the risk of errors
• Need to scan against private artifact registries and internal services

You need to configure Outpost as an integration at your Endor Labs tenant namespace. You can only configure Outpost as an integration at your root namespace. After you configure the integration and complete the Kubernetes cluster configuration, all the monitoring scans in your Endor Labs tenant are run on your Kubernetes cluster.

The following sections describe how you can configure and deploy Outpost.

• Outpost requirements
• Outpost authentication
• Outpost configuration

6.1 - Outpost requirements

You need a Kubernetes cluster with nodes that have at least 8 cores and 32 GB of RAM. Nodes should be running Linux.

You must create a namespace in your Kubernetes cluster to deploy Outpost. Use this namespace when you configure Outpost integration in Endor Labs.

Outpost currently supports the following Kubernetes distributions:

• Azure Kubernetes Service (AKS)
• Google Kubernetes Engine (GKE)
• Amazon Elastic Kubernetes Service (EKS)
• Self-hosted Kubernetes clusters

The total number of nodes in the cluster depends on the number of projects that you want to scan and the number of scans that you want to run in a day. On average, each scan is expected to take approximately one hour to complete.

The total scans per node per day (24 hours) is calculated based on the following formula.

Total scans per node per day = (CPU cores in the node ÷ 8) × 24

You can use the following formula to calculate the number of nodes required.

Number of nodes = Total projects ÷ Total scans per node per day

The following table shows the number of nodes required for different combinations of projects and scans.

Network requirements for Outpost

Ensure the following network requirements are met for Outpost:

• Egress Access: Required. Allow outbound traffic to Endor Labs platform, toolchains, and package managers.
• DNS Resolution: Required. Allow list of necessary domains.
• Network Policies: Required. Allow outbound traffic to Endor Labs platform, toolchains, and package managers.

6.2 - Outpost authentication

Outpost can use the following authentication mechanisms:

• API Key: You can generate an Endor Labs API key and secret and configure Outpost to use it.
• Azure Managed Identity: You can configure Outpost to use an Azure managed identity for authentication. Applicable if you use Azure Kubernetes Service (AKS). You must also configure a corresponding authorization policy in Endor Labs.
• GCP Service Account: You can configure Outpost to use a GCP service account for authentication. Applicable if you use Google Kubernetes Engine (GKE). You must also configure a corresponding authorization policy in Endor Labs.

API key authentication for Outpost

You can create an Endor Labs API key and secret to authenticate your Outpost configuration. See API keys for more information.

Ensure that you select On-prem Scheduler as the API key permissions.

Azure Managed Identity authentication for Outpost

Perform the following steps to configure Outpost to use an Azure managed identity for authentication.

1. Enable workload identity in the AKS cluster.

2. Enable OIDC provider in the AKS cluster.

3. Create an Azure managed identity.

   az identity create -g endor-group -n endor-identity
   

   The command creates a zero-permission managed identity. Store the clientId for later use.

4. Run the following command to retrieve the OIDC Issuer from AKS.

   OIDC_ISSUER=$(az aks show -g endor-group -n onprem-cluster --query "oidcIssuerProfile.issuerUrl" -o tsv)
   

   The command fetches the OIDC issuer URL for federated authentication. Ensure that you enable OIDC and Workload Identity in the AKS cluster.

5. Create federated credentials for workloads.

   Run the following command to create the scheduler federated credential.

   az identity federated-credential create \
    --name scheduler-federated-identity \
    --identity-name endor-identity \
    --resource-group endor-group \
    --issuer $OIDC_ISSUER \
    --subject system:serviceaccount:onprem-cluster:onprem-scheduler-account
   

   Run the following command to create the endorctl federated credential.

   az identity federated-credential create \
    --name endorctl-federated-identity \
    --identity-name endor-identity \
    --resource-group endor-group \
    --issuer $OIDC_ISSUER \
    --subject system:serviceaccount:onprem-cluster:onprem-scheduler-endorctl-account
   

   Warning

   The onprem-cluster specified in the commands is the name of the Kubernetes namespace where Outpost is to be deployed. Replace it with the actual namespace name where you want to deploy Outpost. Ensure that you create the namespace before running the commands.

   The commands link the managed identity to Kubernetes service accounts and enable secure access without static credentials.

6. Configure an authorization policy in Endor Labs with configuration from Azure.

7. Configure the Outpost integration with Managed Identity Client ID.

   See Outpost configuration for more information on how to configure the Outpost integration.

GCP Service Account authentication for Outpost

Perform the following steps to configure Outpost to use a GCP service account for authentication.

1. Enable workload identity in the GKE cluster.

2. Enable OIDC provider in the GKE cluster.

3. Create a new workload service account.

   gcloud iam service-accounts create endor-compute \
     --description="Endor Labs Compute Service Account" \
     --display-name="Endor Labs Compute Service Account"
   

4. Grant roles/iam.serviceAccountOpenIdTokenCreator to workload service account.

   gcloud projects add-iam-policy-binding endor-experiments \
    --member "serviceAccount:endor-compute@endor-experiments.iam.gserviceaccount.com" \
    --role "roles/iam.serviceAccountOpenIdTokenCreator"
   

5. Create a zero-permission service account for Endor Labs to perform federation.

   gcloud iam service-accounts create endor-federation \
    --description="Endor Labs Keyless Federation Service Account" \
    --display-name="Endor Labs Federation Service Account"
   

6. Allow the Kubernetes service accounts to impersonate the IAM workload service account.

   gcloud iam service-accounts add-iam-policy-binding endor-compute@endor-experiments.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:endor-experiments.svc.id.goog[onprem-scheduler/onprem-scheduler-account]"
   

   gcloud iam service-accounts add-iam-policy-binding endor-compute@endor-experiments.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:endor-experiments.svc.id.goog[onprem-scheduler/onprem-scheduler-endorctl-account]"
   

7. Configure an authorization policy in Endor Labs with configuration from Google Cloud.

8. Configure the Outpost integration with Service Account Name.

   See Outpost configuration for more information on how to configure the Outpost integration.

   Warning

   Replace endor-experiments with the actual project name where you want to deploy Outpost. Replace endor-federation and endor-compute with the actual service account names.

6.3 - Outpost configuration

After you set up your Kubernetes cluster and set up authentication, you can configure the Outpost integration.

Configure Outpost integration

Perform the following steps to configure Outpost:

1. Select Integrations from the left sidebar.

2. Click Configure under On-Prem Integration.

   

3. Choose from the following authentication methods:

   • API: Use the Endor Labs API key authentication for Outpost. You need to enter the API key and API secret. See API key authentication for Outpost for more information.
   • Azure: Use the Azure managed identity authentication for Outpost. You need to enter the Azure managed identity client ID. See Azure Managed Identity authentication for Outpost for more information.
   • Google: Use the GCP service account authentication for Outpost. You need to enter the GCP service account name. See GCP Service Account authentication for Outpost for more information.

4. Click Advanced to configure the following parameters:

   • Select Enable Build Tool Caching to enable build tool caching. Bazel remote cache is installed and the build tools are cached in the cluster.
   • Enter the number of concurrent scans that Outpost can run in Max Running Scans.
   • Enter the maximum duration of a scan in Max Duration.

5. Click Enable Scheduler to store the configuration and enable Outpost.

6. Click Download Helm Values to download the endor-outpost-values.yaml file.

   You can choose to customize the values before you deploy Outpost.

   You can also extract the chart from oci://endorcipublic.azurecr.io/charts/onprem-scheduler and refer the default values.yaml for all the available options. See Helm Chart Values for more information.

7. Run the following command to deploy Outpost in your Kubernetes cluster.

   helm install endorlabsscheduler oci://endorcipublic.azurecr.io/charts/onprem-scheduler \
   -n <your Kubernetes namespace> \
   -f endor-outpost-values.yaml
   

   The command installs the Outpost scheduler on your Kubernetes cluster.

   Important

   If you use GCP service account authentication, you need to configure annotations for the service account in Helm values.

   Add the following annotations to the Helm chart values before you run the Helm install command.

   scheduler:
    .
    .
     serviceAccount:
       create: true
       annotations:
         iam.gke.io/gcp-service-account: "endor-scanner-sa@project-name-123456.iam.gserviceaccount.com"
   endorctl:
    .
    .
     serviceAccount:
       create: true
       annotations:
         iam.gke.io/gcp-service-account: "endor-scanner-sa@project-name-123456.iam.gserviceaccount.com"
   

   If you run the Helm command directly, update the generated Helm command to set the annotations with the following options:

   --set scheduler.serviceAccount.annotations.iam.gke.io/gcp-service-account="<GCP_SERVICE_ACCOUNT_NAME>@<GCP_PROJECT_NAME>.iam.gserviceaccount.com"
   --set endorctl.serviceAccount.annotations.iam.gke.io/gcp-service-account="<GCP_SERVICE_ACCOUNT_NAME>@<GCP_PROJECT_NAME>.iam.gserviceaccount.com"
   

   For example:

   helm install endorlabsscheduler oci://endorcipublic.azurecr.io/charts/onprem-scheduler \
       -n <k8s-ns> \
       --set endorAPI=https://api.staging.endorlabs.com \
       --set endorNamespace=nryn \
       --set auth.gcpServiceAccountName=endor-scanner-sa@project-name-123456.iam.gserviceaccount.com \
       --set scheduler.serviceAccount.annotations.iam.gke.io/gcp-service-account="endor-scanner-sa@project-name-123456.iam.gserviceaccount.com" \
       --set endorctl.serviceAccount.annotations.iam.gke.io/gcp-service-account="endor-scanner-sa@project-name-123456.iam.gserviceaccount.com" \
       --set bazelremote.install=false
   

   The annotations are automatically added to the Helm chart values if you choose Azure managed identity authentication.

8. If you do not want to customize the values, the Helm command with your configured values appears in the user interface. You can copy the command and run it on your Kubernetes cluster.

   For example, the following command appears on the user interface when you configure the integration on the endor Kubernetes namespace with the Azure managed identity authentication and build tool caching enabled. The root Endor Labs namespace is endor.

   helm install endorlabsscheduler oci://endorcipublic.azurecr.io/charts/onprem-scheduler \
        -n endor \
        --set endorAPI=https://api.endorlabs.com \
        --set endorNamespace=endor \
        --set auth.azureManagedIdentityClientID=12a34b56-7c89-0d1e-2f34-567g890h1234 \
        --set bazelremote.install=true
   

   You can copy the command and run it on your Kubernetes cluster to deploy Outpost.

Update Outpost configuration

To update the Outpost configuration, you need to uninstall the existing Helm chart and install a new one with the updated values.

Run the following command to uninstall the existing Helm chart.

helm uninstall endorlabsscheduler -n <your namespace>

You can update the configuration in the user interface to generate a new Helm chart or command, or you can manually update the values in the endor-outpost-values.yaml file. We recommend that you update the configuration in the user interface even if you manually update and install the Helm chart.

Perform the following steps to update the configuration in the user interface:

1. Select Integrations from the left sidebar.
2. Click Manage under On-Prem Integration.
3. Update the configuration and click Enable Scheduler to update the configuration.
4. Apply the updated values with the helm install command as described in Configure Outpost integration.

Generally, you need to update the configuration when the authentication expires. API keys have a maximum validity period of one year. The expiry of Azure managed identity and GCP service accounts depends on the expiry of the corresponding authorization policy.

View Outpost logs

You can view the Outpost logs in the Endor Labs platform.

Perform the following steps to view the Outpost logs:

1. Select Integrations from the left sidebar.

2. Click View Logs under On-Prem Integration.

   

   You can copy the logs to the clipboard or download the logs.

   By default, the logs are brief logs are displayed. You can select Show Verbose Logs to view the detailed logs.

   The log level is set as All by default. You can select Info to view the info logs and Debug to view the debug logs.

Helm Chart Values

Run the following command to extract the default values for the Outpost Helm chart.

helm pull oci://endorcipublic.azurecr.io/charts/onprem-scheduler --untar

The values.yaml file in the onprem-scheduler directory contains the default values for the Outpost Helm chart.

The following yaml file shows the default values in the values.yaml file.

# Default values for onprem-scheduler.
# This file is YAML-formatted.

# Base URL for the Endor Labs platform [Do not modify]
endorAPI: "https://api.endorlabs.com"

# Your organization's namespace in Endor Labs [Do not modify unless there is a change in your tenant]
endorNamespace: "required"

# Log level for scheduler and endorctl. Optional.
logLevel: "info"

# Log output format for scheduler. Optional.
logOutput: "json"

# Authentication configuration - use ONE of the following methods.
# NOTE: Only one authentication method (apiKey & apiSecret, gcpServiceAccountName,
# or azureManagedIdentityClientID) must be set.
auth:
  # Option 1: API Key authentication. Enter the Endor Labs API key and secret.
  apiKey: ""
  apiSecret: ""

  # Option 2: GCP Service Account authentication. Enter the GCP service account name.
  # NOTE: Ensure service accounts are created with workload identity annotations.
  gcpServiceAccountName: ""

  # Option 3: Azure Managed Identity authentication. Enter the Azure managed identity client ID.
  # NOTE: Ensure service accounts are created with workload identity annotations.
  azureManagedIdentityClientID: ""

scheduler:
  # Maximum number of scans that you want to run concurrently. Optional.
  maxRunningJobs: 20

  # Scheduler container image settings.
  image:
    # Container repository for the scheduler image [Do not modify]
    repository: "endorcipublic.azurecr.io/scheduler"

    # Image version to use [Do not modify]
    tag: "latest"

    # Image pull policy [Do not modify]
    pullPolicy: "Always"

  # Labels for the scheduler deployment. Optional.
  labels: {}

  # Annotations for the scheduler deployment. Optional.
  annotations: {}

  # Labels for the scheduler pod. Optional.
  podLabels: {}

  # Annotations for the scheduler pod. Optional.
  podAnnotations: {}

  serviceAccount:
    # Specifies whether a service account should be created. Optional.
    create: false

    # Name of the service account to use for scheduler. Optional.
    name: ""

    # Labels for the scheduler service account. Optional.
    labels: {}

    # Annotations for the scheduler service account. Optional.
    annotations: {}

  # Pod-level security context for the scheduler. Optional.
  podSecurityContext: {}

  # Container-level security context for the scheduler. Optional.
  securityContext: {}

  # Resource constraints for the scheduler pod. Optional.
  resources: {}

  healthProbes:
    # Port used to perform health checks. Optional.
    port: 8080

    # Readiness probe for the scheduler pod. Optional.
    readinessProbe:
      enabled: true
      failureThreshold: 2
      successThreshold: 1
      periodSeconds: 5
      timeoutSeconds: 1
      initialDelaySeconds: 10

    # Liveness probe for the scheduler pod. Optional.
    livenessProbe:
      enabled: true
      failureThreshold: 4
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
      initialDelaySeconds: 0

  # Node selector for the scheduler pod. Optional.
  nodeSelector: {}

  # Tolerations for the scheduler pod. Optional.
  tolerations: []

  # Affinity settings for the scheduler pod. Optional.
  affinity: {}

  # Volumes for the scheduler pod. Optional.
  volumes: []

  # Volume mounts for the scheduler pod. Optional.
  volumeMounts: []

  # Additional environment variables for the scheduler pod. Optional.
  additionalEnvs: []

endorctl:
  # Maximum runtime duration in minutes for a scan. Optional. Default value is 60.
  maxDuration: 1440

  bazelRemote:
    # Bazel remote cache service name.
    # Refer bazelremote values below. Optional.
    serviceName: "bazel-remote-cache"

    # Bazel remote cache GRPC service port.
    # Refer bazelremote values below. Optional.
    servicePort: 9092

  # Endorctl container image settings.
  image:
    # Container repository for the endorctl image [Do not modify]
    repository: "endorcipublic.azurecr.io/endorctl_bare"

    # Image version to use [Do not modify]
    tag: "latest"

    # Image pull policy [Do not modify]
    pullPolicy: "Always"

  # Labels for the endorctl job. Optional.
  labels: {}

  # Annotations for the endorctl job. Optional.
  annotations: {}

  # Labels for the endorctl pod. Optional.
  podLabels: {}

  # Annotations for the endorctl pod. Optional.
  podAnnotations: {}

  serviceAccount:
    # Specifies whether a service account should be created. Optional.
    create: false

    # Name of the service account to use for endorctl. Optional.
    name: ""

    # Labels for the endorctl service account. Optional.
    labels: {}

    # Annotations for the endorctl service account. Optional.
    annotations: {}

  # Pod-level security context for the endorctl. Optional.
  podSecurityContext: {}

  # Container-level security context for the endorctl. Optional.
  securityContext: {}

  # Resource constraints for the endorctl job. Optional.
  resources: {}

  # Node selector for the endorctl pod. Optional.
  nodeSelector: {}

  # Tolerations for the endorctl pod. Optional.
  tolerations: []

  # Affinity settings for the endorctl pod. Optional.
  affinity: {}

  # Volumes for the endorctl pod. Optional.
  volumes: []

  # Volume mounts for the endorctl pod. Optional.
  volumeMounts: []

  # Backoff limit for the endorctl job. Optional.
  backoffLimit: 0

  # TTL seconds after finished for the endorctl job. Optional.
  ttlSecondsAfterFinished: 100

  # Additional environment variables for the endorctl pod. Optional.
  additionalEnvs: []

#
# DEPENDENCIES
#

# Bazel remote cache configuration. Optional. Not enabled by default.
bazelremote:
  # Whether to install the Bazel remote cache component
  install: false

  image:
    # Container repository for the Bazel remote cache image [Do not modify]
    repository: "buchgr/bazel-remote-cache"

    # Specific version of the Bazel remote cache to use [Do not modify]
    tag: "v2.4.1"

    # Image pull policy [Do not modify]
    pullPolicy: "IfNotPresent"

  # Full name of the chart. Optional.
  fullnameOverride: "bazel-remote-cache"

  # Bazel-remote config to provision inside of the container. Optional.
  conf: |-
    # https://github.com/buchgr/bazel-remote#example-configuration-file
    dir: /data
    max_size: 500
    experimental_remote_asset_api: true
    access_log_level: all
    port: 8080
    grpc_port: 9092    

  ## For advanced bazel-remote configuration options,
  ## Refer https://github.com/slamdev/helm-charts/tree/master/charts/bazel-remote#readme

Post-deployment configuration

After you deploy Outpost, you can set up Endor Labs apps depending on your source code manager. You can install Endor Labs apps for the following source code managers:

• GitHub Cloud
• GitLab Cloud and self-hosted
• Bitbucket Data Center
• Bitbucket Cloud
• Azure DevOps Cloud

7 - Set up Jenkins pipeline for supervisory scans

Use the Endor Labs Jenkins pipeline to scan all the repositories in your organization and view consolidated findings. This pipeline runs on your organization’s Jenkins infrastructure and enables administrators to run organization-level supervisory scans easily. It is designed to work in GitHub Cloud and GitHub enterprise server environments.

The Jenkins pipeline carries out the following actions.

• Pulls the Endor Labs Docker image required to perform the scan.
• Synchronizes GitHub organization repositories to a specified namespace on the Endor Labs platform.
• Retrieves the project list or the GitHub repositories for the given tenant’s namespace.
• Groups the projects into batches to optimize scan execution.
• Runs endorctl scans on each batch of projects simultaneously.

Scan the repositories in your organization

The Jenkins Pipeline script is available in the github-org-scan-docker.groovy file.

To scan the repositories in your organization:

1. Generate Endor Labs API credentials
2. Configure GitHub cloud or GitHub enterprise server credentials
3. Configure the Jenkins job

Configure GitHub credentials

Configure the required credentials needed to access GitHub and Endor Labs in the Jenkins pipeline script. You can configure these values from the Jenkins user interface.

• GITHUB_TOKEN- Enter the GitHub token that has permission to access all the repositories in the organization.
• ENDOR_LABS_API_KEY- Enter the Endor Labs API key that you generated.
• ENDOR_LABS_API_SECRET- Enter the Endor Labs API secret generated while creating the Endor Labs API key.

Configure GitHub cloud credentials

Configure the following GitHub cloud parameters in the Jenkins pipeline script.

Required Parameters for GitHub cloud

• AGENT_LABEL- This is a string parameter. Enter the label used to identify the Jenkins agents. The Jenkins job will run on the agents that have this label.
• GITHUB_ORG- This is a string parameter. Enter the organization name in GitHub.
• ENDOR_LABS_NAMESPACE- This is a string parameter. The namespace of your organization tenant in Endor Labs.

Optional Parameters for GitHub cloud

• ENDOR_LABS_API- This is a string parameter. This is only required if the tenant namespace is configured on the Endor Labs staging environment.
• ADDITIONAL_ARGS- This is a string parameter. Use this field to pass any additional parameter to the endorctl scan.
• NO_OF_THREADS- This is a string parameter. Enter the number of Jenkins agents that can be used in parallel for the endorctl scan. If you have 10 Jenkins agents configured with the given AGENT_LABEL, you can enter this value as 9, 1 agent is used for the main job. If not specified, this value defaults to 5.
• ENDORCTL_VERSION- This is a string parameter. Specify the version of the endorctl Docker container. Defaults to the latest version.
• SCAN_TYPE- This is a string parameter. Set this to git to scan commits or github to fetch info from the GitHub API. Defaults to [git, analytics].
• SCAN_SUMMARY_OUTPUT_TYPE- This is a string parameter. Use this field to set the desired output format. Supported formats: json, yaml’, table, summary. Defaults to table.
• LOG_LEVEL- This is a string parameter. Use this field to set the log level of the application. Defaults to info.
• LOG_VERBOSE- This is a string parameter. Use this field to make the log verbose.
• LANGUAGES- This is a string parameter. Use this field to set programming languages to scan. Supported languages: c#,go, java, javascript, php, python, ruby, rust, scala, typescript. Defaults to all supported languages.
• ADDITIONAL_ARGS- This is a string parameter. Use this field to pass any additional parameters to the endorctl scan.

Configure GitHub enterprise server credentials

Configure the following GitHub enterprise server parameters in the Jenkins pipeline script.

Required Parameters for GitHub enterprise server

• AGENT_LABEL - This is a string parameter. Enter the label used to identify the Jenkins agents. The Jenkins job will run on the agents that have this label.
• GITHUB_ORG - This is a string parameter. Enter the organization name in GitHub.
• ENDOR_LABS_NAMESPACE - This is a string parameter. The namespace of your organization tenant in Endor Labs.
• GITHUB_API_URL - This is a string parameter. Enter the API URL of the GitHub enterprise server. This is normally in the form of <FQDN of GitHub Enterprise Server>/api/v3. For example, https://ghe.endorlabs.in/api/v3.

Optional Parameters for GitHub enterprise server

• ENDOR_LABS_API - This is a string parameter. This is only required if the tenant namespace is configured on the Endor Labs staging environment.

• GITHUB_DISABLE_SSL_VERIFY - This is a boolean parameter. This should be used when you want to skip SSL Verification while cloning the repository.

• GITHUB_CA_CERT - This is a multi-line string parameter. This should be used to provide the content of the CA Certificate (PEM format) of the SSL Certificate used on the GitHub Enterprise Server.

• PROJECT_LIST - This is a multi-line string parameter. This should be used to provide a list of projects to scan.

• SCAN_TYPE - This is a string parameter. Set this to git to scan commits or github to fetch info from the GitHub API. Defaults to [git, analytics].

• SCAN_SUMMARY_OUTPUT_TYPE - This is a string parameter. Use this field to set the desired output format. Supported formats: json, yaml*, table, summary. Defaults to table.

• LOG_LEVEL - This is a string parameter. Use this field to set the log level of the application. Defaults to info.

• LOG_VERBOSE - This is a string parameter. Use this field to generate verbose logs.

• LANGUAGES - This is a string parameter. Use this field to set programming languages to scan. Supported languages: c#, go, java, javascript, php, python, ruby, rust, scala, typescript. Defaults to all supported languages.

• ADDITIONAL_ARGS - This is a string parameter. Use this field to pass any additional parameters to the endorctl scan.

• PROJECT_LIST - This is a multi-line string parameter. List of projects to scan. Even though all projects are synchronized, scans run only on the provided projects.

• SCAN_PROJECTS_BY_LAST_COMMIT - This is a string parameter. This parameter is used to filter projects based on the date of the last commit. Enter a number (integer) value for this parameter. The value of 0 means that projects won’t be filtered based on last commit date. Any positive integer is used to calculate the duration in which a commit will add the project for further scanning. If a project did not have a commit in that interval, it will be skipped. If a proper SSL Certificate (a certificate issued by a well-known CA) is not used for GitHub Enterprise, the sync-org command fails and Endor Labs cannot fetch the projects or repositories to scan from the GitHub enterprise server. You can use this field to provide the list of projects or repositories to scan one per line. For example:

          https://github-test.endorlabs.in/pse/vuln_rust_callgraph.git
          https://github-test.endorlabs.in/pse/vulnerable-golang.git
          https://github-test.endorlabs.in/pse/java-javascript-vulnerable-repo.git
          https://github-test.endorlabs.in/pse/multi-lang-repo.git
  

• EXCLUDE_PROJECTS - This is a multi-line string parameter. Use this parameter to list projects or repositories to exclude from the scan.

• NO_OF_THREADS - This is a string parameter. Enter the number of Jenkins agents that can be used in parallel for the endorctl scan. If you have 10 Jenkins agents configured with the given AGENT_LABEL, you can enter this value as 9. If not specified, this value defaults to 5.

Configure the Jenkins job

Use the following procedure to configure the Jenkins pipeline and scan the repositories in your organization.

1. Sign in to Jenkins
2. Configure an Endor Labs API Key and GitHub credentials correctly for your environment.
3. Click + New Item, to create a new Jenkins job.
4. Enter the name of the new pipeline
5. Select Pipeline and click OK.
6. Select This project is parameterised and add the parameters based on your requirements.
7. From the Pipeline section, for Definition, select Pipeline script from SCM
8. For SCM select Git
9. For the Repository URL, enter either git@github.com:endorlabs/jenkins-org-scan.git or https://github.com/endorlabs/jenkins-org-scan.git.
10. For Credentials, enter the credentials required for cloning the repository entered in the previous step.
11. In Branches to build, enter */main.
12. For Script Path, enter github-org-scan-docker.groovy.
13. Select Lightweight checkout.
14. Click Save.

The Jenkins pipeline is highly customizable and adaptable to various GitHub environments and scanning requirements. It streamlines the process of running endorctl scans on your repositories efficiently.