Skip to content
This repository has been archived by the owner on Mar 20, 2024. It is now read-only.

Technical Project Overview

JonZeolla edited this page Jan 16, 2020 · 9 revisions

easy_sast is a project that is designed to fit into the integration pipelines of other projects, streamlining their use of SAST tools. This project was created with the following goals in mind:

  1. provide a simple, standard SAST integration that requires minimal configuration,
  2. provide an enterprise-ready method for integrating with SAST tools that is heavily configurable, and
  3. implement robust security and validation practices in code, within a containerized environment.

In order to accomplish these goals, the project uses a multi-stage docker container and a Makefile as its primary environmental setup components. The docker container pushes most project dependencies into a standardized environment regardless of the building or testing system's configuration, and maintains an extremely similar environment between a developer's local environment and the CI pipeline.

The Makefile simplifies the building and running of the container stages, and supports the general maintenance of the project. The various Docker stages are used to provide linting and testing of the final container itself, and the final container supports a list of pre-defined workflows that cover common developer use cases when integrating with a SAST product. General project maintenance tasks are simplified to commands such as make format which will run black (a Python code formatter) on all of the .py files, or make requirements which will update the appropriate requirements.txt files.

If you see any part of this project which isn't being appropriately tested or validated, that would be a great way to contribute. For further examples of how this is currently accomplished, see Ruggedness below.

Workflows

Currently there are two built-in workflows: check_compliance and submit_artifacts.

  • check_compliance will query the SAST product and identify whether or not the provided app_id is considered 'in compliance' with the policy of the SAST vendor. easy_sast makes no judgements as to what compliance means, and defers to the configuration of the application within the SAST vendor.
  • submit_artifacts creates a new scan within the SAST product and submits all of the provided artifacts (see the build_dir configuration option) appropriately.

Runtime

The entrypoint for the easy_sast container is the top level main.py, which is a Python script that uses SAST module(s), such as veracode/. Any arguments provided after your docker run command are passed directly to main.py, which is then applied to the configuration appropriately.

In order to override the default entrypoint, you can use --entrypoint during your docker run statement to run anything that exists within the image.

Configuring your development environment

This project intentionally requires very little of the host in order to develop it by using Docker. You should be able to install the prereqs as outlined in the README and be ready to do any testing or development on the project.

However, there is at least one additional configuration to consider. Running make init after cloning the repository will set up pre-commit, commit-msg, and post-rewrite local git hooks for validation in line with the current easy_sast ci pipeline.

Python

The veracode/ folder is a standalone module, as we expect in the future there may be numerous vendors which this project can integrate with. All of the associated tests for the veracode/ folder are currently located in tests/ and are prefaced by test_.

In order to install the dependencies of the project at the time it was committed (which, in the typical use case, is entirely done within a Docker container to reduce host dependencies), you will need to pip3 install -r requirements.txt -r veracode/requirements.txt. These are generated files using make requirements, and are purposefully separated so that the top level requirements.txt is for testing and linting, whereas veracode/requirements.txt contains only the requirements needed to use the veracode/ module. When building the docker container, the dependency installations are handled via the builder and ci stages specified in the Dockerfile.

On each PR, we also ask that updated reports are included, by running make report. This is important not only to keep the reports up-to-date, but also to be able to look back in time at what the reports were able to tell us about the project at that point in time.

Below is a quick tour of veracode/:

  • __init__.py has some global properties of the project, most importantly the version.
  • api.py supports the API integration and related object validation.
  • check_compliance.py provides the check_compliance workflow.
  • config.py provides methods to get and apply the various information used to create the effective config.
  • constants.py provides constants for use during runtime.
  • submit_artifacts.py provides the submit_artifacts workflow.

Ruggedness

This project takes ruggedness very seriously. For an example, you can look at veracode/api.py, which calls the is_valid_attribute function via the _validate private method for each and every get or set of its properties to ensure the values are properly validated during use and change. Also, a custom @validate decorator was created to provide a similar function against arguments to any function, ensuring that objects and other data being passed between functions is valid. In the typical use cases, this causes the same information to be validated multiple times. This is intentional because (1) the overall performance impact is negligible and (2) we are attempting to anticipate customized use of functions which may not follow the default project flow.