We have a lengthy security detection flow need to pass before each time we have new release to production, in the part of this flow, there are Black Duck and Fortify On Demand scans.

In this article, I would like to record the research of how to use the APIs of both scans to achieve our automation solution.

This research was conducted for this automation solution, if you have interest you can visit it, these 2 articles don’t have dependency, so you can only read this article.

Notice, we don’t record all API endpoints here, only write the APIs which achieved our solutions, if you want to research more, please go to official Black Duck and Fortify On Demand document.

When I was researching how to use both scans APIs, I found there were less reference and articles, so I hope this article can help you.

Table of Content

Here is like the table of content.

Fortify On Demand (FoD)

Fortify On Demand (FoD), it has the public official Swagger document, offers many APIs and the convenient feature is using swagger to test the API endpoint, so if you get many Errors from response, you can try swagger UI first.

Here is the API Endpoints I want to introduce:

• Oauth Authentication: https://api.ams.fortify.com/oauth/token
• Get Application: https://api.ams.fortify.com/api/v3/application
• Get Release (with filter): https://api.ams.fortify.com/api/v3/release or https://api.ams.fortify.com/api/v3/application/<application_id>/release

Oauth Authentication

Before request other FoD APIs, you need to request this Oauth API for getting authorized token, then request other API with this returned token.

// POST https://api.ams.fortify.com/oauth/token
// Header
{
    "scope": "api-tenant", // visit setting to check scope
    "grant_type": "password", 
    "username": "<tenant>\\<user_name>", // check tenant and user name in settings and make sure \ and \\. 
    // Correct: <tenant>\<user_name>.
    "password": "<personal_token>" // This is not fill your password, this fill the token you generate in FoD settings.
}

response

{
    "access_token": "ZGHJ3274JDK" // You will get authorized token, this is only example.
}

Get Application

After you got authorized token, we can request other APIs with this token. First, get all application.

// GET https://api.ams.fortify.com/api/v3/application
// Header
{
    "Content-Type": "application/json", 
    "Accept": "application/json", 
    "Authorization": "Bearer <token>", // With header, attach the token you got from Oauth response.
}

response

{
    ...,
    "items": [{
        "applicationName": "",
        "applicationId": "",
        ...
    }, {
        ...
    }
    ]
}

Through this API, you will get many Application, you can know the applicationName, applicationId and related information.

This API support filter, you can find the application you want, I will introduce it in next API, the usage of filter is similiar cross APIs.

Get Release (with filter)

You can pick to get release or release belongs with specific application, the second API, you need to bring applicationId.

// GET https://api.ams.fortify.com/api/v3/release or https://api.ams.fortify.com/api/v3/application/<application_id>/release
// Header
{
    "Content-Type": "application/json", 
    "Accept": "application/json", 
    "Authorization": "Bearer <token>", // With header, attach the token you got from Oauth response.
}

// Params -> Notice: this is Parameters, not Body, here is GET method, you need to put it in parameters. In Python, please put requests.get(params=<params>). In postman, please put Params.
{
    "filter": "ReleaseName:<service_name>" // Here is single filter example, if you have multiple condition, please use + connect with them. <condition1>:<value1>+<condition2>:<value2>
}

The format of filter is <filter field1>:<filter value1>+<filter field2>:<filter value2>, so if I want to filter ReleaseName is ppt-scanner then I put

“filter”: “ReleaseName:ppt-scanner”

The official document doesn’t record detailed possible filter fields, you can observe the properties in API response and try filter with properties.

Black Duck (BD)

Black Duck scan (BD), the official document is private, you can go the the ? -> REST API Developer Guide in the Black Duck service right top side panel, the document is very clear.

In this article, I will introduce the following API endpoint, please instead host_name to your server host name.

• Authentication: https://<host_name>/api/tokens/authenticate
• Query project: https://<host_name>/api/projects
• Get version: https://<host_name>/api/projects/<project_id>/versions
• Generate Version Details Report: https://<host_name>/api/versions/<version_id>/reports
• Download Version Details Report: https://<host_name>/api/projects/<project_id>/versions/<version_id>/reports/<report_id>/download
• Visit components with filter: https://<host_name>/api/projects/<project_id>/versions/<version_id>/components

Authentication

Similiar with above FoD scan, you need to request Authentication endpoint to get authorized Bearer token and use it for requesting other APIs.

// POST https://<host_name>/api/tokens/authenticate
// Header
{
    "Accept": "application/vnd.blackducksoftware.user-4+json", 
    "Authorization": "token <personal_token>", // You can get the personal_token in Black Duck website -> the setting -> Access token -> generate personal token, if you need to generate report, you need to open the write permission.
    // Authorization value, please put "token" before your personal token, I stuck here much time. example: "Authorization": "token GH2395JKls=="
}

response

{
    "bearerToken": "ZGHJ3274JDK" // After requested, you get find the returned bearerToken if status 200.
}

Query project

After we got the bearerToken, we can request other APIs. first. we want to get projects with filter (or just get all projects)

// GET https://<host_name>/api/projects
// Header
{
    "Authorization": "Bearer <bearerToken>", // Put the authorized bearerToken here.
}
// Params -> Yeah, its Parameters, this is GET method, so don;t put in body, doesn't work.
// If you need limit or filter with project, then you need this parameters.
{
    "limit": 5, // Only get 5 records
    "q": "name:<filter_name>" // Yeah, filter key just a q, and put <filter field>:<filter value>, not sure filter with other properties, you can check properties in response.
}

response

{
    ...,
    "items": [{
        "name": "",
        "_meta": {
            "href": ""
        },
        ...
    }, {
        ...
    }
    ]
}

And I found, the project information in response doesn’t include project_id, so I found project_id by split the url in item[_meta][href], then split("/")[-1] split the href to get the last segment, it is project_id.

Get version

After you got the project_id, now we would like to get all versions under this project.

// GET https://<host_name>/api/projects/<project_id>/versions
// Header
{
    "Authorization": "Bearer <bearerToken>", // Put authorized BearerToken
}

Remember to put the project_id in request url.

response

{
    "items": [{
        "versionName": "master",
        ...,
        "_meta": {
            "href": ""
        },
    },
    {
        "versionName": "version_1",
        ...,
        "_meta": {
            "href": ""
        },
    }
    ]
}

Yeah, it doesn’t return version_id, so we need to split by item[_meta][href] with split("/")[-1] to get version_id

The _meta.href will like https://<host_name>/api/projects/<project_id>/versions/<version_id>, so split by “/” then get the last segment then you will get version_id.

Generate Version Details Report

So now we have project_id and version_id, we can generate Version Detail report based on it.

If you want to generate other type reports, please refer the official website.

// POST https://<host_name>/api/versions/<version_id>/reports
// Yes, we don't need to put project_id, its little weird.
// Header
{
    "Content-Type": "application/json",
    "Authorization": "Bearer <bearerToken>", // Put the authorized bearerToken
}

// Body -> this is Body, because of the POST method.
{
    "reportFormat": "CSV", // here we put CSV, but there is JSON format can choose.
    "reportType": "VERSION",
    "caregories": ["VERSION", "COMPONENTS", "SECURITY", "PROJECT_VERSION_CUSTOM_FIELDS"] // You can check what categories you need when you generate reports in website then fill in here.
}

If you are not sure the part of categories, you can generate a report in website and open the Developer Tool -> Network then click generate, the developer tool will show the request and response, you can check the value of Header and Body in it.

response

{
    "links": {
        "download": {
            "url"
        }
    }
}

You can through links -> download -> url to get report_id.

And again, using split to get the last segment of url (split("/")[-1]), that is the report_id

Maybe there are other ways to get project_id, version_id and report_id, but I didn’t get it.

Download Version Details Report

After generated report, it doesn’t finish immediately, so if you write program to do automation, you can write while loop and pending 7-10 seconds. (It will finish at 30 seconds)

If the report haven’t done, it will return HTTP status 412, but if raise other errors, it will also return 412, so need to check message in response.

Here is the request for downloading the report.

// GET https://<host_name>/api/projects/<project_id>/versions/<version_id>/reports/<report_id>/download
// Weird again, we don't need project_id to generate report, but need it when download report.
// Header
{
    "Authorization": "Bearer <bearerToken>", // Put authorized bearerToken.
}

The response content is bytes, here is the python code to download it, in python 3.9.

# If download at local computer
import io
import zipfile

response = requests.get(url, headers=headers)
if response.status_code == 200:
    z = zipfile.ZipFile(io.BytesIO(response.content))
    z.extractall("./")

# If you want to save into AWS S3 bucket. 
import io
import boto3
import zipfile

s3_client = boto3.client("s3")

s3_bucket = "<bucket_name>"

response = requests.get(url, headers=headers)
if response.status_code == 200:
    z = zipfile.ZipFile(io.BytesIO(response.content))
    for file_info in z.infolist():
        file_name = file_info.filename

        # Read file content
        file_content = z.read(file_name)

        save_key = f"<prefix>/{file_name}"

        # Upload content to S3
        s3_client.upload_fileobj(io.BytesIO(file_content), s3_bucket, save_key)

Visit components with filter

The last one, I want to introduce visit components with specific project_id and version_id.

In this API, we can add many filters, but I only add one.

// GET https://<host_name>/api/projects/<project_id>/versions/<version_id>/components
// Fill the project_id and version_id

// Header
{
    "Authorization": "Bearer <bearerToken>", // Put bearerToken
}

// Params -> Parameters, because GET method, not Body.
{
    "limit": 10, // limit 10 or the number you can set.
    "filter": "bomPolicy:in_violation" // filter -> field_key: field_option
}

Here, I tried to get any In violation components.

response

// If there is not match in filter, it will return [] in items.
{
    "items": [{
        "componentName": "",
        "componentVersionName": ""
    },
    {
        "componentName": "",
        "componentVersionName": ""
    }
    ]
}

So, you can put filters in Params to filter components.

If you are not sure how to write filter content, you can visit the Black Duck website, enter the components page -> open Develoepr Tool -> Network, then you add any filter, you can see the Network catch the request and response, you can read the Parameters in Network.

Or get infromation in the url https://<host_name>/api/projects/<project_id>/versions/<version_id>/components?filter=bomPolicy:in_violation, the filters information is after ?filter= words and you can try in Params with Python or Postman.

Alright, these are the research and introduction of Fortify On Demand and Black Duck APIs.

Hope this help you and thanks for reading.