Listener API Clients

Introduction

Orangebeard provides listener api clients for Java, Javascript/Typescript, .Net and Python to easily connect test tools and frameworks to Orangebeard's listener API. Our clients are open source so you can adapt them to your own specific needs.

This document describes the clients built for Listener API V3.

Source code

JavaScript Client
Java Client
.Net Client
Python Client

The clients allow to easily connect your javascript based test framework to the Orangebeard listener API. We provide integrations for multiple test tools, but if you need to roll your own, the clients are a convenient base to start from.

Installation

Package managers

See the client repositories readme files for detailed instructions. The clients can be installed from public package management repositories.

Java

Maven:

<dependency>
    <groupId>io.orangebeard</groupId>
    <artifactId>java-client</artifactId>
    <version>3.0.0</version>
</dependency>

Gradle:

implementation 'io.orangebeard:java-client:3.0.0'

JavaScript

npm install --save-dev @orangebeard-io/javascript-client

C#

dotnet add package Orangebeard.Client

Python

pip install orangebeard-client

Or build your client from source.

Configuration

Usually, the client is used from a listener that connects the client to a test tool. Listeners can provide their own OrangebeardParameters object.
If no configuration object is provided, the client will try to auto-configure in the following ways:

Auto-config

The automatic configuration looks for a config file named orangebeard.json in the current working directory or higher in the hierarchy (it will scan all the way up to /). The json file is expected to contain an OrangebeardParameters object:

Java

{
    endpoint: "https://my.orangebeard.app",
    token: "xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx",
    project: "my-project",
    testset: "Test set name",
    description: "Test run description",
    attributes: [
        {
            key: "Key 1",
            value: "Some value"
        },
        {
            value: "Tag value"
        }
    ],
    ref_url: "https://my-build-reference.url"
}

tip

The Java Client supports automatic configuration through system properties. These can be set from a orangebeard.properties in the resource directory. In that case the hierarchy is that automatic configuration will read (and overwrite if applicable) properties in the following order:

1. Orangebeard.properties
2. System properties
3. Orangebeard.json
4. Environment variables

JavaScript

{
    endpoint: "https://my.orangebeard.app",
    token: "xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx",
    project: "my-project",
    testset: "Test set name",
    description: "Test run description",
    attributes: [
        {
            key: "Key 1",
            value: "Some value"
        },
        {
            value: "Tag value"
        }
    ],
    ref_url: "https://my-build-reference.url"
}

C#

 {
    endpoint: "https://my.orangebeard.app",
    accessToken: "xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx",
    project: "my-project",
    testset: "Test set name",
    description: "Test run description",
    attributes: [
        {
            key: "Key 1",
            value: "Some value"
        },
        {
            value: "Tag value"
        }
    ],
    ref.url: "https://my-build-reference.url"
}

Python

{
    endpoint: "https://my.orangebeard.app",
    token: "xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx",
    project: "my-project",
    testset: "Test set name",
    description: "Test run description",
    attributes: [
        {
            key: "Key 1",
            value: "Some value"
        },
        {
            value: "Tag value"
        }
    ],
    ref_url: "https://my-build-reference.url"
}

Environment variables

The auto config will scan for environment variables:

ORANGEBEARD_ENDPOINT=https://my.orangebeard.app
ORANGEBEARD_TOKEN=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
ORANGEBEARD_PROJECT=my-project
ORANGEBEARD_TESTSET=test set name
ORANGEBEARD_DESCRIPTION=Test run description
ORANGEBEARD_ATTRIBUTES=key:value; value;
ORANGEBEARD_REF_URL=https://my-build-reference.url

tip

If automatic client configuration is used and a value is present in both orangebeard.json and in the environment settings, the environment setting will take precedence.

The environment variables can be both written as orangebeard_varname and orangebeard.varname.

Usage

The client can be used by constructing an instance in any of the supported languages:

Java

//Using the client's auto config:
OrangebeardAsyncV3Client client = new OrangebeardAsyncV3Client();

//Or pass a configuration object
OrangebeardAsyncV3Client client = new OrangebeardAsyncV3Client(
    new OrangebeardProperties (
        orangebeardEndpoint,
        accessToken,
        projectName,
        testSetName,
        description, //OPTIONAL test run description
        attributes, //OPTIONAL test run attributes - Set<Attribute>
        logLevel, //OPTIONAL LogLevel (and up)  to report 
        false //OPTIONAL if enable: send test logs at the end of a test
        );
);

JavaScript

//Using the client's auto config:
this.client = new OrangebeardAsyncV3Client();

//or pass a configuration object:
this.client = new OrangebeardAsyncV3Client(
    {
        endpoint: eorangebeardEndpoint,
        token: accessToken,
        project: projectName,
        testset: testSetName,
        description: description,
        attributes: attributes, // Attribute[]
        ref_url: referenceUrl
    });

C#

// Using the client's auto config:
_client = new OrangebeardAsyncV3Client(new OrangebeardConfiguration());

// Or pass a configuration object:
var configProperties = new Dictionary<string, object>();

configProperties.Add("orangebeard.endpoint", endpoint);
configProperties.Add("orangebeard.project", projectName);
configProperties.Add("orangebeard.accessToken", accessToken);
configProperties.Add("orangebeard.testset", testSetName);
configProperties.Add("orangebeard.description", description);
configProperties.Add("orangebeard.ref.url", referenceUrl);

_client = new OrangebeardAsyncV3Client(
    new OrangebeardConfiguration(new Configuration(configProperties))
);

// Or by passing the minimal configuration:
_client = new OrangebeardAsyncV3Client(
    endpoint, 
    accessToken,
    projectName,
    true
);

Python

# Using the client's auto config:
self.client = OrangebeardClient()

# Or pass a configuration object:
self.client = OrangebeardClient(
    OrangebeardParameters(
        token=access_token,
        project=project_name,
        testset=test_set_name,
        description=description,
        attributes=attributes, # list[Attribute]
        ref_url=reference_url
    )
)

Client API

The client exposes all public methods required to report a test run to Orangebeard.

Starting a test run

Starts a test run in Orangebeard and returns the UUID (Guid) of the test run that was started.

Java

StartV3TestRun startTestRun = new StartV3TestRun(
    testSetName,
    testRunDescription,
    testRunAttributes
);

UUID testRunUUID = orangebeardClient.startTestRun(startTestRun);

JavaScript

const startTestRun = {
    testSetName, //String
    description, // String
    startTime, // String Formatted as a ZonedDateTime.now().withFixedOffsetZone()
    attributes // Array<Attribute>
}

const testRunUuid = this.client.startTestRun(startTestRun);

C#

var startTestRun = new StartTestRun()
    {
        StartTime = DateTime.UtcNow,
        TestSetName = testSetName, // string
        Description = description ?? "", //string
        Attributes = testRunAttributes, // ISet<Attribute>
    }

var testRunGuid = _client.StartTestRun(startTestRun);                

Python

from pytz import reference
    
tz = reference.LocalTimezone()

start_testrun = StartTestRun(
    test_set_name, # string
    datetime.now(tz),
    test_run_description, # string
    attributes # List[Attribute]
)

test_run_uuid = client.start_test_run(start_testrun)

Starting an announced test run

Marks a test run that was previously announced in Orangebeard as started (Sets status to IN_PROGRESS).

Java

orangebeardClient.startAnnouncedTestRun(testRunUUID);

JavaScript

this.client.startAnnouncedTestRun(testRunUUID);

C#

_client.startAnnouncedTestRun(testRunGuid);

Python

client.start_announced_test_run(test_run_uuid)

Finishing a test run

Waits for all reporting calls to complete and marks the test run finished in Orangebeard.

Java

orangebeardClient.finishTestRun(testRunUUID, new FinishV3TestRun());

JavaScript

const finishTestRun = {
    endTime // String formatted as a ZonedDateTime.now().withFixedOffsetZone()
}

this.client.finishTestRun(testRunUuid, finishTestRun);

C#

_client.FinishTestRun(testRunGuid, new FinishTestRun());

Python

client.finish_test_run(test_run_uuid, FinishTestRun(datetime.now(tz)))

Starting a suite

Starts (a structure of) suite(S) in the test run and returns a list of UUID's for the suites that were reported.
At least a top level suite is mandatory.

Java

StartSuite startSuite = new StartSuite(
    testRunUUID,
    parentSuiteUUID, // if applicable
    description,
    attributes, // Set<Attribute>
    suiteNames // List<String>
);

List<UUID> suiteUUIDs = orangebeardClient.startSuite(startSuite);

JavaScript

const startSuite = {
    testRunUUID,
    parentSuiteUUID, // if applicable
    description,
    attributes, // Array<Attribute>
    suiteNames // Array<string>;
};

const suiteUUIDs = this.client.startSuite(startSuite);

C#

var startSuite = new StartSuite
{
    TestRunUUID = testRunGuid,
    ParentSuiteUUID, // if applicable
    Description = suiteDescription,
    Attributes = attributes, // HashSet<Attribute>
    SuiteNames = suiteNames,
};

var startedSuiteGuids = _orangebeard.StartSuite(startSuite);

Python

start_suite = StartSuite(
    test_run_uuid, 
    suite_names, # list[str]
    parent_suite_uuid, # if applicable
    description,
    attributes # list[Attribute]
)

suite_uuids = client.start_suite(start_suite)

note

As opposed to test runs, tests or test steps, suites are structure elements. Much like folders in a file system. They are used to group logical sets of tests together and thus have no start- or end time, nor a status. This information is derived from the underlying elements (tests).
Because of this, there is no need to explicitly finish a suite and the API provides no method to do so.

Starting a test

Start a test (inside a test suite). Returns the UUID (Guid) of the started test.

Java

StartTest startTest = new StartTest(
    testRunUUID,
    suiteUUID, 
    testName,
    testType, // TestType
    description,
    attributes, // Set<Attribute>
    startTime // i.e. ZonedDateTime.now()
);

UUID testUUID = orangebeardClient.startTest(startTest);

JavaScript

const startTest = {
    testRunUUID,
    suiteUUID,
    testName,
    testType, // TestType
    description,
    attributes, // Array<Attribute>
    startTime // String formatted as a ZonedDateTime.now().withFixedOffsetZone()
};

const testUUID = this.client.startTest(startTest);

C#

var startTest = new StartTest
{
    TestRunUUID = testRunGuid,
    SuiteUUID = suiteGuid,
    TestName = testName,
    TestType = testType, //TestType
    Description = description,
    Attributes = attributes, // HashSet<Attribute>
    StartTime = startTime, // DateTime
};

var testGuid = _orangebeard.StartTest(startTest);

Python

start_test = StartTest(
    test_run_uuid, 
    suite_uuids[len(suite_uuids)-1], 
    test_name, 
    datetime.now(tz), 
    test_type, # TestType
    test_description,
    test_attributes # list[Attribute]
)

test_uuid = client.start_test(start_test)

tip

TestType is an enumeration with the following values:

TEST, BEFORE, AFTER

Where before and after are used to report set-up or tear down routines.

Finishing a test

Finish a test and reports its final status.

Java

FinishTest finishTest = new FinishTest(
    testRunUUID,
    testStatus, // TestStatus
    endTime
);

orangebeardClient.finishTest(testUUID, finishTest);

JavaScript

const finishTest = {
    testRunUUID,
    status, // FinishTest.Status
    endTime // String formatted as a ZonedDateTime.now().withFixedOffsetZone()
};

this.client.finishTest(testUUID, finishTest);

C#

var finishTest = new FinishTest
{
   TestRunUUID = testRunGuid,
   Status = TestStatus, // TestStatus
   EndTime = DateTime.UtcNow,
});

_client.FinishTest(testGuid, finishTest);    

Python

finish_test = FinishTest(
    test_run_uuid, 
    test_status, # TestStatus
    datetime.now(tz)
)

client.finish_test(test_uuid, finish_test)

tip

The test's status is an enumeration with the following values:

PASSED, FAILED, SKIPPED, STOPPED, TIMED_OUT

Starting a step

Start a step (inside a test). Returns the UUID (Guid) of the started step.

Java

StartStep startStep = new StartStep(
    testRunUUID,
    testUUID,
    parentStepUUID, // if applicable
    stepName,
    description,
    startTime // i.e. ZonedDateTime.now()
);

UUID stepUUID = orangebeardClient.startStep(startStep);

JavaScript

const startStep = {
    testRunUUID,
    testUUID,
    parentStepUUID, // if applicable
    stepName,
    description,
    startTime // String formatted as a ZonedDateTime.now().withFixedOffsetZone()
};

const stepUUID = this.client.startStep(startStep);

C#

var startStep = new StartStep
{
    TestRunUUID = testRunGuid,
    TestUUID = testGuid,
    ParentStepUUID = parentStepGuid, // if applicable
    StepName = stepName,
    Description = description,
    StartTime = startTime, // DateTime
};

var stepGuid = _orangebeard.StartStep(startstep);

Python

start_step = StartStep(
    test_run_uuid, 
    test_uuid,
    parent_step_uuid,
    step_name, 
    step_description
    datetime.now(tz)
)

step_uuid = client.start_step(start_step)

Finishing a step

Finish a step and reports its final status.

Java

FinishStep finishStep = new FinishStep(
    testRunUUID,
    testStatus, // TestStatus
    endTime
);

orangebeardClient.finishStep(stepUUID, finishStep);

JavaScript

const finishStep = {
    testRunUUID,
    status, // FinishStep.Status
    endTime // String formatted as a ZonedDateTime.now().withFixedOffsetZone()
};

this.client.finishStep(stepUUID, finishStep);

C#

var finishStep = new FinishStep
{
   TestRunUUID = testRunGuid,
   Status = stepStatus, // TestStatus
   EndTime = DateTime.UtcNow,
});

_client.FinishStep(stepGuid, finishStep);    

Python

finish_step = FinishStep(
    test_run_uuid, 
    test_status, # TestStatus
    datetime.now(tz)
)

client.finish_step(step_uuid, finish_step)

tip

The step's status is an enumeration with the following values:

PASSED, FAILED, SKIPPED, STOPPED, TIMED_OUT

The status enum is identical to the test status enum.

Sending a log entry

Reports a log entry to a step or test and return the UUID of the log entry that was created.

Java

Log logEntry = new Log(
    testRunUUID,
    testUUID,
    stepUUID, // if applicable
    message, // String
    logLevel, // LogLevel
    logTime,
    logFormat // LogFormat
);

UUID logUUID = orangebeardClient.log(logEntry);

JavaScript

const logEntry = {
    testRunUUID,
    testUUID,
    stepUUID, // if applicable
    logTime,
    message, // str
    logLevel, // LogLevel
    logFormat //LogFormat
};

const logUUID = this.client.log(logEntry);

C#

var logEntry = new Log
{
    TestRunUUID = testRunGuid,
    TestUUID = testGuid,
    StepUUID = stepGuid,
    LogTime = DateTime.UtcNow,
    LogLevel = logLevel, // LogLevel
    Message = message,
    LogFormat = logFormat // LogFormat
};

var logGuid = _orangebeard.Log(logEntry);

Python

log_entry = Log(
    test_run_uuid, 
    test_uuid, 
    log_message, 
    log_level, # LogLevel
    log_format, # LogFormat
    step_uuid, 
    datetime.now(tz)
)

log_uuid = client.log(log_entry)

tip

LogLevel is an enumeration with the following values:

DEBUG, INFO, WARN, ERROR

LogFormat is an enumeration with the following values:

PLAIN_TEXT, MARKDOWN, HTML

Attaching a file to a log

Java

AttachmentFile file = new AttachmentFile(
    fileName,
    fileContent, // byte[]
    contentType // String
);

AttachmentMetaData metaData = new AttachmentMetaData(
    testRunUUID,
    testUUID,
    stepUUID, // if applicable
    logUUID,
    attachmentTime
);

Attachment attachment = new Attachment{
    file,
    metaData
);

UUID attachmentUUID = orangebeardClient.sendAttachment(attachment);

JavaScript

const file = {
    name,
    content, // Float32Array;
    contentType // str
};

const metaData = {
    testRunUUID: UUID;
    testUUID: UUID;
    logUUID: UUID;
    stepUUID? : UUID;
    attachmentTime: string;
};

const attachment = {
    file,
    metaData
};

const attachmentUUID = this.client.sendAttachment(attachment);

C#

var file = new AttachmentFile
{
    Name = attachmentFileName,
    Content = attachmentData, // byte[]
    ContentType = mimeType, // string
};

var metaData = new AttachmentMetaData
{
    TestRunUUID = testRunGuid,
    TestUUID = testGuid,
    StepUUID = stepGuid, // if applicable
    LogUUID = logGuid,
    AttachmentTime = DateTime.UtcNow,
}

var attachment = new Attachment
{
    File = file,
    MetaData = metaData,
};

_ = _orangebeard.SendAttachment(attachment);

Python

attachment_file = AttachmentFile(
    file_name, 
    open(file_path, 'rb').read()
)

attachment_meta = AttachmentMetaData(
    test_run_uuid, 
    test_uuid, 
    log_uuid,
    step_uuid # if applicable
)

attachment = Attachment(
    attachment_file, 
    attachment_meta
)

client.send_attachment(attachment)

The Python client will determine the content-type using mimetypes.guess_type and return application/octet-stream if that fails.