Skip to main content
Skip table of contents

Cleaning up execution data

As part of the Hyperscale execution run some data files and objects are created which should be cleaned on the execution completion. These files and objects are:

  1. The system will create data files (unload service) and masked files (masking service) on the file server. As the data size can be large (2 times of source data) and include sensitive information, therefore, it is important to clean up this data.

  2. The system will create multiple data objects like connectors, rulesets, file formats, jobs, etc on the respective Continuous Compliance Engines. Objects created by Hyperscale should be cleaned once Hyperscale execution is complete.

  3. Additionally unload service, masking service, and load service will also store transient internal data for the execution while running it. This data is not required once execution is completed.

Following are the three ways this data will be/can be cleaned.

  1. Using retain_execution_data
    While setting up a Hyperscale Job (POST /jobs), you can set the value for retain_execution_dataproperty to the intimate system when it should clean up data automatically based on the table below.

EXECUTION_STATUS

RETAIN_EXECUTION_DATA

CLEAN UP AUTOMATICALLY?

NA(SUCCESS/FAILED/CANCELED)

NO

YES

SUCCESS

ON_ERROR

YES

FAILED

ON_ERROR

NO

CANCELED

ON_ERROR

NO

NA(SUCCESS/FAILED/CANCELED)

ALWAYS

NO

2. Manual clean up
Hyperscale exposes a delete API (DELETE /executions/{id}) to manually clean up data for execution if it's not already cleaned.

3. Start a new execution
While starting a new execution, Hyperscale will first validate if the previous execution data is cleaned. If it’s not cleaned, then Hyperscale will trigger cleanup before starting new execution.

Limitations

As part of the clean-up process, applications created on the Continous Compliance Engines by Hyperscale job execution are not deleted. You must use the Continuous Compliance Engine APIs to delete the application.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.