Understanding the Functionality of the AWS Export App

The AWS Export App functions as a means of exporting specified file types of your Portal content to an AWS S3 Bucket, without the content being affected. It acts as a one-way bridge that you can modify or control via its configuration.  

The AWS Export App can be activated and configured from your Portal's application settings. During configuration, you enter your AWS credentials, assign an S3 Bucket to contain the exported files, define the export criteria and preferences if you are opting for automatic export, and lastly select the content files you want to export.

Automatic Export 

A crucial setting to consider during configuration is the Automatic Export option. If this option is on and the AWS Export App is enabled, then the content on your Portal that meets the export criteria will undergo the export activity immediately. The automatic export activity takes place on the eligible content once, the content does not undergo the activity again unless the criteria is changed, or the application is re-enabled. The AWS S3 Export app does this by initializing a continuous workflow.  

The workflow continuously monitors all content on your Portal, including content added or created through processing, ingestion, upload, clipping, and more. If the content meets the export criteria, then its content files are automatically exported to the destination S3 Bucket. However, you have the option to make the automatic export activity exclusive to new Portal content, see Export Preference below for more. 

Inside the AWS Export App’s settings, you will also find the option to set an Export Preference for the content present on your Portal when automatic export is turned on. This setting determines which content will be eligible for automatic export depending on when the Export App was enabled or disabled. 

Two options are available to control this setting. You can enable the application with the Export Preference set to "Already Present." By doing so, the application will automatically export the content files of all content present on the Portal according to the defined export criteria. This also applies to past Portal content, and export-eligible content added to the Portal after the Export App has been enabled. 

Alternatively, you can choose "Export After Enable" to restrict the automatic export activity only to content that is added to your Portal after the Export app is enabled from the Integration App page. 

        2. Post Export Options   

Users can determine what happens to their Portal content after it goes through an automatic export activity. The application can be configured to either delete the content and move it to the Portal's Recycle Bin or keep it as it is by performing no action.  

Aside from automatic export, users can export the files of their Portal content manually, which is also referred to as "on-demand" Export. The options to perform an on-demand export are available regardless of whether automatic export is enabled or disabled. You can also manually export content regardless of the Export Preference selected in the application settings.   

The user can perform a manual export on Content present in their Portal's Evidence Library or Media Manager. However, the export app only exports the content files selected in its configuration. 

Manual Export also allows you to "update" the content files on your S3 Bucket. For instance, if new files were generated when an already exported content was processed, then performing an on-demand export on its content will export those newly generated files to the content's named folder in the S3 Bucket.

When an export activity begins (either manually or automatically), the Export App generates requests according to the Portal content selected by the user or the defined export criteria. For each request, the Export App adds a record to your Portal's Activity Logs.  

A log or record provides details, such as the status of the export activity. Initially, the status of the activity is "Pending," which means that the content is in queue and being checked to see if it meets the export criteria. If the criteria is met, then the export activity proceeds with the content files being exported to the destination S3 Bucket and the tag being updated to "Processing." However, if the criteria is not met, the export request is rejected, and the log is marked as "Rejected."  

After the export activity is finished, the status tag is updated to "Complete," "Failed," or "Content Not Found," depending on whether any issues were encountered.  

You can view the progress of your export activity from your Portal's Activity Log. The log also consists of all instances of export activities that took place on your Portal. It provides detailed information for each instance of export activities, including the number of exported files, errors encountered, who initiated the export activity, and more. You can learn more here: Understanding App Activity Logs.

1. It creates batches of all the export requests it receives for a specified duration. The size of a single batch can be up to 100 requests.
2. Scans a batch and checks if the selected Portal content meets the export criteria.
3. It approves the eligible ones for processing and rejects the rest.
4. The application then divides the approved processing requests into chunks, which are then processed in parallel.
5. When all of the requests in the batch are processed, the Export App then moves on to process subsequent batches.
6. After processing all the batches, the application waits for a specific duration (5 minutes) before starting the next export cycle.

If you're exporting content that was ingested from an S3 bucket with its folder hierarchy preserved, then the application will recreate the same folder hierarchy on the destination S3 bucket.    

If the folder hierarchy contains content that's eligible for export, then the application will export its files to the destination S3 bucket. The exported files will be present on the same path (or location) as the original content in the exported folder hierarchy.    

If the content's files (namely supporting files) are present in another folder within the hierarchy on the source S3 bucket, then the application preserves its location during the export activity.    

With the entire folder hierarchy maintained on the destination S3 bucket, the supporting files are present in the same folder, path, or directory as the source. However, if any new files are created when the export-eligible content is processed, they will be saved in the same folder that contains the original content.    

Figure 1 illustrates how content present in a folder hierarchy on a source S3 bucket was ingested to a VIDIZMO Portal and then exported to a Destination S3 Bucket. 

When a user exports a VIDIZMO Folder containing Portal content, the Export App creates a folder on the destination S3 Bucket for each eligible content it finds inside the folder. The exported content will be organized into folders with the same name as the corresponding Portal content, making it easy for users to identify which folder contains which files. It then exports the specified file types inside of those folders according to the Portal Content. 

If the same content is processed for an activity that creates new files and then exports them, then the named folder on the destination S3 bucket will be updated with the latest files.  

Figure 2 shows how the content files are exported to the Destination S3 bucket when the export-eligible content is present inside a folder hierarchy in VIDIZMO. 

If the user decides to re-configure the application and selects “Case” as a value in the Formats field instead of “Folder”, then the application runs the export activity on all eligible content present in Cases across the Portal, even if they are present inside a Folder or Folder hierarchy.  

To run the export activity on eligible content present in both Folders and Cases, the user needs to add both “Case” and “Folder” as values for the Formats field.