Remote Cloud Storage

A Remote Cloud Storage is a type of Endpoint that represents a cloud provider and associated object storage service used by your organization or a partner. It can be used as either a Source Endpoint or Destination Endpoint in Pipeline configurations.

The Endpoint stores the Rclone remote configuration for the selected cloud provider and object storage service, and is used to initiate cloud transfers. It defines the connection parameters and credential variables that UDMG requires to authenticate and connect. Because Remote Cloud Storage Endpoints use a flexible, Rclone-based configuration model, UDMG can interact with multiple cloud providers through a unified interface.

Before You Begin

Supported Providers

UDMG supports integration with multiple cloud providers. Each provider requires a specific Configuration JSON structure that defines the connection settings and authentication method.

These are the supported providers:

• Amazon (AWS) S3
• Google Cloud Storage
• Microsoft Azure Blob Storage
• Microsoft Azure Files

The Type field identifies the cloud provider and associated object storage service associated with the Endpoint. This value is primarily used in the UDMG Admin UI, helping Users quickly recognize which cloud provider the Endpoint is intended to represent.

The actual connection behavior is determined by the Configuration JSON, which defines the cloud storage backend used by the Endpoint.

Cloud Storage Base Path

The Cloud Storage Base Path is an optional field that can be used to define the root location within the Endpoint. For object storage, this path is appended to the bucket or container as a suffix or will be used as the bucket or container if none was supplied in the Configuration JSON. It helps define the base storage location that associated Pipelines can read from and write to within the configured cloud provider. Locations outside this base path are not accessible to associated Pipelines.

All Pipeline paths configured for this Endpoint are resolved relative to the bucket or container and Cloud Storage Base Path. Be cautious when leading with a '/' as it may result in unintended outcomes depending on other parts of the constructed path. See the impact of unnamed folders for more details.

Valid formats:

• "" (blank)
• dir
• dir/

info

Regardless of the UDMG Server host operating system, the path must use forward slashes / (Linux path formatting).

Configuration JSON

The Configuration JSON defines the Rclone remote configuration for the selected cloud provider and object storage service. This field requires a JSON object containing the provider parameters and credential variables. UDMG uses these parameters to authenticate and connect to the cloud provider.

For detailed instructions, examples, and provider-specific requirements, see Configuration JSON.

Adding an Endpoint

To add a Remote Cloud Storage Endpoint, follow these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click Add Endpoint.
3. Select Remote Cloud Storage as the Endpoint Type.
4. Complete the form using the Field Descriptions table as a guide.
5. Click Add.

tip

After adding the Endpoint, you can validate the configuration to confirm that the JSON configuration is in the correct format for the specific-cloud provider type. For more information, see Validating the Configuration.

Field Descriptions

The following table describes the fields that are configured for a Remote Cloud Storage Endpoint:

Name	Description	Specifications	Required
Endpoint Type	The type of Endpoint indicates the type of file transfer. Select: Remote Cloud Storage.	Cannot be modified after creation.	Yes
Name	The name of the Endpoint.	Must be unique. Must follow the Standard Naming Pattern.	Yes
Description	The description of the Endpoint.		No
Business Service	Assigning a Business Service to this Endpoint restricts updating, deleting, and other actions so only Users with the same Business Service can perform them (role permissions still apply). For details, see Business Services.	Only Users with the System Administrator, Domain Administrator, or Pipeline Management role can assign Business Services to Endpoints. Select an already-created Business Service. To select it, your User must have that Business Service already assigned.	No
Type	Identifies the cloud provider and object storage service associated with this Endpoint. This field is displayed in the UDMG Admin UI for management purposes. Options: AWS S3 Google Cloud Storage Microsoft Azure Blob Microsoft Azure File Storage		Yes
Cloud Storage Base Path	The root location within the Endpoint. For object storage, the path is appended (as a suffix) to the bucket or container path entered in the JSON Configuration. For details, see Cloud Storage Base Path.	warning Modifying this path after initial setup may disrupt associated Pipelines and file references.	No
Configuration JSON	Provide the Rclone remote configuration for the selected cloud provider, formatted as JSON. For details, see Configuration JSON.		Yes
Enable Endpoint	When enabled, the Endpoint is active and can participate in file transfers. For more information, see Enabling and Disabling an Endpoint.	Default value: Enabled.	Yes

Editing an Endpoint

To edit an Endpoint, follow these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click the Name field of the Endpoint you want to edit.
3. Click the Edit button above the Endpoint details.
4. Edit details for the Endpoint, using the Field Descriptions table as a guide.
5. Click Update.

Managing an Endpoint

Viewing Endpoint Details

To view the details of an Endpoint, follow these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click the Name of the Endpoint you want to view.
3. Additionally, click the Pipelines tab to see all the Pipelines the Endpoint is associated with.

Endpoint Metadata

Remote Cloud Storage Endpoint details include all parameters given in the Field Descriptions table, plus the following read-only metadata:

Name	Description
UUID	Universally Unique Identifier of this Endpoint.
Version	Version number of the configuration. Every change increases the number.
Test Status	Status of the "Validate" Endpoint configuration test. See Validating the Configuration.
Created	Date and time this Endpoint was created.
Updated	Date and time this Endpoint was last updated.
Use Forward Proxy	Always set to False, as Forward Proxy is not supported for this Endpoint.

Validating the Configuration

The Validate action checks whether the Remote Cloud Storage Endpoint is configured correctly, including that the credential placeholders follow the correct format ({{ pipeline_credentials.key }} / {{ pipeline_credentials.value }}).

Because authentication credentials are associated with the Pipeline, Endpoint validation through the UDMG Admin UI, does not validate the Credential used for file transfers. As a result, a successful validation confirms that the Endpoint configuration is correctly structured, but it does not guarantee that all transfer operations will succeed at runtime.

To validate an Endpoint, follow these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click the Name field of the Endpoint you want to validate.
3. Click the Validate button above the Endpoint details.

After clicking the Validate button, one of the following messages appears:

• Endpoint test has [passed]. The Endpoint configuration is correct.
• Endpoint test has [failed]. The Endpoint is not correct. Verify that the Configuration JSON formatting and values are correct.

To test Endpoint connectivity with credentials outside of the UDMG Admin UI, use the API endpoint and request below. By including credentialsName in the request, UDMG will resolve the credentials, connect to the cloud provider, and verify access to the configured path.

API Endpoint

POST /api/v1/domains/${DOMAIN_UUID}/endpoints/${Endpoint_Name}/test

Request Example

rest /api/v1/domains/primary/endpoints/aws-cloud/test 
-X POST -d @- << JSON
{
  "credentialsName": "aws-s3-credential"
}

Response Example - Success

{
  "status": "success",
  "message": "Successfully connected to S3 storage",
  "details": {
    "accessible": true,
    "bucket": "udmg-qa",
    "cloudType": "s3",
    "path": "udmgs3bucket",
    "provider": "AWS",
    "region": "us-east-1"
  }
}

Response Example - Failure

{
  "status": "error",
  "message": "Failed to access S3 storage path: failed to list path '': operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 83ACBVPV5P3FA17Y, HostID: s40Wqa17xJyB4l6Q7vkutFCuUyskwarh8atlC6yjlpW2TR0Qla1r3zPSD3OzW76/yI/OrXyff8Q=, api error InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.",
  "details": {
    "authError": true,
    "bucket": "udmg-qa",
    "cloudType": "s3",
    "path": "udmgs3bucket",
    "provider": "AWS",
    "region": "us-east-1"
  }
}

Enabling and Disabling an Endpoint

Remote Cloud Storage Endpoints can be Enabled or Disabled to control their active status and ability to participate in file transfers.

• Enabled (default): The Endpoint is active and can participate in file transfers.
• Disabled: The Endpoint is inactive and cannot participate in file transfers.

The status is set during initial creation and changed following these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click the Name of the Endpoint you want to enable or disable.
3. Click Edit.
4. Move the Enable Endpoint toggle.
5. Click Update.

info

Changes to the Endpoint's Enabled status are not active until the Account reconnects.

Deleting an Endpoint

To delete an Endpoint, follow these steps:

1. From the Sidebar, select Configuration > Endpoints.
2. Click the Name of the Endpoint you want to delete.
3. Click the Delete button above the Endpoint details.
4. You will be asked to confirm the deletion. Click Continue.

warning

UDMG prevents the deletion of a Configuration Item if it is currently referenced by another Configuration Item (i.e., Pipeline). The Configuration Item must be edited or the reference removed before you can delete the Endpoint. Deletion cannot be undone.