In CData Arc, data travels through flows as Messages that consist of a body and metadata headers. For example, if one connector passes a file to another, the file is transmitted as a Message that contains the file in the body and metadata about the file and associated connectors as headers. Messages can be grouped together in Batch Groups, which allow multiple messages to travel together as a single unit. Some uses for Batch Groups include moving data between systems and processing multiple purchase orders at once. When a Message is part of a Batch Group, it is referred to as a Batch Message.
Key Capabilities
- Intelligent message grouping based on correlation IDs and batch size limits
- Configurable minimum and maximum batch sizes for optimal processing
- Scheduled and immediate batch creation modes
- Support for custom correlation headers for advanced grouping logic
Overview
The Batch Create connector produces Batch Groups based on Messages that arrive in the Send folder (the Transactions tab of the connector settings pane). When a Batch Group is created, it is passed down the flow through the Transactions tab, where it can be processed by other connectors as needed.Connector Configuration
This section contains all of the configurable connector properties.Settings Tab
Configuration
Settings that determine how the connector can be accessed.| Setting | Description |
|---|---|
| Connector Id | The static, unique identifier for the connector. |
| Connector Type | Displays the connector name and a description of what it does. |
| Connector Description | An optional field to provide a free-form description of the connector and its role in the flow. |
Advanced Settings
Settings that specify Batch Grouping options.Advanced Tab
Message
Message settings determine how the connector searches for messages and manages them after processing.
Note: The settings below are deprecated and hidden by default. They appear only on connectors where they were previously enabled or configured to a non-default value. To retain copies of successfully processed files, right-click the connector in the flow, choose Show Success Path, and connect the success path to a File connector.
| Setting | Description |
|---|---|
| Save to Sent Folder (Deprecated) | Copies files processed by the connector to the Sent folder. Disabled by default. Files saved to the Sent folder are not covered by Encryption at Rest. |
| Sent Folder Scheme (Deprecated) | Groups files in the Sent folder by the selected time interval. Only relevant when Save to Sent Folder is enabled. |
Logging
Settings that govern the creation and storage of logs.
| Setting | Description |
|---|---|
| Log Level | The verbosity of logs generated by the connector. When you request support, set this to Debug. |
| Log Subfolder Scheme | Instructs the connector to group files in the Logs folder according to the selected interval. The Weekly option (which is the default) instructs the connector to create a new subfolder each week and store all logs for the week in that folder. Leaving this setting blank tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders helps keep logs organized and improves performance. |
| Log Messages | Check this to have the log entry for a processed file include a copy of the file itself. If you disable this, you might not be able to download a copy of the file from the Transactions tab. |
Other Settings
Miscellaneous
Miscellaneous settings are for specific use cases.
| Setting | Description |
|---|---|
| Other Settings | Enables you to configure hidden connector settings in a semicolon-separated list (for example, setting1=value1;setting2=value2). Normal connector use cases and functionality should not require the use of these settings. |
Automation Tab
Automation Settings
Settings related to the creation and sending of Batch Groups.Automation Status
Performance
Settings related to the allocation of resources to the connector.
| Setting | Description |
|---|---|
| Max Workers | The maximum number of worker threads consumed from the threadpool to process files on this connector. If set, this overrides the default setting on the Performance Settings portion of the Advanced Settings page. |
| Max Files | The maximum number of files sent by each thread assigned to the connector. If set, this overrides the default setting on the Performance Settings portion of the Advanced Settings page. |
Alerts Tab
SLAs Tab
Correlation Id Header Example
When you enter a header name in the Correlation Id Header field, the connector looks for headers that match the name you enter, then groups messages according to the values they have assigned to that header. For example, the following table displays six incoming messages:- The first batch group contains the messages with MessageId 1 and 2, since they both have the value
001for the Batch-Correlation-Id header. - The second batch group contains messages with MessageId 3 and 4, since they both have the value
002for the header. - The third batch group contains messages 5 and 6, since neither of them has the Batch-Correlation-Id header.