The CSV connector converts comma-separated values (CSV) files into XML and generates CSV files from XML.
Key Capabilities
- Bidirectional CSV/XML conversion with configurable delimiters and encoding
- Header detection and custom column mapping capabilities
- Support for various character encodings and special character handling
Overview
Connector Configuration
This section contains all of the configurable connector properties.
Settings Tab
Configuration
Settings related to the connector Id and description.
| Setting | Description |
|---|
| Connector Id | The static, unique identifier for the connector. |
| Connector Type | Displays the connector name and a description of what it does. |
| Connector Description | An optional field to provide a free-form description of the connector and its role in the flow. |
Connector Settings
Settings related to the core operation of the connector.
Other Settings
Settings not included in the previous categories.
| Setting | Description |
|---|
| Local File Scheme | A scheme for assigning filenames to messages that are output by the connector. You can use macros in your filenames dynamically to include information such as identifiers and timestamps. For more information, see Macros. |
| Line Separator | The character sequence to use to denote line endings. By default, this is determined by the system, but you can use this setting to override the system behavior. Use CR or \r to denote carriage return, and LF or \n to denote line feed. |
Sample Files Tab
Uploading a sample file allows for easy mapping in any connected CSV Map or XML Map connectors.
Use the Sample Files tab to generate a source or destination XML template for use in a CSV Map or XML Map connector. Click Add Sample File.
- If you already have an example file that reflects the expected format and structure of your data, choose File Upload and browse to your file. Alternatively, select Manually Create to create a file directly in the application. If you choose Manually Create, give your sample file a name. Click Next.
- If you browsed to an existing file, the View File tab is populated with its contents. If you chose to manually create a file, enter your sample data on this tab. Click Next.
- The Translate and Preview tab displays a preview of your sample data converted to XML. When you are satisfied with your sample, click Add.
Advanced Tab
| Setting | Description |
|---|
| Processing Delay | The amount of time (in seconds) by which the processing of files placed in the Transactions tab is delayed. This is a legacy setting. Best practice is to use a File connector to manage local file systems instead of this setting. |
Logging
Settings that govern the creation and storage of logs.
| Setting | Description |
|---|
| Log Level | The verbosity of logs generated by the connector. When you request support, set this to Debug. |
| Log Subfolder Scheme | Instructs the connector to group files in the Logs folder according to the selected interval. The Weekly option (which is the default) instructs the connector to create a new subfolder each week and store all logs for the week in that folder. Leaving this setting blank tells the connector to save all logs directly in the Logs folder. For connectors that process many transactions, using subfolders helps keep logs organized and improves performance. |
| Log Messages | Check this to have the log entry for a processed file include a copy of the file itself. If you disable this, you might not be able to download a copy of the file from the Transactions tab. |
Miscellaneous
Miscellaneous settings are for specific use cases.
| Setting | Description |
|---|
| Other Settings | Enables you to configure hidden connector settings in a semicolon-separated list (for example, setting1=value1;setting2=value2). Normal connector use cases and functionality should not require the use of these settings. |
Automation Tab
Automation Settings
Settings related to the automatic processing of files by the connector.
| Setting | Description |
|---|
| Send | Whether messages arriving at the connector are automatically processed. |
Settings related to the allocation of resources to the connector.
| Setting | Description |
|---|
| Max Workers | The maximum number of worker threads consumed from the threadpool to process files on this connector. If set, this overrides the default setting on the Performance Settings portion of the Advanced Settings page. |
| Max Files | The maximum number of files sent by each thread assigned to the connector. If set, this overrides the default setting on the Performance Settings portion of the Advanced Settings page. |
Alerts Tab
SLAs Tab
Converting CSV to XML
When a CSV file is transformed into XML, the resulting XML has the following structure:
Each row (record) in the original file becomes a child of the root element Items. The name of all record elements is determined by the Record Name option set in Connector Settings. Each record element then has child elements corresponding to the values in each row of the input file.
Some CSV files include a line of header information that provides context to the values in the file. When the First line is header information setting is enabled, this header line is parsed, and the parsed headers are used as the element names for the value elements (the children of the record elements). Otherwise, the value elements are given generic names such as field_0, field_1, and so on.
Converting XML to CSV
To convert XML to a CSV file, the input XML must have a ‘flat’ structure. This means that, disregarding the Items root element, the depth of the XML structure is two. For example:
The XML is interpreted as follows:
- Children of the root element are treated as records (rows) in the resulting file
- Children of each record element are treated as the values in each row
If the First line is header information option is enabled, a header row is inserted into the resulting CSV file with the names of each value element to provide context to the values. In the example above, this header row would consist of title, year, and runtime.
Print View
CSV Operations
In addition to the Operations provided with Arc, connectors can provide operations that extend functionality into ArcScript.
These connector operations can be called just like any other ArcScript operation, except for two details:
- They must be called through the
connector.rsc endpoint.
- They must include an auth token.
For example, calling a connector operation using both of these rules might look something like this:
Operations specific to the functionality of the CSV connector are listed below.
csvListRecords
Loops over every record in a specified CSV file or string. See Functions for more details.
Required Parameters
- file: The path to the CSV file.
Optional Parameters
- data: If the CSV data exists as a string rather than stored in a CSV file, use this parameter instead of file.
- columns: The comma-separated list of columns to include in the output (if unspecified, all columns are included).
- requireheader: By default, the first row of data is interpreted as column headers; pass false to this parameter to use generic column names (for example, c1, c2, c3).
Output
Any script in a csvListRecords operation executes multiple times: once for each record/row in the input CSV file/data. Within the operation, individual CSV values are accessible using the csv formatter. This formatter takes a column name as a parameter, and outputs the value in that column for the current record.
For example, imagine the CSV input data contains a set of items purchased in an order, and the name of the item is held in the ItemName column. The following script generates XML containing each ItemName value in an Item element:
Macros
Examples
