Overview

The File Import Module allows users to manually import data or set up an automated data import stream. This article will cover the process of creating a Manual Import profile and importing data into iTwin IoT. If configured correctly, data for multiple sensors of the same type can be imported from a single file.

Setting Up a Manual File Import Profile

1. Navigating to File Import Module- Click on the File Import Icon along the left module navigation sidebar to navigate to the File Import module. This will open the page below.

2. New Import Profile- Clicking the blue "+Import" icon opens the File Import configuration page, allowing the user to configure a new import profile (seen below).

3. Import Name- Name the Import profile.

4. Import Type- Select "Manual" for Import Type.

5. File Upload- A file can be drag-and-dropped into the upload file or the user can click the blue "Choose a file" button to open a file-selection window. Select the file to be imported. Once a file has been selected, the window will be populated with the file data on the right side of the screen, Illustrated below, and the additional configuration of the import can be completed.

6. Parsing- Parsing rules may need to be adjusted in order for the file data to be correctly imported. Parsing rules are automatically applied to the data preview on the right, allowing the user to ensure that the correct parsing rules are selected for the file to be imported. When configured correctly, the data on the right side preview should look similar to the image below.

6a. Delimiter- Select the delimiter the file is using as the field qualifier.

6b. Text Qualifier- Select the qualifier the file is using for text.

6c. Text Encoding- Define the text encoding manner of the file.

7. Headers- The Headers selection allows the user to identify the rows containing label and unit data.

7a. Labels- If the data sheet has a label row, it can be selected. Once selected, the row will be highlighted green.

7b. Units-  If the data sheet has a unit row for the data being imported, it can be selected. Once selected, the row will be highlighted green. In the example above, labels and headers are on the same row.

8. Time Configuration- Properly configuration of the time stamp data will ensure accurate data import.

 

8a. Time Zone- Select the time zone via dropdown of the time stamps for the data being imported.

8b. Timestamp Column- Select the correct column for the time/date data from the checkbox list. Once selected, the column will be highlighted light grey on the data preview. Ensure that the format auto-populated after selecting the correct column matches the data preview. An example of the timestamp format is available below.

9. Sensor Configuration- The sensor configuration allows the user to select the Sensor ID column within the import file, to select the sensor within iTwin IoT to import the data into, or create a new sensor to store the import data. The sensor configuration step is where you will configure the import of multiple sensors, if required.

9a. Sensor ID- Select the data column housing the sensor name or names. Once selected, the column will be highlighted light grey on the data preview. 

9b. Sensors- The sensors selection allows the user to select where the data will be imported. If there are multiple sensor names in the selected sensor column, multiple sensor configuration windows will populate, as illustrated below.

The two initial options for each sensor available are "Existing" for sensors already added to iTwin IoT or "New" in order to create a new sensor for the import of data.

Existing Sensor- An existing sensor is a sensor that has been added to the platform when establishing a new IOT connection, adding an IOT device/sensor, or a previously created import sensor. Clicking the "Existing"  button opens a sensor selection window, illustrated below, allowing the user to select from existing sensors. Clicking a sensor in the left column will select it, moving it to the selected sensor column, and marking it as the sensor the data will be imported to.

After navigating back to the Import profile, the user will need to match import data to the required data for the selected existing sensor, as illustrated below. Note: If the import data doesn't match the data requirements on the left, the import data is likely a calculated metric. Raw sensor metrics must be imported for existing sensors.

New Sensor- The "New" option allows the user to create a new sensor within iTwin IoT to provide an import option for data or sensors not associated with an IoT-connected sensor. 

1. Sensor Name- By default, the sensor name will be the sensor name provided in the Sensor row configured earlier. This can be changed at this time and will be the sensor name used for the sensor in iTwin IoT.

2. Device Assignment- The device is the middle of the hierarchy of the IoT network. The device groups sensors by connection, or allows the user to group import sensors manually, within the Connectivity Module. There are two options available for device assignment when importing data.

The "Existing" selection option opens a window, illustrated below, where the user can select a device for the imported sensor data to be imported into.

The "New" option allows the user to create and name a new Device for the import.

3. Sensor Type Selection- The sensor type selection dictates whether the data will be imported into an existing sensor type (defined metrics based on sensors) or into a newly created custom sensor (allowing the user to select their own data metrics for import).

 

Existing Type Sensor- Selecting Existing type reveals a dropdown allowing the user to select a sensor type.

After selecting an existing sensor type and navigating back to the Import profile, the user will need to match import data to the required data for the selected existing sensor, as illustrated below. Note: If the import data doesn't match the data requirements on the left, the import data is likely a calculated metric. Raw sensor metrics must be imported for existing sensors.

Custom Sensor- The custom sensor selection allows the user to create any metric they want by inputting a metric and unit. Additional metrics can be added by clicking the "+Metric label" button.

After configuring custom metrics, the custom metrics must be matched to the data import file columns.

Once the configuration of the Manual Import profile is complete, the "Start Import" button will turn blue, indicating that the Import can be completed. Clicking Save will save the configuration of the Import Profile, but when saving the selected file will be removed from the profile.