Overview

The File Import Module allows users to manually import data or set up an automated data import stream. This article will cover the process of creating a Manual Import profile and importing data into iTwin IoT. The manual import consists of bringing one file in at a time via the created import profile. If configured correctly, data for multiple sensors of the same type can be imported from a single file.

 

Navigating to File Import Tool

1. Navigating to File Import Tool- Click on the Sensor Icon along the left module navigation sidebar to open the Sensors sidebar. 

2. File Import Tool- Click the "File import" button on the Sensors sidebar.

 

 

You will now be in the File Import Main page, as shown below.

 

 

Configuring an Import Profile 

 

 

1. New Import Profile- Once on the File Import main page, clicking the blue "+Import" icon opens the File Import configuration page, allowing the user to begin configuring a new import profile (example seen below). In the screenshot below, I have added a file via file upload so that all configuration options are visible. I've also collapsed the "Automation" dropdown, which is not required for manual export, and I've expanded the "Advanced Parsing" dropdown to show additional configuration options within.

 

   

2. Import Name- Name the Import profile.

3. File Upload- A file can be drag-and-dropped into the upload file or the user can click the blue "Choose a file" button to open a file-selection window. Select the file to be imported. Once a file has been selected, the window will be populated with the file data on the right side of the screen, Illustrated below, and the additional configuration of the import can be completed.

 

 

4. Header Label Row- The Header Label Row dropdown selection allows you to select a row from the spreadsheet that houses information about the data in each subsequent column. This selection will help

5. Time Stamp Column- Select the column of data that holds the sensor data timestamp (date and time of data collection). Once a selection is made, a box will appear below the selected column, and you will need to configure the timestamp. For more information about Timestamp configuration, please visit link here.

 

 

6. Time Zone- Select the time zone of the data in the import file. (It is important to remember that data in iTwin IoT will be stored in a certain time zone, but data visualization in iTwin IoT will convert that data to reflect the time zone selected on the computer.)

7. Advanced Parsing- Parsing rules may need to be adjusted in order for the file data to be correctly imported. When configured correctly, the data on the right side preview should look similar to the image below. Parsing rules are automatically applied to the data preview on the right, allowing the user to ensure that the correct parsing rules are selected for the file to be imported. If the data in the data preview window doesn't look correctly parsed, you may need to adjust the Advance Parsing settings, explained below.

 

 

7a. Delimiter- Select the delimiter the file is using as the field qualifier from the dropdown options. Comma is the default selection for iTwin IoT. 

7b. Text Qualifier- Select the qualifier the file is using for text.

7c. Text Encoding- Define the text encoding manner of the file.

7d. Header Units Row- If there is an additional row just for sensor measurement units, you can select that row here.

 

8. Sensor Configuration- The sensor configuration allows you to match the data in your import file to existing sensors in your project, or you can create new sensors for the data to be held in on the project. This is where you will also define the sensor type and match the spreadsheet data to metrics and metric units in iTwin IoT. The section below will explain how to set sensor configuration.

 

 

8a. Sensor ID- The Sensor ID dropdown selection allows you to select a column on the import file that is populated with sensor names. This is most common if there are multiple sensors on a single file. If the file doesn't have a Sensor ID column, you can select "None". Selecting "None" will result in all of the data in the file being imported into a single sensor.

Once a Sensor ID is selected, the Sensors will be populated with a sensor configuration box for each sensor in the file, as shown below. The example below has one sensor in the file, called "Piezometer Demo".

8b. Sensors- After the "Sensor ID" configuration, you should see a number of sensors coincident with the number of sensors in your "Sensor ID" column. Each sensor will have two configuration options, "Existing" or "New". Each configuration will be covered below.

• Existing Sensor- An existing sensor is a sensor that has been added to the platform when establishing a new IOT connection, adding an IOT device/sensor, or a previously created import sensor. Clicking the "Existing"  button opens a sensor selection window, illustrated below, allowing the user to select from existing sensors. Clicking a sensor in the left column will select it, moving it to the selected sensor column, and marking it as the sensor the data will be imported to.

 

 

After navigating back to the Import profile, the user will need to match import data to the required data for the selected existing sensor, as illustrated below. Note: If the import data doesn't match the data requirements on the left, the import data is likely a calculated metric. Raw sensor metrics must be imported for existing sensor types. The example below is a fairly complex reflectorless prism sensor import.

 

 

• New Sensor- The "New" option allows the user to create a new sensor within iTwin IoT to provide an import option for data or sensors not associated with an IoT-connected sensor. Selecting "New" will open a popout window similar to the one below. 

 

 

• • 1. Sensor Name- By default, the sensor name will be the sensor name provided in the Sensor row configured earlier. This can be changed at this time and will be the sensor name used for the sensor in iTwin IoT.
  • 2. Device Selection- The device is the middle of the hierarchy of the IoT network. The device groups sensors by connection, or allows the user to group import sensors manually, within the Connectivity Module. There are two options available for device assignment when importing data.

The "Existing" selection option opens a window, illustrated below, where the user can select a device for the imported sensor data to be imported into.

 

 

The "New" option allows the user to create and name a new Device for the sensor import.

 

• • 3. Sensor Type Selection- The sensor type selection dictates whether the data will be imported into an existing sensor type (defined import metrics based on sensors integrated with loggers) or into a newly created custom sensor (allowing the user to select their own data metrics for import). 

Existing Type Sensor- An existing type sensor is a sensor that has previously been integrated into the iTwin IoT system through a logger device. These sensor types require specific raw metrics be imported to function. Required import metrics can be confirmed by selecting a sensor. An existing sensors benefit is that these sensors often have calculated metrics available within the platform. Selecting Existing type reveals a dropdown allowing the user to select the sensor.

 

After selecting an existing sensor type and navigating back to the Import profile, the user will need to match import data to the required data for the selected existing sensor, as illustrated below for a vibrating wire piezometer. Note: If the import data doesn't match the data requirements on the left, the import data is likely a calculated metric or may be the wrong sensor selection. Raw sensor metrics must be imported for existing sensors. For questions or assistance, please reach out to our support team

 

 

Custom Sensor- The custom sensor selection allows the user to create any metric they want by inputting a metric and unit. After selecting custom, you will need to input a metric name and the unit of the import data. Additional metrics can be added by clicking the "+Metric label" button, as shown in the example below where two metrics are ready for input. 

 

 

After configuring custom metrics, the custom metrics must be matched to the data import file columns. The example below shows the Dam Discharge and Reservoir Level custom metrics ready to be matched to a import file column.

 

 

9. Alarms- Alarms can be configured to let you know if there were issues with the import. 

• Success Rate- An alarm will be raised on the import if a some rows of the import file aren't imported. The dropdown allows you to select a success threshold of some, most, or all rows parsed. 
• Maximum Timeout- This setting should only be used for automated imports.

 

10. Start Import/Save- Once the configuration of the Manual Import profile is complete, the "Start Import" button will turn blue, indicating that the Import can be completed. Clicking "Start Import" will begin the import process. Clicking Save will save the configuration of the Import Profile for import later, but when saving the selected file will be removed from the profile.