Industrial Internet of Things Ingestion Service (IIoT IS)


Widget size: 2 x 1, 2 x 2

Use the IIoT IS widget to add data to the Amulet database. It can collect data in the following ways:

IIoT IS classes are written for specific customer requirements using a library of helper functions for extracting data from an imported files and inserting values into Amulet.

When creating a IIoT IS widget, it is associated with a specific IIoT IS class; every file received by the widget is processed by this class. The expected frequency of uploads can be specified and the user can receive an email of the status or notified if an upload is not received within the expected time frame. File types are generally Excel or csv, but can be any type of data.

Upload File

Manually upload a file to be queued for processing. Once the file has been uploaded, it will be displayed in the queue. As the file is processed, the progress bar and Progress Details provide real-time feedback.

Click the Details button to display errors generated during the Import process. You can download the file by clicking on the file link and saving the file. The number of files displayed on this screen is limited. To see every file processed, use the Old Uploads page.

Individual tasks can be deleted from the Upload File and the Old Uploads list, by clicking the Delete button in the Delete All row. Tasks will then be removed from both lists.

Clicking the Delete All button clears all tasks from both the Upload File and Old Uploads lists.

Old Uploads

Every file imported is available on this page.

Clicking Details brings up a pop-up containing details of the reason for failure.

Clicking Logs is an advanced feature available only to SuperUsers of the system. This provides access to logs generated from the IIoT IS background processes.

Background Process

A background process does the following:

Automatic Email Collection

Files can be automatically imported by collecting emails. Every email attachment received is queued for processing by the IIoT IS class.

The system keeps track of files already processed; no files are removed from the server.

Automatic FTP Collection

Import files automatically from an FTP server. Every file collected is queued for processing by the IIoT IS.

Automatic Folder Collection

Files can be automatically imported from a designated folder. Every file collected is queued for processing by the File Importer.

Notifications

The system can notify (email) people when certain events happen.

Settings

Each IIoT IS may have its own specific settings that can be manipulated to configure the ingestion service. These differ for each IIoT IS. Please refer to the specific ingestion services’s documentation for more details.

Below is an example of a settings screen for an Azure IOT Hub IS. These will be different for each IIoT IS depending on what configuration settings that IIoT IS requires.

If the parameters for Azure IOT Hub do not appear on your Settings tab, click the Quick Config icon, expand the File Type drop-down and choose the Azure IoT Data Ingestion option. Click Save.

There are a few settings that are common to most ingestion services:

Name Default Value Comments
Allow standarddata updates True Overwrite existing values if they already exist for a particular timetag. Setting to False would import new values only.
Rollback dependent calcs True Rollback (remove) calculations' values that depend on changed values.
Max old uploads 1000 The maximum number of uploads and files to be retained.
Max allowed task run time 3600 The maximum duration (in seconds) that a task will run, before it is terminated.

Schedule

Tasks can be performed on a schedule from within the Bigguy framework. If the task is an Email collection, FTP collection, or Folder collection, you do not need to set the schedule as the collection frequency is already set up on that specific tab.

Note: For additional information, see IIoT Framework.

This schedule is for all other ingestion services that have their own method of connecting to the data source, such as a direct database connection.

Logs

The logs tab is only available to a SuperUser.

There are two ways to access the logs.

Widget Customisation

Value Default Comment
Widget Heading File Import Enter the name of the widget as it is to appear in the heading.
File Type Test IIoT IS class that is associated with the widget. All files received by the widget are processed using this class. The contents of this list is controlled by Menu>IIoT Framework.
Refresh Period No widget refresh Select a refresh period from the drop down menu from between 15 seconds to 1 hour, or no refresh at all.

Installing New IIoT IS Classes

New IIoT IS classes can be created by implementing the interface 'com.c3.bigguy.interfaces.IDashFileImporter4' which allows the background process to hook into the IIoT IS and process the file.

The Menu>IIoT Framework page takes care of installing the new IIoT IS into the system. It also provides feedback about the version of IIoT IS classes installed.

The page also checks the jar files for duplicate classes. Useful for checking for multiple versions of the same File Importer.