Cloud Portal for File Storage and Cataloging

Stephen Hanly

Objective

Develop a cloud portal that .ide recording files can be imported to and accessed globally. Enable cataloging of files so that a basic understanding of the file can be achieved before directly opening/accessing it.

Description

Users are beginning to perform tests over long periods of time with multiple devices that generate hundreds, even thousands of recording files. It can be very time consuming to find particular recording files of interest and also understand how the environment has changed from test to test without directly opening the file.

As the product line begins to evolve towards a wireless product offering, it will become important to provide a centralized file storage system that can be accessed by devices and by customers globally (who may not be in the same location).

Testing and analysis is often a collaborative process where the engineer performing the analysis is doing so on a team and has an end customer who also wants to occasionally access data files. To promote collaboration between internal and external stakeholders this portal should also be very "shareable" so that a link/key can be provided to others to enable sharing of data.

Most importantly, the files themselves need to be "described" with details about the device and sensors used to generate the data and even summary metrics themselves. In the short term this information can be useful to users looking to find the points of interest in a test; but in the long run this same infrastructure can be used in monitoring applications to show in semi-real time how important metrics are changing over time.

Requirements

• Cloud storage
• Canned cataloging of recording files (recorder serial number, time stamp, sensors enabled, sample rate, max/min/mean, RMS, dominant frequency and corresponding amplitude, second dominant frequency and corresponding amplitude)
• GUI in desktop software to upload/download and view list of files (with summary metrics)
• API keys to access files and portal and share specific files with custom scripts
• Custom cataloging function capability

Deliverables

A few stages would be utilized to receive feedback and buy-in:

• Cloud storage (simple online file access - "Dropbox for .ide files") - this includes a desktop based GUI to interact with the files/portal.
• Canned cataloging when files are uploaded
• Documented API to access files with custom scripts and receive API pull/push requests when new files are available
• Interface (could be through desktop app) to provide a custom script that catalogs files with custom numeric metrics as they come into the portal (also allow for post-processing and cataloging of files to fine tune the analysis functions). Documentation and examples will be needed to demonstrate the functions/objects/outputs the system is looking for and providing.
• Browser-based interface to view contents of the database, upload/download data, and manage billing

Cost to the Customer

This system will have inherent costs based upon total storage, upload/download frequency and size, and computing requirements. As such the cost is elastic depending on usage. We would look to have an elastic and transparent pricing model where the user pays for what they use. We recognize the concern over receiving surprise bills so we would include a user-defined maximum monthly limit. Some grace period would be provided with warnings when the limit is being approached and warn that over usage is occurring.

Baring in mind that the cost is elastic depending on use, we'd expect the cost to the average customer to range between $50 and $250 a month per recording device.

Example Use Cases

1. Post-processing of many files: A mechanical engineer has performed a test over the course of several days with multiple devices. The test was a shipment from the facility in the US to one in Europe where an engineering team and/or end customer can download the data from the devices. But this data now needs to be accessible by both teams to understand how the test/shipment went. And they need to collaborate on how best to analyze these recording files. The test has generated hundreds of recording files and they need to understand how things changed over time, and find specific files of interest.
2. Monitoring application: This use case requires a product line that is interfacing with the cloud portal to directly upload files from the device. In this instance, the team has set up the test at their customer's site that has internet access but it is a separate location than the "home base." The engineering team returns to their respective offices after setting up the test and now expects to see/analyze data as it is being continually generated and uploaded to the portal. The engineering team utilizes the portal and the API to develop a custom dashboard with alerts they provide to their end customer to notify them of issues with the test/system as they arise.