Designed for ease-of-use by data analysts and with an intuitive drag-and-drop interface, EIQ2 is a data platform tool, automating workflow tasks in a simple and transparent fashion and removing the need for developing ETL tasks or manually writing code and validation checks. Shifting the burden of handling common and repetitive tasks from the user to technology, EIQ2 leverages smart technology to optimise and intelligently accelerate performance tuning on the fly.
EIQ2 is in effect, an engineering environment for the accurate movement and manipulation of data, handling a variety of sourced data volumes and formats, including SQL, Oracle and ODBC databases, excel xml and delimited or fixed width flat files, as well as textual data from converted image and word related documents, i.e.: (pdf, jpg etc.). In addition, and furnished with a rich set of tools and functionality, EIQ2 provides profiling capabilities, a graphical reporting suite, full end-to-end auditing and a built in library of transformation, cleansing and data governance measures, that can be easily added or changed, according to user requirements.
The EIQ2 application is split into five sections
The Data Source, which can reside on a remote or the local machine, is a facility for storing data. It can be sophisticated as a complex database or as simple as a file with rows and columns. The EIQ2 interface provides differing, connection methods to establish / modify a secure link and periodically extract and collect changed records into the landing pad staging area.
The landing pad or staging area is an intermediate storage area used for data processing and sits between the sourced data sources and the ODS (data warehouse). When data sources happen to change, the staging area becomes stale and to ensure data timeliness, the landing pad tables need to be refreshed on a periodical basis. The naive approach of simply reloading the staging area is obviously inefficient. Typically, only a small fraction of sourced data may change during the loading cycles and so it is therefore desirable to only capture these changes and refresh the landing pad incrementally. The staging area should only be accessed by the loading of ETL processes. It should never be available to end users, as it is not intended for data presentation and may contain incomplete or invalid data.
The ODS (Data Warehouse) has been designed to integrate data from multiple sources and allow additional operations to be applied. During this stage, this solution provides the means for data cleaning, resolving redundancy issues and checking and applying transformations rules for integrity. The data transformations and cleaning steps are some of the most important as they ensure the quality of data made available to the end users. The ODS table is an architectural component of a data warehouse, and is used for immediate reporting and with current operational data. ODS population covers data extraction from dependent staging and other ODS tables. The main objective is to retrieve only new, updated or deleted data from the dependent tables and with as little resources as possible. EIQ2 utilises hierarchical dependency checks and moves these processes upstream before transactions into the ODS can be executed.
In addition to providing administrative reports and the tools to interrogate or generate and save ad-hoc reporting, EIQ2 provides the means to easily monitor and filter data against the EIQ2 environment. In challenging times, good decision-making becomes critical. The best decisions are made when all the relevant data available is taken into consideration.
A Data Platform solution requires a high level of maintenance and support and so, the real work of taking output from such an environment, depends largely on how it is managed. As data platform projects are very expensive, it is extremely desirable that it gives the desired results and functions smoothly.