Graph Abstraction for Databases
The extraction and transformation of various types of data from different sources to a single format and database is a time consuming, expensive and redundant task in Data Analytics. Dexlock was keen to take up the challenge as an R&D project to result in performing data analytics tasks with more speed, security and efficiency.
- The process of identifying the queries and converting the user defined scripts to the corresponding database queries for processing data.
- The efficiency while converting these scripts into queries and running the queries in the respective databases.
- A semantic web analyzer was built to map data from different databases using a graph structure. The graph structure acts as a commonplace for queries to be performed.
- An extension to an existing framework was custom designed to read data from different types of databases.
- Custom graphs optimization tools were created to improve and monitor the overall performance of the platform.
- The need of writing separate queries for each type of database was replaced with user-defined scripts. A custom solution designed to translate the user-defined scripts into the appropriate query type and format was implemented to achieve the same.
- The platform overall is a major innovation in the field of Data Analytics.
- The complex and time consuming process of ETL is simplified by this solution.
- The redundant operations such as transformation of data into another format, separate query operations and the need for a separate database is no longer needed.
- Data Integrity is maintained with the absence of moving data from the original location.
- Apache TinkerPop
- Spring Boot