NNI Technologies
The Inception
In 2019, we began a joint research project with JAXA(Japan Aerospace Exploration Agency) to address the issue of managing the massive time-series data that accumulate daily.
In 2023, we achieved the development of the NNI algebra, which facilitates interactive combining, overviewing (searching, aggregating, and sorting), storing, and transferring of massive table-structured big data.
In March 2023, three members of the joint research group founded NNI Technologies and launched initiatives to disseminate our findings around the world.
The obstacles that we surmounted
１. 2 Types of Big Data
HOT Big Data
Big Data
deployed on EXPENSIVE HW
for fast CPU access
10～20%
Various solutions exist
COLD Big Data
← DEF. →
Big Data
stored on LOW-COST STORAGE
not optimized for CPU access
← Fraction →
80～90%
Limited accessibility limits utilization
２. Why Cold Big Data is UNSEEN and UNUSABLE ?
-
10-year data is made up of 3,652 daily data sets
-
Trillions of Records
-
Tens of Thousands of Columns
-
Unindexed data is too time-consuming and costly
The solution that we created
3. Multi-Source Composer（MSC）
Big Data Library
On Your Desktop
MSC is a system that enables users to create purpose-specific databases from various big data around the world stored in the D5A file format, and enables to search, aggregate, and sort all columns in real time.
As a result, MSC solves the problems of UNSEEN and UNUSABLE at once, enabling interactive use of COLD BIG DATA.
4. The Realization of a COLD Big Data-Driven Society
A COLD Big Data-Driven Society is realized,
where Distributed COLD Big Data is
combined on demand,
to purpose-specific databases,
with indexes on all columns.
Information
Demo Video (Youtube)
We are demonstrating a method for creating a purpose-specific database with 3.15 trillion records by combining 100 sets of 315 billion records collected over one year with a 1 ms sampling rate and 1,000 sets of 31.5 billion records collected over one year with a 10 ms sampling rate. The database can then be used for sorting, searching, and aggregation.
Multi-Source Composer enables interactive dialogue with COLD BIG DATA (slides)
Slides updated Feb. 22, 2024
Interact with COLD BIG DATA using NNI Algebra
(Multi-Source Composer)