Forum Discussion
Proposal for a new data structure that extremely reduces data sizes for data in which two item types
I propose a new data structure that reduces data sizes for data in which two item types have many-to-many relations. The proposed data structure newly introduces container variables related to many values of both items, and these container variables record many-to-many relations between them. The proposed data structure maintains data normalization and integrity and is independent of indexing methods conventionally used for relational databases, allowing simultaneous use of both.
When one item type has N items and the other item type has M items and all of N is related to all of M, the conventional RDB requests N×M rows, whereas the proposed data structure requests N+M rows. When N=100,000 and M=10,000,000, the conventional RDB requests 1,000,000,000,000 rows, whereas the proposed data structure requests only 10,100,000 rows.
In detail, please show the journal article linked by
https://www.iaiai.org/journals/index.php/IEE/article/view/589
, or, please show the US Patent No.11294961. In the patent, upper item or item group index is used as anther name of container variable.
The testing results for the proposed method using data in the real world are published. These results show that the proposed methods are efficient also for the data in the real world. The used data are news documents and words in the documents written in Japanese. The data size for the relations between the documents and words is reduced approximately 43%. The search time from a word to the related documents becomes 31% when data on main memory used, whereas 438% when data on relational database in hard disk. These results shows that the proposed method is useful when data on main memory are used, for example, Redis which is one of the famous NoSQL databases.
Reference; Tsuneo Kuwabara and Kazumi Saito, Data Reduction and Data Searching Time Using Container Variables Relation between News Documents and Word used in the Documents, Science Journal of Kanagawa University, Vol.35, pp49-52 (2024.7)
https://www.sci.kanagawa-u.ac.jp/about/pdf/SJKU_2024_vol35.pdf
- A patent concerning this technologies is newly registered in Japan on November 29, 2022. The registered Number of the patent in Japan is 7185264. Concerning this technologies, there are two other patents registered in Japan. Those registered Number in Japan are 6269884 and 4853797, which are registered on January 12, 2018 and November 4, 2011, respectively.
T.kuwbara Have you ever seen the show Silicon Vally? I feel like they came up with a similar idea.
Middle Out Algorithm --> Youtube
Edit: Because I should likely mention NSFW, but 'aight if you've got something of a wrong sense of humor.
- Dir Virle
Thank you for your reply to my submission. I regret the Yutube URL you mentioned cannot seen in my environment. I would like to emphasize that the technologies I submitted is not a simple idea but US patent already registered and published. T.KuwabaraI went and read the overview, not in its entirety as I'm just kind of checking some notifications and trying to move on to a project. I downloaded the paper to read-at later. Just wanted to say that I had thought similar to 'storage and time' while getting fairly deep into playing with Javascript, specifically in regards to its prototyping (I still have my test scripts around here somewhere) I found that I could, via prototypical inheritance, allow an object to be 'knowledgeable' of and read values from both its 'past' prototypical state to and from its own 'future' states. Future states could reach back to a previous state and read from it. I could write, overwrite, and recall prototype constructors and then recall them from a past state, as well as being able to roll them up and roll them back out again* (I'd figured this out while attempting to convert these objects to valid JSON strings (because, obviously, having so much chained structure disallowed conversion to and from in any normal way). *Can't remember if I was completely successful)
Disclaimer, what looked something like time-like data traversal really might have just been typical proto behavior (have never found a needed to use the behavior in the extent I was playing with then (AngularJS had some issues with the way it handled data flow some while back, did some fairly deep trickery there to patch it for a project but the trickery was only similar in nature the these prior objects, Angular mostly fixed the issue themselves and is not much of a problem any longer).
- I moved on to another project and stored the script. Best to let smarter people handle such things, I'm sure I had no idea what I was doing. 😉 I do always intend to get back to experimenting with the things I play with. Sometimes I do, sometimes I don't.
