The Hadoop Honeymoon Is Over

Here Comes “Associative Data Management!”

According to Gartner there is “continuing enthusiasm for the big data phenomenon”. However, “demand for Hadoop specifically is not accelerating.” So what’s the story?

All NoSQL systems are not ‘Associative Technologies’. Their definition of associative involves the fact that a subject is connected to an object through an association (predicate). “Billy loves cheese” for example whether in a triple “A” -> “B” -> “C”, or an XML: (simplified)

<ENTITY> Billy

<ACTION> Loves

<ENTITY> Cheese

These are considered ‘associative' because a piece of data is connected to another piece of data through another piece of data.

The fact is they are ‘associative' at the data level. But they are not associative at the storage level, in other words they are not 'Associative Technologies’. They are at best an associative
layer on top of an old technology.

Every statement, triple, K-V pair, is still stored in some hiding system, where the structure and namespace are in between the user and the information.

For instance to understand that Billy has a relationship with cheese, someone has to know what kind of relationships Billy has with everything. Then one has to search through all the
relationships Billy has. Then one has to decide what is pertinent to the interest one has in Billy. Once one gets all the relationships Billy has with the Universe, then one must check each thing’s relationships to find out if anything has relationships to something related (possibly several degrees away) to the said interest in Billy.

In a real 'Associative Technology’, one which is associative on all levels, (storage, data, information, intelligence, user), one need only select one’s interest area and Billy. The network of relationships, (that in fact are the ‘Associative Technology’), are live, accessible directly and generically from each data element, and, (since everything is automatically contextualized upon ingestion, generating a virtual contextual graph of all minimal pathways between all contexts (not at the data level like all NoSQL systems), can provide an automated means to answer every question one can ask about anything.

In every other ‘technology’ you can only find what you specifically ask for in a unique namespace bound query created in some complex language like OWL or SparQL… "What does Billy Love?" one could ask and in the result set one would find cheese, but Billy could ‘like’ of ‘enjoy’ or ‘really like’ or 'is enamored with’ or ‘adores’ or ‘whatever’ cheese.

How could one possibly know what to ask to find out that there is some sentiment with cheese?

One would need to encode some ontology to deal with the ideas of sentiments, affective states, and proclivities as well as foods, food groups, dairy products, etc., to begin with. Then comes the taxonomies…

What if the interest was farms, and one wanted to know if Billy has any interest in farms. The knowledge of farms, farm products, distribution networks and grocery stores would have to be factored into the data store, and intelligently presented so that a query could path it’s way from Billy to the farm, through his love of cheese.

The AtomicDB system is an Associative Information System that mimics human memory capabilities. It can function as a database but calling it a database is like calling a human a consumer, sure a human can consume but there is far more to being human that just consuming, (or at least one would hope), all relationships are facts and all relationships are in dimensions of association that correspond directly to the many and varied contexts of the information about the data.

Finding common information about any number of things is simply part of the associative memory function capabilities. It ‘resolves’ relationships in a completely general and generic (non-namespace bound) way.

Just the beginning...