Design Pattern Approach for Data Lakes Increases Insight and Opportunity

Teradata, a big data analytics and marketing applications company, recently announced a new "design pattern" approach for data lake deployment to help clients build and benefit from data lakes. The design pattern approach is a new method that may help business users, data scientists and IT professionals establish data lakes that produce greater business value.

Organizations are exploring the functionality of data lakes to create insight and opportunity from exploding data volumes, yet serious problems entangle their IT teams, including a lack of best practices, the shortage of data scientists and even confusion regarding the definition of a data lake. In addition to these challenges, technology choices are multiplying. For example, data lakes are typically assumed synonymous with Hadoop – which is an excellent choice for many data lake workloads – however, a data lake can be built on multiple technologies such as Hadoop, NoSQL, Amazon Simple Storage Service (S3), a relational database (RDBMS) or various combinations thereof. And while technologies are critical to the outcome, a successful data lake needs a plan. A data lake design pattern is an example of a possible plan. The design pattern consists of intellectual property based on enterprise-class best practices combined with products co-developed from a stream of successful customer engagements.

"A data lake is different from an operational data store. Teradata's value proposition stems from practical, on-the-ground experience helping customers cope with managing data in heterogeneous environments," said Tony Baer, senior analyst, OVUM.

By having access to new data such as customer service records, clickstream data, IP traffic, log information and sensor data stored in a data lake, users can address cases that generally require multiple, simultaneous interpretations of the data to be tested against each other. Two examples of possible implementation are:

1. Generating improved customer churn detection models by extracting text from customer service calls stored in the data lake, then applying predictive text analytics methods.
2. Providing for trend analytics against combinations of vast streams of machine data with consumer data. In the utility industry, for example, data lakes pave the way for running multiple data models against each other to examine the impact of installing energy-efficient appliances and the latent effect, months later, of reducing electricity consumption.

"Who hasn't heard about data lake implementation nightmares?” said Ron Bodkin, president of Think Big, a Teradata company. “Sadly, so many companies find big data landmines the hard way. We get customers out of crisis mode and help business, IT and data scientists plan, execute and benefit from data lakes that actually generate great business value."