Category Archives: Big Data

Runners as a metaphor for agile

Agile Techniques Are Helpful with Databases

picture by Kanban Tool via Flickr

I’ve been practicing agile database techniques for about twenty years now. My use of agile techniques didn’t start as an explicit plan. Rather it evolved over time as I was working on consulting projects. It made sense to look for ways of working faster and better and with greater customer interaction.

I can think of at least three kinds of agile database techniques.

  • For data modeling.
  • For data warehouse development.
  • For database reverse engineering.

Continue reading Agile Techniques Are Helpful with Databases

cover for Agile Data Warehouse Design

Agile Data Warehouse Design Video Course

My new video course — Agile Data Warehouse Design — is now on the market.

A data warehouse is a database dedicated to decision support and business analysis. The inputs to a data warehouse are data from the day-to-day operational systems. A data warehouse integrates the input data and restructures it so that it is amenable to data mining.

With an agile approach developers build a data warehouse rapidly to get it into the hands of business users so they can give feedback. Where possible, SQL code substitutes for programming and ETL scripts.

The course is organized about a threaded case study for the retail industry.

Runners as a metaphor for agile

Agile Analytics

picture by Kanban Tool via Flickr

We define agile analytics as the iterative and rapid processing of data for decision support. There is minimal upfront investment. Developers instead process source data and build queries incrementally as business needs emerge.

Big Data and data warehouses are both technologies for realizing agile analytics. They differ in several ways.

Continue reading Agile Analytics

What Is Big Schema?

picture by Tom Woodward via Flickr

“Big Data” has become a popular buzzword. Big Data is data that is so massive that it is difficult to manage. For example, the volume of search engine queries, online retail sales, and Twitter messages exceed the capabilities of traditional databases.

There’s a complement to “Big Data” that we call “Big Schema”. Today’s data can not only have vast quantities and fast rates, but can also have diverse structure. Big Schema can arise with enterprise data models, large data warehouses, and scientific data.
Continue reading What Is Big Schema?