The focus of week two is data management and use of collaborative tools in the work environment.
Data management from a technology perspective seems much easier to understand compared to the world of data coming to life in the real world.
Two books helped me immensely when I tried to understand why data management groups got re-titled as information management groups and how database technologies were evolving at major database technology vendors until Y2K,
First was Alan Simon's work - Strategic Database technology: management for the year 2K that was so valuable for gaining clarity. The online GC algorithm for achieving availability, the partitioning of datasets algorithms for achieving scalability, schema evolution, were all explained in a simple fashion with diagrams in this book. I could view the information management group organized right out of this book for the most part.
The second was the Price water house technology forecasts for the year 2K - talked more about data warehousing and several other data applications - all dominant and equally important as the database technologies themselves. The data applications industry was a very mature industry by 2K.
There was a period when combinations of object oriented systems with the traditional systems was also in vogue (ORDMS) and were believed to be the next big thing for databases.
In summary, database systems and their applications have become matured and dominant industries by Y2K and even before. MySQL (open source) came in providing relief to many as a free offering.
In summary, database systems and their applications have become matured and dominant industries by Y2K and even before. MySQL (open source) came in providing relief to many as a free offering.
Soon, web data management, web database management took on but it seemed that everything about data is about back end processing. Multi tier applications started showing the traits of database engines with applications also behaving like robust engines with the necessary ingredients (data storage and retrieval logic, process logic and presentation logic) including security layer. This is the essence of the enterprise applications layout. I won't use the word enterprise application architecture here - that is a bigger term for including the entire infrastructure layout along with the application engines on the top layers.
Who could imagine that the front end of the enterprise takes on with data - capturing data at the source (mobile, social data) and data accessible anywhere and everywhere (cloud). Facebook and Twitter were products of the mid y2K decade. Understanding behaviors and patterns of browsing and buying habits of consumers and customers - all outside the organization. Their noise is the source of data deluge.
Fast forward to mid Y2K's - the dormant business intelligence took shape to analytics (social, mobile, big data) and now the era of big data and cloud and the convergence of all these leading to new platforms.
I started paying attention to the applications industries reading books on business intelligence, analytics and now Big data (SQL and noSQL, structured or not structured data diferences). Until now my focus has been to view them from the customer and management perspective to some extent. Big data and cloud convergence will be interesting - this will require the platform thinking. Social and mobile created the data noise, the data issue have to be resolved by big data solutions and data management taking more of a scientific perspective.
More on the staying power of platforms and the business of software in a separate post.
Competing with analytics: The new science of winning by Thomas Davenport released around 2006 - 2007 time frame I think. Then followed another titled Analytics at work, Smarter decisions, better results. These books showed insights into data driven companies such as Amazon, Netflix (which were seemingly simple e businesses to start with). The type of questions that are asked in a data driven culture are presented very well in these books. HBR's articles in the Oct 2012 issue on big data cover most of these topics in a nutshell.
For my notes, I relied heavily on these two books and brought some flavor to data products starting with recommendation systems.
For references on line, I suggest the following:
Additional Readings:
End of theory, the data deluge makes scientific theory obsolete
Reshaping the workforce with new analytics
Transforming collaboration with social tools
