Friday, February 3, 2017

Big data for Machine Learning


There is much to unlearn about computer and information systems for understanding machine learning.
Traditionally, business intelligence using Information processing systems has been about data warehousing, online analytical processing, meta-data repositories. While all of them use past data and experience for knowledge and discovery – there are differences. The one thing in common is that they use sorting and searching algorithms with sequences of instructions which lead to definite responses from the systems. Machine learning is different.

Data mining models and applications come closest – except they use very large data bases. Typically, data mining tasks are grouping people or objects based on selected characteristics, estimating for inferring or guessing reasonably, predicting expecting behaviors as a consequence of previous actions. The list goes on with more statistical models with deeper selective techniques for evaluating relationships and associations. 

The talk about machine learning is around predictions using Big data.  A super market chain or Amazon would prefer to know the customer’s next choice of items based on previous purchases. Customer too prefers to get to the items he needs faster. These recommendations and matching systems work with recognizing behavioral patterns obtained from the data. What is derived from the pattern recognition is a response of likelihood of the behavior. Detecting frauds, stock market predictions with changing data and environment require learning and intelligence more than just depending on the system design using a large database. Applications recognizing speech and faces, robotics, biometrics use machine learning techniques. Machine learning broadly encompasses supervised learning (through association, regression, classification) and unsupervised learning for pattern recognition – an engineering term for classifying.  
     
Questions to ponder:
What can go right and wrong with machine learning systems?
What work is suitable for learning machines in platforms?
How does machine learning contribute to Platform Value (its feedback system, market places, data orchestration, network effects) ?

References:
Loshin, David. Business Intelligence, The Savvy Manager’s guide, Morgam Kauffman Publishers, 2003.
Alpaydin, Ethem. Introduction to Machine Learning, edited by Ethem Alpaydin, MIT Press, 2014.