When machine replace human Recently, a good friend sent me an article from Harvard Business Review called "Beyond Automation", written by Thomas H. Davenport... Read More
Incorporate domain knowledge into predictive model As a data scientist / consultant, in many cases we are being called in to work with domain experts who has in-depth business knowledge of in... Read More
Common Text Mining workflow In this post, I want to summarize a common pattern that I have used in my previous text mining projects. Text mining is different in that it... Read More
Estimating statistics via Bootstrapping and Monte Carlo simulation We want to estimate some "statistics" (e.g. average income, 95 percentile height, variance of weight ... etc.) from a population. ... Read More
Exploration and Exploitation "Cold Start" is a common problem happens quite frequent in recommendation system. When a new item enters, there is no prior histo... Read More
Six steps in data science "Data science" and "Big data" has become a very hot term in last 2 years. Since the data storage price dropped signific... Read More
Text Processing (part 1) : Entity Recognition Entity recognition is commonly used to parse unstructured text document and extract useful entity information (like location, person, brand)... Read More
Optimization in R Optimization is a very common problem in data analytics. Given a set of variables (which one has control), how to pick the right value such... Read More
Detecting Communities in Social Graph In analyzing social network, one common problem is how to detecting communities, such as groups of people who knows or interacting frequentl... Read More
Machine Learning in Gradient Descent In Machine Learning, gradient descent is a very popular learning mechanism that is based on a greedy, hill-climbing approach. Gradient Desce... Read More
Measuring similarity and distance function Measuring similarity or distance between two data points is very fundamental to many Machine Learning algorithms such as K-Nearest-Neighbor,... Read More
Predictive Analytics: Evaluate Model Performance In previous posts, we have discussed various machine learning techniques including linear regression with regularization, SVM, Neural networ... Read More