What are currently the hot topics in Machine Learning research and in real applications?

What are currently the hot topics in Machine Learning research and in real applications?

1. Deep learning seems to be getting the most press right now. It is a form of a Neural Network (with many neurons/layers). Articles are currently being published in the New Yorker [1] and the New York Times[2] on Deep Learning.

2. Combining Support Vector Machines (SVMs) and Stochastic Gradient Decent (SGD) is also interesting. SVMs are really interesting and useful because you can use the kernel trick [10] to transform your data and solve a non-linear problem using a linear model (the SVM). A consequence of this method is the training runtime and memory consumption of the SVM scales with size of the data set. This situation makes it very hard to train SVMs on large data sets. SGD is a method that uses a random process to allow machine learning algorithms to converge faster. To make a long story short, you can combine SVMs and SGD to train SVMs on larger data sets (theoretically).

3. Because computers are now fast, cheap, and plentiful, Bayesian statistics is now becoming very popular again (this is definitely not "new"). For a long time it was not feasible to use Bayesian techniques because you would need to perform probabilistic integrations by hand (when calculating the evidence). Today, Bayesist are using Monte Carlo Markov Chains, Grid Approximations, Gibbs Sampling, Metropolis Algorithm 

4. Any of the algorithms described in the paper "Map Reduce for Machine Learning on a Multicore". This paper talks about how to take a machine learning algorithm/problem and distribute it across multiple computers/cores. It has very important implications because it means that all of the algorithms mentioned in the paper can be translated into a map-reduce format and distributed across a cluster of computers. Essentially, there would never be a situation where the data set is too large because you could just add more computers to the Hadoop cluster. This paper was published a while ago, but not all of the algorithms have been implemented into Mahout yet.