The Next Wave of Deep Learning Applications
September 14, 2016 Nicole Hemsoth
Last week we described the next stage of deep learning hardware developments in some detail, focusing on a few specific architectures that capture what the rapidly-evolving field of machine learning algorithms require. This week we are focusing in on a trend that is moving faster than the devices can keep up with; the codes and application areas that are set to make this market spin in 2017.
It was with reserved skepticism that we listened, not even one year ago, to dramatic predictions about the future growth of the deep learning market—numbers that climbed into the billionsdespite the fact that most applications in the area were powering image tagging or recognition, translation, and other more consumer-oriented services. This was not to say that the potential of deep learning could not be seen springing from these early applications, but rather, the enterprise and scientific possibilities were just on the edge of the horizon.
In the meantime, significant hardware and algorithmic developments have been underway, propping up what appears to be an initial Cambrian explosion of new applications for deep learning frameworks in areas as diverse as energy, medicine, physics, and beyond.
What is most interesting is that in our careful following of peer-reviewed research over the last couple of years, it was only just this past month that a large number of deep learning applications in diverse domains have cropped up. These breathe new life into the market figures for deep learning that seemed staggering, at best—at worst, woefully optimistic.
These also help explain why companies like Intel are keen to make acquisition for both the hardware and software stacks from companies like Nervana Systems and Movidius, why Nvidiahas staked its future on deep learning acceleration, and why a wealth of chip startups with everything from custom ASICs, FPGAs, and other devices have rushed to meet a market that until very recently, just hasn’t been present in sufficient volume to warrant such hype. As a counterbalance to that statement, a significant uptick in research employing various deep learning frameworks does not create a market out of thin air either, but the point is that there is momentum in areas of high enterprise and scientific value—and it keeps building.
In the last two weeks alone we have seen research that breaks new ground in each of the following domains via neural networks and advanced machine learning frameworks. The listing below provides just a few select examples of the wave that hit the publication shores since the summer. Take note of the emphasis on medical applications for neural networks and machine learning. This appears to be where the most aggressive publishing is happening results-wise—and for an emerging market putting all the right tooling and coding in place, it starts to put some substance behind those billion dollar projections.
Advanced Melanoma Screening and Detection
Researchers at the University of Michigan are putting advanced image recognition to work, detecting one one of the most aggressive, but treatable in early stages, types of cancer. Melanoma can not only be deadly, but it can also be difficult to screen accurately. The team trained a neural network to isolate features (texture and structure) of moles and suspicious lesions for better recognition. The team says “the experimental results of qualitative and quantitative evaluations demonstrate that the method can outperform other state-of-the-art algorithms” for detecting melanoma known to date.
Neural Networks for Brain Cancer Detection
A team of French researchers note that spotting invasive brain cancer cells during surgery is difficult, in part because of the effects of lighting in operating rooms. They found that using neural networks in conjunction with Raman spectroscopy during operations allows them to detect the cancerous cells easier and reduce residual cancer post-operation. In fact, this piece is one of many over the last few weeks that matches advanced image recognition and classification with various types of cancer and screening apparatus–more in the short list below.
Machine Learning for Ultrasound Images, Pre-Natal Care
A collaborative team of researchers from the UK and Australia have applied image recognition and machine learning techniques to automatically interpret signs of fetal distress and to guide pre-operative strategies to mitigate potentially unhealthy conditions in the womb. Although limited by limited training sets for the neural networks, this research shows promise for further exploration, according to the authors.
Weather Forecasting and Event Detection
This traditional area for large-scale supercomputers is now becoming a hotbed for neural network development, particularly when it comes to weather event (pattern) detection. In one such use case, computational fluid dynamics codes are matched with neural networks and other genetic algorithm approaches to detect cyclone activity.
Energy Market Price Forecasting using Neural Networks
Researchers in Spain and Portugal have applied artificial neural networks to the energy grid in effort to predict price and usage fluctuations. The daily and intraday markets for the region are organized in a daily session where next-day sale and electricity purchase transactions are carried out and in six intraday sessions that consider energy offer and demand, which may arise in the hours following the daily viability schedule fixed after the daily session. In short, being able to make adequate predictions based on the patterns of consumption and availability yields to far higher efficiency and cost savings. More on how this model was put together and deployed here.
More on energy and wind generator prediction models can also be found in this paper, as well asthis one for the Canadian energy market, both published last week. Yet another, also published this week, does similar work in determining load and balancing for hybrid power facilities.
Neural Networks in Space Mission Efforts
An Italian team of researchers focused on CubeSats (a new category of space systems for missions in low Earth orbit) face several technical challenges on a number of different fronts. Their research focuses on the “attention on event detection capabilities, with the intent of enabling autonomous operations for a nanosatellite mission by presenting an artificial intelligence algorithm based on neural network technology, and applies it to a future mission used as a case study.” This is, as it sounds, a particularly complex, dense paper with a lot of unknowns, but worth a read to see how neural networks are being considered to solve optimization and other problems.
Neural Networks in Finance
Futures markets have seen a phenomenal success since their inception both in developed and developing countries during the last four decades. This success is attributable to the tremendous leverage the futures provide to market participants. This study analyzes a trading strategy which benefits from this leverage by using the Capital Asset Pricing Model (CAPM) and cost-of-carry relationship. The team applies the technical trading rules developed from spot market prices, on futures market prices using a CAPM based hedge ratio. Historical daily prices of twenty stocks from each of the ten markets (five developed markets and five emerging markets) are used for the analysis. Popular technical indicators, along with artificial intelligence techniques like neural networks and genetic algorithms, are used to generate buy and sell signals for each stock and for portfolios of stocks.
Trading and risk management are two areas where we would expect to see developments for neural networks. Also of note this last week was the application of neural networks to predict corporate bankruptcies (matched against other predictive approaches). Another interesting piecethis week looks at using neural networks to determine much larger-scale banking and financial health.
Neural Networks in Civil and Mechanical Engineering
This study from a team in Indonesia utilizes artificial neural networks to predict structural response (story drift) of multi-story reinforced concrete building under earthquake load in the region of Sumatera Island. Modal response spectrum analysis is performed to simulate earthquake loading and produce structural response data for further use in the ANN. The ANN architecture comprises of 3 layers: an input layer, a hidden layer, and an output layer. Earthquake load parameters from 11 locations in Sumatra Island, soil condition, and building geometry are selected as input parameters, whereas story drift is selected as output parameter for the ANN. As many as 1080 data sets were used to train the ANN and 405 data sets for testing. The trained ANN is capable of predicting story drift under earthquake loading at 95% rate of prediction and the calculated Mean-Squared Errors (MSE)
Also for civil engineers and city planning purposes, neural networks are being deployed to helppredict traffic speed conditions in various settings and another that looks at classification patterns for traffic accidents.
There are quite a few more examples to highlight, but these were some we handpicked to showcase the diversity of applications. Below is a yet another pared-down list of other application areas for neural networks all from the last couple of weeks of published research.
Biological/Earth Sciences
Estimation of Chlorophyll Concentration Index at Leaves using Artificial Neural Networks
Background Categorization for Automatic Animal Detection in Aerial Videos Using Neural Networks
Solar radiation prediction using fuzzy logic and neural networks
Automated Detection of Deep-Sea Animals
Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
Prediction of Water-Level in the Urmia Lake Using the Extreme Learning Machine Approach
Electronics, Sensors, Equipment
Materials, Manufacturing, and Industry
Intelligent integrated optimization of mining and pre-dressing grades in metal mines
Prediction of Dust Dispersion by Drilling Operation Using Artificial Neural Networks
Robust Adaptive Voltage Control of Electric Generators for Ships
Sociology, Psychology, and the Humanities
The application of artificial neural networks in predicting children’s giftedness
Short Story Popularity Prediction using Neural Networks with Time Series-Based Circular Dependencies
Additional Medical Advancements using Neural Networks
Quantifying Radiographic Knee Osteoarthritis Severity using Deep ConvolutionalNeural Networks
Again, remember that this is not a comprehensive list, but that it is notable in that there have been so very many new additions to the base of literature from many disciplines added in just the last few weeks. Just one year ago, we pulled the hype hat over our eyes to some extent–after all, this was most useful in tagging images on social sites and getting machines to paint pictures. The potential for higher purposes was there (the supercomputing world is seeing it too) but just beyond reach.
We are, it is safe to say, at the real beginning of mainstream applications for deep learning.
