No, We Need a Neural Network tells the story of a computer science professor, referred to as M.A., asked to develop a neural net for something that most certainly did not require, or benefit from, such technology:
M.A’s first few weeks were spent analyzing the existing system and its processes. Water testing equipment throughout the region sent data in a variety of different formats to the central processing server. The data was logged in a single database table with three columns: Station Number, Recording Date, and Raw Data. Each week, a team of seven technicians would spend a day manually parsing the Raw Data — be it in CSV, fixed-width, XML, etc. — and inserting the formatted data into another database table. A script would then parse the standardized data and create the weekly water reports.Management wanted to create a Neural Network that would learn how to parse the raw data and free up the technicians so they could do their regular job. Armed with all the details of the process, M.A. explained that this was a very bad idea. He argued that:
- Neural Networks are not designed to handle tasks like this
- If developed, such a Neural Network would need a lot of training
- An algorithmic version could be developed in a fraction of the time and do the job better
Management disagreed. They insisted that Neural Networks were the wave of the future and wanted M.A. to go forward with developing it. He grudgingly agreed and got to work designing the system.
[...]
Nearly a year later, the system was complete. And it was a horrible mess. It required a server cluster to run the analysis, produced and executed horribly ugly and slow code at runtime, consumed a ridiculous amount of RAM, and needed at least a full week of training for each node. But it worked. Before heading back to academia, M.A. tried one last time to convince them to drop the neural network. They wouldn’t budge.Fast forward three years, and the programming firm was bought and given a completely new management team. The new CTO got wind of the behemoth that M.A. created and called him up to find out the reason behind such an “eccentric” implementation. M.A. explained everything and even forwarded the CTO a proposal that he created for a traditional solution long before starting the neural network.
The new CTO was more than pleased and wanted to bring M.A. in once again to develop the traditional solution. A few weeks later — and after receiving another generous paycheck — the new solution runs on a single server and is able to calculate, alert, and report on any issues within two minutes of a station updating their log file.