Developing Machine Learning applications is hard, but not because of the complex math behind them, as most people think. Unless you are doing research on new problems or actively debugging a very tricky model, you don't need much math at all.
Let's first look at why this is the case and then at what makes ML practice hard today.
Back in the day, when ML was becoming a popular field, very little code had been written. We couldn’t access libraries such as scikit-learn, Pytorch, or Keras. If you wanted to try a new method or even sometimes build your neural network, you had to code the thing from scratch. This is a list of things that you might need to know back in the day to use a neural network, in no particular order:
If you took a formal course in ML, you likely had to learn and implement some things by hand. It’s almost a rite of passage, but it can’t be further from the day-to-day practice.
If you are tasked with creating machine learning models, you probably don’t need to write gradient descent from scratch. Instead, you are most likely doing something along the following lines:
In all of the above cases, even though knowing the mathematics behind the models you are using is helpful, it’s not MANDATORY. You can get pretty far without it.
Where does the complexity of ML come from? Software, software, software.
I have fought with this notion several times in my professional career but always come to the same conclusion. Software is hard.
It is one of the most complex abstractions that humans have ever built. It’s hard to make, hard to maintain, and hard to debug (fixing errors).
Software that interacts with the internet is harder because it only works on a “best effort” basis (yes, it does, trust me, it’s insane). That means you need to plan what someone might do all across the planet or consider a scenario you have never seen before but could potentially bring your system down.
In 2024, even my work notes app is an internet application and I can't work without a connection. If your ML model serves customers in real-time, it is most likely an Internet application.
All this to say, creating and maintaining good and reliable software is hard, and machine learning is built on software by adding extra layers. Consequently, we inherit all the problems of building software and then some.
Machine Learning models are massive piles of algebra, and for the most part, why they work is unknown to us. Yes, we have explainable AI, yes, we have shapely values, and yes, we are fairly good at looking at simpler models (linear regressions or regression trees). Yet, I would bet that anyone who claims to know what a given weight of a neural network represents on any decently sized network (10k+ parameters) is outright lying.
That means that when your ML system trained to remove the background from an image suddenly fails to crop a whole region, it’s not as easy as to find "the bug 🐛, which lives in line 324". That behavior is buried in a pile of math, and it will take considerable effort to understand why that happened if you notice it (given that monitoring live systems is its domain with complexities).
We are moving towards more "no-code" ML tools such as Vertex AI, SageMaker, and AutoGluOn. Will we eventually get there? I hope so, but I'm not very optimistic. We have had no-code website builders for quite a while now. I have tried a few and always end up disappointed. They are decent at what they do, but their limited nature applies to specific use cases. I fear that ML will be similar.
Should this discourage you when thinking about building AI within your company? I hope not! But I also would love you to consider your goal.
If it’s a prototype, having a Ph.D. in math might get the job done. You might need more than that if you need a live system to serve thousands of customers.