← back

Missing the M(L)ark

Created on: 2024-02-13

This article has the following ideas at its core:

The first is a fairly trivial observation occurring in all life aspects.

A story to showcase what happens when you put the technology before the horse

Recently, I wanted to generate automatic notifications whenever a deposit with certain characteristics arrived at my checking account. My bank does not provide an open API, so some tinkering would be involved.

After poking around my bank’s website, it dawned on me that if I could log in, it was fairly straightforward to download a CSV of the most recent transactions. With that CSV coupled with a few lines of Python + Pandas, I should have been able to generate the desired report. The question now was how to log into my bank to download the report automatically.

To achieve this, you need to automate your browser, often achieved using a tool such as Puppeteer or Selenium. These tools are not hard to learn but very hard to maintain. Particularly because they depend on the particular layout of a website, something very easy to change. Moreso, if on the other end is a bank that actively does not want you to log in automatically (most likely to prevent fraud).

Long story short:

The funny thing is that logging into my bank and downloading the CSV takes about 1 minute on my laptop. That meant I could manually download the file once daily for a year, and I would still have used 10x less time.

There are several problems with the situation described above. The first one is that the problem that needed solving was ill-defined:

What could have been done differently?

This was a clear case where the technology (automatic browsing of the web) and failure to define clear requirements (how often does this need to run) transformed what could have been a simple yet powerful solution into a waste of time and effort.

Editor's note: the solution has not been used since the last fight lost to the bank. Kudos to them!

Same same but different?

The other key point of this article is that similar problems in Machine Learning might look similar but be very different in complexity.

As a side note, when I lived in Paris, it always fazed me how two neighboring bars that looked exactly alike could have a 100% difference in their beer price. It was always a roulette: will you pay $4 or $9 for your pint?

Back to ML, let's examine the problem: extract key personal information from a text file. That I can define the problem this way hints at how close the two problems are: they fall under the exact "definition.”

Let’s get concrete: the two key pieces of information we need to extract are phone numbers and last names. The following example provides an instance of this problem:

It was the year 1984 when Jane Smith was born at Very Bright Hospital in Madrid. Since then, she has started her agency, which you can reach at (555)-123-456. The agency is doing remarkably well with 3 employees and generating $1000000 in revenue.

Now, how do we solve the above problem? Those with a background in Computer Science will recognize the phone number as the canonical example for regular expressions. Regular Expressions are a non-ML tool that can be used to find patterns in text. In this case, we are looking at the pattern (DDD)-DDD-DDD, where D is a digit between 0 and 9. We can even omit the parenthesis and the dashes and search for DDDDDDDD. This runs into the problem of getting confused with the revenue figure, but we can solve that problem by asking that the character before the first D is not a “$.” This is A SINGLE LINE OF CODE for someone in the knowhow.

Take the other problem instead: how do we know if Smith is a last name? We might first think about using the fact that the first letter is uppercase, but that also finds Jane. We might also think that we can count consecutive words that start with uppercase, and if it’s the second, then it’s a last name. That doesn't work, either. In our example, Bright also fits that rule.

In fact, you might realize that we can only tell if something is a last name by the context in which it was written. Understanding context in natural language needs Natural Language Processing (NLP), a subdiscipline of ML. That makes the problem 100x harder (or more).

There is an uncountable number of similar problems that look similar, and yet one is easy, and the other problem is very hard: when finding items on an image, the size of the object can change the difficulty of the problem drastically.

Wrapping Up

To conclude, whenever you think you have a problem that needs to be solved with ML, do two things:

  1. Define the problem in terms of the requirements, independently of the technology.
  2. Consult an expert before spending 9999+ hours on a very hard problem when a similar and much simpler solution could solve your original problem.

P.S. The main idea of this article is very related to the XY problem. If you have never read before about it, you can do so here.