After 6+ years of experience with machine learning, both in research and industry, it’s very interesting to see how far the field has gotten over the years. I still remember sitting in seminar rooms, listening to lectures on everything ML: deep learning, reinforcement learning, random forests, neural networks, natural language processing, …
One particular NLP lecture stands out in my memory, where we discussed the rapid advancements in the field. We were discussing vanilla attention mechanisms the week before and were now looking at Transformer-based approaches. The great tutor showed us graphs with the parameter counts of the models. We then looked up the then-recent advances, and it became clear: any figure is outdated within a month. That was when my ML journey had barely started, and already the amount of innovation that was published again and again was insane.
Since then, my journey proceeded in several phases. From exchange with other ML people, it appears that they have a similar experience: From a beginner to seasoned practitioner, the journey can be divided into the three following phases:
Phase I: Beginner (~1 year; this article)
Phase II: Intermediate (~1 to 3 years; upcoming)
Phase III: Advanced (~5+ years; upcoming)