This article will explain how to use Pipeline and Transformers correctly in Scikit-Learn (sklearn) projects to speed up and reuse our model training process.
This piece complements and clarifies the official documentation on Pipeline examples and some common misunderstandings.
I hope that after reading this, you’ll be able to use the Pipeline, an excellent design, to better complete your machine learning tasks.
There’s a famous dish in Chinese restaurants around the world called “General Tso’s Chicken,” and I wonder if you’ve tried it.
One characteristic of “General Tso’s Chicken” is that each piece of chicken is processed by the chef to be the same size. This ensures that:
- All pieces are marinated for the same amount of time.
- During cooking, each piece of chicken reaches the same level of doneness.
- When using chopsticks, the uniform size makes it easier to pick up the pieces.
This preprocessing includes washing, cutting, and marinating the ingredients. If the chicken pieces are cut larger than usual, the flavor can change significantly even if stir-fried for the same amount of time.
So, when preparing to open a restaurant, we must consider standardizing these processes and recipes to ensure that each plate of “General Tso’s Chicken” has a consistent taste and texture. This is how restaurants thrive.
Back in the world of machine learning, Scikit-Learn also provides such standardized processes called Pipeline. They solidify the data preprocessing and model training process into a standardized workflow, making machine learning projects easier to maintain and reuse.
In this article, we’ll explore how to use Transformers correctly within Scikit-Learn’s Pipeline, ensuring that our data is as perfectly prepared as the ingredients for a fine meal.