How to Process 10k Images in Seconds | by Florian Trautweiler | Jan, 2025

Editor
1 Min Read


Efficient image operations with multiprocessing in Python

Dataset for Image Processing

Manual, repetitive tasks. Egh. One of the things I hate the most, especially if I know they can be automated. Imagine you need to edit a bunch of images with the same cropping and resizing operation. For a couple of images you might just open an image editor and do it by hand. But what about doing the same operation for a thousands or tens of thousands of images? Let’s see how we can automate such an image processing task with Python and OpenCV, as well as how we can optimize this data processing pipeline to run efficiently on a sizeable dataset.

For this post, I created a toy example where I extracted 10,000 frames from a random video of a beach I recorded, where the goal is to crop the image to a square aspect ratio around the center and then resize the image to a fixed size of 224×224.

This roughly resembles part of a pre-processing step that might be required for a dataset when training a machine learning model.

Illustration of Crop and Resize Operation
Share this Article
Please enter CoinGecko Free Api Key to get this plugin works.