Accelerate Pandas with GPU Using RAPIDS cuDF

...without any code changes.

Advertise to 76k readers | Deep Dives

Two of the biggest problems with Pandas is that:

• It always adheres to a single-core computation on a CPU.

• It creates bulky DataFrames.

While many libraries (Polars, for instance) do address these limitations, they are still limited to CPU-driven computations.

NVIDIA’s RAPIDS cuDF library allows Pandas users to supercharge their Pandas workflow with GPUs.

How to use it?

Within a GPU runtime, do the following :

• Load the extension: %load_ext cudf.pandas

• Import Pandas: import pandas as pd

• Done! Use Pandas’ methods as you usually would.

Just loading the extension provides immense speedups. This is evident from the gif below.

As per NVIDIA’s official release, this can be as fast as 150x.

In my personal experimentation, however, I mostly observed it to range between 50-70x, which is still pretty good.

The good thing is that the extension accelerates most Pandas’ methods.

Yet, if needed, it can automatically fall back to the CPU.

How does it work?

Whenever cudf.pandas is enabled, the import pandas as pd statement does not import the original Pandas library which we use all the time.

Instead, it imports another library that contains GPU-accelerated implementations of all Pandas methods.

This is evident from the image below:

This alternative implementation preserves the entire syntax of Pandas. So if you know Pandas, you already know how to use cuDF’s Pandas.

Isn’t that cool?

You can find the code here: Google Colab.

To learn CUDA programming from scratch, check this: Implementing (Massively) Parallelized CUDA Programs From Scratch Using CUDA Programming.

👉 Over to you: What are some other ways to accelerate Pandas operations in general?

---

• 1 Referral: Unlock 450+ practice questions on NumPy, Pandas, and SQL.

• 2 Referrals: Get access to advanced Python OOP deep dive.

• 3 Referrals: Get access to the PySpark deep dive for big-data mastery.

Get your unique referral link:

---

Are you preparing for ML/DS interviews or want to upskill at your current job?

Every week, I publish in-depth ML deep dives. The topics align with the practical skills that typical ML/DS roles demand.

Join below to unlock all full articles:

Here are some of the top articles:

• [FREE] A Beginner-friendly and Comprehensive Deep Dive on Vector Databases.

• Implementing Parallelized CUDA Programs From Scratch Using CUDA Programming

• Understanding LoRA-derived Techniques for Optimal LLM Fine-tuning

• 8 Fatal (Yet Non-obvious) Pitfalls and Cautionary Measures in Data Science.

• 5 Must-Know Ways to Test ML Models in Production (Implementation Included).

• A Beginner-Friendly Guide to Multi-GPU Model Training.

• 11 Powerful Techniques To Supercharge Your ML Models.

• Don’t Stop at Pandas and Sklearn! Get Started with Spark DataFrames and Big Data ML using PySpark.

Join below to unlock all full articles:

---

👉 If you love reading this newsletter, share it with friends!

👉 Tell the world what makes this newsletter special for you by leaving a review here :)