Why Don't We Invoke model.forward() in PyTorch?

Feb 21, 2025

Announcement: This Saturday (22nd Feb 2025), we shall not release a new issue. You'll hear next from us on Monday (24th Feb 2025).

In PyTorch, we never invoke the forward() method to run the forward pass:

Instead, we always invoke the model (which is a class object) as if it was a function:

We can also verify that model is a class object:

So how can a class object be invoked like a function?

Let’s understand!

Consider a quadratic class, which has an evaluate method:

While this approach is fine, there is a better way to implement it by defining the __call__() magic method.

This magic method allows you to define the behavior of the class object when it is invoked like a function (like this: object()).

For instance, by renaming evaluate() to __call__(), we can invoke the class object directly instead of explicitly invoking a method:

This way, we can use objects flexibly and intuitively.

This is precisely what happens in PyTorch.

PyTorch implicitly declares the __call__() method in the model class, and the forward method is invoked from it:

This way, we get to invoke the model object like a function—model().

We can verify that we get the same output in all three ways:

And that is why we don’t invoke the model.forward() method in PyTorch?

We covered such advanced OOP stuff in detail in a recent deep dive here if you wish to level up your OOP skills: Object-Oriented Programming with Python for Data Scientists.

👉 Over to you: What are some other cool Python OOP tricks?

Thanks for reading and we’ll see you next week!

At the end of the day, all businesses care about impact. That’s it!

We have discussed several other topics (with implementations) that align with such topics.

Here are some of them:

Learn sophisticated graph architectures and how to train them on graph data: A Crash Course on Graph Neural Networks – Part 1.
So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here: Bi-encoders and Cross-encoders for Sentence Pair Similarity Scoring – Part 1.
Learn techniques to run large models on small devices: Quantization: Optimize ML Models to Run Them on Tiny Hardware.
Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust: Conformal Predictions: Build Confidence in Your ML Model’s Predictions.
Learn how to identify causal relationships and answer business questions: A Crash Course on Causality – Part 1
Learn how to scale ML model training: A Practical Guide to Scaling ML Model Training.
Learn techniques to reliably roll out new models in production: 5 Must-Know Ways to Test ML Models in Production (Implementation Included)
Learn how to build privacy-first ML systems: Federated Learning: A Critical Step Towards Privacy-Preserving Machine Learning.
Learn how to compress ML models and reduce costs: Model Compression: A Critical Step Towards Efficient Machine Learning.

All these resources will help you cultivate key skills that businesses and companies care about the most.

Daily Dose of Data Science