Simplify Python Imports with Explicit Packaging

A step towards better Python project development.

Denmark's Gefion: The AI supercomputer that puts society first [NOT SPONSORED]

This week at AIport (a newsletter about global AI news which I often contribute to), we talked about Denmark’s new AI supercomputer, Gefion.

The project intends to serve as a more socially conscious model for investing in AI infrastructure.

From the way it’s been funded to the way it’s powered, Gefion challenges how we might measure “returns” on AI investments—is it short-term profits or long-term societal impact?

Daniel Nest

from the ​Why Try AI newsletter​ (a newsletter dedicated to hype-free and hands-on AI for non-techies) has put together an interesting story as a guest author of ​AIport​ about this topic.

​Read the insightful discussion here →

Let’s get to today’s issue now!

---

Simplify Python Imports with Explicit Packaging

Python lets us package a project by adding an __init__.py file inside a directory.

If you have ever been confused about the internal details of the __init__.py file, today’s issue with help.

Let’s understand!

---

Packaging in Python

Simply put, if a project is packaged, you can import stuff from it.

While Python 3.3+ provides Implicit Namespace Packages — a directory with modules is considered a package by default, it is still advised to create an explicit __init__.py file.

A couple of major benefits of doing this are that it helps in:

• Explicitly specifying which classes/functions can be imported from the package.

• Avoiding redundant imports.

---

Some terminology before proceeding ahead:

• Module: A Python file.

• Package: A collection of Python files in a directory.

• Library: A collection of Packages.

---

Consider this is our directory structure (and we are using Python 3.3+):

• train.py has a Training class.

• test.py has a Testing class.

As we are using Python 3.3+, we can directly import the Training and Testing class in pipeline.py as follows:

While this will work as expected, the problem is that we have to explicitly import the specific class from each of the modules.

This creates redundant imports.

Defining the __init__.py file can simplify this.

Let’s see how.

As depicted above:

• We first explicitly package the directory by creating an __init__.py file.

• Next, we specify the imports directly in this file.

Now, instead of writing redundant imports, you can directly import the intended classes from the “model” package, as shown below:

In other words, specifying the __init__.py file lets you treat your package like a module.

This simplifies your imports.

Also, as discussed earlier, an __init__.py file lets you explicitly specify which classes/functions can be imported from the package, which, otherwise, will not be evident.

This simplifies things for other users of your project.

Isn’t that cool?

👉 Over to you: What are some other cool Python project development tips you are aware of?

Thanks for reading!

---

---

P.S. For those wanting to develop “Industry ML” expertise:

We have discussed several other topics (with implementations) in the past that align with such topics.

Here are some of them:

• Learn how to build real-world RAG apps, evaluate, and scale them: A crash course on building RAG systems—Part 3 (With Implementation).

• Learn sophisticated graph architectures and how to train them on graph data: A Crash Course on Graph Neural Networks.

• Learn techniques to run large models on small devices: Quantization: Optimize ML Models to Run Them on Tiny Hardware

• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust: Conformal Predictions: Build Confidence in Your ML Model’s Predictions.

• Learn how to identify causal relationships and answer business questions: A Crash Course on Causality.

• Learn how to scale ML model training: A Practical Guide to Scaling ML Model Training.

• Learn techniques to reliably roll out new models in production: 5 Must-Know Ways to Test ML Models in Production (Implementation Included)

• Learn how to build privacy-first ML systems: Federated Learning: A Critical Step Towards Privacy-Preserving Machine Learning.

• Learn how to compress ML models and reduce costs: Model Compression: A Critical Step Towards Efficient Machine Learning.

All these resources will help you cultivate key skills that businesses and companies care about the most.

---

SPONSOR US

Get your product in front of 110,000 data scientists and other tech professionals.

Our newsletter puts your products and services directly in front of an audience that matters — thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., who have influence over significant tech decisions and big purchases.

To ensure your product reaches this influential audience, reserve your space here or reply to this email to ensure your product reaches this influential audience.