Better accuracy, smaller models, less data - enabled by perforated learning

Introduction

Perforated AI (PAI) is a PyTorch library that adds artificial dendrites to your neural network - the same branching structures that give real biological neurons their computational power. PAI restructures your network during training and only requires minimal code changes. Perforated AI is part of the PyTorch Landscape.

📊 Key Results

• Up to 70% accuracy improvements on challenging tasks
• Up to 90% compression without sacrificing accuracy
• 50% reduction in data requirements for similar accuracy scores

These results come from the full Perforated Backpropagation™ system, available in the [full Perforated AI Suite][contact-us]. [Resources][resources] and our published paper include full benchmark methodology.

🚀 Quick Start

Requirements: Python 3.7+ · PyTorch 1.9+

pip install perforatedai

Integrate PAI into your training script with these five steps:

from perforatedai import globals_perforatedai as GPA
from perforatedai import utils_perforatedai as UPA
import torch

model = YourModel()

# 1. Wrap your model after initialization
model = UPA.perforate_model(model)

# 2. Register your optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
GPA.pai_tracker.set_optimizer_instance(optimizer)

# 3. Run until dendritic optimization completes (replace your training loop with while True)
while True:
    train(model, optimizer)
    val_score = validate(model)

    # 4. Report validation score and handle model restructuring
    model, restructured, training_complete = GPA.pai_tracker.add_validation_score(val_score, model)
    model = model.to(device)  # re-send to device if PAI restructured the model

    # 5. React to PAI's outcome - exit when done, or reset optimizer after restructuring
    if training_complete:
        # Best model has been saved - training is done!
        break
    elif restructured:
        # Model was restructured with new dendrites - reinitialize optimizer (and scheduler) using your initial setup
        optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
        GPA.pai_tracker.set_optimizer_instance(optimizer)

Need help with integration? See the API documentation for full integration details, including how to use our Claude skill or MCP server for AI coding assistants to get instant guidance.

💡 Examples

PAI works with many popular architectures and frameworks:

Base Examples

• MNIST - Classic computer vision with dendritic enhancement
• TD3 Reinforcement Learning - Continuous control with dendrites

Advanced Examples

• ImageNet ResNet-18 - ResNet-18 with dendritic optimization (pretrained model on Hugging Face)
• Edge Impulse Block - Keyword spotting in audio; across 800 hyperparameter sweeps, dendritic models were consistently more accurate at every parameter count

Framework Integration

• PyTorch Lightning - Using PAI with popular training frameworks
• Hugging Face - Transformer models with dendritic layers

🧠 How It Works

Traditional artificial neurons are point neurons - they simply sum weighted inputs and apply an activation function. Real biological neurons have dendrites - branching structures that perform sophisticated computations before signals reach the cell body.

Perforated AI adds dendritic modules to artificial neurons:

1. Automatic Perforation: PAI analyzes your network and perforates it with dendrites for selected modules
2. Validation-Driven Growth: Identifies when to add dendrites based on validation score improvements
3. Optimized Training: Automatically adjusts learning rates for optimal dendritic optimization
4. Multi-Layer Support: Unlike other dendritic methods, PAI works on multiple neuron layers simultaneously

This approach delivers:

• Significant accuracy improvements on challenging tasks
• Superior parameter efficiency compared to standard scaling
• Reduced data requirements for achieving target performance
• Robustness to noise - particularly effective in real-world conditions
• Drop-in compatibility - works with existing PyTorch code

See our Papers directory for detailed comparisons with other dendritic learning methods.

📚 Documentation

💎 Need more? Consider the full Perforated AI Suite

This repository is the open-source version of Perforated AI under Apache 2.0 - the dendritic architecture, trained with standard backpropagation. Free to use, fully self-serve.

For larger performance gains and a smoother setup experience, we also offer a commercial version that adds:

• Perforated Backpropagation™ - our patented training algorithm (replaces standard backprop) that produces our headline accuracy, compression, and data-efficiency results
• Perforated Studio - a GUI for configuring runs and inspecting results, so you can get up and running without hand-rolling the integration in code

Get in touch to learn more about the commercial version.

🤝 Contributing

We welcome all contributions from the community!

Adding Examples:

• Follow the MNIST example template
• Include before/after results with visualizations
• Provide complete running instructions
• Provide baseline code as NAME.py and perforated code as NAME_perforated.py

Reporting Issues:

• GitHub Issues
• [email protected]

Modifying Code:

• Describe what the change does and the benefit it provides
• Use Black Python formatting
• Include comments within code to describe processes

💬 Community

• 💬 Discord - Get help and share results
• 📧 Newsletter - Stay updated on dendritic AI research
• 🤝 LinkedIn - Follow our latest developments
• 🤗 Hugging Face - Try our pretrained models on your datasets

Citation

If you use Perforated AI in your research, please cite:

@article{brenner2025perforated,
  title={Perforated Backpropagation: A Neuroscience Inspired Extension to Artificial Neural Networks},
  author={Brenner, Rorry and Itti, Laurent},
  journal={arXiv preprint arXiv:2501.18018},
  year={2025}
}

Python Version Support

We support Python 3.7+ and PyTorch 1.9+. We are committed to supporting Python versions for at least six months after their official end-of-life (EOL) date.

License

Apache License 2.0

Alternative Training Mechanisms

Not a Contribution. If you would like to get additional performance boosts from dendritic architectures through Perforated Backpropagation^TM please get in touch at perforatedai.com. Details on this approach can be found in our original paper. This open source code does not include the perforatedbp library and the perforated_backpropagation variable is set to False so the functions of that library will not be called without a license. The Perforated Backpropagation libraries and functionality are not part of this release, are not a contribution to this release, and are not released under any open source license.