Paperless-ngx: Building a Document Management System That Actually Works
Paper has a way of piling up — utility bills, insurance documents, tax records, medical forms. Scanning them into a folder structure solves the physical problem but not the organizational one: you still end up with scan_2024_03_final_REAL.pdf in a directory tree you half-remember.
Paperless-ngx solves this. It’s a document management system that ingests PDFs and images, runs OCR to make them full-text searchable, and lets you tag, classify, and find anything instantly. This guide covers deploying it in Docker Compose, building an automated scanning pipeline, and configuring the features that make it genuinely useful.
What Paperless-ngx Does
Paperless-ngx is a fork of the original Paperless project, actively maintained with a modern React UI. Its core loop is simple:
- Consume — watch an inbox folder (or receive email) for new documents
- OCR — extract text with Tesseract so PDFs become searchable
- Classify — auto-assign tags, correspondents, document types, and storage paths via rules
- Store — archive the original file in a structured layout
The result is a web interface where you can search for “electricity bill 2023” and find the exact document in milliseconds, regardless of what you named the file.
Architecture
Paperless-ngx runs as several cooperating containers:
- webserver — Django application + React frontend
- worker — Celery task queue for OCR and classification
- broker — Redis for the task queue
- db — PostgreSQL for metadata storage
- gotenberg (optional) — converts Office documents to PDF
- tika (optional) — extracts text from complex document types
For most home users, the base four containers are sufficient.
Docker Compose Setup
|
|
Note: The
./consumedirectory is where you drop new documents — Paperless-ngx watches it and processes anything that appears there. The./exportdirectory is for the built-in backup/export command.
Environment File
|
|
First Start
|
|
Visit http://localhost:8000 and log in. The dashboard will be empty — let’s start feeding it documents.
The Consume Pipeline
Paperless-ngx processes documents placed in the consume directory. Understanding the full pipeline helps you tune it for your needs.
Consume Directory Watching
Drop any PDF, image (JPG, PNG, TIFF), or even a ZIP of images into ./consume/ and within seconds Paperless-ngx detects and begins processing it:
- File is detected by inotify (or polling on non-Linux systems)
- File is moved to a temporary processing directory
- OCR runs if the document lacks a text layer (or always, depending on
PAPERLESS_OCR_MODE) - Classification rules execute against the document
- Document is stored in the media directory
- Metadata is written to PostgreSQL
- The consume file is deleted (or moved to
PAPERLESS_CONSUMPTION_DIR_TRASH_DIR)
Directory-Based Tag Assignment
You can pre-assign tags based on the subdirectory a file is placed in:
|
|
Enable this with:
|
|
This is useful when scanning from a device (like a Brother scanner) that can send to specific directories based on a scan profile.
Email Consumption
Paperless-ngx can pull documents from an IMAP mailbox — useful for e-bills and statements sent as PDF attachments:
|
|
Configure email accounts under Settings → Mail in the UI. Each mail rule specifies:
- Which folder to watch
- Which senders/subjects to match
- What to do with the email after processing (delete, mark read, move)
- Tags to assign to matched documents
For example: match all emails from billing@electricity-company.com, attach the PDF, apply tag “utilities”, and mark the email as read.
Organizing with Tags, Correspondents, and Document Types
Paperless-ngx has four classification dimensions:
| Dimension | Example | Purpose |
|---|---|---|
| Tags | tax, important, 2024 |
Flexible multi-label classification |
| Correspondent | IRS, Blue Cross, State Farm |
Who sent the document |
| Document Type | Invoice, Statement, Contract |
What kind of document |
| Storage Path | Finance/Tax/{year} |
Where on disk to store it |
The power comes from automation rules that assign these based on content matching.
Automation Rules
Navigate to Settings → Document Classification to create rules. Each rule has:
- Name: descriptive label
- Order: rules run in order; lower numbers first
- Sources: apply to consumed documents, API uploads, or both
- Filter: match on filename, correspondent, document type, tag, or full-text content
- Assignment: what to set when the rule matches
Example: Auto-Tag Utility Bills
Rule: Electric Company Statements
- Filter: match “from correspondents” →
Pacific Gas & Electric - Assign tag:
utilities - Assign document type:
Statement - Assign storage path:
Utilities/Electric/{created_year}
Rule: IRS Documents
- Filter: match “if content contains” →
Internal Revenue Service - Assign correspondent:
IRS - Assign tag:
tax - Assign document type:
Government - Assign storage path:
Tax/{created_year}
Example: Date Extraction
Paperless-ngx can extract the document date from content rather than using the file date:
|
|
Alternatively, embed the date in the filename using the format YYYY-MM-DD:
2024-01-15 Electric Bill.pdf
Paperless-ngx will parse this automatically.
Custom Consumption Scripts
For advanced pre-processing, place scripts in the consume directory or use the PAPERLESS_PRE_CONSUME_SCRIPT and PAPERLESS_POST_CONSUME_SCRIPT environment variables:
|
|
The post-consume script receives the document ID as its first argument, allowing you to trigger webhooks, send notifications, or integrate with other systems:
|
|
Scanner Integration
Brother Scanners (Scan to FTP/Network)
Many Brother scanners support scanning to a network folder. Point it at your consume directory via SMB or NFS:
|
|
Create scan profiles on the scanner for different destinations (e.g., “Tax Docs” → consume/tax/, “Medical” → consume/medical/).
iPhone/Android with Scanning Apps
Apps like Adobe Scan, Microsoft Lens, or Genius Scan can save directly to a network folder or cloud location. The workflow:
- Scan document with phone app
- App saves to Nextcloud/Dropbox folder
- Rclone or Nextcloud sync copies to consume directory
- Paperless-ngx processes automatically
Dedicated Scanning Hardware
A dedicated document scanner (ScanSnap, Fujitsu ix500, Brother ADS series) with automatic document feeder makes bulk scanning practical. Configure it to save to the consume directory directly over SMB.
For the ScanSnap series, the ScanSnap Cloud service can route documents to Dropbox/Google Drive, which you then sync to your consume directory.
Storage and File Organization
Filename Format
The PAPERLESS_FILENAME_FORMAT variable controls how archived files are named. Use these variables:
| Variable | Description |
|---|---|
{title} |
Document title |
{correspondent} |
Correspondent name |
{document_type} |
Document type |
{created} |
Full creation date (YYYY-MM-DD) |
{created_year} |
Year only |
{created_month} |
Month only |
{added} |
Date added to Paperless-ngx |
{asn} |
Archive serial number |
{tags} |
Comma-separated tags |
A useful format for tax documents:
|
|
This produces paths like:
2024/IRS/Tax Return/2024-04-15-1040 Tax Return.pdf
Archive Serial Numbers
Enable ASNs to give each document a permanent physical label:
|
|
Print the ASN on a sticky label and attach it to the physical document before filing. Later, searching by ASN instantly retrieves the digital version — useful for documents you need to keep physical copies of.
Storage Path Objects
Create named storage paths under Settings → Storage Paths. These support the same variables as PAPERLESS_FILENAME_FORMAT and can be assigned via automation rules:
- Finance:
Finance/{created_year}/{correspondent}/{title} - Medical:
Medical/{created_year}/{correspondent}/{title} - Home:
Home/{correspondent}/{created_year}/{title}
Documents without an assigned storage path use PAPERLESS_FILENAME_FORMAT as the fallback.
OCR Configuration
Language Configuration
For multilingual households or businesses:
|
|
OCR language data must be installed. In the Docker image, additional languages are available as packages:
|
|
OCR Mode
| Mode | Behavior | Use case |
|---|---|---|
skip |
Skip OCR if text layer exists | Fastest; use for already-searchable PDFs |
skip_noarchive |
Skip OCR but still create archive copy | Good default |
redo |
Re-OCR everything regardless | Fix existing documents |
force |
Force OCR even with text layer | Correct bad existing OCR |
|
|
Image Preprocessing
For documents with skewed text or poor quality:
|
|
PDF/A Output
Converting documents to PDF/A ensures long-term archival compatibility:
|
|
PDF/A embeds all fonts, color profiles, and metadata — documents remain readable without the original software that created them.
Backup Strategy
Document Export
The built-in export command creates a portable archive of all documents and metadata:
|
|
The export includes:
- All original files
- A
manifest.jsonwith all metadata - Can be re-imported to a fresh Paperless-ngx instance
Database Backup
|
|
Add to cron:
|
|
Offsite Replication with Rclone
|
|
Restoring from Backup
|
|
Traefik Integration
For HTTPS via Traefik:
|
|
Remove the ports: mapping from the webserver service — Traefik handles routing.
Advanced Configuration
Custom Classifier Training
Paperless-ngx uses a Naive Bayes classifier to automatically suggest correspondents, document types, and tags based on document content. It trains automatically as you manually classify documents.
To improve accuracy:
- Manually classify all existing documents consistently
- Use specific correspondent names rather than generic ones (“Pacific Gas & Electric” not “Utility”)
- Give the classifier time — it improves significantly after 50-100 classified documents
Trigger retraining manually:
|
|
API Access
The REST API enables programmatic access:
|
|
The API is useful for:
- Integrating with home automation (Home Assistant can upload documents automatically)
- Bulk uploading historical archives
- Building custom mobile upload workflows
Webhook Notifications (via Post-Consume Script)
Send a notification to a Home Assistant webhook when a new document arrives:
|
|
|
|
Full-Text Search Tuning
Paperless-ngx uses PostgreSQL’s full-text search. For better multilingual search, ensure you’ve set the right search stemming:
|
|
For non-English primary languages, the full-text search configuration in PostgreSQL may need adjustment for optimal stemming.
Performance Tuning
Worker Configuration
OCR is CPU-intensive. Tune based on your hardware:
|
|
TASK_WORKERS controls how many documents are processed concurrently. Each worker can use multiple threads for page-parallel OCR.
Redis Memory
For large document collections, Redis may need more memory:
|
|
PostgreSQL Tuning
For collections over 10,000 documents:
|
|
Migrating Existing Documents
Bulk Import
For importing an existing folder of PDFs:
|
|
For very large archives, rate-limit the copy to avoid overwhelming the OCR queue:
|
|
From Evernote
Export notebooks from Evernote as .enex files, then use the evernote2paperless tool:
|
|
From DevonThink
Export as PDFs from DevonThink, then bulk-import. DevonThink exports preserve original file names which Paperless-ngx uses as initial titles before classification.
From a Folder Structure
If you have an existing Year/Category/Document.pdf structure, use a script to pre-tag based on path:
|
|
Maintenance
Updating Paperless-ngx
|
|
Always read the release notes before major version upgrades — some versions require manual migration steps.
Sanity Checks
|
|
Cleaning Up Orphaned Files
Occasionally files in the media directory may not correspond to database records:
|
|
Security Considerations
Network Exposure
Paperless-ngx contains sensitive documents — do not expose it directly to the internet without authentication. Options:
- VPN-only access: Only accessible from Tailscale/WireGuard network
- Traefik with forward auth: Require Authentik/Authelia SSO before reaching Paperless-ngx
- Cloudflare Access: Put it behind Cloudflare Zero Trust
Admin Panel Security
The Django admin panel at /admin/ has broader database access than the Paperless-ngx UI. Restrict it:
|
|
Document Sensitivity Labels
Use tags like confidential, sensitive, or medical to identify high-sensitivity documents. Paperless-ngx doesn’t currently support per-document ACLs (beyond multi-user ownership in newer versions), but tags help you track what needs extra care.
Multi-User Support
Paperless-ngx v1.14+ supports proper multi-user access with ownership and permissions:
- Documents can be owned by a specific user
- Share documents with other users or groups
- Set default visibility (owner-only vs. any logged-in user)
Configure under Settings → User Management.
The Workflow in Practice
After setup, the day-to-day workflow becomes:
- Receive paper document → scan with dedicated scanner or phone app → drops into consume directory automatically
- Receive email with PDF → Paperless-ngx mail rule picks it up → processed automatically
- Download PDF statement → save to consume directory or upload via web UI
- Find any document later → full-text search finds it in under a second
The classification rules do the heavy lifting. After training the classifier with a few hundred documents, new documents arrive pre-tagged with the right correspondent, type, and storage path — you just verify and click Save.
Practical Tips
Start with classification rules early: The more rules you define upfront, the less manual tagging you’ll do on bulk imports.
Use consistent correspondent names: “Pacific Gas & Electric” and “PG&E” are different correspondents to Paperless-ngx. Pick one and stick to it.
Tag with years for tax season: A 2024-taxes tag makes pulling all relevant documents for your accountant trivial.
Create a to-review tag: New documents that need attention get this tag automatically. Reviewing and removing it becomes a weekly habit.
Archive serial numbers for important documents: Anything you might need to physically retrieve (deeds, contracts, birth certificates) should have an ASN label so you can find both the physical and digital copies instantly.
Schedule regular exports: The built-in exporter creates a portable archive that can be read without Paperless-ngx. Run it monthly as insurance against data loss or migration needs.
Going paperless isn’t just about saving space — it’s about having instant access to any document ever received, fully text-searchable, with metadata that makes sense. Paperless-ngx makes that achievable on commodity homelab hardware with a scanning workflow that takes less time than filing the physical document ever did.
Comments