A document archive for the office: Paperless-ngx on your own server
A contract, a supplier’s invoice, a handover report. In the office they sit in binders or in folders full of files with names like “scan_0412.pdf”. When someone asks how much you paid for the air conditioning service two years ago, the search begins. You only know a file’s name if you named it yourself.
The short answer: an electronic document archive that reads the text from scans by itself and lets you search for any word. One program for this is Paperless-ngx: free, open source, and it runs on your own server, so the documents never leave the company. Below I describe what such a program does, what it needs and what to watch out for when you set it up.
What a document archive gives you
Paperless-ngx takes a file from one chosen folder and turns it into a document you can find. It works in four steps:
- Scan. The scanner or a phone saves a PDF file in the input folder on the server. You can also simply drag a file into that folder.
- Text recognition (OCR). The program reads the image and turns it into text. That way a scanned invoice stops being a “photo of a sheet of paper” and becomes text you can search.
- Index. All the text goes into a search engine. You type an invoice number, a supplier’s name or part of a sentence from a contract, and you get a list of documents.
- Tags and filing. Each document gets a sender (correspondent), a type, a date and tags, for example “invoice” or “contract”. The program suggests some of this by itself, based on earlier documents.
The first processing run is the heaviest. After that, searching is fast, because it happens in the index and not in hundreds of files.
What it needs
You need three things.
A server that runs all the time. It can be a small server in the office, a virtual machine or a NAS device. The program runs in containers and has its own database inside (PostgreSQL) plus Redis, a fast buffer for the task queue. It doesn’t need powerful hardware, but text recognition puts some load on the processor for a while when a big batch of scans comes in.
Space for the data. The archive grows with every scan. A plan for a few years is the size of your documents today multiplied by a decent margin.
Backups. This is a condition, not an extra. An archive that exists as a single copy on a single disk is one more place documents can disappear from. Back up the whole data folder and keep a separate backup of the database. I write more about how to set up backups in a small business in the article Backups in a small business: what to check.
How to set it up: an outline
I describe the installation on a TrueNAS SCALE server, because Paperless-ngx is available there as a ready-made app in the catalogue. On an ordinary server with Docker the steps are similar: you set the folder paths, passwords and recognition language in a configuration file instead of a form.
Prepare the folders
Create two folders: one for the program’s data (database, index, thumbnails, processed documents), the other for files waiting to be processed. In TrueNAS you give them the access profile meant for apps. Apps there run as the user and group with the number 568, so the folders must belong to them.
Install the app
In the server panel you open the list of apps, search for paperless-ngx and choose to install the official version. The form has a few places where it is easy to make a mistake.
- Folders in the container. You attach the data folder to the program’s data path, and the input folder to the path the program picks up new files from. The program expects a separate directory for documents (“media”). If you want everything in one place, you can point both to the same folder on the server.
- User. For the container you set the same user and group (568) that own the folders.
- Passwords. Generate the program key, the admin password, the Redis password and the database password at random, ideally in a password manager. Don’t use the
#character in them. - Port. The “Web Port” field is the port you will open the app on in the browser, for example
30070. - Recognition languages. There are two separate fields. English is built in. If your documents are also in another language, enter its code in the field listing extra languages to install, for example
deufor German (separate several languages with a space). In the field for the language used for reading, entereng+deu(join several languages with a plus). Without this second setting the program still reads only in English and loses accented letters such as ä, é or ß. - Time zone. Set your own time zone, for example
Europe/Dublin, so the document dates are right.
An example of the values you enter (names and addresses are examples):
Path on the server: /mnt/pool/paperless/data → in the container: /usr/src/paperless/data
Path on the server: /mnt/pool/paperless/media → in the container: /usr/src/paperless/media
Path on the server: /mnt/pool/paperless/inbox → in the container: /usr/src/paperless/consume
User / group: 568 / 568
Port (Web Port): 30070
OCR languages to install: deu
OCR language for reading: eng+deu
Time zone: Europe/Dublin
Check that it works
When the app shows the status “Running”, you open it in the browser at the server’s address, for example http://192.168.1.10:30070 (the port from the installation form), and sign in as administrator. Then you drop one test PDF into the input folder. After a moment the file should disappear from the folder and show up in the list of documents, already with searchable text. Only that confirms the whole thing works, not just that the page opens.
What can go wrong
Start with this list when something won’t start.
- Permission errors (“permission denied”). The container tries to change the owner of the files and isn’t allowed to. Check that all attached folders belong to user and group 568. If that doesn’t help, there is a workaround going around the community: from the server’s shell, make the user
netdatathe owner of the main Paperless folder (the commandsudo chown netdata: <folder>). Its number matches the PostgreSQL database user in the container, which is why it helps when the database won’t start. Use it knowingly and only for the Paperless folder. - Redis won’t connect. Usually the culprit is a password with a special character, above all
#. Generate a password without it. - Documents aren’t processed. Check that the input folder really is attached to the right path in the container and that the files can be read.
- Accented letters missing in search. Check that your documents’ language (for example
deu) is on the list of languages to install and that it is also set as the language used for reading (for exampleeng+deu). - The app won’t start. Look at the container logs in the server panel. The message usually says plainly what is missing.
One design decision: after processing, Paperless deletes the file from the input folder and saves its own copy in its directory. I advise companies to keep their existing document folders and put copies into Paperless. The program converts everything to its own file layout and its own database, and people still sometimes need the original PDF or Word file to send it or correct it. It costs disk space, but it changes nothing about how your people work today.
When to do it yourself and when to ask for help
If there is someone in the company who can install an app on a server and set folder permissions, the installation takes an afternoon. A small number of documents and one user is a good start: you will see whether it suits you before you bring in the whole team.
Ask for help when things come in that are hard to fix after the fact. Access from outside the office and an encrypted connection through the browser. Taking in documents from email. Rules that assign tags by themselves. Accounts for several people with different permissions. And backups: they are worth setting up straight away, not “some day”. I use the built-in document export, and the database is covered by snapshots of the whole dataset and the app, taken by the TrueNAS server. That way you get back both the files and the descriptions, tags and history that live in the database.
Because the documents stay on your server, whoever maintains it is responsible for them: they update the program, check the backups and keep an eye on disk space. It’s worth agreeing on this at the start, not when the disk is full.
What next
A document archive is one of the tools I describe on the page about automation and your own tools: together with a password manager, files and one sign-in for several apps. They all run on your server, with backups and updates.
Not sure where to start, or whether an archive would pay off for you? Book a free audit. I’ll check what your documents look like today and what can be improved.