pd3f is an open-source PDF text extraction pipeline that is self-hosted and local-first.
pd3f reconstructs the original continuous text: paragraphs, hyphenated words, running headers and footers, footnotes, and OCR for scans.
Each line end, page break and choice between copies of a running header or footer is logged with what settled it, and the uncertain ones are marked for checking.
Some reasons why pd3f is for you
Scans are automatically OCRd
Run it on your computer or on a server
Docker makes installation easy
A longer introduction to pd3f 0.x in German on the site of the Prototype Fund.
pd3f reads PDFs with LiteParse and runs OCR on scanned pages with OCRmyPDF (Tesseract). It then rebuilds the continuous text: it joins lines into paragraphs, rejoins words hyphenated at the end of a line, moves running headers, footers and page numbers out of the text and collects the footnotes at the end. Fixed rules and the spelling the document uses elsewhere settle most of these decisions. Optionally, the character-based language models of Flair settle the rest.
pd3f is especially useful for languages with long words such as German. It was mainly developed to parse German letters and official documents. Besides German, pd3f supports English, Spanish, French and Italian.
pd3f is a Python library, a command line tool and a web app with an HTTP API, packaged as one Docker image.
pd3f is licensed under the GNU Affero General Public License v3.0 and is looking for contributors. Write an email to hi@jfilter.de if you want to cooperate.
pd3f was created by Johannes Filter and financed by the German Federal Ministry of Education and Research as part of their funding for Open-source software with the Prototype Fund.
Johannes Filter is an independent researcher, software developer and data analyst. His work is focused on human-computer interaction, machine learning and natural-language processing. Website, Twitter
Legal Contact / Impressum, Privacy Policy
The project was funded by the German Ministry of Education and Research under the funding reference number 01IS19S18. The author is responsible for the content of this publication.
