Alternative installation method for the OCR backend

[R0Wi]

Currently the app has a direct dependency to the OCRmyPDF commandline tool. So this dependency has to be installed directly onto the Nexcloud system for this app to work.

This discussion is meant to collect various alternative installation methods for the OCR backend needed by this app.

Related issues:

• OCRmyPDF with docker image #49
• Use OCRmyPDF with original nextcloud docker image #50
• "ocrmypdf: not found" using Snap (on Ubuntu) #84
• Feature request: run the OCR process on another server #260

@bahnwaerter @l00v3 @bohrsty FYI

[tommyhlw]

I refer to #50 since it is quite common to use official images without "hacking" them.
Either with docker-compose or k8s I would like to have an explicit service for the ocr-job. In k8s this would even scale without having effects on the resources of nextcloud.
Therefore it would be great to have a domain/service and api-key similar to ownpad:

Flow would be as follows:
PDF-file uploaded -> POST to ocr-service -> Response OCR-PDF -> Replace PDF or just add OCR-PDF with new name

[R0Wi]

Thanks for the hint i thinkt we're going to inspect ownpad and see how they solve it. In the meantime you could just "extend" (not "hack" ;-) ) the official image by just installing ocrMyPdf into it.

Please also note that providing a service backend means much work for us so we cannot promise we can deliver this in the not to distant future.

[tommyhlw]

Don't think you need to develop the service
https://ocrmypdf.readthedocs.io/en/latest/docker.html

Just the communication with the nc-app

[R0Wi]

We know this project but like stated in the docs

The OCRmyPDF web service wrapper is intended for demonstration or development. It provides no security, no authentication, no protection against denial of service attacks, and no load balancing. The default Flask WSGI server is used, which is intended for development only. The server is single-threaded and so can respond to only one client at a time. While running OCR, it cannot respond to any other clients.

this is not meant for production use unfortunately. Nevertheless it could give us a good starting point.

[joho500]

Is there any progress on this? I would like to use Workflow OCR with the AIO version of Nextcloud.

[R0Wi]

No updates so far. As a workaround you can just install ocrMyPdf directly in your NC container, see https://github.com/nextcloud/all-in-one#how-to-add-os-packages-permanently-to-the-nextcloud-container

[joho500]

Thanks for your reply and the pointer.

[R0Wi]

AppAPI and External Apps

Proposal

With the AppAPI and External Apps concept, it is now possible to create a NC app inside of a docker container. The framework already handles the basic infrastructure between the container and the NC ecosystem:

• Deployment of Container(s) in various situations. For example it is possible do deploy the Container on the same host like NC but also a remote host is possible as well as NC AIO.
• Bi-directional Authentication between the Container and Nextcloud
• A predefined interface between NC and the Container which already covers:
  • Various ExApp Skeletons
  • A minimal lifecycle interface
  • A python library nc_py_api (if the Container App is written in python) which contains a nice bootstrapping

So in theory it should be possible to build a Nextcloud ExApp based on the official ocrMyPdf Docker image.

Things to check

Basics like Authentication and Encrypted Communication should already be covered by the ExApp infrastructure. Nevertheless, we should keep in mind the official statement for the OCRmyPDF docker container:

Warning

The OCRmyPDF web service wrapper is intended for demonstration or development. It provides no security, no authentication, no protection against denial of service attacks, and no load balancing. The default Flask WSGI server is used, which is intended for development only. The server is single-threaded and so can respond to only one client at a time. While running OCR, it cannot respond to any other clients.

We should check the aspects from https://ocrmypdf.readthedocs.io/en/latest/cloud.html#ocr-service beforehand. Maybe it also makes sense to also have a closer look into paperless-ngx, since they are using ocrMyPdf in a webservice already.

Restricting CPU usage

Could be implemented by using dockers native CPU Resource Contraints. NC AppAPI would need to add an additional param to the creation process of a container to for example set the property NanoCPUS or CpuPeriod and CpuQuota in ContainerCreate

Temporary Storage

To ensure that ocrMyPdfs Temporary Storage is not affecting the whole docker host (making it unresponsible in worst case), we should also check if there's a way to limit the Containers Volume Amount. The VolumeCreate has a generic parameter DriverOpts, which could be used to create a new, size-restricted, tempfs volume (seems like tempfs is the only type supporting a size parameter) like this:

docker volume create --driver local --opt type=tmpfs --opt device=tmpfs --opt o=size=100m,uid=1000

A container started via docker-compose like this:

services:
  alpine:
    image: alpine
    command: ["tail","-f","/dev/null"]
    volumes:
      - tmpfs:/tmp

volumes:
  tmpfs:
    # For details, see:
    # https://docs.docker.com/engine/reference/commandline/volume_create/#driver-specific-options
    driver: local
    driver_opts:
      o: "size=100M"
      device: tmpfs
      type: tmpfs

Will see:

/ # df -h
Filesystem                Size      Used Available Use% Mounted on
...
tmpfs                   100.0M         0    100.0M   0% /tmp

Again, to implement this, the AppAPI code needs to be adjusted for this. Probably by introducing a second tempfs volume for a newly created container.

Finally we also need to tell ocrMyPdf to use another temp dir by setting the TMPDIR to a path where the tempfs volume is mounted into the container.