adding first pass at Dockerfile by amazingvince · Pull Request #12 · impira/docquery

amazingvince · 2022-09-03T04:48:56Z

No description provided.

toant13 · 2022-09-07T03:55:47Z

Dockerfile

@@ -0,0 +1,18 @@
+FROM ubuntu:latest


Any reason to use an ubuntu image rather than python:3-alpine so you don't have to install python and the image can be smaller?

https://hub.docker.com/_/python

The function pthread_attr_setaffinity_np is not on alpine linux and needed for pytorch. I am not sure if there is a slimmer distro than ubuntu. Open to try something if you have any suggestions.

I'd prefer we use ubuntu and maybe even package the model weights in the dockerfile (you're going to have to download it anyway, and we pin the revision in docquery).

On my local when using it to develop I mount my machines huggingface cache. But I think as a general easy to use docker container that would simplify things and be respectful of huggingfaces resources.

Do we want to build from source than? Or should I just pip install docquery?

I think it depends on how we want to publish it. We should probably get a better release cadence going and pin the Dockerfile to whatever the latest release is. I think the way to do that would be to somehow provide the version as a parameter to the Dockerfile and populate the parameter by looking at https://github.com/impira/docquery/blob/main/src/docquery/version.py.

docker run IMAGE_NAME docquery scan "What is the invoice number?" https://templates.invoicehome.com/invoice-template-us-neat-750px.png

Seems like a really nice way to use the tool.

npappenhagen · 2022-09-10T17:09:15Z

Dockerfile

+FROM ubuntu:latest
+
+RUN apt-get update \
+  && apt-get install -y python3-pip python3-dev \


does this need to be specific about which python3 version to use? I think some of the dependencies here require > 3.6.

RUN apt-get update && apt-get install -y python3.9 python3.9-dev python3-pip

I switched to python:3.10-slim-bullseye. So we will be using 3.10.

ankrgyl · 2022-09-10T17:10:11Z

Ideally 3.10

…

-- Ankur Goyal CEO, Impira Make Meaningful Sent via Superhuman iOS ( ***@***.*** )

On Sat, Sep 10 2022 at 10:09 AM, npappenhagen < ***@***.*** > wrote: ***@***.**** commented on this pull request. In Dockerfile ( #12 (comment) ) : > @@ -0,0 +1,18 @@ +FROM ubuntu:latest + +RUN apt-get update \ + && apt-get install -y python3-pip python3-dev \ does this need to be specific about which python3 version to use? I think some of the dependencies here require > 3.6. RUN apt-get update && apt-get install -y python3.9 python3.9-dev python3-pip — Reply to this email directly, view it on GitHub ( #12 (review) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAEKA47Y6QTY5O7Z5DRLLNDV5S6EPANCNFSM6AAAAAAQDXWIFY ). You are receiving this because you are subscribed to this thread. Message ID: <impira/docquery/pull/12/review/1103155127 @ github. com>

ankrgyl · 2022-09-11T00:37:43Z

@amazingvince how does the scan command work from the docker container? Specifically, if you run something like docker ... scan "What is the invoice number?" /path/to/fs how is it able to access files outside the container?

ankrgyl · 2022-09-11T00:40:44Z

Yeah but what happens if you point to a file on your local filesystem?

amazingvince · 2022-09-11T01:43:58Z

@ankrgyl It would be something like $ docker run -v $(pwd) ... docquery scan "What is the invoice number?" .
Something is missing there and also might change based on system env like windows

ankrgyl

Oh cool, that looks good. I think as long as we document it that works. This is very exciting, and I think we are in the homestretch.

To land this we need to sort out a few more things:

Add a command to make, like make docker, that builds the Dockerfile (with the appropriate container/tag names)
Add documentation to the README that shows how to run the scan command and how to add local files
Add something to the tests (maybe a sanity test that the container can be built, or a mode that builds the container and then runs the tests inside of it)

I've also filed some follow ups: #27 and #28. We can address these in follow ups.

ankrgyl · 2022-09-11T15:09:24Z

Dockerfile

+COPY ["src/", "./src"]
+
+RUN pip install .
+CMD ["python3"]


can we make the default entrypoint python3 -m docquery.cmd (or just docquery)?

ankrgyl · 2022-09-11T15:11:09Z

Dockerfile

+COPY ["README.md", "pyproject.toml", "setup.py", "./"]
+COPY ["src/", "./src"]
+
+RUN pip install .


I think we should either pip install .[all] or a select list of extensions, e.g. [donut] (I'm currently working on adding [web] which will contain extras for web scraping).

We should also manually install transformers (the same version that's suggested in the README)

adding first pass at Dockerfile

a0d017a

toant13 reviewed Sep 7, 2022

View reviewed changes

npappenhagen reviewed Sep 10, 2022

View reviewed changes

amazingvince added 2 commits September 10, 2022 20:22

going to 3.10, adding tesseract 5 and using slim container

a7ca26e

empty endline for pre-commit

2759502

ankrgyl mentioned this pull request Sep 11, 2022

Publish docker image to Dockerhub #27

Open

ankrgyl reviewed Sep 11, 2022

View reviewed changes

Conversation

amazingvince commented Sep 3, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ankrgyl commented Sep 10, 2022 via email

Uh oh!

ankrgyl commented Sep 11, 2022

Uh oh!

ankrgyl commented Sep 11, 2022

Uh oh!

amazingvince commented Sep 11, 2022

Uh oh!

ankrgyl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants