Bump marker-pdf from 1.7.5 to 1.8.0

[dependabot]

Bumps marker-pdf from 1.7.5 to 1.8.0.

Release notes

Sourced from marker-pdf's releases.

Chunk output; custom prompts; structured extraction improvements

Marker 1.8.0

• Marker will now output a flat list of blocks with associated html, which is useful for RAG
• Structured extraction beta is significantly improved, with better performance/accuracy
• New LLM sectionheader processor will correctly label section header levels
• You can pass a prompt to marker in LLM mode to adjust the output
• Marker batch conversion script has somewhat better performance, closer to our inference container - email us at hi@datalab.to if you want to get setup with our inference container (used on prem at top AI research orgs)
• Add an option to filter out blank white page images from output
• Enable keeping pageheader/pagefooter.

Chunking/RAG improvements

• Add chunk output format which is a flat list of chunks with full html in each
• Add an llm sectionheader processor that will redo all the header levels against each other properly

Use the sectionheaderprocessor by setting --use_llm, and the chunk output by setting --output_format chunks.

Structured extraction

• Fix structured extraction, so it works much better than before (requires llm)
• Improve structured extraction test app

You can try with with the streamlit app by running python extraction_app.py.

Promptability/customization!

• Add promptability via block_correction_prompt, which can be used to create custom behavior (requires llm)

Try it by setting the block_correction_prompt config key to a specific prompt.

Misc

• Get the marker script to perform a bit closer to our inference container by default (inference container gets 10-25 pages/s on H100). Will auto-configure worker count to available VRAM.
• Fix where marker would output blank pages as images
• Enable keeping pageheader/pagefooter in the output
• Adjust llm services to enable text-only input
• Add html field to almost every block type

Test pageheader/pagefooter by setting keep_pagefooter_in_output and keep_pageheader_in_output.

What's Changed

• Adding image format arg to OpenAI interface by @​rgeorgi in datalab-to/marker#752
• fix: don't remove leading newlines when paginate=True for markdown by @​zanussbaum in datalab-to/marker#769
• Fix LLM query retry logic and unnecessary sleeping by @​runarmod in datalab-to/marker#772
• Update issue templates by @​tarun-menta in datalab-to/marker#767
• WIP: Add Blank Page Processor by @​tarun-menta in datalab-to/marker#750
• Vik/0625fixes by @​VikParuchuri in datalab-to/marker#768
• Marker v1.8 by @​VikParuchuri in datalab-to/marker#773

... (truncated)

Commits

• edbcb8c Merge pull request #774 from datalab-to/dev
• 17d8c63 Fix schema issues
• bd7add2 Fix attributes
• b10399c Test fix
• 8fb2815 Better support for chunk format
• 720e7f7 Refactor llm services
• 735b610 Merge pull request #675 from MauritsBrinkman/master
• e071e19 Merge pull request #773 from datalab-to/dev
• 553ad2e Fix tests
• a3e4387 Merge pull request #768 from datalab-to/vik/0625fixes
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #306.