Supply url_fetcher to weasyprint to support /media and /static files

[LqdBcnAtWork]

Pull request with changes from a comment made on #9351.

This allows weasyprint to grab files from /media and /static locations by redirecting them to open() calls with respect to MEDIA_ROOT and STATIC_ROOT settings.

Probably could be made better. But at least ensures that the resulting path is always still a child of the parent folder.

[netlify]

✅ Deploy Preview for inventree-web-pui-preview canceled.

Name	Link
🔨 Latest commit	0b1ef04
🔍 Latest deploy log	https://app.netlify.com/projects/inventree-web-pui-preview/deploys/68f0d349717c950008ce4c6c

[SchrodingersGat]

@LqdBcnAtWork FYI there are already template tags available to the reporting system for displaying uploaded images:

• https://docs.inventree.org/en/stable/report/helpers/#media-files
• https://docs.inventree.org/en/stable/report/helpers/#uploaded_image
• https://docs.inventree.org/en/stable/report/helpers/#encode_svg_image
• https://docs.inventree.org/en/stable/report/helpers/#part_image
• https://docs.inventree.org/en/stable/report/helpers/#company_image
• https://docs.inventree.org/en/stable/report/helpers/#report-assets

[SchrodingersGat]

An issue here is that the request URL may or may not exactly match the server's internal URL, depending on proxying settings. Adding a hard-coded URL here is not a good idea.

[LqdBcnAtWork]

Just a clarifying question, would it be possible to use these somehow with markdownify?

That was the "problem" this code was meant to solve. The notes editor uses relative paths (eg: /media/notes/image.png) for the SRC attribute. Which markdownify uses without transformation.

This pull request makes it so weasyprint can grab those images as well for pdf generation. Currently images aren't included when notes are transformed into html by markdownify, as weasyprint has no method of getting the assets. It actually throws an error as there are relative paths with no base path provided.

[SchrodingersGat]

@LqdBcnAtWork yeah I definitely appreciate what you are trying to achieve here. Having another read through, this might not be a terrible idea ;)

• You upload an image into the notes for a part, which then loads an image against /media/images/my_image.png
• When you try to render the same markdown into HTML (for weasyprint report rendering) it fails because it doesn't know how to access the image
• Add a custom fetcher to weasyprint which supports just /media/ and /static/ requests
• This works not just for markdown code, but any HTML we want to render into the report

So, there are still some issues to deal with here:

1. Remove the hard-coded URL prefix - should use the defined site_url
2. Can we support lookup of assets without a HOST prefix e.g. /media/images/my_image.png vs http://server.com/media/images/my_image.png
3. We need to be able to support other types of django storage backends - not just filesystem storage

[matmair]

Let us know if you want pointers how the requested changes might be achieved

[SchrodingersGat]

@LqdBcnAtWork are you still looking into this?

[LqdBcnAtWork]

My apologies, I am still planning on doing more on this. But I've been pulled aside to other projects for the time being.

I'll get back to this eventually. It's becoming a conversation of when, not if, we'll switch to Inventree.

[SchrodingersGat]

@LqdBcnAtWork any interest in this still? I think it would be great to get this implemented

[wolflu05]

Why do we need this at all? Doesn't this work out of the box, because weasyprint can also connect to the server?

[LqdBcnAtWork]

Why do we need this at all? Doesn't this work out of the box, because weasyprint can also connect to the server?

The issue isn't if weasyprint can connect to the server or not. The issue is that weasyprint had no idea it was supposed to connect to a server. No base URL was supplied to weasyprint. As such any relative urls (like image attachments typically are) would error and the image would get ignored.

I suppose giving weasyprint what it needs to connect to the server might be a better solution. I'm getting my fork caught back up so I can poke at this again. I'll give that idea a try in a minute.

[SchrodingersGat]

I don't think connecting to the server is the right approach here. You already have all you need:

• Find a URL which starts with /media/ or /static/
• Substitute these with the local paths to file storage e.g. /opt/inventree/media/
• Rendering pipeline then works as expected

[LqdBcnAtWork]

I got it working without any path substitution magic. This may not necessarily be the best route, but I wanted test it anyway.

I had to copy the request from api.py down to the .render() method. But then I was able to clone it's headers for auth.

This has the benefit of being completely agnostic to storage backend.

Downsides: requires a HttpRequest, and the request needs permissions to all of the resources.

Path substitution magic is probably better. But I have no idea how to make it work with other storage backends.

[LqdBcnAtWork]

I also got this working as a prototype. It wouldn't need any changes made to weasyprint. But it does break images when REPORT_DEBUG_MODE is True. It also leaks the paths. Which isn't great.

I haven't dug into only injecting the extension when the plan is to use weasyprint. I have no idea how that would work to be honest.

This would need to be placed somewhere. (Open for thoughts as to where would be best)

from markdown.extensions import Extension
from markdown import Markdown
from markdown.treeprocessors import Treeprocessor

class ImgSrcFixerTreeProcessor(Treeprocessor):
    def run(self, root):
        for el in list(root.iter("img")):
            src = el.attrib.get("src")
            print(f"img src='{src}'")

            if isinstance(src, str) and src.startswith(MEDIA_URL):
                pth = MEDIA_ROOT.joinpath(src[len(MEDIA_URL):]).as_uri()
                print(f"img path='{pth}'")
                el.attrib["src"] = pth


class ImgSrcFixerExt(Extension):
    def extendMarkdown(self, md: Markdown) -> None:
        md.treeprocessors.register(
            ImgSrcFixerTreeProcessor(md),
            "path-fixer",
            0
        )

Then the MARKDOWNIFY setting would need to be updated as follows:

        'MARKDOWN_EXTENSIONS': ['markdown.extensions.extra', ImgSrcFixerExt()],
        'WHITELIST_PROTOCOLS': ['http', 'https', 'file'],