Python 3.8 AWS Lambda incompatibility (Amazon Linux 2)

[Wenzil]

Hello there!

I'm trying to build and package WeasyPrint and its native dependencies into an AWS Lambda layer for the Python 3.8 runtime.

The steps described in #916 work for the Python 3.7 runtime environment (Amazon Linux 1), but fail with Python 3.8.

Here's as far as I got:

# Dockerfile
FROM lambci/lambda:build-python3.8

# Based on https://aws.amazon.com/premiumsupport/knowledge-center/lambda-linux-binary-package/
RUN yum install -y yum-utils rpmdevtools
WORKDIR /tmp
RUN yumdownloader --resolve \
    libffi \
    libffi-devel \
    cairo \
    pango && \
    rpmdev-extract *rpm

RUN mkdir /opt/lib
WORKDIR /opt/lib
RUN cp -P -R /tmp/*/usr/lib64/* /opt/lib
RUN ln libcairo.so.2 libcairo.so && \
    ln libpango-1.0.so.0 pango-1.0 && \
    ln libpangocairo-1.0.so.0 pangocairo-1.0

WORKDIR /opt
RUN pip3 install weasyprint -t python
RUN zip -r /var/task/weasyprint_lambda_layer.zip ./lib ./python

# Test the package
RUN PYTHONPATH=python python -m weasyprint

And the resulting error:

Traceback (most recent call last):
  File "/var/lang/lib/python3.8/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/var/lang/lib/python3.8/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/var/lang/lib/python3.8/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/opt/python/weasyprint/__init__.py", line 443, in <module>
    from .document import Document, Page  # noqa isort:skip
  File "/opt/python/weasyprint/document.py", line 24, in <module>
    from .fonts import FontConfiguration
  File "/opt/python/weasyprint/fonts.py", line 54, in <module>
    pangoft2 = dlopen(ffi, 'pangoft2-1.0', 'libpangoft2-1.0-0',
  File "/opt/python/weasyprint/text.py", line 253, in dlopen
    return ffi.dlopen(names[0])  # pragma: no cover
  File "/opt/python/cffi/api.py", line 146, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/opt/python/cffi/api.py", line 828, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/opt/python/cffi/api.py", line 823, in _load_backend_lib
    raise OSError(msg)
OSError: cannot load library 'pangoft2-1.0': pangoft2-1.0: cannot open shared object file: No such file or directory.  Additionally, ctypes.util.find_library() did not manage to locate a library called 'pangoft2-1.0'

Now, the problem is I can't find any pangoft2 distribution for Fedora nor any compilation instructions.

Any pointers?

Thanks!

[Wenzil]

Digging a bit further, this is happening only in the Python 3.8 AWS Lambda environment because the distributed version of pango on Amazon Linux 2 is higher than on Amazon Linux 1 (which older AWS Lambda Python runtime environments use).

As seen in the WeasyPrint source code, the newer pango version enables @font-face support, which requires pangoft2:

if pango.pango_version() < 13800:
    warnings.warn('@font-face support needs Pango >= 1.38')
else:
    # No need to try...catch:
    # If there's no fontconfig library, cairocffi already crashed the script
    # with OSError: dlopen() failed to load a library: cairo / cairo-2
    # So let's hope we find the same file as cairo already did ;)
    # Same applies to pangocairo requiring pangoft2
    fontconfig = dlopen(ffi, 'fontconfig', 'libfontconfig',
                        'libfontconfig-1.dll',
                        'libfontconfig.so.1', 'libfontconfig-1.dylib')
    pangoft2 = dlopen(ffi, 'pangoft2-1.0', 'libpangoft2-1.0-0',
                      'libpangoft2-1.0.so', 'libpangoft2-1.0.dylib')

So I am stuck, because I need @font-face support.

[Wenzil]

Ok, I managed to install the dependencies, I was missing a symbolic link for the pangoft2 lib and a few installs. This seems to cover all the dependencies:

RUN yumdownloader --resolve \
    expat \
    glib2 \
    libffi \
    libffi-devel \
    cairo \
    pango && \
    rpmdev-extract *rpm

RUN ln libgobject-2.0.so.0 libgobject-2.0.so && \
    ln libcairo.so.2 libcairo.so && \
    ln libpango-1.0.so.0 pango-1.0 && \
    ln libpangoft2-1.0.so.0 pangoft2-1.0 && \
    ln libpangocairo-1.0.so.0 pangocairo-1.0

Now I'm getting a panic at runtime though. Here are the logs:

(+0s) START RequestId: 572115b6-87b0-4d57-bc6b-0f0bb08d0b0c Version: $LATEST
(+2.34s) Fontconfig error: Cannot load default config file
(+0.11s) (process:8): GLib-GObject-WARNING **: 02:27:47.227: cannot register existing type 'PangoFcFontMap'
(+0s) (process:8): GLib-CRITICAL **: 02:27:47.227: g_once_init_leave: assertion 'result != 0' failed
(+0s) (process:8): Pango-CRITICAL **: 02:27:47.227: pango_fc_font_map_set_config: assertion 'PANGO_IS_FC_FONT_MAP (fcfontmap)' failed
(+0.85s) END RequestId: 572115b6-87b0-4d57-bc6b-0f0bb08d0b0c
(+0s) REPORT RequestId: 572115b6-87b0-4d57-bc6b-0f0bb08d0b0c	Duration: 3318.53 ms	Billed Duration: 3400 ms	Memory Size: 128 MB	Max Memory Used: 101 MB	
(+0s) RequestId: 572115b6-87b0-4d57-bc6b-0f0bb08d0b0c Error: Runtime exited with error: signal: segmentation fault (core dumped) Runtime.ExitError

I'm not familiar with fontconfig. Anyone care to help ?

Thanks!

[Wenzil]

Wonder if the Fontconfig error: Cannot load default config file error is causing all the subsequent ones. A similar issue (#402) appears to have been resolved before. @liZe do you remember what was causing it?

[Tontyna]

@Wenzil have a look at How can I install a font on AWS lambda?.
Can't believe that the missing config file results in segfault, but who knows...

[Wenzil]

Ok, I managed to get rid of the above Fontconfig error, but sadly still getting segfaults.

/opt/python/weasyprint/document.py:35: UserWarning: There are known rendering problems and missing features with cairo < 1.15.4. WeasyPrint may work with older versions, but please read the note about the needed cairo version on the "Install" page of the documentation before reporting bugs. http://weasyprint.readthedocs.io/en/latest/install.html
(process:8): GLib-GObject-WARNING **: 18:19:35.898: cannot register existing type 'PangoFcFontMap'
(process:8): GLib-CRITICAL **: 18:19:35.898: g_once_init_leave: assertion 'result != 0' failed
(process:8): Pango-CRITICAL **: 18:19:35.898: pango_fc_font_map_set_config: assertion 'PANGO_IS_FC_FONT_MAP (fcfontmap)' failed
Error: Runtime exited with error: signal: segmentation fault (core dumped) Runtime.ExitError

[liZe]

@Wenzil Now that you have successfully configured Pango, you probably have to install fonts.

[Wenzil]

Thanks @Tontyna and @liZe . I tried copied the default fonts from the older Amazon Linux based lambda:build-python3.7 image to the built layer. It got rid of a "No fonts configured" error but still getting the segfaults.

I have to put this away for now but I'll try to get back to it when time permits. If anyone wants to keep digging in the mean time here's the Dockerfile I currently have:

FROM lambci/lambda:build-python3.7 AS py37
FROM lambci/lambda:build-python3.8

# Based on https://aws.amazon.com/premiumsupport/knowledge-center/lambda-linux-binary-package/
RUN yum install -y yum-utils rpmdevtools
WORKDIR /tmp
RUN yumdownloader --resolve \
    expat \
    glib2 \
    libffi \
    libffi-devel \
    cairo \
    pango && \
    rpmdev-extract *rpm

RUN mkdir /opt/lib
WORKDIR /opt/lib
RUN cp -P -R /tmp/*/usr/lib64/* /opt/lib
RUN ln libgobject-2.0.so.0 libgobject-2.0.so && \
    ln libcairo.so.2 libcairo.so && \
    ln libpango-1.0.so.0 pango-1.0 && \
    ln libpangoft2-1.0.so.0 pangoft2-1.0 && \
    ln libpangocairo-1.0.so.0 pangocairo-1.0

WORKDIR /opt
RUN pip3 install jinja2 weasyprint -t python

COPY --from=py37 /usr/share/fonts/default /opt/fonts/default
COPY --from=py37 /etc/fonts/fonts.conf /opt/fonts/fonts.conf
RUN sed -i s:/usr/share/fonts:/opt/fonts: /opt/fonts/fonts.conf

RUN zip -r /tmp/weasyprint_lambda_layer.zip \
    ./lib \
    ./python \
    ./fonts

And I set FONTCONFIG_PATH to /opt/fonts

[Tontyna]

@liZe - if there were no fonts, shouldn't weasyprint.fonts.FontConfiguration.__init__ emit a warning when it calls _check_font_configuration() and skip the segfaulting PangoFcFontMap?

WeasyPrint/weasyprint/fonts.py

Lines 268 to 275 in 05a9513

	if _check_font_configuration(self._fontconfig_config):
	self.font_map = ffi.gc(
	pangocairo.pango_cairo_font_map_new_for_font_type(
	cairo.FONT_TYPE_FT),
	gobject.g_object_unref)
	pangoft2.pango_fc_font_map_set_config(
	ffi.cast('PangoFcFontMap *', self.font_map),
	self._fontconfig_config)

pangocairo seems ok, but pangoft2 library seems to have a/be the problem...

[liZe]

if there were no fonts, shouldn't weasyprint.fonts.FontConfiguration.__init__ emit a warning when it calls _check_font_configuration() and skip the segfaulting PangoFcFontMap?

Probably, you're right.

The problem could be that PangoCairo is not built with the FreeType2 support. In this case, pango_cairo_font_map_new_for_font_type returns NULL, and PangoFT2 would be right to say that assertion 'PANGO_IS_FC_FONT_MAP (fcfontmap)' failed.

We should check that self.font_map is not NULL/None before calling the next lines.

[Tontyna]

Not sure whether avoiding pango_cairo_font_map_new_for_font_type in case our self.font_map being NULL/None would prevent the segfault. There is:

 GLib-CRITICAL **: 18:19:35.898: g_once_init_leave: assertion 'result != 0' failed

Which, I suspect, is triggered by gobject.g_object_unref, the destructor parameter of ffi.gc().

Anybody keen enough to setup an AWS Lambda with modified fonts.py to verify or falsify our hypothesis?

Of course, preventing the segfault would be an enhancement, but as @Wenzil said: He needs @font-face support.

[liZe]

Which, I suspect, is triggered by gobject.g_object_unref, the destructor parameter of ffi.gc().

Of course. The real pseudo-code to test is something like:

font_map = pangocairo.pango_cairo_font_map_new_for_font_type(…)
if font_map:
    self.font_map = ffi.gc(font_map, gobject.g_object_unref)
    pangoft2.pango_fc_font_map_set_config(…)
else:
    self.font_map = None

Maybe I can create a new branch / PR tested with another font type unsupported on my system.

[liZe]

Of course, preventing the segfault would be an enhancement, but as @Wenzil said: He needs @font-face support.

You're right. I really think that Pango can't work with FreeType. Linking the FreeType library (as it's done for the other libraries) in the Dockerfile may help.

[fleaz]

Hey everybody,
I'm also in the process of getting a recent version of weasyprint running on Lambda because we also need @font-face support.

I'm currently stuck at the same point like @Wenzil and get the exact same segfault in my function.

Let me know if I can support you with anything to help get this fixed :)

[cpmech]

Thanks @Tontyna and @liZe . I tried copied the default fonts from the older Amazon Linux based lambda:build-python3.7 image to the built layer. It got rid of a "No fonts configured" error but still getting the segfaults.

I have to put this away for now but I'll try to get back to it when time permits. If anyone wants to keep digging in the mean time here's the Dockerfile I currently have:

FROM lambci/lambda:build-python3.7 AS py37
FROM lambci/lambda:build-python3.8

# Based on https://aws.amazon.com/premiumsupport/knowledge-center/lambda-linux-binary-package/
RUN yum install -y yum-utils rpmdevtools
WORKDIR /tmp
RUN yumdownloader --resolve \
    expat \
    glib2 \
    libffi \
    libffi-devel \
    cairo \
    pango && \
    rpmdev-extract *rpm

RUN mkdir /opt/lib
WORKDIR /opt/lib
RUN cp -P -R /tmp/*/usr/lib64/* /opt/lib
RUN ln libgobject-2.0.so.0 libgobject-2.0.so && \
    ln libcairo.so.2 libcairo.so && \
    ln libpango-1.0.so.0 pango-1.0 && \
    ln libpangoft2-1.0.so.0 pangoft2-1.0 && \
    ln libpangocairo-1.0.so.0 pangocairo-1.0

WORKDIR /opt
RUN pip3 install jinja2 weasyprint -t python

COPY --from=py37 /usr/share/fonts/default /opt/fonts/default
COPY --from=py37 /etc/fonts/fonts.conf /opt/fonts/fonts.conf
RUN sed -i s:/usr/share/fonts:/opt/fonts: /opt/fonts/fonts.conf

RUN zip -r /tmp/weasyprint_lambda_layer.zip \
    ./lib \
    ./python \
    ./fonts

And I set FONTCONFIG_PATH to /opt/fonts

Hi All,

I've managed to make it work---thanks to Wenzil's and previous work.

Here you go my files:

Dockerfile

FROM lambci/lambda:build-python3.7 AS py37
FROM lambci/lambda:build-python3.8

# download libraries
RUN yum install -y yum-utils rpmdevtools
WORKDIR /tmp
RUN yumdownloader --resolve \
    expat \
    glib2 \
    libffi \
    libffi-devel \
    cairo \
    pango && \
    rpmdev-extract *rpm

# install libraries and set links
RUN mkdir /opt/lib
WORKDIR /opt/lib
RUN cp -P -R /tmp/*/usr/lib64/* /opt/lib
RUN ln libgobject-2.0.so.0 libgobject-2.0.so && \
    ln libcairo.so.2 libcairo.so && \
    ln libpango-1.0.so.0 pango-1.0 && \
    ln libpangoft2-1.0.so.0 pangoft2-1.0 && \
    ln libpangocairo-1.0.so.0 pangocairo-1.0

# copy fonts and set environment variable
COPY --from=py37 /usr/share/fonts/default /opt/fonts/default
COPY --from=py37 /etc/fonts/fonts.conf /opt/fonts/fonts.conf
RUN sed -i s:/usr/share/fonts:/opt/fonts: /opt/fonts/fonts.conf
ENV FONTCONFIG_PATH="/opt/fonts"

# install weasyprint and dependencies
WORKDIR /opt
RUN pipenv install weasyprint
RUN mkdir -p python/lib/python3.8/site-packages
RUN pipenv lock -r > requirements.txt
RUN pip install -r requirements.txt --no-deps -t python/lib/python3.8/site-packages

# remove warning about cairo < 1.15.4
WORKDIR /opt/python/lib/python3.8/site-packages/weasyprint
RUN sed -i.bak '34,40d' document.py

# run test
WORKDIR /opt
ADD test.py .
RUN pipenv run python test.py

# package lambda layer
WORKDIR /opt
RUN zip -r weasyprint-py38x.zip fonts lib python

test.py

from weasyprint import html

data = """
<!DOCTYPE html>
<html>
<body>
<h1>Hello World</h1>
<p>Just Testing.</p>
</body>
</html>
"""

HTML(string=data).write_pdf("output.pdf")

Bash script to generate lambda-layer

#!/bin/bash

set -e

PYVER=py38x

docker image build -f Dockerfile-$PYVER -t weasyprint-$PYVER .
docker create -ti --name dummy-$PYVER weasyprint-$PYVER bash
docker cp dummy-$PYVER:/opt/weasyprint-$PYVER.zip .
docker cp dummy-$PYVER:/opt/output.pdf ./output-$PYVER.pdf
docker rm dummy-$PYVER

aws lambda publish-layer-version --layer-name weasyprint-$PYVER --zip-file fileb://weasyprint-$PYVER.zip

Lambda function

import json
from weasyprint import HTML

data = """
<!DOCTYPE html>
<html>
<body>
<h1>Hello World</h1>
<p>Just Testing.</p>
</body>
</html>
"""

def lambda_handler(event, context):
    HTML(string=data).write_pdf("/tmp/output.pdf")
    return {
        'statusCode': 200,
        'body': json.dumps('Success!')
    }

Output of my Lambda function

{
  "statusCode": 200,
  "body": "\"Success!\""
}

:-D :-D :-D

I hope it helps!
And thanks again.

[cpmech]

Hi Again,

For reference, the lambda-function code below generates the PDF and returns a URL to download the file from S3. (remember to create the right AWS policies/roles; also, I'm setting the memory to 256Mb and Timeout to 1 min 0 sec).

EDIT The lambda function fails from time to time with SEGFAULT.

lambda function handler Python 3.8

import json
import boto3
from weasyprint import HTML

s3 = boto3.client('s3')
bucket = "testing-weasyprint-output"

def lambda_handler(event, context):
    # generate PDF
    filekey = event['filekey']
    HTML(string=event['data']).write_pdf(f'/tmp/{filekey}')
    
    # upload output to S3
    with open(f'/tmp/{filekey}', 'rb') as f:
        s3.upload_fileobj(f, bucket, filekey, ExtraArgs={'ContentType': 'application/pdf'})
        
    # get download URL
    Params = {'Bucket': bucket, 'Key': filekey}
    url = s3.generate_presigned_url(ClientMethod='get_object', Params=Params, ExpiresIn=60)
    
    # results
    return {
        'statusCode': 200,
        'body': json.dumps({'url': url})
    }

The input to this function is something like this:

{
  "filekey": "test.pdf",
  "data": "<!DOCTYPE html><html><body><h1>Hello WeasyPrint!</h1><p>Just testing this nice tool.</p></body></html>"
}

There's a problem though as shown below:

Problem

START RequestId: 28f8d633-2647-45fe-a0ce-776effb4101e Version: $LATEST

(process:8): GLib-GObject-WARNING **: 02:30:18.438: cannot register existing type 'PangoFcFontMap'

(process:8): GLib-CRITICAL **: 02:30:18.438: g_once_init_leave: assertion 'result != 0' failed

(process:8): Pango-CRITICAL **: 02:30:18.438: pango_fc_font_map_set_config: assertion 'PANGO_IS_FC_FONT_MAP (fcfontmap)' failed
END RequestId: 28f8d633-2647-45fe-a0ce-776effb4101e
REPORT RequestId: 28f8d633-2647-45fe-a0ce-776effb4101e	Duration: 2258.65 ms	Billed Duration: 2300 ms	Memory Size: 256 MB	Max Memory Used: 78 MB	

From the return link, we get the figure below (very nice!)

For future reference: https://gist.github.com/cpmech/b48729ea07e575d678698bb116d1cbba#file-build-weasyprint-python3-8-lambda-layer-md

Thanks for this nice tool

[cpmech]

Hi Again, unfortunately, the code DOES SEGFAULT from time to time... So, it's not working well. Sorry.

So, going back to Python 3.7.

Here you go the Dockefile:

Dockerfile

FROM lambci/lambda:build-python3.7

# download libraries
RUN yum install -y yum-utils rpmdevtools
WORKDIR /tmp
RUN yumdownloader libffi libffi-devel cairo pango \
    && rpmdev-extract *rpm

# install libraries and set links
RUN mkdir /opt/lib
WORKDIR /opt/lib
RUN cp -P -R /tmp/*/usr/lib64/* /opt/lib
RUN ln libpango-1.0.so.0 pango-1.0
RUN ln libpangocairo-1.0.so.0 pangocairo-1.0

# install weasyprint and dependencies
WORKDIR /opt
RUN pipenv install weasyprint
RUN mkdir -p python/lib/python3.7/site-packages
RUN pipenv lock -r > requirements.txt
RUN pip install -r requirements.txt --no-deps -t python/lib/python3.7/site-packages

# remove warning about cairo < 1.15.4
WORKDIR /opt/python/lib/python3.7/site-packages/weasyprint
RUN sed -i.bak '34,40d' document.py

# run test
WORKDIR /opt
ADD test.py .
RUN pipenv run python test.py

# package lambda layer
WORKDIR /opt
RUN zip -r weasyprint-py37.zip lib python

[Florent-Breton]

Hi @cpmech ,
Could you also add the content of your requirements.txt please.

[cpmech]

Sure, here you go.

requirements.txt

-i https://pypi.org/simple
cairocffi==1.1.0
cairosvg==2.4.2
cffi==1.14.0
cssselect2==0.2.2
defusedxml==0.6.0
html5lib==1.0.1
pillow==7.0.0
pycparser==2.20
pyphen==0.9.5
six==1.14.0
tinycss2==1.0.2
weasyprint==51
webencodings==0.5.1

[gligorkot]

Hi all, thank you so much for providing these instructions and lambda layers to get this running on AWS Lambda. I am having some issues where my lambda always generates an empty pdf. It will all work and it'll say successful, but when I return the PDF it's a PDF of blank pages. Anyone else experiencing this issue?

[mightym]

@cpmech thanks a lot for your code and effort. was looking for a long time for a good example running weasyprint on lambda. Works like a charm. The only thing: have you tested if custom fonts/webfonts work in your setup? For me fonts are completely ignored. No matter if I use custom otf fonts via css locally/remote.

[mightym]

Just double checked. The latest yum pango version is 1.28.1. That means no font face support https://github.com/Kozea/WeasyPrint/blob/master/weasyprint/fonts.py#L40-L41 :-/ damn...

[gligorkot]

Hi all, thank you so much for providing these instructions and lambda layers to get this running on AWS Lambda. I am having some issues where my lambda always generates an empty pdf. It will all work and it'll say successful, but when I return the PDF it's a PDF of blank pages. Anyone else experiencing this issue?

Just found out that if I use it to write pdf and upload it to S3 it works, so I've gone that route, rather than trying to deliver the pdf binary back via the lambda. Just in case anyone else stumbles upon this issue and can't get it to work.

Thanks for all the guidelines everyone

[liZe]

Did anyone find a solution to make WeasyPrint work on AWS Lambda with Python 3.8?

[gligorkot]

Just double checked. The latest yum pango version is 1.28.1. That means no font face support https://github.com/Kozea/WeasyPrint/blob/master/weasyprint/fonts.py#L40-L41 :-/ damn...

Has anyone had any luck installing a later version of pango to enable custom font support?

[mightym]

@gligorkot I assume it would work using Python 3.8 because Lambda is using Amazon Linux 2 for that Python runtime. And it seems like there is a newer version of pango available. But as @cpmech mentioned here #1003 (comment) Weasyprint is currently not working reliable on Python 3.8

[gligorkot]

@mightym I might need to go revisit the Python 3.8 container to check the dependencies and if pango is at the 1.38+ version there. Thanks for the reminder!

[mightym]

@gligorkot I didn't have time to so far to check that. Would be great if you could ping me when you found something out :-)

[gligorkot]

@mightym good news, looks like the Python 3.8 container has a version we could use, maybe one of us in this thread will have some time to get it properly working on 3.8 soon?

bash-4.2# yum list pango
Loaded plugins: ovl, priorities
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Available Packages
pango.i686           1.40.4-1.amzn2.0.2           amzn2-core
pango.x86_64           1.40.4-1.amzn2.0.2           amzn2-core

[kalekseev]

Did anyone find a solution to make WeasyPrint work on AWS Lambda with Python 3.8?

Hi, I have open sourced our solution https://github.com/kotify/cloud-print-utils, it builds weasyprint layer for amazon linux 2, here test results of the report sample, feedback would be great.

I believe that problem in @cpmech solution is hard links used to create .so files while that is working solution in general it doesn't work with zip files, zip doesn't preserve links so lambda server end up with two different files for the same library and weasyprint loads .so files while the rest of the stack loads .so.N files that leads to memory problems. We fixed that by patching weasyprint https://github.com/kotify/cloud-print-utils/blob/2c64672d0f20d3b4bae85dff5cfd976eb03b5b0f/layers/weasyprint/builder.sh#L9, I'm going to create pull requests with rationale why it's the right thing to load .so.N files instead of .so.

Another non-trivial thing is to configure pixbuf https://github.com/kotify/cloud-print-utils/blob/2c64672d0f20d3b4bae85dff5cfd976eb03b5b0f/layers/cairo-pixbuf-libffi-pango/builder.sh#L23-L30, otherwise images doesn't work.

[cpmech]

Did anyone find a solution to make WeasyPrint work on AWS Lambda with Python 3.8?

Hi, I have open sourced our solution https://github.com/kotify/cloud-print-utils, it builds weasyprint layer for amazon linux 2, here test results of the report sample, feedback would be great.

I believe that problem in @cpmech solution is hard links used to create .so files while that is working solution in general it doesn't work with zip files, zip doesn't preserve links so lambda server end up with two different files for the same library and weasyprint loads .so files while the rest of the stack loads .so.N files that leads to memory problems. We fixed that by patching weasyprint https://github.com/kotify/cloud-print-utils/blob/2c64672d0f20d3b4bae85dff5cfd976eb03b5b0f/layers/weasyprint/builder.sh#L9, I'm going to create pull requests with rationale why it's the right thing to load .so.N files instead of .so.

Another non-trivial thing is to configure pixbuf https://github.com/kotify/cloud-print-utils/blob/2c64672d0f20d3b4bae85dff5cfd976eb03b5b0f/layers/cairo-pixbuf-libffi-pango/builder.sh#L23-L30, otherwise images doesn't work.

Fantastic! I'll check it out. Thanks!
(may the force be with you)

[gligorkot]

@kalekseev you ABSOLUTE LEGEND!!! Got it working this morning and it's a beauty!

We use serverless to publish our lambda, so I just built the zip, then unzipped in a folder and stuck that as my layer in the serverless stack and it's all go!

[liZe]

Is anyone interested in adding some documentation about AWS?

[liZe]

@kalekseev Should we add a chapter in WeasyPrint’s documentation recommending kotify/cloud-print-utils?

(PS: I think that the current master doesn’t require Kotify’s patch anymore.)

[kalekseev]

@kalekseev Should we add a chapter in WeasyPrint’s documentation recommending kotify/cloud-print-utils?

@liZe I don't see any problem with that, we will support weasyprint layer for the foreseeable future

(PS: I think that the current master doesn’t require Kotify’s patch anymore.)

Great, I will review and release update in a couple of days.

[mojimi]

By the way, aws lambda now supports Docker container images up to 10GB

[Ricardomol]

Hi @kalekseev!

At my company, we are also facing this problem.

@liZe I don't see any problem with that, we will support weasyprint layer for the foreseeable future

Do you mean WeasyPrint will be compatible with Amazon Linux 2 in the near future, out of the box?

[kalekseev]

It's already compatible but you have to add native dependencies on your own, see my repo for inspiration.

…

On Thu, Aug 12, 2021, at 20:19, Ricardo Molina wrote: Hi @kalekseev <https://github.com/kalekseev>! At my company, we are also facing this problem. > @liZe <https://github.com/liZe> I don't see any problem with that, we will support weasyprint layer for the foreseeable future > Do you mean WeasyPrint will be compatible with Amazon Linux 2 in the near future, out of the box? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1003 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACZVG6D6FJRYJD3LQRFB3TT4P7BFANCNFSM4JWGLCEA>.

-- Konstantin Alekseev

[liZe]

The chapter about AWS has been included in documentation, that was the goal of this issue. Feel free to open new issues if you have problems with AWS!

[pimmee]

For anyone facing font rendering issues with kotify's cloud-print-utils lambda layer, you have to set environment variables (which they mention in the README)

Environment variables expected by layer:

    GDK_PIXBUF_MODULE_FILE="/opt/lib/loaders.cache"
    FONTCONFIG_PATH="/opt/fonts"
    XDG_DATA_DIRS="/opt/lib"

[MahmoudElkayal]

I have said the environment variables as mentioned, and it is working and I am getting a pdf outputted but the font is not right and getting the following error

' Function Logs
Fontconfig error: Cannot load default config file: No such file: (null)
/opt/python/lib/python3.12/site-packages/weasyprint/text/fonts.py:63: UserWarning: FontConfig cannot load default config file. Expect ugly output.
warn('FontConfig cannot load default config file. Expect ugly output.')'

Any thoughts why?