Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pillow 11.0.0 and Django: TiffImagePlugin raised 'Invalid dimensions' error #8530

Open
jedie opened this issue Nov 4, 2024 · 11 comments · May be fixed by #8535
Open

Pillow 11.0.0 and Django: TiffImagePlugin raised 'Invalid dimensions' error #8530

jedie opened this issue Nov 4, 2024 · 11 comments · May be fixed by #8535
Labels

Comments

@jedie
Copy link

jedie commented Nov 4, 2024

Seems that v11.0.0 release add a Bug parsing tiff images:

...
  File "django/core/files/images.py", line 63, in get_image_dimensions
    p.feed(data)
  File "PIL/ImageFile.py", line 471, in feed
    im = Image.open(fp)
  File "PIL/Image.py", line 3515, in open
    im = _open_core(fp, filename, prefix, formats)
  File "PIL/Image.py", line 3503, in _open_core
    im = factory(fp, filename)
  File "PIL/TiffImagePlugin.py", line 1153, in __init__
    super().__init__(fp, filename)
  File "PIL/ImageFile.py", line 144, in __init__
    self._open()
  File "PIL/TiffImagePlugin.py", line 1177, in _open
    self._seek(0)
  File "PIL/TiffImagePlugin.py", line 1248, in _seek
    self._setup()
  File "PIL/TiffImagePlugin.py", line 1426, in _setup
    raise ValueError(msg)

more info in Sentry message:

grafik

Runtime: CPython v3.11.10(3.11.10 (main, Oct 19 2024, 01:04:28) [GCC 12.2.0])

It works with Pillow 10.4.0

@radarhere
Copy link
Member

Could you attach a copy of the image?

@jedie
Copy link
Author

jedie commented Nov 4, 2024

no, sorry :( ...

@hugovk hugovk added the TIFF label Nov 4, 2024
@radarhere
Copy link
Member

I'm guessing that you mean you can't for copyright/privacy reasons, but you still have a copy of the image you can test with? You can still replicate the problem on your end? Otherwise, you couldn't have determined that this passed with Pillow 10.4.0.

Are you sure that the image isn't actually invalid? Are you able to open the image using other software?

Are you able to modify your copy of Pillow to print xsize and ysize so we can see exactly how they are invalid?

If you're unable to upload the image, can you run exiftool -v -v over the image and paste the output here?

@jedie
Copy link
Author

jedie commented Nov 4, 2024

Yes, it's about copyright.

I can play with the image locally. It's very strange, i can't reproduce with:

import sys

import PIL
from PIL import Image

print(f'{sys.version=}')
print(f'{PIL.__version__=}')

with open("/tmp/foobar.tiff", "rb") as fp:
    im = Image.open(fp)
    print(f'{im.size=}')

It works in both cases:

sys.version='3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]'
PIL.__version__='10.4.0'
im.size=(4000, 4000)
sys.version='3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]'
PIL.__version__='11.0.0'
im.size=(4000, 4000)

exiftool:

  ExifToolVersion = 12.76
  FileName = foobar.tiff
  Directory = /tmp
  FileSize = 9666494
  FileModifyDate = 1730711096
  FileAccessDate = 1730711097
  FileInodeChangeDate = 1730711096
  FilePermissions = 33204
  FileType = TIFF
  FileTypeExtension = TIF
  MIMEType = image/tiff
  ExifByteOrder = II
  + [IFD0 directory with 22 entries]
  | 0)  ImageWidth = 4000
  |     - Tag 0x0100 (2 bytes, int16u[1])
  | 1)  ImageHeight = 4000
  |     - Tag 0x0101 (2 bytes, int16u[1])
  | 2)  BitsPerSample = 8 8 8
  |     - Tag 0x0102 (6 bytes, int16u[3])
  | 3)  Compression = 8
  |     - Tag 0x0103 (2 bytes, int16u[1])
  | 4)  PhotometricInterpretation = 2
  |     - Tag 0x0106 (2 bytes, int16u[1])
  | 5)  StripOffsets = 8 1460 2912 4488 5986 7407 8887 10362 11801 13256 14743 16231 17[snip]
  |     - Tag 0x0111 (16000 bytes, int32u[4000])
  | 6)  Orientation = 1
  |     - Tag 0x0112 (2 bytes, int16u[1])
  | 7)  SamplesPerPixel = 3
  |     - Tag 0x0115 (2 bytes, int16u[1])
  | 8)  RowsPerStrip = 1
  |     - Tag 0x0116 (2 bytes, int16u[1])
  | 9)  StripByteCounts = 1452 1452 1576 1498 1421 1480 1475 1439 1455 1487 1488 1402 1[snip]
  |     - Tag 0x0117 (16000 bytes, int32u[4000])
  | 10) XResolution = 300 (300/1)
  |     - Tag 0x011a (8 bytes, rational64u[1])
  | 11) YResolution = 300 (300/1)
  |     - Tag 0x011b (8 bytes, rational64u[1])
  | 12) PlanarConfiguration = 1
  |     - Tag 0x011c (2 bytes, int16u[1])
  | 13) ResolutionUnit = 2
  |     - Tag 0x0128 (2 bytes, int16u[1])
  | 14) Software = Adobe Photoshop Elements 6.0 Windows
  |     - Tag 0x0131 (37 bytes, string[37])
  | 15) ModifyDate = 2021:11:12 10:46:02
  |     - Tag 0x0132 (20 bytes, string[20])
  | 16) Predictor = 2
  |     - Tag 0x013d (2 bytes, int16u[1])
  | 17) ApplicationNotes (SubDirectory) -->
  |     - Tag 0x02bc (3670 bytes, undef[3670])
  | + [XMP directory, 3670 bytes]
  | | XMPToolkit = XMP Core 5.5.0
  | | Format = image/tiff
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/dc:format'
  | | ColorMode = 3
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/photoshop:ColorMode'
  | | History = 
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/photoshop:History'
  | | ICCProfileName = sRGB IEC61966-2.1
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/photoshop:ICCProfile'
  | | CreateDate = 2011-03-30T14:11:25+02:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmp:CreateDate'
  | | CreatorTool = Adobe Photoshop Elements 6.0 Windows
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmp:CreatorTool'
  | | MetadataDate = 2021-11-12T10:46:02+01:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmp:MetadataDate'
  | | ModifyDate = 2021-11-12T10:46:02+01:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmp:ModifyDate'
  | | DocumentID = xmp.did:C9172B3515206811994CDBA4105FCF75
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:DocumentID'
  | | InstanceID = uuid:8511B8538A5BE011A5A9C61357626A13
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:InstanceID'
  | | OriginalDocumentID = xmp.did:C9172B3515206811994CDBA4105FCF75
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:OriginalDocumentID'
  | | HistoryAction = created
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 10/xmpMM:action'
  | | HistoryInstanceID = xmp.iid:C9172B3515206811994CDBA4105FCF75
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 10/xmpMM:instanceID'
  | | HistorySoftwareAgent = Adobe Photoshop CS4 Macintosh
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 10/xmpMM:softwareAgent'
  | | HistoryWhen = 2011-03-30T14:11:25+02:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 10/xmpMM:when'
  | | HistoryAction = produced
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 11/xmpMM:action'
  | | HistorySoftwareAgent = Affinity Photo 1.10.1
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 11/xmpMM:softwareAgent'
  | | HistoryWhen = 2021-11-08T14:43:02+01:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 11/xmpMM:when'
  | | HistoryAction = produced
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 12/stEvt:action'
  | | HistorySoftwareAgent = Affinity Photo 1.10.1
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 12/stEvt:softwareAgent'
  | | HistoryWhen = 2021-11-12T10:46:02+01:00
  | | - Tag 'x:xmpmeta/rdf:RDF/rdf:Description/xmpMM:History/rdf:Seq/rdf:li 12/stEvt:when'
  | 18) PhotoshopSettings (SubDirectory) -->
  |     - Tag 0x8649 (28 bytes, undef[28])
  | + [Photoshop directory, 28 bytes]
  | | IPTCDigest = .............B~
  | | - Tag 0x0425 (16 bytes)
  | 19) ExifOffset (SubDirectory) -->
  |     - Tag 0x8769 (4 bytes, int32u[1])
  | + [ExifIFD directory with 3 entries]
  | | 0)  ColorSpace = 1
  | |     - Tag 0xa001 (2 bytes, int16u[1])
  | | 1)  ExifImageWidth = 4000
  | |     - Tag 0xa002 (2 bytes, int16u[1])
  | | 2)  ExifImageHeight = 4000
  | |     - Tag 0xa003 (2 bytes, int16u[1])
  | 20) ICC_Profile (SubDirectory) -->
  |     - Tag 0x8773 (596 bytes, undef[596])
  | + [ICC_Profile directory with 11 entries, 596 bytes]
  | | ProfileHeader (SubDirectory) -->
  | | + [BinaryData directory, 128 bytes]
  | | | ProfileCMMType = lcms
  | | | - Tag 0x0004 (4 bytes, string[4])
  | | | ProfileVersion = 1072
  | | | - Tag 0x0008 (2 bytes, int16s[1])
  | | | ProfileClass = mntr
  | | | - Tag 0x000c (4 bytes, string[4])
  | | | ColorSpaceData = RGB 
  | | | - Tag 0x0010 (4 bytes, string[4])
  | | | ProfileConnectionSpace = XYZ 
  | | | - Tag 0x0014 (4 bytes, string[4])
  | | | ProfileDateTime = 2021 11 12 9 44 25
  | | | - Tag 0x0018 (12 bytes, int16u[6])
  | | | ProfileFileSignature = acsp
  | | | - Tag 0x0024 (4 bytes, string[4])
  | | | PrimaryPlatform = MSFT
  | | | - Tag 0x0028 (4 bytes, string[4])
  | | | CMMFlags = 0
  | | | - Tag 0x002c (4 bytes, int32u[1])
  | | | DeviceManufacturer = 
  | | | - Tag 0x0030 (4 bytes, string[4])
  | | | DeviceModel = 
  | | | - Tag 0x0034 (4 bytes, string[4])
  | | | DeviceAttributes = 0 0
  | | | - Tag 0x0038 (8 bytes, int32u[2])
  | | | RenderingIntent = 0
  | | | - Tag 0x0040 (4 bytes, int32u[1])
  | | | ConnectionSpaceIlluminant = 0.9642 1 0.82491
  | | | - Tag 0x0044 (12 bytes, fixed32s[3])
  | | | ProfileCreator = lcms
  | | | - Tag 0x0050 (4 bytes, string[4])
  | | | ProfileID = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  | | | - Tag 0x0054 (16 bytes, int8u[16])
  | | 0)  ProfileDescription = sRGB IEC61966-2.1
  | |     - Tag 'desc' (34 bytes, type 'mluc')
  | | 1)  ProfileCopyright = No copyright, use freely
  | |     - Tag 'cprt' (48 bytes, type 'mluc')
  | | 2)  MediaWhitePoint = 0.9642 1 0.82491
  | |     - Tag 'wtpt' (20 bytes, type 'XYZ ')
  | | 3)  ChromaticAdaptation = 1.04788 0.02292 -0.05022 0.02959 0.99048 -0.01707 -0.00[snip]
  | |     - Tag 'chad' (44 bytes, type 'sf32')
  | | 4)  RedMatrixColumn = 0.43604 0.22249 0.01392
  | |     - Tag 'rXYZ' (20 bytes, type 'XYZ ')
  | | 5)  BlueMatrixColumn = 0.14305 0.06061 0.71391
  | |     - Tag 'bXYZ' (20 bytes, type 'XYZ ')
  | | 6)  GreenMatrixColumn = 0.38512 0.7169 0.09706
  | |     - Tag 'gXYZ' (20 bytes, type 'XYZ ')
  | | 7)  RedTRC = para..ff...Y...[
  | |     - Tag 'rTRC' (32 bytes, type 'para')
  | | 8)  GreenTRC = para..ff...Y...[
  | |     - Tag 'gTRC' (32 bytes, type 'para')
  | | 9)  BlueTRC = para..ff...Y...[
  | |     - Tag 'bTRC' (32 bytes, type 'para')
  | | 10) Chromaticity (SubDirectory) -->
  | |     - Tag 'chrm' (36 bytes, type 'chrm')
  | | + [BinaryData directory, 36 bytes]
  | | | ChromaticityChannels = 3
  | | | - Tag 0x0008 (2 bytes, int16u[1])
  | | | ChromaticityColorant = 0
  | | | - Tag 0x000a (2 bytes, int16u[1])
  | | | ChromaticityChannel1 = 0.64 0.33
  | | | - Tag 0x000c (8 bytes, fixed32u[2])
  | | | ChromaticityChannel2 = 0.3 0.60001
  | | | - Tag 0x0014 (8 bytes, fixed32u[2])
  | | | ChromaticityChannel3 = 0.14999 0.06
  | | | - Tag 0x001c (8 bytes, fixed32u[2])
  | 21) Exif_0xc7e0 = 0 255 75 83 2 0 112 79 120 69 1 0 9 0 0 0 49 115 116 112 79 1 0 0[snip]
  |     - Tag 0xc7e0 (110 bytes, int8u[110])

But this runs on a different machine...

@jedie
Copy link
Author

jedie commented Nov 4, 2024

I also tried with Python 3.11:

sys.version='3.11.10 (main, Oct 16 2024, 04:23:21) [Clang 18.1.8 ]'
PIL.__version__='11.0.0'
im.size=(4000, 4000)

Strange. Any idea?

@radarhere
Copy link
Member

If you're asking how Django could have manipulated the image before passing it to Pillow, then that sounds like a Django question more than a Pillow question.

If you're asking how to debug this further,

  1. I would suggest modifying either Django or Pillow locally to write the image data to disk at the exact point that it is passed to Pillow.
  2. If that then gives you an image that causes a problem without Django, then I suggest checking if that image opens in other software. If it doesn't, then it sounds like the image is corrupt, and not necessarily something we need to be opening successfully.
  3. If it does, then I suggest modifying Pillow to print xsize and ysize just before the ValueError is raised.

@jedie
Copy link
Author

jedie commented Nov 4, 2024

I can not say what's happening here... But i can say that's the combination between Django v4.2.16 and pillow '10.4.0' vs. '11.0.0'

>>> import django
>>> django.__version__
'4.2.16'
>>> Image.open((x.thumbnail.open('rb'))).width
4000
>>> import PIL
>>> PIL.__version__
'11.0.0'
>>> x.thumbnail.width
/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/TiffImagePlugin.py:935: UserWarning: Corrupt EXIF data.  Expecting to read 2 bytes but only got 0. 
  warnings.warn(str(msg))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/django/core/files/images.py", line 20, in width
    return self._get_image_dimensions()[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/django/core/files/images.py", line 30, in _get_image_dimensions
    self._dimensions_cache = get_image_dimensions(self, close=close)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/django/core/files/images.py", line 63, in get_image_dimensions
    p.feed(data)
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/ImageFile.py", line 471, in feed
    im = Image.open(fp)
         ^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/Image.py", line 3515, in open
    im = _open_core(fp, filename, prefix, formats)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/Image.py", line 3503, in _open_core
    im = factory(fp, filename)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/TiffImagePlugin.py", line 1153, in __init__
    super().__init__(fp, filename)
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/ImageFile.py", line 144, in __init__
    self._open()
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/TiffImagePlugin.py", line 1177, in _open
    self._seek(0)
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/TiffImagePlugin.py", line 1248, in _seek
    self._setup()
  File "/root/.local/share/virtualenvs/foobar/lib/python3.11/site-packages/PIL/TiffImagePlugin.py", line 1426, in _setup
    raise ValueError(msg)
ValueError: Invalid dimensions
>>> 

Think it's related to the way how Django tries to get the image size: https://github.com/django/django/blob/968397228fe03968bb855856532569586c8a8a1c/django/core/files/images.py#L35-L89

@jedie
Copy link
Author

jedie commented Nov 4, 2024

Yes, it's the combination of how Django tries to get the image size via django.core.files.images.get_image_dimensions() here: https://github.com/django/django/blob/968397228fe03968bb855856532569586c8a8a1c/django/core/files/images.py#L35-L89 and this change: e6e5ef5#diff-6ad43f85f1a075181d4d8cfcd97ae27bba1eccf5c3db5a3457160f98218eba6eR1404 that added the raise ValueError:

        xsize = self.tag_v2.get(IMAGEWIDTH)
        ysize = self.tag_v2.get(IMAGELENGTH)
        if not isinstance(xsize, int) or not isinstance(ysize, int):
            msg = "Invalid dimensions"
            raise ValueError(msg)

@jedie
Copy link
Author

jedie commented Nov 4, 2024

I created https://code.djangoproject.com/ticket/35884 and django/django#18764

@jedie jedie changed the title Pillow 11.0.0: TiffImagePlugin raised 'Invalid dimensions' error Pillow 11.0.0 and Django: TiffImagePlugin raised 'Invalid dimensions' error Nov 4, 2024
@sarahboyce
Copy link

Hello 👋 Django Fellow here

In Django we are using Pillow as documented to parse an image: https://pillow.readthedocs.io/en/stable/reference/ImageFile.html#example-parse-an-image

Using this this file python-logo.tiff and following the example

from PIL import ImageFile

fp = open("python-logo.tiff", "rb")

p = ImageFile.Parser()

while 1:
    s = fp.read(1024)
    if not s:
        break
    p.feed(s)

im = p.close()

im.save("copy.tiff")

This raises the ValueError: Invalid dimensions error on Pillow 11

Traceback (most recent call last):
  File "/project-path/./main.py", line 11, in <module>
    p.feed(s)
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/ImageFile.py", line 471, in feed
    im = Image.open(fp)
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/Image.py", line 3520, in open
    im = _open_core(
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/Image.py", line 3503, in _open_core
    im = factory(fp, filename)
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/TiffImagePlugin.py", line 1153, in __init__
    super().__init__(fp, filename)
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/ImageFile.py", line 144, in __init__
    self._open()
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/TiffImagePlugin.py", line 1177, in _open
    self._seek(0)
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/TiffImagePlugin.py", line 1248, in _seek
    self._setup()
  File "/project-path/.venv/lib/python3.10/site-packages/PIL/TiffImagePlugin.py", line 1426, in _setup
    raise ValueError(msg)
ValueError: Invalid dimensions

No error is raised on Pillow 10.4.0

We can add new error handling in Django but before doing so, I would like to confirm that this is not an issue in Pillow 11 and that the documented example is missing error handling also

@radarhere
Copy link
Member

Hi. Thanks for breaking the problem down. The code change that led to this wasn't intended to change behaviour, and your use case is reasonable.

I've created #8535 to resolve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants