The reference frame associated to an image is not the best one to perform a shear transformation

[empet]

Shear transformations are defined here https://en.wikipedia.org/wiki/Shear_mapping.

using Images, CoordinateTransformations, ImageTransformations, Plots

function  vertical_shear(α)  #vertical shear because it preserves the direction e₂=[0,1]
    M=[1 0; tan(α) 1]
    LinearMap(M)
end 
img = load("xorimg.png")

imgw = warp(img, vertical_shear(π/9), fillvalue=1)

Surprise!! The vertical shear acted as an horizontal shear transformation:

function horizontal_shear(α)  #preserves the horizontal direction,  [1,0]
    M =[1 tan(α); 0 1]
    LinearMap(M) 
end 

I spent two hours to find out why this simple linear map led to an unexpected effect.
Since the shear transformation reversed the vertical to the horizontal preserved direction,
and conversely, I concluded that in the warp implementation, the image is referenced by default to the system
of coordinates with origin at the upper-left corner, the left side is xaxis, and the top side is yaxis,
i.e. for a pixel in the position (i, j), x=i, y=j.
This particular system of coordinates works well in the case of rotations, because the rotation has no invariant direction.
But it doesn't work for a vertical and horizontal shear, because xaxis isn't associated to the horizontal direction, and yaxis to the vertical one.

In order to get the right shear, as well as any linear or affine transformation of an image,
that image should be referenced to the orthonormal frame with origin D, as in the next figure, xaxis Dxₚ,
yaxis Dyₚ

To check my hypothesis I wrote the same code in Python, to performe the shear transformation
via scipy.ndimage, and got the right transformed image.

[johnnychen94]

A good catch. This might be due to the inconsistency of the axis order. In JuliaImages, we take the following order convention(x as the first axis, y as the second axis), so vertical_shear is indeed an horizontal_shear.

I think this is mostly a documentation issue and should be explained somewhere here or in the main JuliaImages documentation. See also JuliaImages/juliaimages.github.io#173

[empet]

I'm wondering why you made this choice for the image coordinate system. I've worked with images in C/C++, Java, Python, Matlab and all of them consider the image coordinate system as in my last figure posted above.
Also Makie displays the images upside down, most likely due to an improper choice of the reference frame

[johnnychen94]

I'm wondering why you made this choice for the image coordinate system.

I'm not the first one to introduce this notation, but I'll try to explain.
This is perhaps due to the fact that JuliaImages tries to make things dimensionality-agnostic -- many algorithms intrinsically support n-dimensional input. And if we take this into consideration, we typically call the axes "the first dimension", the second dimension", and "the i-th dimension". Furthermore, it's itself consistent to name the first dimension as the "x-axis", the second dimension as the "y-axis", the third "z-axis". In this sense, the x/y/z-notation is for computation in contrast to the commonly used camera coordinate system as you expected.

Also Makie displays the images upside down, most likely due to an improper choice of the reference frame

I believe all Julia image IO backends does this axis re-order, for instance, PNGFiles, JpegTurbo.jl so image(testimage("cameraman")) gives some strange display..

I'm not sure if it's a good idea to enforce the camera coordinate system into JuliaImages -- it's perhaps too late to do this kind of breaking change. Maybe we'll just need to document these two types of coordinates: one for computation, and one for visualization/camera.

also ping @timholy @zygmuntszpak for some inputs

[juliohm]

Adding to the conversation. I think the OP may be interested in Meshes.jl Grid subtypes and the transforms we define over there. We map the x/y/z as described but rotate the plots in MeshViz.jl when necessary. Em seg., 16 de jan. de 2023 10:19, Johnny Chen ***@***.***> escreveu:

…

I'm wondering why you made this choice for the image coordinate system. I'm not the first one to introduce this notation, but I'll try to explain. This is perhaps due to the fact that JuliaImages tries to make things dimensionality-agnostic -- many algorithms intrinsically support n-dimensional input. And if we take this into consideration, we typically call the axes "the first dimension", the second dimension", and "the i-th dimension". Furthermore, it's itself consistent to name the first dimension as the "x-axis", the second dimension as the "y-axis", the third "z-axis". In this sense, the x/y/z-notation is for computation in contrast to the commonly used camera coordinate system as you expected. Also Makie displays the images upside down, most likely due to an improper choice of the reference frame I believe all Julia image IO backends does this axis re-order, for instance, PNGFiles <https://github.com/JuliaIO/PNGFiles.jl/blob/5e12270422c692933706018dec1dc611f2c9216c/src/io.jl#L216-L217>, JpegTurbo.jl <https://github.com/JuliaIO/JpegTurbo.jl/blob/33d53e772cb3c11c0f49082d6493b06bec6cbaea/src/decode.jl#L100-L104> so image(testimage("cameraman")) gives some strange display.. I'm not sure if it's a good idea to enforce the camera coordinate system into JuliaImages -- it's perhaps too late to do this kind of breaking change. Maybe we'll just need to document these two types of coordinates: one for computation, and one for visualization/camera. also ping @timholy <https://github.com/timholy> @zygmuntszpak <https://github.com/zygmuntszpak> for some inputs — Reply to this email directly, view it on GitHub <#162 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZQW3OGMPBHVRHVYZFJVITWSUVDJANCNFSM6AAAAAAT4A5K5E> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[empet]

Thank you, I can work very well with ImageTransformation.jl and except the shear transformation I haven't encountered any problem. I managed very quickly to define structures for transforming an image by a complex function, in order to highlight the properties of functions defined on a disk or a rectangle in the complex plane, then transforming an image into a triangular one, polar transformations, and more.

[timholy]

I've worked with images in C/C++, Java, Python, Matlab and all of them consider the image coordinate system as in my last figure posted above.

We use a similar convention as Matlab. It comes from the fact that (1) both Matlab & Julia store matrices in column-fastest ("column-major") order, and that (2) for a matrix, in A[i, j] i means row (the vertical coordinate) and j means column (the horizontal coordinate). Hence we display images in a graphical window in the same way they show in the REPL.

The origin of the failure is, I believe, linear algebra notation dating back a few hundred years, which introduced a conflict between Cartesian ordering and matrix representations. So definitely send them a bug report 😉 .