How to properly handle situation when you need to rotate an image, drawImage() another image in that rectangle, then rotate the image back?

[MissedSte4k]

I need your guys help. I have 1 Image, that I want to blur/not blur some parts off, but there are cases, where some blur/not blur overlap. For unbluring - I'm just taking the original image, and pasting the rectangle back in the place.

Let's take this image for example:

`using (SixLabors.ImageSharp.Image originalImage = SixLabors.ImageSharp.Image.Load(buffer.ToArray()))
using (SixLabors.ImageSharp.Image bluredImage = SixLabors.ImageSharp.Image.Load(buffer.ToArray()))
using (SixLabors.ImageSharp.Image resultImage = SixLabors.ImageSharp.Image.Load(buffer.ToArray()))
{
    //generate blurred iamge
    bluredImage.Mutate(image => image.GaussianBlur(5, new SixLabors.ImageSharp.Rectangle(0, 0, originalImage.Size.Width, originalImage.Size.Height))); 
    for (int i = 0; i < resultList.Count; i++)
{
	var blur = pairs[i].Blur;
	foreach (var item in resultList[i])
	{
		if (blur)
		{
			if (item.Rotation == 0)
			{
				resultImage.Mutate(img => img.DrawImage(bluredImage, new SixLabors.ImageSharp.Point(item.Rect.X, item.Rect.Y), new SixLabors.ImageSharp.Rectangle(item.Rect.X, item.Rect.Y, item.Rect.Width, item.Rect.Height), 1));
			}
			else
			{
				ImageHelper.CopyAndRotateImageSharp(bluredImage, resultImage, item.Rect, item.Rotation);
			}
		}
		else
		{
			if (item.Rotation == 0)
			{
				resultImage.Mutate(img => img.DrawImage(originalImage, new SixLabors.ImageSharp.Point(item.Rect.X, item.Rect.Y), new SixLabors.ImageSharp.Rectangle(item.Rect.X, item.Rect.Y, item.Rect.Width, item.Rect.Height), 1));
			}
			else
			{
				ImageHelper.CopyAndRotateImageSharp(originalImage, resultImage, item.Rect, item.Rotation);
			}
		}
	}
    
    
}
`

Everything works fine, when rotation is not involved:

I have predetermined locations where I want to blur/unblur the objects. But in some cases - there is rotation involved. With System.Drawing.Graphics something like this looks possible, but - for some reason, graphics seems not to be rotated when copying:

public static void CopyAndRotateImage(Bitmap fromBitmap, Graphics destGraphics, Rectangle rect, float rotationAngle)
{
	using (var g = Graphics.FromImage(fromBitmap))
	{
	Graphics g; //source graphics
	Graphics destGraphics 
	g.TranslateTransform(rect.X + rect.Width / 2, rect.Y);
	g.RotateTransform(+rotationAngle);
	g.TranslateTransform(-rect.X - rect.Width / 2, -rect.Y);
	destGraphics.TranslateTransform(rect.X + rect.Width / 2, rect.Y);
	destGraphics.RotateTransform(+rotationAngle);
	destGraphics.TranslateTransform(-rect.X - rect.Width / 2, -rect.Y);
	destGraphics.DrawImage(fromBitmap, rect, rect, GraphicsUnit.Pixel);
	g.ResetTransform();
	destGraphics.ResetTransform();
	}
}

The Image part of rectangle gets copied, but it get's copied not rotated, so I had to switch libraries...

As you see in this case - I want to blur the car, but keep the License plates visible.
So - I've just translated the same code over here:

fromImage.Mutate(img => img.Transform(new AffineTransformBuilder()
	.AppendTranslation(new SixLabors.ImageSharp.PointF(rect.X + rect.Width / 2, rect.Y))
	.AppendRotationDegrees(rotationAngle)
	.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.X - rect.Width / 2, -rect.Y))));

toImage.Mutate(img => img.Transform(new AffineTransformBuilder()
	.AppendTranslation(new SixLabors.ImageSharp.PointF(rect.X + rect.Width / 2, rect.Y))
	.AppendRotationDegrees(rotationAngle)
	.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.X - rect.Width / 2, -rect.Y))));

toImage.Mutate(img => img.DrawImage(fromImage, new SixLabors.ImageSharp.Point(rect.X, rect.Y), new SixLabors.ImageSharp.Rectangle(rect.X, rect.Y, rect.Width, rect.Height), 1));

toImage.Mutate(img => img.Transform(new AffineTransformBuilder()
	.AppendTranslation(new SixLabors.ImageSharp.PointF(rect.X + rect.Width / 2, rect.Y))
	.AppendRotationDegrees(-rotationAngle)
	.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.X - rect.Width / 2, -rect.Y))));

fromImage.Mutate(img => img.Transform(new AffineTransformBuilder()
	.AppendTranslation(new SixLabors.ImageSharp.PointF(rect.X + rect.Width / 2, rect.Y))
	.AppendRotationDegrees(-rotationAngle)
	.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.X - rect.Width / 2, -rect.Y))));

The result was...

It seems, that the view get's stretched, so the new coordinates are no longer the same ones.
I've played around with more ideas that I got when looking at results, but got to more or less the same spot.

Like: half merging mutate:

	toImage.Mutate(img => img.Transform(new AffineTransformBuilder()
		.AppendTranslation(new SixLabors.ImageSharp.PointF(rect.X + rect.Width / 2, rect.Y))
		.AppendRotationDegrees(rotationAngle)
		.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.Width / 2, 0)))
		.DrawImage(fromImage, new SixLabors.ImageSharp.Rectangle(rect.X, rect.Y, rect.Width, rect.Height), 1)
		.Transform(new AffineTransformBuilder()
		.AppendTranslation(new SixLabors.ImageSharp.PointF(+rect.Width / 2, 0))
		.AppendRotationDegrees(-rotationAngle)
		.AppendTranslation(new SixLabors.ImageSharp.PointF(-rect.X - rect.Width / 2, -rect.Y))
		)
	);

Which perhaps did the work, just that the coordinate shift made it not visible and the from image was not actually rotated. I've also tried more ideas - nothing worked so far, so I came here.

Anyone got any ideas how problem like this could be solvable?

[JimBobSquarePants]

I haven’t looked at your sample in detail but one thing immediately stands out. You don’t need to apply the translation like System.Drawing. We already rotate around the centre.

[MissedSte4k]

My main issue probably is the fact, that I want to do the rotations around the center of rectangle that I want to copy-paste later. What I wanted to do System.Drawing was basically:

1. Go to the center of rectangle in both images
2. Rotate everything by x degrees
3. Copy from source to destination
4. Rotate back by x degrees
5. go to starting location

In ImageSharp as far as I managed to accomplish after finding some related discussions (sadly I stumbled on them a bit after creating the post):

1. create a new image by cloning the old one that is rotated based on my point

using var rotatedClone = fromImage.Clone(x => x.Transform(new AffineTransformBuilder().AppendRotationDegrees(rotationAngle, new Vector2(rect.X + rect.Width / 2, rect.Y + rect.Height / 2))));

2. rotate the result image on rectangle center point, then draw the image, rotate the image back

`toImage.Mutate(img => img.Transform(new AffineTransformBuilder()
	.AppendRotationDegrees(rotationAngle, new Vector2(rect.X + rect.Width / 2, rect.Y + rect.Height / 2)))
	//TODO the center point and rectangle position changed in both rotatedClone and toImage at this point and I don't know how to calculate them again
	.DrawImage(rotatedClone, new SixLabors.ImageSharp.Rectangle(rect.X, rect.Y, rect.Width, rect.Height), 1)
	.Transform(new AffineTransformBuilder()
	.AppendRotationDegrees(-rotationAngle, new Vector2(rect.X + rect.Width / 2, rect.Y + rect.Height / 2))
	)
	//TODO will probably need to cut the autogenerated empty space, that appeared after rotations
);`

If I could use Vector2 or something else to also decide the rotation when doing DrawImage() - would be great, but it seems - I can't. So in my case the center position changes and I've got stuck trying to find the new positions, or DrawImage()/Rotation is done at the wrong place. If it would be in the center of the image - wouldn't be hard to calculate them, but since it can be anywhere - I'm a bit overwhelmed by the math.

[tocsoft]

i've just create a samples repo PR with an sample for producing this effect with out all the repeated transforms SixLabors/Samples#33.

The linked PR/sample use our clip api to build up the final complex polygon you want the blur (or any processing operation) applied to as a single operation, no need to have lots of parts of images that you need to manually need to draw.

It wasn't 100% clear from your sample the final desired behaviour with layering multiple blur/unblur operations with multiple overlapping regions so there are 2 versions of the helper in the sample for the 2 behaviours I could think might be wanted.

Basically the process is as such, you create a single complex polygon that covers all parts of the image you want blurred, then call Clip() with the polygon and the operation you applied inside the polygon.

Source image

Result image

Direct link to sample program code

https://github.com/SixLabors/Samples/blob/85702b5354a1fb6cf1c39488f301f5d9e378bf79/ImageSharp/BlurMultiplePartsOfImage/Program.cs
https://github.com/SixLabors/Samples/blob/67ffd43bf3dae46a659fa4deb9eaa23af6ca7302/ImageSharp/BlurMultiplePartsOfImage/Program.cs (updated sample)

[MissedSte4k]

Wow, you guys are awesome. Thank you so much, that's more or less what I needed. The only addition that still bothers me is the fact, that some of my rectangles have rotation assigned to them, the BlurRegion in my case looks like this (sorry for not specifying instantly):

public class BlurRegion
        {
            public RectangleF Rect { get; set; }
            public bool Blur { get; set; }
            public float Rotation {get; set}
        }

So the end result I'm searching for is something like:

In this case - I blur the face, but unblur the phone. As you see - both are independently rotate from one to another.

I've looked into SixLabors.ImageSharp.Drawing, but couldn't find any other shapes beside RegularPolygon that can have rotation assigned to them. The only alternative I see is Polygon, which will require doing some additional work, but it's way better, than what I had before I guess.

Thank you once again and if you would have any ideas for rotated rectangle would be great, but if not - that's ok too, I will try to solve my issue using your sample and with Polygon and Points (if that's possible), that will require some trigonometry, but way less than I've been dealing with until now.

[tocsoft]

All paths/polygons has a transform method that can be used with a Matrix to rotate them once they are created.

Once I'm back on a computer I'll update my sample to all rotating each region.

[tocsoft]

Sample has been updated with rotation support

relevant code you will need to generate each region of interest to combine is this.

...
public IPath AsPath()
    => new RectangularPolygon(Rect.X, Rect.Y, Rect.Width, Rect.Height)
        .Transform(
            Matrix3x2Extensions.CreateRotationDegrees(
                Rotation,
                new PointF((Rect.Width / 2 + Rect.X), (Rect.Height / 2 + Rect.Y))
            )
        );
...

https://github.com/SixLabors/Samples/blob/67ffd43bf3dae46a659fa4deb9eaa23af6ca7302/ImageSharp/BlurMultiplePartsOfImage/Program.cs#L25-L32

[MissedSte4k]

You guys are awesome. Thank you so much ❤