Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a faster deepcopy library #2030

Merged
merged 1 commit into from
Nov 22, 2024
Merged

Use a faster deepcopy library #2030

merged 1 commit into from
Nov 22, 2024

Conversation

Juneezee
Copy link
Contributor

PR Details

Description

Related Issue

Closes #2029.

Motivation and Context

Benchmark:

package excelize

import (
	"testing"

	olddeepcopy "github.com/mohae/deepcopy"
	newdeepcopy "github.com/tiendc/go-deepcopy"
)

func BenchmarkCopyXlsxCol(b *testing.B) {
	src := xlsxCol{}

	b.Run("old", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			olddeepcopy.Copy(src)
		}
	})

	b.Run("new", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			var dst xlsxCol
			newdeepcopy.Copy(&dst, src)
		}
	})
}

func BenchmarkCopyXlsxRow(b *testing.B) {
	src := xlsxRow{}

	b.Run("old", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			olddeepcopy.Copy(src)
		}
	})

	b.Run("new", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			var dst xlsxRow
			newdeepcopy.Copy(&dst, src)
		}
	})
}

func BenchmarkCopyXlsxWorksheet(b *testing.B) {
	f, err := prepareTestBook1()
	if err != nil {
		b.Fatal(err)
	}

	idx, err := f.NewSheet("CopySheet")
	if err != nil {
		b.Fatal(err)
	}

	fromSheet := f.GetSheetName(idx)
	src, err := f.workSheetReader(fromSheet)
	if err != nil {
		b.Fatal(err)
	}

	b.Run("old", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			olddeepcopy.Copy(src)
		}
	})

	b.Run("new", func(b *testing.B) {
		for i := 0; i < b.N; i++ {
			var dst xlsxWorksheet
			newdeepcopy.Copy(&dst, src)
		}
	})
}

Result:

goos: linux
goarch: amd64
pkg: github.com/xuri/excelize/v2
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
BenchmarkCopyXlsxCol/old-16      	  446193	      2483 ns/op	     224 B/op	      13 allocs/op
BenchmarkCopyXlsxCol/new-16      	  963056	      1148 ns/op	     128 B/op	       3 allocs/op
BenchmarkCopyXlsxRow/old-16      	  344492	      3135 ns/op	     344 B/op	      16 allocs/op
BenchmarkCopyXlsxRow/new-16      	  909142	      1371 ns/op	     192 B/op	       3 allocs/op
BenchmarkCopyXlsxWorksheet/old-16         	   49108	     23097 ns/op	    3656 B/op	     128 allocs/op
BenchmarkCopyXlsxWorksheet/new-16         	  176534	      7085 ns/op	     808 B/op	       6 allocs/op
PASS
ok  	github.com/xuri/excelize/v2	7.377s

How Has This Been Tested

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

This commit closes #2029.

Signed-off-by: Eng Zer Jun <[email protected]>
Copy link

codecov bot commented Nov 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.20%. Comparing base (5f446f2) to head (1e5744d).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2030   +/-   ##
=======================================
  Coverage   99.20%   99.20%           
=======================================
  Files          32       32           
  Lines       29952    29958    +6     
=======================================
+ Hits        29714    29720    +6     
  Misses        158      158           
  Partials       80       80           
Flag Coverage Δ
unittests 99.20% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Copy link
Member

@xuri xuri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for your contribution.

@xuri xuri merged commit c936188 into qax-os:master Nov 22, 2024
41 checks passed
@xuri xuri added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Nov 22, 2024
@arp242
Copy link

arp242 commented Nov 27, 2024

Did anyone audit this library?

The previous library was unmaintained, but it was simple, worked, and was trusted. This is a random project from more or less a random person without any reputation that I can see. While I'm 99% sure it's fine, that's not 100%. I'm not very enthusiastic bringing this in our dependency tree.

And we're talking about nanoseconds here. The benchmarks didn't really demonstrate this makes a meaningful difference.

@xuri
Copy link
Member

xuri commented Nov 28, 2024

Thanks for your comments, although the new deepcopy library not popular and not widely used currently, but after I checked the code of that, it seems fine. I think if bug reports before next release of excelize, we can revert this or using encoding/gob for deepcopy. @Juneezee what you think of this?

@Juneezee
Copy link
Contributor Author

Hi @arp242, thank you for your feedback and concerns. I did review the source code of the suggested library before making the proposal in #2029.

The previous library was unmaintained, but it was simple, worked, and was trusted. This is a random project from more or less a random person without any reputation that I can see.

You’re correct that the previous library code is simpler. However, the new library also includes unit tests to ensure functionality. Are your main concerns related to the security or trustworthiness of the code?

The benchmarks didn't really demonstrate this makes a meaningful difference.

The performance improvement is most noticeable in the Copy Worksheet function. For the benchmark, I used a small worksheet from the test Excel file (containing only two charts and a few rows and columns with data). Even with this relatively simple example, we see significant improvements in execution time and memory allocations. These benefits will scale and become even more impactful when working with larger worksheets.

@Juneezee
Copy link
Contributor Author

Thanks for your comments, although the new deepcopy library not popular and not widely used currently, but after I checked the code of that, it seems fine. I think if bug reports before next release of excelize, we can revert this or using encoding/gob for deepcopy. @Juneezee what you think of this?

@xuri I agree. This PR introduces only a small number of code changes. If it leads to any new bugs or regressions, we can easily revert to the previous library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
Status: Performance
Development

Successfully merging this pull request may close these issues.

Use a faster deepcopy library
3 participants