Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADBDEV 6061 Add test for doRestoreAgent in gpbackup #102

Open
wants to merge 37 commits into
base: master
Choose a base branch
from

Conversation

dkovalev1
Copy link

@dkovalev1 dkovalev1 commented Jul 31, 2024

Add tests for doRestoreAgent with interface mocking.

This PR proposes to add mocking interfaces and unit test for the
doRestoreAgent() function in the gprestore.

When we look at the gprestore code, and into doRestoreAgent() in particular,
there is not so trivial logic which might be difficult to cover with the
end-to-end and integration tests. From the other side this logic relies on
numerous external dependencies which makes unit testing nearly impossible.

This commit proposes using of well-known mocking technique for unit testing of
gpbackup code with an example based on gprestore and doRestoreAgent(). With
interface mocking it becomes possible to:

  • extend test coverage with more specific use cases
  • make testing closer to the golang code, resulting in fast tests with no
    external dependencies, tests will be more stable
  • test is using a single process which makes it faster and easier to debug

However, this approach only complements existing end-to-end and integration
tests, it's not intended to replace them. The outcome heavily relies on the
clarity and quality of mock interfaces; it's easy to achieve unintended results.

The idea for mocking is to replace complex objects with their test imitation.
Unfortunately, gpbackup is written in solid manner, and it's not possible to
just replace built-in functions. To mitigate it, internal dependencies are split
with introducing of interfaces for dependencies external related to code under
test. Production code is using production interface implementation, while in
testing code we use test implementation, which does do only things related to
the test and return values exactly for the test scenario.

This commit adds the following interfaces:

  • IRestoreHelper - an abstraction for restore helper functions located in
    helper/restore_helper.go
  • IRestoreReader - an abstraction for file reader used for restoring data
  • IPluginCmd - an abstraction needed to keep track of readable stderr and
    whether the command has already been ended

This is a minimum change for the supplied tests. It's expected that when more
methods or interfaces will be needed they will be created of necessity. The
changes for the interface introduction are minimal and straightforward.

Test logic is located in the helper/helper_test.go.

Base automatically changed from 1.30.5-sync to master August 2, 2024 06:41
@RekGRpth

This comment was marked as resolved.

helper/helper_test.go Outdated Show resolved Hide resolved
@RekGRpth

This comment was marked as resolved.

@whitehawk

This comment was marked as resolved.

@whitehawk
Copy link

The same should be done for doBackupAgent()

helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
@whitehawk

This comment was marked as resolved.

@whitehawk

This comment was marked as resolved.

@whitehawk

This comment was marked as resolved.

@whitehawk

This comment was marked as resolved.

@whitehawk

This comment was marked as resolved.

helper/helper_test.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
helper/restore_helper.go Outdated Show resolved Hide resolved
helper/helper.go Outdated Show resolved Hide resolved
helper/helper.go Outdated Show resolved Hide resolved
@whitehawk

This comment was marked as resolved.

helper/restore_helper.go Outdated Show resolved Hide resolved
helper/helper_test.go Outdated Show resolved Hide resolved
whitehawk
whitehawk previously approved these changes Nov 15, 2024
@RekGRpth

This comment was marked as resolved.

@dkovalev1
Copy link
Author

bender build

Comment on lines +84 to +88
if !h.started {
h.started = true
} else {
h.currentStep++
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be remove started and put h.currentStep++ before return ret?

Comment on lines +90 to +91
Expect(h.currentStep).To(BeNumerically("<", len(h.expectedSteps)))
ret := h.expectedSteps[h.currentStep]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Expect(h.currentStep).To(BeNumerically("<", len(h.expectedSteps)))
ret := h.expectedSteps[h.currentStep]
ret := h.getCurStep()

@@ -26,10 +25,9 @@ import (
var (
CleanupGroup *sync.WaitGroup
version string
pipesMap map[string]bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused var?

endbyte: %d
`, dataLength+18, dataLength+18+18)
fToc, err := os.Create(tocFile)
Expect(err).ShouldNot(HaveOccurred())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if error occurred then file is not closed!

*tocFile = fmt.Sprintf("%stest_toc.yaml", testDir)
writeTestTOC(*tocFile)
defer func() {
_ = os.Remove(*tocFile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if writeTestTOC does not create file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeTestTOC has

	fToc, err := os.Create(tocFile)
	Expect(err).ShouldNot(HaveOccurred())
```, then it will fail inside `writeTestTOC`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants