-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dp/memory leak fix #136
Dp/memory leak fix #136
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got part way through writing some of these review items before it dawned on me that maybe the separation of the channel and pool implementations were intended to be separated. Or maybe they weren't. In any case I see at least two ways this could be implemented:
- Based on the
maxSize
either rely on thesync.Pool
or on the channel, but not really both. - Always run the
sync.Pool
and channel modes regardless of themaxSize
. You would always pull from channel first and sync.Pool second. WhenmaxSize
is 0, you'd virtually never get a value on the channel and always default to the pool -- perfect. IfmaxSize is > 0
, then there's a chance that theres a module waiting for you on the channel, otherwise you get it from the pool.
The advantage I see to option 2 is that it requires less cognitive overhead and its behavior is almost identical to option 1 in practice and the differences are really only theoretical.
pool.go
Outdated
new func() (*module[T], error) | ||
pool sync.Pool | ||
maxSize uint32 | ||
new func(*module[T]) (*module[T], error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does new
accept a *module[T]
as a parameter? Seems like it should only return new modules.
if p.maxSize == 0 { | ||
p.pool.Put(m) | ||
} else { | ||
select { | ||
case p.ch <- m: | ||
default: | ||
// Channel is full, call the close function | ||
p.close(m) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be safe to always push to the channel and default to the pool. If the channel is unbuffered (which is the case when maxSize = 0
) then you'd automatically drop back to the pool. I personally would always let the pool do the module deallocation when it wants to and never manually call p.close
the whole point of the pool (imo) is that allocated modules can be shared and I think there are some wins there even when maxSize > 0
.
Both approaches are pretty close to equivalent in practice, this approach (if valid) is probably just simpler to reason about.
if p.maxSize == 0 { | |
p.pool.Put(m) | |
} else { | |
select { | |
case p.ch <- m: | |
default: | |
// Channel is full, call the close function | |
p.close(m) | |
} | |
} | |
select { | |
case p.ch <- m: | |
default: | |
p.pool.Put(m) | |
} |
pool.go
Outdated
close: func(m *module[T]) { | ||
m.Close() | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this necessary if we have runtime finalizers calling m.Close
already?
m, _ := p.pool.Get().(*module[T]) | ||
return m, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like if the pool is empty, this function returns a nil module. Since the sync.Pool
's new property isn't specified, the pool will not automatically create its own module.
pool.go
Outdated
if ok && m != nil { | ||
return m, nil | ||
} | ||
return p.new(m) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we got to this line, m
would have been nil
and p.new
would likely panic.
for key := range r.activeModules { | ||
r.activeModules[key].instantiatedModule.CloseWithExitCode(r.config.context, 0) | ||
delete(r.activeModules, key) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An entirely stylistic suggestion that doesn't functionally matter in the slightest.
for key := range r.activeModules { | |
r.activeModules[key].instantiatedModule.CloseWithExitCode(r.config.context, 0) | |
delete(r.activeModules, key) | |
} | |
for key, module := range r.activeModules { | |
module.instantiatedModule.CloseWithExitCode(r.config.context, 0) | |
delete(r.activeModules, key) | |
} |
pool.go
Outdated
new: func(m *module[T]) (*module[T], error) { | ||
m.SetFinalizer() | ||
return newModule[T](ctx, template) | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I can tell, calls to modulePool.new actually pass null pointers, so it seems like this should be panicing. This approach seems closer.
new: func(m *module[T]) (*module[T], error) { | |
m.SetFinalizer() | |
return newModule[T](ctx, template) | |
}, | |
new: func() (*module[T], error) { | |
m := newModule[T](context.Background(), template) | |
m.SetFinalizer() | |
return m, nil | |
}, |
pool.go
Outdated
maxSize: maxSize, | ||
new: func(m *module[T]) (*module[T], error) { | ||
m.SetFinalizer() | ||
return newModule[T](ctx, template) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ctx
being used here is the context provided in calls to newModulePool
. I'm not familiar with where the context passed to newModulePool
ultimately comes from, but if it comes from the user, the context might be dead by the time this pool wants to allocate new modules in the future.
pool.go
Outdated
if p.maxSize == 0 { | ||
m, ok := p.pool.Get().(*module[T]) | ||
if ok && m != nil { | ||
return m, nil | ||
} | ||
return p.new(m) | ||
} | ||
|
||
// Use buffered channel | ||
select { | ||
case m := <-p.ch: | ||
return m, nil | ||
default: | ||
// Channel is empty, create a new module? | ||
m, _ := p.pool.Get().(*module[T]) | ||
return m, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's what this code could look like under the option 2 approach.
if p.maxSize == 0 { | |
m, ok := p.pool.Get().(*module[T]) | |
if ok && m != nil { | |
return m, nil | |
} | |
return p.new(m) | |
} | |
// Use buffered channel | |
select { | |
case m := <-p.ch: | |
return m, nil | |
default: | |
// Channel is empty, create a new module? | |
m, _ := p.pool.Get().(*module[T]) | |
return m, nil | |
select { | |
case m := <-p.ch: | |
return m, nil | |
default: | |
m, ok := p.pool.Get().(*module[T]) | |
if ok && m != nil { | |
return m, nil | |
} | |
return p.new() |
see: #132