Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gioutil.ListModel memory leak #154

Open
abenz1267 opened this issue Sep 5, 2024 · 13 comments
Open

gioutil.ListModel memory leak #154

abenz1267 opened this issue Sep 5, 2024 · 13 comments

Comments

@abenz1267
Copy link

see #126 (comment)

Apparently ListModel leaks memory.

@abenz1267
Copy link
Author

this sounded like a quick fix 😂 ...

@diamondburned
Copy link
Owner

Sorry, I think I just completely forgot about this issue until now @.@

The bug isn't being clear to me at all. Just splice() shouldn't be leaking at all. Does it not leak if you replace that with just appending and deleting? Do you also have a minimally reproducing code?

@diamondburned
Copy link
Owner

diamondburned commented Nov 3, 2024

Branch debug-listmodel (commit babdedb) contains more debug logging to figure this out. It would be wonderful if you could provide the logs for your application on this commit.

Running your app with G_MESSAGES_DEBUG=all should work, but the domains for the ListModel type should be fairly obvious to search for (they all start with Gotk4Gbox).

@abenz1267
Copy link
Author

Sorry, I think I just completely forgot about this issue until now @.@

The bug isn't being clear to me at all. Just splice() shouldn't be leaking at all. Does it not leak if you replace that with just appending and deleting? Do you also have a minimally reproducing code?

Using Remove and Append instead of Splice also leaks. I don't have minimal code, but this is the offending line: https://github.com/abenz1267/walker/blob/master/internal/ui/interactions.go#L798

Branch debug-listmodel (commit babdedb) contains more debug logging to figure this out. It would be wonderful if you could provide the logs for your application on this commit.

Running your app with G_MESSAGES_DEBUG=all should work, but the domains for the ListModel type should be fairly obvious to search for (they all start with Gotk4Gbox).

Stupid question: where is the log output supposed to be? Stdout doesn't print anything, neither does journalctl -f.

@diamondburned
Copy link
Owner

Stupid question: where is the log output supposed to be? Stdout doesn't print anything, neither does journalctl -f.

It should be using log/slog, but if your app isn't already using slog, then the default of it is to ignore debug logs. A quick snippet like this playground should work (in addition to the env var).

@abenz1267
Copy link
Author

When using debug-listmodel I get a segfault:

unexpected fault address 0x1008
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1008 pc=0x4e04c0]

goroutine 1 gp=0xc0000061c0 m=0 mp=0x1325820 [running, locked to thread]:
runtime.throw({0xcdbec7?, 0xb?})
	/usr/lib/go/src/runtime/panic.go:1067 +0x48 fp=0xc000520978 sp=0xc000520948 pc=0x54c908
runtime.sigpanic()
	/usr/lib/go/src/runtime/signal_unix.go:914 +0x26c fp=0xc0005209d8 sp=0xc000520978 pc=0x54e3ac
indexbytebody()
	/usr/lib/go/src/internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc0005209e0 sp=0xc0005209d8 pc=0x4e04c0
runtime.findnull(0x54f365?)
	/usr/lib/go/src/runtime/string.go:543 +0x79 fp=0xc000520a38 sp=0xc0005209e0 pc=0x533259
runtime.gostring(0x1008)
	/usr/lib/go/src/runtime/string.go:329 +0x1c fp=0xc000520a70 sp=0xc000520a38 pc=0x54f31c
github.com/diamondburned/gotk4/pkg/glib/v2._Cfunc_GoString(...)
	_cgo_gotypes.go:704
github.com/diamondburned/gotk4/pkg/glib/v2.(*LogField).Value(0xc0005981e0?)
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/glib/v2/glib.go:33800 +0x28 fp=0xc000520a90 sp=0xc000520a70 pc=0x7a0688
github.com/diamondburned/gotk4/pkg/glib/v2.init.1.func1.newSlogWriterFunc.1(0x80, {0xc000468820, 0x4, 0x8?})
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/glib/v2/glib.go:33847 +0x105 fp=0xc000520bd0 sp=0xc000520a90 pc=0x7a10e5
github.com/diamondburned/gotk4/pkg/glib/v2.init.1.func1(0x1323f60?, {0xc000468820?, 0x0?, 0x58?})
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/glib/v2/glib.go:33821 +0x3a fp=0xc000520c10 sp=0xc000520bd0 pc=0x7a0f9a
github.com/diamondburned/gotk4/pkg/glib/v2._gotk4_glib2_LogWriterFunc(0x80, 0x7fffc23481d0, 0x4, 0x4ef15b?)
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/glib/v2/glib_export.go:181 +0x142 fp=0xc000520c68 sp=0xc000520c10 pc=0x7a0de2
_cgoexp_0e499e6aadca__gotk4_glib2_LogWriterFunc(0x7fffc23480b0)
	_cgo_gotypes.go:12603 +0x28 fp=0xc000520c98 sp=0xc000520c68 pc=0x7a19e8
runtime.cgocallbackg1(0x7a19c0, 0x7fffc23480b0, 0x0)
	/usr/lib/go/src/runtime/cgocall.go:442 +0x28b fp=0xc000520d58 sp=0xc000520c98 pc=0x4e284b
runtime.cgocallbackg(0x7a19c0, 0x7fffc23480b0, 0x0)
	/usr/lib/go/src/runtime/cgocall.go:361 +0x11a fp=0xc000520da8 sp=0xc000520d58 pc=0x4e253a
runtime.cgocallbackg(0x7a19c0, 0x7fffc23480b0, 0x0)
	<autogenerated>:1 +0x29 fp=0xc000520dd0 sp=0xc000520da8 pc=0x556c89
runtime.cgocallback(0xc000520e30, 0x546615, 0xa20360)
	/usr/lib/go/src/runtime/asm_amd64.s:1084 +0xcc fp=0xc000520df8 sp=0xc000520dd0 pc=0x55464c
runtime.systemstack_switch()
	/usr/lib/go/src/runtime/asm_amd64.s:479 +0x8 fp=0xc000520e08 sp=0xc000520df8 pc=0x552888
runtime.cgocall(0xa20360, 0xc000520e68)
	/usr/lib/go/src/runtime/cgocall.go:185 +0x75 fp=0xc000520e40 sp=0xc000520e08 pc=0x546615
github.com/diamondburned/gotk4/pkg/core/gioutil._Cfunc_gotk4_gbox_list_splice(0x1b749ab0, 0x0, 0x0, 0x1bbc7270)
	_cgo_gotypes.go:242 +0x45 fp=0xc000520e68 sp=0xc000520e40 pc=0x9a0de5
github.com/diamondburned/gotk4/pkg/core/gioutil.(*ListModel[...]).Splice.func2(0x0?, 0x0, 0x1bbc7270, 0x0)
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/core/gioutil/listmodel.go:85 +0x88 fp=0xc000520ea8 sp=0xc000520e68 pc=0x9af4e8
github.com/diamondburned/gotk4/pkg/core/gioutil.(*ListModel[...]).Splice(0xe3e5e0?, 0x0?, 0x0?, {0xc000530008?, 0x3f?, 0x0?})
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/core/gioutil/listmodel.go:85 +0x23e fp=0xc0005210f8 sp=0xc000520ea8 pc=0x9af39e
github.com/abenz1267/walker/internal/ui.setInitials()
	/home/andrej/Documents/walker/internal/ui/interactions.go:878 +0x3db fp=0xc000521748 sp=0xc0005210f8 pc=0x9a82db
github.com/abenz1267/walker/internal/ui.process()
	/home/andrej/Documents/walker/internal/ui/interactions.go:557 +0xe7 fp=0xc0005217b0 sp=0xc000521748 pc=0x9a5ea7
github.com/diamondburned/gotk4/pkg/core/glib._gotk4_goMarshal(0x1b7e79b0, 0x0, 0x1, 0x7fffc2348bc0, 0xc0000061c0?, 0x1b76eb70)
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/core/glib/glib.go:239 +0xed fp=0xc000521a30 sp=0xc0005217b0 pc=0x78c58d
_cgoexp_ce1cbf65a778__gotk4_goMarshal(0x521626?)
	_cgo_gotypes.go:2891 +0x2c fp=0xc000521a70 sp=0xc000521a30 pc=0x793eec
runtime.cgocallbackg1(0x793ec0, 0x7fffc2348960, 0x0)
	/usr/lib/go/src/runtime/cgocall.go:442 +0x28b fp=0xc000521b30 sp=0xc000521a70 pc=0x4e284b
runtime.cgocallbackg(0x793ec0, 0x7fffc2348960, 0x0)
	/usr/lib/go/src/runtime/cgocall.go:361 +0x11a fp=0xc000521b80 sp=0xc000521b30 pc=0x4e253a
runtime.cgocallbackg(0x793ec0, 0x7fffc2348960, 0x0)
	<autogenerated>:1 +0x29 fp=0xc000521ba8 sp=0xc000521b80 pc=0x556c89
runtime.cgocallback(0xc000521c08, 0x546615, 0x9c90f0)
	/usr/lib/go/src/runtime/asm_amd64.s:1084 +0xcc fp=0xc000521bd0 sp=0xc000521ba8 pc=0x55464c
runtime.systemstack_switch()
	/usr/lib/go/src/runtime/asm_amd64.s:479 +0x8 fp=0xc000521be0 sp=0xc000521bd0 pc=0x552888
runtime.cgocall(0x9c90f0, 0xc000521c40)
	/usr/lib/go/src/runtime/cgocall.go:185 +0x75 fp=0xc000521c18 sp=0xc000521be0 pc=0x546615
github.com/diamondburned/gotk4/pkg/gio/v2._Cfunc_g_application_run(0x1b659960, 0x1, 0x1b65dcb0)
	_cgo_gotypes.go:14099 +0x4b fp=0xc000521c40 sp=0xc000521c18 pc=0x7aefcb
github.com/diamondburned/gotk4/pkg/gio/v2.(*Application).Run.func3(0x1b659960, 0x1, 0x1b65dcb0)
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/gio/v2/gio.go:42200 +0x67 fp=0xc000521c70 sp=0xc000521c40 pc=0x7ce687
github.com/diamondburned/gotk4/pkg/gio/v2.(*Application).Run(0xc00023fc80, {0xc0000241b0, 0x1, 0x1})
	/home/andrej/go/pkg/mod/github.com/diamondburned/gotk4/[email protected]/gio/v2/gio.go:42200 +0x1f1 fp=0xc000521d18 sp=0xc000521c70 pc=0x7ce571
	```

@abenz1267
Copy link
Author

package main

import (
	"os"
	"time"

	"github.com/diamondburned/gotk4/pkg/core/gioutil"
	"github.com/diamondburned/gotk4/pkg/gio/v2"
	"github.com/diamondburned/gotk4/pkg/glib/v2"
	"github.com/diamondburned/gotk4/pkg/gtk/v4"

	coreglib "github.com/diamondburned/gotk4/pkg/core/glib"
)

type Item struct {
	Value string
}

func main() {
	app := gtk.NewApplication("com.github.diamondburned.gotk4-examples.gtk4.simple", gio.ApplicationFlagsNone)
	app.ConnectActivate(func() { activate(app) })

	if code := app.Run(os.Args); code > 0 {
		os.Exit(code)
	}
}

var items = gioutil.NewListModel[Item]()

func activate(app *gtk.Application) {
	window := gtk.NewApplicationWindow(app)
	window.SetTitle("gotk4 Example")

	selection := gtk.NewSingleSelection(items)

	factory := gtk.NewSignalListItemFactory()

	list := gtk.NewListView(selection, &factory.ListItemFactory)

	go func() {
		for {
			time.Sleep(time.Millisecond * 10)

			n := []Item{
				{Value: "Hello"},
				{Value: "World"},
				{Value: "Foo"},
				{Value: "Bar"},
			}

			glib.IdleAdd(func() {
				items.Splice(0, int(items.NItems()), n...)
			})
		}
	}()

	factory.ConnectSetup(func(object *coreglib.Object) {
		item := object.Cast().(*gtk.ListItem)
		box := gtk.NewBox(gtk.OrientationVertical, 0)
		item.SetChild(box)
	})

	factory.ConnectBind(func(object *coreglib.Object) {
		item := object.Cast().(*gtk.ListItem)
		valObj := items.Item(item.Position())
		val := gioutil.ObjectValue[Item](valObj)
		child := item.Child()

		box, _ := child.(*gtk.Box)

		label := gtk.NewLabel(val.Value)

		box.Append(label)
	})

	window.SetChild(list)
	window.SetDefaultSize(400, 300)

	window.Show()
}

That's a minimal example. Running this will increase memory usage over time.

@abenz1267
Copy link
Author

I think there's actually 2 leaks here:

  • the list model
  • factory.ConnectSetup is also leaking

@diamondburned
Copy link
Owner

I believe ConnectSetup is the cause of the leak, actually. Here's a smaller reproducing snippet with a newer gotk4 commit that fixes the crash:

package main

import (
	"log/slog"
	"os"

	"github.com/diamondburned/gotk4/pkg/core/gioutil"
	"github.com/diamondburned/gotk4/pkg/gio/v2"
	"github.com/diamondburned/gotk4/pkg/glib/v2"
	"github.com/diamondburned/gotk4/pkg/gtk/v4"
)

func init() {
	slog.SetDefault(slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
		Level: slog.LevelDebug,
	})))
}

type Item struct {
	Value string
}

func main() {
	app := gtk.NewApplication("com.github.diamondburned.gotk4-examples.gtk4.simple", gio.ApplicationFlagsNone)
	app.ConnectActivate(func() { activate(app) })

	if code := app.Run(os.Args); code > 0 {
		os.Exit(code)
	}
}

var items = gioutil.NewListModel[Item]()

func activate(app *gtk.Application) {
	factory := gtk.NewSignalListItemFactory()
	// factory.ConnectBind(func(object *coreglib.Object) {
	// 	listItem := object.Cast().(*gtk.ListItem)
	// 	ourItem := gioutil.ObjectValue[Item](items.Item(listItem.Position()))
	// 	slog.Debug("Binding", "item", ourItem)
	// })

	selection := gtk.NewSingleSelection(items)
	list := gtk.NewListView(selection, &factory.ListItemFactory)

	window := gtk.NewApplicationWindow(app)
	window.SetTitle("gotk4 Example")
	window.SetChild(list)
	window.SetDefaultSize(400, 300)
	window.Show()

	n := []Item{
		{Value: "Hello"},
		{Value: "World"},
		{Value: "Foo"},
		{Value: "Bar"},
	}

	glib.IdleAdd(func() {
		for i := 0; i < 100; i++ {
			slog.Debug("Splicing", "n", n)
			items.Splice(0, int(items.NItems()), n...)
		}
	})
}

With the Connect call commented out, the items are actually finalized:

time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg=Splicing n="[{Value:Hello} {Value:World} {Value:Foo} {Value:Bar}]"
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Added 4 objects using splice()" priority=7 glib_domain=Gotk4GboxList
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Freed gbox object 4106 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Freed gbox object 4105 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Freed gbox object 4103 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Freed gbox object 4104 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg=Splicing n="[{Value:Hello} {Value:World} {Value:Foo} {Value:Bar}]"
time=2024-11-03T10:26:24.627-08:00 level=DEBUG msg="Added 4 objects using splice()" priority=7 glib_domain=Gotk4GboxList
time=2024-11-03T10:26:24.628-08:00 level=DEBUG msg="Freed gbox object 4099 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.628-08:00 level=DEBUG msg="Freed gbox object 4101 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.628-08:00 level=DEBUG msg="Freed gbox object 4102 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject
time=2024-11-03T10:26:24.628-08:00 level=DEBUG msg="Freed gbox object 4100 in gotk4_gbox_object_finalize" priority=7 glib_domain=Gotk4GboxObject

These log messages aren't seen when the Connect call is used.

@abenz1267
Copy link
Author

Oh! Indeed! I had to fully comment out the whole ConnectSetup call and not just it's body. Leak gone.

@diamondburned
Copy link
Owner

This is probably the same old gotk4 reference memory leak that's been happening elsewhere as well. It seems like my original comment was wrong.

@diamondburned
Copy link
Owner

This logging suggests that too:

time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg=Splicing n="[{Value:Hello} {Value:World} {Value:Foo} {Value:Bar}]"
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Created gbox object 0x1003 in gotk4_gbox_object_new" priority=7 code_file=listmodel.c code_line=39 code_func=gotk4_gbox_object_new glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Created gbox object 0x1004 in gotk4_gbox_object_new" priority=7 code_file=listmodel.c code_line=39 code_func=gotk4_gbox_object_new glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Created gbox object 0x1005 in gotk4_gbox_object_new" priority=7 code_file=listmodel.c code_line=39 code_func=gotk4_gbox_object_new glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Created gbox object 0x1006 in gotk4_gbox_object_new" priority=7 code_file=listmodel.c code_line=39 code_func=gotk4_gbox_object_new glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Added 4 objects using splice()" priority=7 code_file=listmodel.c code_line=119 code_func=gotk4_gbox_list_splice glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1003, ref=1->2" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1003, ref=2->3" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1004, ref=1->2" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1005, ref=1->2" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1006, ref=1->2" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Adding new reference onto object 0x1003, ref=3->4" priority=7 code_file=listmodel.c code_line=71 code_func=gotk4_gbox_list_get_item glib_domain=Gotk4GboxObject
time=2024-11-03T12:13:36.224-08:00 level=DEBUG msg="Fetching item" item[0]->ptr=829091136 item[0]->type=Gotk4GboxObject

In this code, I'm calling gioutil.ObjectValue() on just the first item, and only that item gets its reference count incremented a lot without any finalizing happening.

@diamondburned
Copy link
Owner

I reckon once I (have enough time to actually) fix the main memory leak issue, this bug will magically fix itself as well.

diamondburned added a commit that referenced this issue Nov 3, 2024
Also add g_debug logging to listmodel.c for #154
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants