Update Zig toolchain #105

mcadariu · 2025-12-11T11:53:20Z

PR to close #104

urso · 2025-12-16T11:41:04Z

Hi, thank you for giving this a try.

The removal of usingnamespace is quite unfortunate and we essentially end up with pg.c.c here. As @cImport will also go away I was thinking to move the header translation to build.zig and finally expose that module via pg.c.

The project also uses nix to provide a stable development environment. See flake.nix. We need to update zig-stable to 0.15.2 and use nix flake update to update zig-overlay and zls dependencies. Otherwise this change will break downstream projects.

mcadariu · 2025-12-16T12:57:17Z

@urso thanks for your feedback, I will try it out.

urso

Nice. The move to TranslateC is a great improvement. There are still a few things we can improve on, though. I still wonder if we can reduce the amount of breaking code.

src/pgzx/c.zig

urso · 2025-12-17T12:14:07Z

src/pgzx.zig

 // Export common set of postgres headers.
-pub const c = @import("pgzx_pgsys"); // keep for backwards compatibility
-pub const pg = c;
+pub const pg = @import("pgzx_pgsys");


I was wondering if it is possible to use pg = @import("c_translated") here. the pgzx_pgsys module basically was c.zig that is now replaced by headers.h. The goal of pgzx_pgsys was to have a base-module just doing the C header translation.

src/pgzx/elog.zig

urso · 2025-12-18T11:58:45Z

The Zig changes look good, but something seems to be wrong with the nix setup :(

Will take me some time to figure this out, unless you want to give that a go as well.

urso · 2025-12-19T13:10:50Z

flake.nix

    };
    zls = {
-      url = "github:zigtools/zls";
+      url = "github:zigtools/zls/0.15.0";


Oh, I see. ZLS normally follow the development version. This bit me as well a few times in the past. Normally I'm updating the dependency like so:

nix flake update zls --override-input github:zigtools/zls/0.15.0

With ZLS I kinda agree updating the URL here. It is quite clear what to update when upgrading to new zig version. Let's leave it here as is.

I wonder about the pre-commit-hooks-nix dependency though. Why did we need to pin it? A breaking change? For that dependency I would prefer to pin it in the flake.lock file only via --override-input. If there is a breaking change we eventually have to address that in the future.

Edit: good news, CI now succeeds

Without the pin, it throws this error:

error: evaluation aborted with the following error message: 'lib.customisation.callPackageWith: Function called without required argument "zizmor" at /nix/store/r1wrhwj8m29ni4j36dxha1pkzpjdg3jy-source/nix/tools.nix:105'

Should we try to upgrade nixpkgs? (to try to avoid the pinning, but it might be a good idea even for its own sake)

oh, good point. We still use 2405. Current is already 2511. Yeah, I think we should try to update this instead.

urso · 2025-12-22T10:22:13Z

Updating nixpkgs also updates the linters who seem to be more strict 🤦

The linters are correct. In .envrc we should add #!/usr/bin/env bash. This is not really used by direnv, but explicitely document that direnv actually uses a bash sub-shell to load the .envrc file.

The deadnix linter is also correct. in nixpkgs.nix the config at line 2 and the pkgs at line 75 are unused.

You can run the linters locally via: nix develop --command -- pre-commit run --all-files --color always

mcadariu · 2025-12-22T11:13:27Z

Probably was already considered on your side. I was thinking if the following would be a good idea. Set up a pipeline that runs let's say weekly. Downloads the latest Zig and tries to build the project and the example extensions and other such checks. Fails if there's any issues with that.

urso · 2025-12-22T12:16:29Z

Probably was already considered on your side. I was thinking if the following would be a good idea. Set up a pipeline that runs let's say weekly. Downloads the latest Zig and tries to build the project and the example extensions and other such checks. Fails if there's any issues with that.

Yeah, we initially considered that. On the other hand Zig language and standard library is still changing quite a bit between releases. Plus some breaking changes only become apparent when functions are actually used to to Zig 'lazy' nature of compiling code. For this reason we decided to pin to a stable release and even keep to a stable release a little longer in case of critical bugs being discovered. It is still a balance.

The ZLS dependency is also a pain to track as this project normally builds against main. A few times we got caught up in incompatibilities because Zig and ZLS overlays/releases were not well in sync.

I think this level of automation still make sense for patch releases.

urso · 2025-12-22T12:21:35Z

Oh no, looks like postgres headers can't be found after updating nixpkgs:

install
+- install generated to pghostname_zig.so
   +- compile lib pghostname_zig Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^
error: the following command failed with 1 compilation errors:
/nix/store/dbkgn0cfg79318znr71kc4lxjxlz14zr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/runner/.cache/zig --listen=- -I /home/runner/work/pgzx/pgzx/examples/pghostname_zig/../.././src/pgzx/c/include/ -I /usr/include/postgresql/14/server -I /usr/include/postgresql /home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h
install
+- install generated to spi_sql.so
   +- compile lib spi_sql Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^
error: the following command failed with 1 compilation errors:
/nix/store/dbkgn0cfg79318znr71kc4lxjxlz14zr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/runner/.cache/zig --listen=- -I /home/runner/work/pgzx/pgzx/examples/spi_sql/../.././src/pgzx/c/include/ -I /usr/include/postgresql/14/server -I /usr/include/postgresql /home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h
install
+- install generated to sqlfns.so
   +- compile lib sqlfns Debug native
      +- translate-c 1 errors
/home/runner/work/pgzx/pgzx/src/pgzx/c/include/headers.h:1:10: error: 'postgres.h' file not found
#include "postgres.h"
         ^

mcadariu · 2025-12-22T16:22:11Z

I think it's due to this change:

postgresql and libpq don’t provide pg_config by default anymore. Instead, pg_config is available via postgresql.pg_config or libpq.pg_config. This allowed implementing it as a shell script, which can be built for both the build and host systems when cross-compiling. If your build fails to find pg_config, add postgresql.pg_config or libpq.pg_config to nativeBuildInputs.

Source: https://nixos.org/manual/nixpkgs/stable/release-notes?#sec-nixpkgs-release-25.05-incompatibilities

mcadariu · 2025-12-23T16:12:19Z

pglocal now fails with

+ /home/runner/work/pgzx/pgzx/out/default/bin/pg_ctl initdb -D /home/runner/work/pgzx/pgzx/out/default/var/postgres/data -o --encoding=UTF8
initdb: error: file "/home/runner/work/pgzx/pgzx/out/default/share/postgresql/postgres.bki" does not exist

I think it is now confused about where to find files due to the presence of both pkgs.postgresql_16_jit and pkgs.postgresql_16_jit.pg_config

urso · 2025-12-29T10:15:47Z

Good news: tests compile now.
Bad news: Tests seem to crash postgres.

See:

INFO:  Running test: testMemoryContext_reset

server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

…hat it's the culprit

…mand array

urso · 2026-01-05T10:14:15Z

Happy new year.

This is a lot of back and forth creating a commit for every change. While I appreciate the effort I can imagine this being quite frustrating on your side.

If you check the check.yaml github workflow you will find that it is quite easy to run the CI scripts. Enter a development shell via nix develop and then run ./ci/setup.sh once to create a local postgres instance. The postgres setup will be copied into your local development folders and does not mess with your systems postgres installation (well, besides trying to use the same socket). Once CI is setup you run ./ci/run.sh. We wanted to make it easy to run everything local.

In case you don't have nix installed on your dev machine you can do everything via docker as well. User ./dev/docker/build.sh to build the docker image with nix. Then start a the docker based development shell via ./dev/docker/run.sh (the container will be deleted when you quit). Now you can run ./ci/setup.sh or ./ci/run.sh as usual.

I hope this helps a little.

mcadariu · 2026-01-05T11:26:12Z

Happy new year!
I think indeed Docker saves the day.

With Docker I managed to create a local setup and with the latest commit the ./ci/run.sh passes here.

...
Run example CI script ./examples/sqlfns
Build extension sqlfns
Create extension sqlfns
CREATE EXTENSION
Run regression tests: sqlfns
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig translate-c -lc --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --listen=- -I /home/dev/workdir/examples/sqlfns/../.././src/pgzx/c/include/ -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include /home/dev/workdir/src/pgzx/c/include/headers.h
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig build-exe -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -Mroot=/home/dev/workdir/tools/gennodetags/main.zig -lc --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --name gennodetags --zig-lib-dir /nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/lib/ --listen=-
install -C /home/dev/workdir/examples/sqlfns/extension/sqlfns.control /home/dev/workdir/examples/sqlfns/zig-out/share/postgresql/extension/sqlfns.control
install -C /home/dev/workdir/examples/sqlfns/extension/sqlfns--0.1.sql /home/dev/workdir/examples/sqlfns/zig-out/share/postgresql/extension/sqlfns--0.1.sql
CLICOLOR_FORCE=1 pkg-config --list-all
/nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/bin/zig build-lib -ODebug -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server --dep pgzx -Mroot=/home/dev/workdir/examples/sqlfns/src/main.zig -ODebug --dep c_translated --dep gen_node_tags -Mpgzx=/home/dev/workdir/src/pgzx.zig -cflags -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -- /home/dev/workdir/./src/pgzx/c/libpqsrv.c -lpq -ODebug -I /home/dev/workdir/examples/sqlfns/../.././src/pgzx/c/include/ -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include/server -I /nix/store/ys1j7wdz8rwndiqxc3lvyimm19b9yv64-postgresql-16.11-dev/include -L /nix/store/9rynhw9pyiz2jpmhvyrgkh67q7xx49yx-postgresql-16.11-lib/lib -Mc_translated=.zig-cache/o/30fcf3312dbbc667efbd952ab3f6ee08/headers.zig --dep c_translated -Mgen_node_tags=.zig-cache/o/64e397b0ab8da351e868c2cd824e57c5/nodetags.zig -lc -fallow-shlib-undefined --cache-dir .zig-cache --global-cache-dir /home/dev/.cache/zig --name sqlfns -dynamic --version 0.1.0 --zig-lib-dir /nix/store/zicpmzdq8cdm0j5w4s18ww9dqr4vn9qr-zig-0.15.2/lib/ --listen=-
install -C .zig-cache/o/a6c15ec3d6d9e764aef383a120736354/libsqlfns.so.0.1.0 /home/dev/workdir/examples/sqlfns/zig-out/lib/sqlfns.so
# using postmaster on Unix socket, port 5432
ok 1         - sqlfns_test                                16 ms
1..1g_regress
# All 1 tests passed.
Drop extension sqlfns
DROP EXTENSION
Success!
Stopping PostgreSQL cluster ''
waiting for server to shut down.... done
server stopped

urso · 2026-01-09T11:08:13Z

With Docker I managed to create a local setup and with the latest commit the ./ci/run.sh passes here.

That's great. But we still have some tests commented out. Can you reenable and see if they run through?

mcadariu · 2026-01-09T12:31:45Z

Good catch, if I uncomment the pgzx.mem.TestSuite_Mem it fails with the following output. I can investigate and work on it locally. At least we have narrowed it down.

mcadariu · 2026-01-10T14:43:53Z

Still passes here.. one option would be to revert to commit 8972e51 which was passing the CI but had something pinned.

mcadariu · 2026-01-22T20:09:39Z

@urso shall I revert this PR to commit 8972e51 (which was passing the CI pipeline)? I can not work actively anymore however was wondering if that would be an acceptable way to wrap up this PR.

urso · 2026-01-23T09:00:23Z

Hi,
Sorry for the late reply, I've been a little busy the last weeks.
I would say it is better to keep in all the commits. I will pick up this PR when I have time and see if I can reproduce and fix the last issue. Having all history available to do so will be helpful I think.

Thank you for all the effort you've put into it. I understand it was quite time consuming and maybe a little frustrating at times. The more I appreciate the dedication and time you spend on this PR. I'm sure we're this close to get it merged.

mcadariu added 2 commits December 11, 2025 12:55

fix breaking changes

8e19d18

cleanup

8843195

apply review comments

64bcf11

urso reviewed Dec 17, 2025

View reviewed changes

remove c.zig

8656618

mcadariu added 2 commits December 18, 2025 16:24

pin pre-commit-hooks.nix

4063394

apply Zig formatter and pin zls

8972e51

urso reviewed Dec 19, 2025

View reviewed changes

update nixpkgs to 2511 and remove pin on pre-commit-hooks.nix

904debe

fix lint errors

588e1d3

mcadariu added 2 commits December 22, 2025 13:43

fix postgres packages

a13b114

cleanup path export

61beaaa

add pkgs.postgresql_16_jit.pg_config

29593ec

mcadariu added 3 commits December 23, 2025 16:45

create unified postgres package

797f945

bring back pglocal changes

b175ef2

fix bitcode directory copying issue

933a06b

mcadariu added 4 commits December 29, 2025 17:49

temporarily disable node tests in order to determine with certainty t…

c6ad224

…hat it's the culprit

New experiment: disable memory tests temporarily

77f961b

fix lint issue

3251787

more logging around process spawning

8cd6106

use {any} which works on any type instead of {s} for printing the com…

1921d8f

…mand array

fix pg_regress path

1dc82c4

ensure fresh data directory

6be6f20

Update Zig toolchain #105

Are you sure you want to change the base?

Update Zig toolchain #105

Uh oh!

Conversation

mcadariu commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Dec 16, 2025

Uh oh!

mcadariu commented Dec 16, 2025

Uh oh!

urso left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

urso Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

urso commented Dec 18, 2025

Uh oh!

urso Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

mcadariu Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

urso Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

urso commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 22, 2025

Uh oh!

urso commented Dec 22, 2025

Uh oh!

urso commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 22, 2025

Uh oh!

mcadariu commented Dec 23, 2025

Uh oh!

urso commented Dec 29, 2025

Uh oh!

urso commented Jan 5, 2026

Uh oh!

mcadariu commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Jan 9, 2026

Uh oh!

mcadariu commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcadariu commented Jan 10, 2026

Uh oh!

mcadariu commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

urso commented Jan 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mcadariu commented Dec 11, 2025 •

edited

Loading

mcadariu Dec 18, 2025 •

edited

Loading

urso Dec 19, 2025 •

edited

Loading

mcadariu Dec 19, 2025 •

edited

Loading

mcadariu commented Jan 5, 2026 •

edited

Loading

mcadariu commented Jan 9, 2026 •

edited

Loading

mcadariu commented Jan 22, 2026 •

edited

Loading