Based on this thread it’s the deduplication that requires a lot of RAM.

See also: https://wiki.freebsd.org/ZFSTuningGuide

Edit: from my understand the pool shouldn’t become inaccessible tho and only get slow. So there might be another issue.

Edit2: here’s a guide to check whether your system is limited by zfs’ memory consumption: https://github.com/openzfs/zfs/issues/10251

Then why didn’t you contact the devs or opened a bug report on GitHub?

Additionally this isn’t the community where this needs to be addressed. Either contact the admins or open an issue on GitHub.