graalvm8: upgrade to 20.0.0

[bennyandresen]

Issue description

I would like to request a package upgrade to it.

https://github.com/graalvm/graalvm-ce-builds/releases/tag/vm-20.0.0

My system is choking trying to compile it, so I didn't even attempt to update it.

Additional question: Maybe a graalvm8-ce could be introduced based on the ce-builds above? Would that be accepted? That way one can test graalvm without the large and long compilation process. I'm willing to work on that.

@hlolli @volth

[jlesquembre]

The problem with graalvm*-ee is its license. To me, it makes sense to create 2 new packages, graalvm8-bin and graalvm11-bin, using the binaries provided by graalvm-ce-builds. We have similar solutions for other packages. For example, for firefox there is also a -bin version.

I agree that ideally graalvm should be built from source, but right now hydra fails to build it:
https://hydra.nixos.org/job/nixpkgs/trunk/graalvm8.x86_64-linux
I'd like to fix it and also add a new build based on openjdk11, but the derivation is pretty complex, there are a lot of patches and I have no idea where to start.

My suggestion is to add a -bin version. This way, packages depending on graalvm (like clj-kondo) could temporarily use it until someone has the time to fix and update the source version. Even after that, we can keep the -bin version, since it should be relatively easy to maintain.

I want to avoid blocking people depending on packages built with graalvm. As far as I know, right now we have only 2 of those packages, clj-kondo and babashka, but I guess that soon we'll have more binaries built with graalvm.

Thanks to @volth and @hlolli for their work on graalvm8, you put a lot of effort into it. I'd like to know if my suggestion makes sense to you or you see some problems.

[hlolli]

I'll have a look this week and next on graalvm8. But yes I agree, the derivation is a mess of patches. This is a beast. And despite 10MB of logs, we can't see what went wrong because the logs are piped to logfiles from java, and if those logs were shows, the logfile would increase 10x.

The idea of graalvm8 and graalvm11, I think the java11 support was brand new when I made the last update, and it didn't work if I remember correctly, but now a major version later, it could work fine.

I'll try to refactor and comment the derivation, but I'm not sure I can decrease the complexity much.

[bennyandresen]

Just to note: I was able to compile graalvm8 on a recent (less than 1 week old) master several times due to #84354 and #84350

What I observed, because I had swap on zfs zvol deadlocks (known upstream), is that the build process for graalvm8 using the nix flags --cores 1 -j 1 still consumes 24GB of memory and takes over an hour on my system. (i7-8550U)
I have no idea about hydra but 24GB memory is a lot, so maybe the OOM-Killer gets involved?

[hlolli]

@BAndresen the OOM issues pop up differently, we can see that every time @GrahamcOfBorg tries to build graalvm8. It's likely a gcc/clang compilation error happening in the process below, just need to see the logs and it should be easy to debug, but as you mention, it really is an hour process.

[hlolli]

I've been working on refactoring last days. Trying to seperate the primary imports. I think it seems clear to me that patching mx.py isn't a good long term solution. mx is a build tool that goes against nix in so many ways. There needs to be developed a nix specific mx build tool imo.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/graalvm-compile-from-source-after-upgrading-to-20-03/6848/2

[hlolli]

#85902

Need help here

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/issue-building-graalvm-19-2-1-dependency-for-the-clj-kondo-package/6932/4

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/building-graalvm8-fails-with-unable-to-load-native-library-error/8950/5

[bennyandresen]

I personally don't require graalvm8 any longer. With #99631 we've got graalvm8 and graalvm11 and the main application I use that require graalvm, are babashka and clj-kondo and they both basically require graalvm11 by now.

I will close it for now. Please reply if someone wants me to re-open this.