Replace wine by makensis for Linux and use finalization to sign (un)installer

[rainerjung]

Nullsoft installers for Windows can be produced on Linux using the same NSIS scripts. Of course one needs to install makensis.

The other reason for using wine in build.xml was running the temporary installer used to create the uninstaller in order to be able to sign it. But NSIS now allows to define a callback used to sign installer and uninstaller files while creating the installer. So the temporary installer is not needed any more.

First tests look good for me, but I am not using the official signing service used by release managers.

What is not very nice is, that in tomcat.nsi we are calling ant from inside the tomcat.dist directory. So we have to point it to the directory with the build.xml. I am using ../.. which will break if someone uses a non-default tomcat.dist. Since tomcat.nsi is produced by the version filter, in principal we could patch the path to build.xml into tomcat.nsi, but I didn't find a property containing the absolute basedir path.

It would be nice if release managers could test this. I can provide it for the other release branches as well (the patch needs some simple adjustments, because the jsign task uses different attributes in the other branches).

[ChristopherSchultz]

I think we can't build-and-sign the installer and uninstaller in this way and still produce reproducible builds. As long as makensis can be used to produce an unsigned uninstaller and installer with our build producing detached signatures, then I'm okay with this. Oh, and makensis needs to produce byte-for-byte identical binaries as nsis.exe, of course.

[rainerjung]

Why do you think that it wouldn't be reproducible? My patch should still provide the detached signature and use it it already available. And the makensis linux binary should create the same installer as the makensis windows exe. So it would be nice, it you could test it. I tried it with locally signing with a non-official cert/key and it seemed to work.

Whether the installers are cross platform reproducible I don't know. We have to test. But I also do not know whether anyone tested the reproducability using wine versus native windows.

[rainerjung]

What I will try to investigate and improve:

• make wine usable optionally => OK, new prooperty codesigning.tool (wine or makensis)
• check whether "ant" can be used in tomcat.nsi on Windows/Wine, or whether it must be ant.exe, could be patched via a filter => "ant" seems OK, ant binary zip download contains a binary named "ant" in the bin folder
• check whether ../../build.xml in tomcat.nsi can be replaced by something more stable (absolute path for our basedir), could be patched via a filter => OK, added; ${basedir} is absolute even when defining as "." in build.xml.
  Nevertheless it would be useful to get feedback on the patch as is.

[markt-asf]

I'll run some tests with the official signing service and report back.

[markt-asf]

I am testing the signing on Linux and then checking the file produced (and whether the build is reproducible) on Windows.

• The installer works and is correctly signed
• The uninstaller works and is correctly signed
• The build using the existing signatures fails on Windows - currently trying to figure out why

[rainerjung]

I added wine support (without intermediate installer) plus path improvement.
I will stop adding features now.
Let me know your failure messages if you like help in investigation. Thanks for testing!!

[ebourg]

The build using the existing signatures fails on Windows - currently trying to figure out why

Reproducibility issue maybe?

[markt-asf]

The signing failure was a configuration issue. The signing process works (the detached signatures are inserted) but the resulting signatures are not valid. Next step is to check overall reproducibility. Because I'm not working from a tag, there is a greater chance of a line ending difference or similar triggering a reproducibility issue. Looking now...

[markt-asf]

The issue is that the installer binaries are different. May be due to version differences between Ubuntu's standard packages (3.09-4) and Windows (3.10). Looking further.

[rainerjung]

Sorry for the work but thanks a lot. Some of the NSIS version differences should be mitigated by us using the DLLs from the 3.0.8 download of NSIS, that out ant build.xml does. Our NSISDIR point to that 3.0.8 and also the nsis.exe. So it should be 3.0.8 (Windows/Wine) against the local Unix makensis. The local Windows platform installed one should not be in use.

[markt-asf]

The Ant build should download and use NSIS 3.10. I'm going to try building the 3.10 src package for Linux locally. If that doesn't give us repeatable builds then I think we have to continue using wine.

[rainerjung]

Oups, I noticed I branched off main somewhere in the past, not from the recent head...

[rainerjung]

Building a minimal makensis from an nsis source download on Linux should work using:

scons UNICODE=yes PREFIX=/path/to/install/nsis-3.10 SKIPSTUBS=all SKIPPLUGINS=all SKIPUTILS=all SKIPMISC=all NSIS_CONFIG_CONST_DATA_PATH=no install-compiler

(taken from the INSTALL file). This will not need Windows dependencies and also not try to install DLLs etc. We don't need them in the locally installed NSIS, because we take them from our download anyways (and let NSISDIR point there).

[rainerjung]

I can confirm, that the (unsigned) installer is different, if being build with makensis Linux 3.08 and makensis Linux 3.10. Even if both are using the same downloaded zip for DLLs and both are build as minimal makensis standalone binaries without dependencies on the same machine.
So at least it is expected, that one must use the same nsis version.
I will also try to setup the build chain on a Windows system to be able to compare the build installers.

[rainerjung]

Note: when experimenting with a minimal makensis build, the path detection of makensis in our build.xml fails, because executing the bare makensis gives an error. As a short term workaround I set the unix environment variable NSISDIR to our ${nsis.home} to make the experiment work.

[markt-asf]

Progress. When building makensis you need something like:

scons UNICODE=yes PREFIX=/home/mark/libs/nsis-3.10/bin SKIPPLUGINS=all SKIPUTILS=all SKIPMISC=all NSIS_CONFIG_CONST_DATA_PATH=no VERSION=3.10 install-compiler

Key points to note:

• you need to specify the version (this ends up in the installer binary so it needs to match)
• the prefix needs to be the bin directory of the NSIS download else the paths aren't right.

I now have something that is very close but all the changes mean the signatures are no longer valid. I need to start again with a clean build so test the signatures.

Note I am still working with your first commit. I haven't pulled in the subsequent commits.

[rainerjung]

Thanks. Unfortunately my own experiments, using a minimal self-build makensis on Linux and the same version on Windows seem to show, that even when just running makensis.exe /D... and makensis -D... using the exact same tomcat.nsi (I removed the two finalizers) and the exact same output/dist directory produce slightly different installer files. I hope you get a better result, but for me it seems, that makensis and makensis.exe do not produce binary identical results :(

[markt-asf]

Worked perfectly. Release built and signed on Linux without wine. Build 100% reproducible on Windows. Signatures good. Installer and uninstaller work perfectly. That was only with your first commit. I still need to test the subsequent commits.

[ChristopherSchultz]

Worked perfectly. Release built and signed on Linux without wine. Build 100% reproducible on Windows. Signatures good. Installer and uninstaller work perfectly. That was only with your first commit. I still need to test the subsequent commits.

This requires a custom build of makensis, though, right?

[markt-asf]

Yes, I had to build makensis from source - but that was trivial.

[markt-asf]

I think this is an excellent addition to the build script. It doesn't change the current behaviour and it provides makensis as an alternative - one that I very much welcome as getting wine set up and running on MacOS M1 was non -trivial. If this lets me avoid that (to be confirmed) that would be fabulous. Even if it doesn't, I'll be switching to makensis on linux. I particularly like the switch to using callbacks from the installer to do the signing.

I've applied this manually to 12.0.x and have made a few minor changes after a review. My plan is to commit this with all due credit to @rainerjung and then retest repeatable builds across Windows, Linux and MacOS.

[rainerjung]

Thanks for your great support! Integrating it can of course wait until after the possible quick interim release for the JDKCompat fix.

[markt-asf]

Happy to help and thanks for all the work you have put in. I can commit this to 12.0.x now for testing but not back-port it until a) we are happy with it and b) 11.0.4 has been tagged.

[rainerjung]

I am adding a few lines to BUILDING.txt. Concerning your scons argument "PREFIX=/home/mark/libs/nsis-3.10/bin": this nsis path comes from a separate binary download (for Windows), that you extracted there? It is not the path to which ant downloads and extracts nsis during the ant run, correct?

So putting it there is "just" needed, because otherwise calling makensis without arguments leads to an error, because it can't find files it needs and this in turn leads to ant failing to accept it during checking for makensis in the path. Correct?

[markt-asf]

That location is where Ant downloads NSIS on my linux box (I use base.path=/home/mark/libs). I don't have a local NSIS install on my Linux box.

[markt-asf]

For the record this works for both Linux and MacOS. I have an original build of 12.0.x on Linux that, if I copy the sig files to Windows and/or MacOS I can reproduce the build exactly. Nice work.

[ChristopherSchultz]

Yes, I had to build makensis from source - but that was trivial.

I'm asking because a custom build of a part of a toolchain would look suspicious to anyone auditing our release process. Obviously, we can document the process of taking an official (source) release and modifying it in non-shady ways, but it's just One More Thing that both release managers and release verifiers must do in order to build and verify the release.

The beauty of being able to just "ant release" right now is really great.

I suppose the use of makensis is a user decision, so we are supporting those who wish to use that particular tool instead of Wine. Would the "official" release process then be to use nsis.exe via Wine so that our Official Release Process doesn't require this special build? (It does, however, have a Wine requirement... :/)

[ChristopherSchultz]

Progress. When building makensis you need something like:

scons UNICODE=yes PREFIX=/home/mark/libs/nsis-3.10/bin SKIPPLUGINS=all SKIPUTILS=all SKIPMISC=all NSIS_CONFIG_CONST_DATA_PATH=no VERSION=3.10 install-compiler

Key points to note:

* you need to specify the version (this ends up in the installer binary so it needs to match)

* the prefix needs to be the bin directory of the NSIS download else the paths aren't right.

I now have something that is very close but all the changes mean the signatures are no longer valid. I need to start again with a clean build so test the signatures.

Note I am still working with your first commit. I haven't pulled in the subsequent commits.

@markt-asf When you have this working to your satisfaction, could you please update the https://cwiki.apache.org/confluence/display/TOMCAT/Environment+Setup+for+Releases page to include instructions for building makensis for non-Windows platforms? I'd be happy to try it myself as well.

[rainerjung]

@ChristopherSchultz Note that in ignorance of the wiki I (and Mark) already updated BUILDING.txt in main with details on the Windows installer build and on the custom makensis build. If there is a need to have it in the wiki as well, one could probably copy parts of that.

[markt-asf]

I'm asking because a custom build of a part of a toolchain would look suspicious to anyone auditing our release process.

I'd counter that in two ways:

• We are building from source. That is arguably safer than building using an opaque Windows binary.
• The build is repeatable. Build it with the Windows binary and you'll get the exact same bits.

[markt-asf]

This is now working perfectly. Thanks to @rainerjung for all the effort put into this. Being able to build Tomcat releases on MacOS without having to install Wine means I can now easily repeat Tomcat builds across MacOS, Linux and Windows.