FELIX-6708 upgrade bnd to 7.0.0 #322

paulrutter · 2024-05-25T16:44:29Z

See https://issues.apache.org/jira/browse/FELIX-6708 and bndtools/bnd#2227 (comment)

To be able to handle Multi-Release jars as dependencies in an OSGi bundle.
This prevents exceptions like:

build	24-May-2024 18:40:01	[ERROR] Bundle com.abc:jersey-bundle:bundle:55.0-SNAPSHOT : Classes found in the wrong directory: {META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport$NonLoomishExecutors.class=org.glassfish.jersey.innate.VirtualThreadSupport$NonLoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/virtual/LoomishExecutors.class=org.glassfish.jersey.innate.virtual.LoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport$Java21LoomishExecutors.class=org.glassfish.jersey.innate.VirtualThreadSupport$Java21LoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport.class=org.glassfish.jersey.innate.VirtualThreadSupport}

To be able to handle Multi-Release jars as dependencies in an OSGi bundle. This prevents exceptions like: ``` build 24-May-2024 18:40:01 [ERROR] Bundle com.abc:jersey-bundle:bundle:55.0-SNAPSHOT : Classes found in the wrong directory: {META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport$NonLoomishExecutors.class=org.glassfish.jersey.innate.VirtualThreadSupport$NonLoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/virtual/LoomishExecutors.class=org.glassfish.jersey.innate.virtual.LoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport$Java21LoomishExecutors.class=org.glassfish.jersey.innate.VirtualThreadSupport$Java21LoomishExecutors, META-INF/versions/21/org/glassfish/jersey/innate/VirtualThreadSupport.class=org.glassfish.jersey.innate.VirtualThreadSupport} ```

Update to java 17 as thats required by bnd

Revert unneeded change

paulrutter · 2024-05-25T18:54:19Z

Probably this code can be removed now as well, as bnd can handle it.

felix-dev/tools/maven-bundle-plugin/src/main/java/org/apache/felix/bundleplugin/BundlePlugin.java

Line 2236 in 6bb228d

    
           protected static void includeJava9Fixups(MavenProject currentProject, Analyzer analyzer)

It suppresses the errors as warnings via fixupmessages.
This didn’t work for me due to having this line in a parent pom.xml:

<_fixupmessages>^Classes found in the wrong directory: \{META-INF/versions/(9|10|11|12)/</_fixupmessages>

It didn’t cover java 21. Still, it would be nice to move to 7.0.0 as it claims to support multi release jars properly.

paulrutter · 2024-05-30T12:11:28Z

Two integration tests fail still, but one of these also failed on 6.3.1.

[INFO] Building: existing-metadata-no-update\pom.xml
[INFO] run pre-build script prebuild.groovy
[INFO] run post-build script verify.groovy
[INFO]   The post-build script did not succeed. assert manifest.isEmpty()
       |        |
       |        false
       Manifest-Version: 1.0
       Bnd-LastModified: 1717071194581
       Build-Jdk-Spec: 21
       Bundle-Description: This project has been built before (i.e. a cache fil
        e is there), the current pom.xml has not been touched, nor any of the b
        uilt files
       Bundle-DocURL: https://wcm.io
       Bundle-License: http://www.apache.org/licenses/LICENSE-2.0.txt
       Bundle-ManifestVersion: 2
       Bundle-Name: existing-metadata-no-update
       Bundle-SymbolicName: org.apache.felix.bundleits.existing-metadata-no-upd
        ate
       Bundle-Vendor: wcm.io
       Bundle-Version: 1.0.0.SNAPSHOT
       Created-By: Apache Maven Bundle Plugin 5.1.10-SNAPSHOT
       Tool: Bnd-7.0.0.202310060912
[INFO]           existing-metadata-no-update\pom.xml .............. FAILED (28.9 s)
[INFO] Building: no-test-scoped-imports\pom.xml
[INFO] run post-build script verify.groovy
[INFO]   The post-build script did not succeed. Import-Package instruction contains version number from test dependency
[INFO]           no-test-scoped-imports\pom.xml ................... FAILED (8.2 s)

paulrutter · 2024-06-02T07:52:11Z

@hboutemy @rotty3000 any feedback on this pr, as you both worked on upgrading bnd last time it was upgraded.

rotty3000

LGTM

paulrutter · 2024-06-03T15:39:38Z

What do you think about the failing Integration tests?

rotty3000 · 2024-06-03T16:17:43Z

@paulrutter do this to fix the test:

diff --git a/tools/maven-bundle-plugin/src/it/no-test-scoped-imports/pom.xml b/tools/maven-bundle-plugin/src/it/no-test-scoped-imports/pom.xml
index 3a15568ced..2eedf2350c 100644
--- a/tools/maven-bundle-plugin/src/it/no-test-scoped-imports/pom.xml
+++ b/tools/maven-bundle-plugin/src/it/no-test-scoped-imports/pom.xml
@@ -53,6 +53,11 @@ under the License.
                 <artifactId>maven-bundle-plugin</artifactId>
                 <version>@project.version@</version>
                 <extensions>true</extensions>
+                <configuration>
+                    <instructions>
+                        <_noimportjava>true</_noimportjava>
+                    </instructions>
+                </configuration>
             </plugin>
         </plugins>
     </build>

rotty3000 · 2024-06-03T16:20:02Z

In fact, @paulrutter if you want to preserve the original behaviour of maven-bundle-plugin not to import java.* you may wish to set that bnd flag default to true such that users opt-in by setting it to false.

ref: https://bnd.bndtools.org/instructions/noimportjava.html

laeubi · 2024-06-03T16:24:03Z

I think java packages should simply be always imported, I don't see a reason to disable that, especially as its the default for bnd.

rotty3000 · 2024-06-03T16:30:25Z

whatever you prefer! 🤷🏻

paulrutter · 2024-06-03T17:25:08Z

Great, i will adjust the PR and rerun the tests with that instruction.

+ <configuration> + <instructions> + <_noimportjava>true</_noimportjava> + </instructions> + </configuration>

paulrutter · 2024-06-04T07:07:49Z

The only IT still failing was also failing with bnd 6.3.1.

[INFO] --- invoker:3.2.1:run (integration-test) @ maven-bundle-plugin ---
[INFO] Building: archiver\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           archiver\pom.xml ................................. SUCCESS (13.3 s)
[INFO] Building: dep-reduced\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           dep-reduced\pom.xml .............................. SUCCESS (4.9 s)
[INFO] Building: embed-multiple-artifacts\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           embed-multiple-artifacts\pom.xml ................. SUCCESS (4.4 s)
[INFO] Building: existing-metadata-no-update\pom.xml
[INFO] run pre-build script prebuild.groovy
[INFO] run post-build script verify.groovy
[INFO]   The post-build script did not succeed. assert manifest.isEmpty()
       |        |
       |        false
       Manifest-Version: 1.0
       Bnd-LastModified: 1717484827643
       Build-Jdk-Spec: 21
       Bundle-Description: This project has been built before (i.e. a cache fil
        e is there), the current pom.xml has not been touched, nor any of the b
        uilt files
       Bundle-DocURL: https://wcm.io
       Bundle-License: http://www.apache.org/licenses/LICENSE-2.0.txt
       Bundle-ManifestVersion: 2
       Bundle-Name: existing-metadata-no-update
       Bundle-SymbolicName: org.apache.felix.bundleits.existing-metadata-no-upd
        ate
       Bundle-Vendor: wcm.io
       Bundle-Version: 1.0.0.SNAPSHOT
       Created-By: Apache Maven Bundle Plugin 5.1.10-SNAPSHOT
       Tool: Bnd-7.0.0.202310060912
[INFO]           existing-metadata-no-update\pom.xml .............. FAILED (28.2 s)
[INFO] Building: no-test-scoped-imports\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           no-test-scoped-imports\pom.xml ................... SUCCESS (7.9 s)
[INFO] Building: reproducible\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           reproducible\pom.xml ............................. SUCCESS (2.9 s)
[INFO] Building: reproducible-manifest\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           reproducible-manifest\pom.xml .................... SUCCESS (5.7 s)

paulrutter · 2024-06-04T08:51:48Z

I think we should update the major version when we release this, due to the minimum jvm version updated to 17.

laeubi · 2024-06-04T08:59:34Z

I think we should update the major version when we release this, due to the minimum jvm version updated to 17.

bnd major update > felix major update regardless of java version (but plugin minium java version must be set to 17 then)

…proper manifest is created with bnd 7.0.0

rotty3000 · 2024-06-04T13:25:39Z

The only IT still failing was also failing with bnd 6.3.1.

[INFO] --- invoker:3.2.1:run (integration-test) @ maven-bundle-plugin ---
[INFO] Building: archiver\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           archiver\pom.xml ................................. SUCCESS (13.3 s)
[INFO] Building: dep-reduced\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           dep-reduced\pom.xml .............................. SUCCESS (4.9 s)
[INFO] Building: embed-multiple-artifacts\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           embed-multiple-artifacts\pom.xml ................. SUCCESS (4.4 s)
[INFO] Building: existing-metadata-no-update\pom.xml
[INFO] run pre-build script prebuild.groovy
[INFO] run post-build script verify.groovy
[INFO]   The post-build script did not succeed. assert manifest.isEmpty()
       |        |
       |        false
       Manifest-Version: 1.0
       Bnd-LastModified: 1717484827643
       Build-Jdk-Spec: 21
       Bundle-Description: This project has been built before (i.e. a cache fil
        e is there), the current pom.xml has not been touched, nor any of the b
        uilt files
       Bundle-DocURL: https://wcm.io
       Bundle-License: http://www.apache.org/licenses/LICENSE-2.0.txt
       Bundle-ManifestVersion: 2
       Bundle-Name: existing-metadata-no-update
       Bundle-SymbolicName: org.apache.felix.bundleits.existing-metadata-no-upd
        ate
       Bundle-Vendor: wcm.io
       Bundle-Version: 1.0.0.SNAPSHOT
       Created-By: Apache Maven Bundle Plugin 5.1.10-SNAPSHOT
       Tool: Bnd-7.0.0.202310060912
[INFO]           existing-metadata-no-update\pom.xml .............. FAILED (28.2 s)
[INFO] Building: no-test-scoped-imports\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           no-test-scoped-imports\pom.xml ................... SUCCESS (7.9 s)
[INFO] Building: reproducible\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           reproducible\pom.xml ............................. SUCCESS (2.9 s)
[INFO] Building: reproducible-manifest\pom.xml
[INFO] run post-build script verify.groovy
[INFO]           reproducible-manifest\pom.xml .................... SUCCESS (5.7 s)

I did not get this failure when I locally ran the IT tests on your branch yesterday. Could someone else maybe confirm?

paulrutter · 2024-06-04T13:27:01Z

Maybe it's platform related? I'm running on a Windows 11 based machine.

rotty3000 · 2024-06-04T13:39:17Z

Maybe it's platform related? I'm running on a Windows 11 based machine.

.. and I'm running on Ubuntu Linux. Bnd has often had weird windows issues because it's the least tests platform.

paulrutter · 2024-06-04T13:42:11Z

If you don't see the failing test, then i'd say we're good to go.

paulrutter · 2024-09-03T18:53:34Z

Since it would be a major version upgrade for the plugin (6.0.0?), i don't see a problem with requiring 17.
Also, since the bundle plugin can be used apart from felix itself, we probably don't need to enforce 17 as a baseline for felix framework itself.

cziegeler · 2024-09-04T05:45:17Z

I agree that updating to bnd 7 and requiring java 17 for the tool chain is probably not a huge problem. However, I'm too far away from the bundle-plugin code to make a call here.

paulrutter · 2024-09-04T06:28:05Z

@rotty3000 maybe?

rotty3000 · 2024-09-04T13:06:23Z

man... I've not looked at this for such a long time I would be remise to make a recommendation such out of date info. Perhaps @jbonofre would have better insight?

stbischof · 2024-09-04T13:19:23Z

@paulrutter since this plugin is justr a wrapper around bnd why not using bnd-maven-plugin?

paulrutter · 2024-09-04T13:28:25Z

I looked at bnd-maven-plugin and although that would probably work, i find the configuration (in a CDATA tag in the pom.xml) less convenient/readable than the maven-bundle-plugin.
Also, we have a lot of OSGi bundles (40+) which would be quite some rework (with regression risk).

So ideally, i would like to have an updated maven-bundle-plugin that uses bnd 7.0.0.

laeubi · 2024-09-04T14:13:51Z

Sad to say but bnd-maven plugin even misses some features of felix-maven-plugin, beside that +1 from me to simply upgrade (with or without major version increment) it all looks fine and sane and should has no impact to the rest of felix.

paulrutter · 2024-09-23T16:45:35Z

Who could give the final go for this to get merged and released? Would be a shame to leave it on the shelve unused, but maybe I'm just too impatient 😉

stbischof · 2024-11-18T09:30:08Z

Lgtm

stbischof · 2024-11-18T11:59:58Z

I can merge, but i am not in the release process.
@jbonofre may knows more.

Sorry for that.

For all other Users i highly recommend to use bnd direct.

laeubi · 2024-11-18T12:03:11Z

For all other Users i highly recommend to use bnd direct.

Sadly maven-bundle-plugin still has some features not supported by bnd-maven-plugin and alike directly (or its quite complex to emulate).

chrisrueger · 2024-11-18T14:32:08Z

Sadly maven-bundle-plugin still has some features not supported by bnd-maven-plugin and alike directly (or its quite complex to emulate).

@laeubi Could you list which features these are? That may help to address them in the future.

laeubi · 2024-11-18T14:43:57Z

I think most noteable is <Embed-Dependency> .

(one can potentially try to work around this by a combination of copy-dependecy and bnd instructions but thats really cumbersome if more than one jar is involved)

Then bnd-jar can only work when extension is on (and then mangles with default maven executions) instead of using it as a standalone execution, one can use bnd-process (if it not would be confused by the existing jar) with additional jar executions but again thats quite cumbersome and can give surprising results.

Last but not least, even though its a bit a style question, the usage of XML elements to represent the Instructions instead of an embedded CDATA or dedicated bnd.bnd file is often more "natural" to maven/non-osgi users.

rotty3000 · 2024-11-18T15:06:47Z

<Embed-Dependency> is trivially simple in bnd-maven-plugin

https://github.com/bndtools/bnd/blob/b2deb7f91312a4d37f0504d8cfd904847ca028a2/maven-plugins/bnd-maven-plugin/src/it/test-wrapper-bundle/bnd/wrapper.bnd#L7

The embedding can take many forms.

In the case showed, the "maven dependency" osgi.annotation-*.jar is embedded into the lib/ directory (i.e. lib:=true) AND is appended to the Bundle-ClassPath header (which is created for you automatically with a base of . if not already specified so as not to break the bundle's normal class loading structure).

You could embed in a flattened way using (i.e. the classes and packages are directly beside your own):

-includeresource: @osgi.annotation-*.jar

and so on... there are lots of possible things you can do like extracting certain files and whatnot!

laeubi · 2024-11-18T15:22:48Z

is trivially simple in bnd-maven-plugin

one would think so, but if you

don't know it by name
don't check in the jar into your code repository

it instantly become incredibly hard. As you mentioned here

Sadly, there is no way to specify transitive inclusion natively in bnd

Also includeresource can not filter for scopes (as far as I know) and you have to know the name of the artifact as described here.

So yes that's all doable but far from convenient to the not so experienced users.

Also please keep in mind this important portion of my answer:

but that's really cumbersome if more than one jar is involved

so all these incredible easy example always ever seem to cover exactly one jar where it in fact becomes quite easy.

So to convince people that bnd-maven-plugin is a full replacement, I think the best would be to add docs that cover the examples mentioned here:

<!-- embed all compile and runtime scope dependencies -->
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>

<!-- embed any dependencies with artifactId junit and scope runtime -->
<Embed-Dependency>junit;scope=runtime</Embed-Dependency>

<!-- inline all non-pom dependencies, except those with scope runtime -->
<Embed-Dependency>*;scope=!runtime;type=!pom;inline=true</Embed-Dependency>

<!-- embed all compile and runtime scope dependencies, except those with artifactIds in the given list -->
<Embed-Dependency>*;scope=compile|runtime;inline=false;artifactId=!cli|lang|runtime|tidy|jsch</Embed-Dependency>

<!-- inline contents of selected folders from all dependencies -->
<Embed-Dependency>*;inline=images/**|icons/**</Embed-Dependency>

rotty3000 · 2024-11-18T15:50:59Z

Fair, but these are incorrect assumptions:

don't know it by name

You can use a regex which gets all dependencies? matching is a regex across the names of all artifacts.. so just match all!

don't check in the jar into your code repository

You don't need to care where the jar is, only that maven will ultimately already have gotten it for you!

The rest may be arguably true, but this is widely used in very large projects with MANY jars (please, if you dare, look at this repo and search for -includeresource).

rotty3000 · 2024-11-18T15:53:25Z

anyway.. I do see your point.

laeubi · 2024-11-18T16:04:33Z

The rest may be arguably true, but this is widely used in very large projects with MANY jars (please, if you dare, look at this repo and search for -includeresource).

And it is using.... maven-bundle-plugin ;-)

Don't get me wrong its all possible for sure, it was just asked why people using it and I gave some examples, you should not assume that every maven user that wants to wrap a small (or medium) jar can afford to dive into the details of bnd instruction magic, those people only search for a fast way to archive their goal without much hassle and the plugin is offering that for them.

So if these one-liner examples above can be transformed into a one-liner bnd instruction (or better maven configuration option) it is fine and probably there is just a lack of let people know, if I look here:

https://github.com/bndtools/bnd/tree/master/maven-plugins/bnd-maven-plugin

I don't find any good hint to archive things similar to felix-maven-plugin, (e.g. migration guide) nor how to archive the embedding in a similar way.

cziegeler · 2024-11-18T16:07:58Z

While I really value this discussion and I dont want to stop it, I think this PR is really not the right place. No one will find this here. So please move it to our mailing list - this is the place where discussions really should happen

chrisrueger · 2024-11-18T17:59:34Z

I created a topic here https://bnd.discourse.group/t/bnd-maven-plugin-missing-features-compared-to-felix-maven-bundle-plugin/488 where we can continue the discussion. @laeubi @rotty3000

- Skip this test on windows see, #322 (comment)

paulrutter added 5 commits May 8, 2024 18:31

Update pom.xml

969a5d3

Merge branch 'apache:master' into master

1f1316a

Update pom.xml

93b43b7

Update to java 17 as thats required by bnd

Update pom.xml

efc79a7

Revert unneeded change

paulrutter mentioned this pull request May 25, 2024

3.1.7 OSGi bundle issue eclipse-ee4j/jersey#5661

Closed

paulrutter changed the title ~~FELIX-6708 upgrade bnd~~ FELIX-6708 upgrade bnd to 7.0.0 May 25, 2024

paulrutter added 3 commits May 25, 2024 22:14

Remove fixupmessages

065c1cc

Remove fixup test

6267785

Fix error with conversion

3d7a035

rotty3000 approved these changes Jun 3, 2024

View reviewed changes

Add noimportjava instruction

15ac258

+ <configuration> + <instructions> + <_noimportjava>true</_noimportjava> + </instructions> + </configuration>

paulrutter requested a review from rotty3000 June 4, 2024 08:51

Add integration test that includes multi-release jar and validates a …

f77d087

…proper manifest is created with bnd 7.0.0

paulrutter added 4 commits November 17, 2024 09:16

Merge branch 'apache:master' into maintenance/upgrade-bnd

87aca5f

Update changelog.txt

13686ea

Create README.md

1712d5c

Update README.md

33f94de

laeubi approved these changes Nov 18, 2024

View reviewed changes

stbischof self-requested a review November 18, 2024 11:56

stbischof approved these changes Nov 18, 2024

View reviewed changes

stbischof merged commit cab1b31 into apache:master Nov 18, 2024
1 check passed

paulrutter deleted the maintenance/upgrade-bnd branch November 18, 2024 12:44

paulrutter added a commit that referenced this pull request Nov 25, 2024

FELIX-6708 upgrade bnd to 7.0.0

368df58

- Skip this test on windows see, #322 (comment)

FELIX-6708 upgrade bnd to 7.0.0 #322

FELIX-6708 upgrade bnd to 7.0.0 #322

Conversation

paulrutter commented May 25, 2024 • edited Loading

paulrutter commented May 25, 2024 • edited Loading

paulrutter commented May 30, 2024 • edited Loading

paulrutter commented Jun 2, 2024

rotty3000 left a comment

Choose a reason for hiding this comment

paulrutter commented Jun 3, 2024

rotty3000 commented Jun 3, 2024

rotty3000 commented Jun 3, 2024

laeubi commented Jun 3, 2024

rotty3000 commented Jun 3, 2024

paulrutter commented Jun 3, 2024

paulrutter commented Jun 4, 2024

paulrutter commented Jun 4, 2024

laeubi commented Jun 4, 2024

rotty3000 commented Jun 4, 2024

paulrutter commented Jun 4, 2024

rotty3000 commented Jun 4, 2024

paulrutter commented Jun 4, 2024

paulrutter commented Sep 3, 2024 • edited Loading

cziegeler commented Sep 4, 2024

paulrutter commented Sep 4, 2024

rotty3000 commented Sep 4, 2024

stbischof commented Sep 4, 2024

paulrutter commented Sep 4, 2024 • edited Loading

laeubi commented Sep 4, 2024

paulrutter commented Sep 23, 2024 • edited Loading

stbischof commented Nov 18, 2024

stbischof commented Nov 18, 2024

laeubi commented Nov 18, 2024

chrisrueger commented Nov 18, 2024 • edited Loading

laeubi commented Nov 18, 2024

rotty3000 commented Nov 18, 2024

laeubi commented Nov 18, 2024

rotty3000 commented Nov 18, 2024 • edited Loading

rotty3000 commented Nov 18, 2024

laeubi commented Nov 18, 2024

cziegeler commented Nov 18, 2024

chrisrueger commented Nov 18, 2024

paulrutter commented May 25, 2024 •

edited

Loading

paulrutter commented May 25, 2024 •

edited

Loading

paulrutter commented May 30, 2024 •

edited

Loading

paulrutter commented Sep 3, 2024 •

edited

Loading

paulrutter commented Sep 4, 2024 •

edited

Loading

paulrutter commented Sep 23, 2024 •

edited

Loading

chrisrueger commented Nov 18, 2024 •

edited

Loading

rotty3000 commented Nov 18, 2024 •

edited

Loading