Maven plugin to produce complex matchers?

[beargiles]

Problem Statement

One of the biggest hassles when introducing Hamcrest to a team is the fact that the matchers are often field-by-field. That's hard to maintain as objects change, it can cause frustrating loops since you only see one error at time, etc.

Reflection helps but it can go too far the other way - it can check things that don't matter.

The best solution is writing custom matchers that understand your classes - what's important, what's not - but it can be difficult for teams to figure out how to do this and again you're back to the question of maintainability.

Ideal solution

A maven plugin that's automatically run during the 'generate test source' phase of the build. It reads the existing source code or jars and implements appropriate matchers.

I don't know how this is handled in gradle but I'm sure there's an analogous method.

Possible Solution

I have built a POC but wanted to ask a few questions first.

Preferred approach.

I see two obvious candidates:

• java reflection during the build process
• a custom javadoc doclet implementation

In both cases the tool should produce java source code since that allows customization. (It could be something simple, it could be something pretty advanced like the way spring data handles extensions to the generated code.)

My POC has taken the first approach but it requires the use of a third party library in order to navigate the newly built jar. The javadoc approach does not require an external tool but it doesn't have the same natural fit as the reflection-based tool. However it would be easy to have the mojo support putting the generated code into the expected location of generated test source code.

Identifying fields to match

We don't want to use reflection because sometimes objects have elements that we don't care about.

We already have a solution in the persistence world - there is a @Transient annotation that tells the ORM to ignore that field.

Since Hamcrest is often used to test persistence mechanisms it makes sense that the maven plugin could use the same JPA annotations. It wouldn't need the full annotation - simply knowing that something is a @Value tells the plugin to include the field in the matcher.

In addition JPA already has support for parent-child and many-to-many checks. This could be used to provide matchers for nested fields and collections.

The JPA annotation could be used anywhere - even when it's not used in the actual persistence mechanisms - since it's just a standard set of annotations in the core JVM packages.

The alternative is to define our own set of annotations. There's definitely some benefits for that, esp. when annotating classes that are not persisted, but I think the benefits of reducing the friction of introducing these new tests by building on annotations that are often already present outweighs the cost of adding the new annotations elsewhere.

Most importantly this removes any new dependencies in the existing 'main' classes.

Therefore my POC uses the JPA annotations.

Code generation

The final question involves the code generation. The java reflection or doclets will create a model - how is it converted to source code?

There are several suitable libraries. The safest bet is probably velocity. It's well tested, widely used, etc. However it's pretty old and doesn't have the same features as newer template engines. More importantly fewer and fewer developers will have a deep understanding of velocity templates vs. newer templates.

I'm leaning towards jinja2 since it's widely used. However I haven't found a library with the same level of trust as org.apache, org.spring, etc. There's a real risk of unknown vulnerabilities but I think that's offset by the control we have over the input and the fact that the generated code will only be used in tests, not production.

I'll wait a few days for feedback then submit a PR for further discussion. I have used this plugin for my own projects and it's definitely nice to get more robust testing and reports with a single 'assert()' statement but some people may feel it's too far from the core project for it to be a good fit here. Or there will be a strong preference for using doclets over reflection.

Bear