MockServer/MockHandler performance

[sturose]

I am currently attempting to do some performance testing using gatling against an application, and some of the mock servers have a lot of different Scenarios to match against (more than 50, less than 100).

I'm finding that that MockHandler.isMatchingScenario is using more processing than the actual application logic I'm trying to performance test. Sometimes the responses coming back from MockServer will take over 1000 ms when there is no delay configured.

As best as I can tell the vast majority of slowness is coming from JS.eval, which leads me to believe that I should perhaps look to use a different MockServer solution, but I thought I'd see if this is an expected outcome for the way MockServer is designed?

I did attempt to put the more likely to match scenarios at the top, which improved the situation dramatically, but it did not solve the problem of inconsistent response times. At that point performance was still hindered by things like initEngine which seems to occur for every incoming request into the mock server.

I realize this probably isn't a bug report you can use at this time, but I thought it might be worth letting you know.

[ptrthomas]

@sturose thanks for the report and I will keep it open. it is quite likely we can use code from #2530 to replicate

besides JS eval, there is a global lock for every request which in future we need to make user-opt-out via a tag:

karate/karate-core/src/main/java/com/intuit/karate/core/MockHandler.java

Line 153 in 30e93ed

public synchronized Response handle(Request req) { // note the [synchronized]

I'm pretty sure the work we are doing to replace the JS engine will help in case it is the JS eval which is the bottleneck. refer: #2546

finally for completeness - there is this "alternate" way to implement mocks, but my guess is you may want to stick to the simpler way: https://github.com/karatelabs/karate/wiki/Karate-JavaScript-Mocks

[marcinmilewski93]

@sturose

I was facing similar performance problems in MockServer, especially when dealing with a large number of scenarios. It sounds like the JS evaluation (JS.eval) is indeed adding significant overhead, which can lead to inconsistent response times.

In my fork of the repository, I implemented a few optimizations that might help address these issues:

1. Removed Synchronization: I found that synchronization was contributing to performance bottlenecks, so I removed it. After testing, I haven't faced any issues, and this change improved the performance.
2. Optimized Scenario Evaluation: Instead of evaluating all scenarios sequentially (which, in your case, would be 50–100 scenarios), I introduced logic that skips unnecessary evaluations. Specifically, mock scenarios are now skipped if the feature name doesn't match the first part of the URL. This approach narrows the scope of evaluation and has a noticeable impact on response times.

In your situation, where you're testing an application with numerous scenarios, applying this kind of context-path filtering could prevent MockServer from having to evaluate each scenario.

Feel free to explore these improvements if they might be useful in your setup.

3608333