Improve error handling and reporting in query side

[gortiz]

Error handling and reporting can be improved in Pinot. Sometimes, error messages are not easy for users to understand, and the fact that each query engine notifies errors differently makes it even more confusing. Here is a nonexclusive list of improvements we need to apply:

• On errors, Pinot controller query endpoints inconsistently return broker-like JSON error payloads (if the broker detects the error) or plain text (if the controller itself detected the error)
• The Pinot controller detected errors are not logged.
• MSE errors include the stack trace and don't include the actual error code. Instead, error code 200 is always returned.
• MSE errors are logged several times in the same server.
• MSE checks and SSE checks are different. Some are copied and pasted in both places, but some constraints are enforced differently.
  • MSE doesn't check if tables are disabled

[gortiz]

Opened #14951 to fix the first two problems

[real-mj-song]

MSE errors are large and don't include the actual error code. Instead, error code 200 is always returned.

💯 agreed. We definitely need a throwable/exception class containing Pinot error code info. I described the reasoning in this comment.

[vrajat]

MSE errors include the stack trace and don't include the actual error code. Instead, error code 200 is always returned.

This is true for SSE as well. For example:

"exceptions": [
    {
      "message": "QueryExecutionError:\nQuery execution error on: Server_100.81.71.117_7050 org.apache.pinot.spi.exception.EarlyTerminationException: Interrupted while processing next block",
      "errorCode": 200
    },

The error code should have been 503 or QUERY_CANCELLATION_ERROR_CODE

[gortiz]

Please take a look at #14994