Kuiper 1.2.1 beta.1 version

Overview

Kuiper 1.2.1 beta.1 added the support of binary reading and fix some problems of the support of Edgex v2 message. It is a pre-release for integration test.

Features & fixes

• Support binary reading of Edgex v2 message
• Update to the latest Edgex v2 modules

Kuiper 1.2.0 version

Overview

Kuiper 1.2.0 added external function to map existing REST, gRPC or msgpackrpc services into user defined functions by configuration. It is an easy to use extension method to complement plugin. It is more recommened than plugin extension for all scenarios except performance critical scenario. External function can be defined in a batch by a json file and one or multiple schema files (only support protobuf format now). It is possible to hot reload by CLI or REST API. Addionally, http options are supported in the proto schema to better define the REST services.

A new type of source table was added in this release. It works as a snapshot at the current time point for the source. Users can use it as a lookup dictionary or referenced state by joining it with a stream.

We also added a new SQL expression: case expression to evaluates a list of conditions and returns one of multiple possible result expressions. Several enhanements were also added to the built-in functions. And the wildcard is now supported as the parameter of user defined functions. Finally, we fixed several issues for both the product and document.

Features & fixes

• Support external function to invoke exiting services in SQL by configuration only
  • Implement external function configuration parsing and execution of REST services, gRPC services and msgpackrpc services
  • Support protobuf as the schema type, support google http api options in the protobuf to better define the REST services
• Support table type to represent the current state of the stream
  • Support table type, statements and management CLI/REST API
  • Add file source which is typically used as a static table
  • Allow to use existing source type as a continuous table
• Support case expression
• Support * as parameter of custom functions
• Support string parameter for json functions
• Allow bigint, float and string as the parameter of format_time function
• Rename "reengine_field" to "kuiper_field" for non-specified fields
• Build
  • Fix a permission problem when injecting configuration through mouted volumes by Docker
  • Add check for markdown directory
  • Add check for deb/rpm
  • Update tdengine version
• Fixes
  • Avoid printing password in logs
  • Let meta fields ignore cases
  • Fix problems when joining without stream name in the field ref
  • Fix alias for same field names in multiple stream problem
  • Fix access subfolder failure problem in absolute mode
  • Fix stream describe problem for multi-level embedded structs
• Documentation fixes
  • Fix some broken links
  • Fix several typos
  • Update json expression doc

Thanks

• @jrtitus provides several fixes for the documentation
• @tmactao provides a fix for the documentation

Kuiper 1.2.1 beta version

Overview

Kuiper 1.2.1 beta added the support of Edgex v2 message. It is a pre-release for integration test.

Features & fixes

• Support Edgex v2 message

Kuiper 1.1.2 version

Overview

Kuiper 1.1.2 added a sample plugin LabelImage to demonstrate the integration with machine learning framework TensorFlow Lite. Additionally, function plugin was enhanced to support defining multiple functions in one plugin to organize functions in category and reduce the complexity to create functions. To build the extended plugins, the build tag --trimpath is now required in order to improve the compatibility. We also support more built in functions like geohash related functions and cardinality , support more expressions like negative array index and support more rule properties like sendError. Finally, we fix several issues for both the product and document.

Features & fixes

• Integration with TensorFlowLite
  • Sample plugin LabelImage to leverage pre-trained TensorFlowLite model to label images
  • Tutorial about Run TensorFlow Lite model with Kuiper function plugin
• Plugin enhancements
  • Support multiple functions in one function plugin
  • Support to get Kuiper root path in context ctx.GetRootPath()
  • Refactor image functions to one plugin
  • Add GeoHash plugin to support several geo hash functions
• Build
  • Add --trimpath build option for kuiperd build. Since this release, this property is required to build extended plugins
  • Run process/container as kuiper user to avoid security risk
• Support negative array index such as array[:-1]
• Add built in function cardinality(array) to get the length of array
• Add rule property sendError to specify if runtime errors will send to the sink
• Fixes
  • Fix stop rule memory clean up problem
  • Restore httpPull dataSource support
  • Fixed describe stream issue for bytea type
  • Fixed arbitrary session window error at the first input
  • Remove unused empty folders inside data
• Documentation fixes
  • Fix several links
  • Fix an index error for built-in functions doc

Thanks

• @beaufrusetta provides a fix to run process/container as kuiper user
• @noahlaux provides a fix for the documentation

Kuiper 1.1.1 version

Overview

Kuiper 1.1.1 is a minor version that fixes an EdgeX ZeroMQ issue.

Features & fixes

• Upgraded EdgeX message bus, and fixed EdgeX ZeroMQ issue.

Kuiper 1.1.0 version

Overview

Kuiper 1.1.0 add an image processing plugin, and support to process image in rule engine; Also optimized the SQL execution plan. This version also fixed several document issues.

Features & fixes

• Binary data type support
  • Based on the binary data supported in last release, now Kuiper supports 2 image process functions.
  • Added an FVT scenario for binary processing.
• Support streams & rules update in kubernetes-tools
• SQL plan optimization:
  • Enhance PushDownPredicate for join on condition, if the condition is related to one source only for inner join, it will be executed earlier.
  • Add ColumnPruning rule, all unused columns or metas will be dropped in preprocessor to reduce memory usage
• Removed shared-subcription configuration for MQTT source. We suggest to use the MQTT topic for shared subscription directly in MQTT source.
• Documents
  • Fixed several 404 document issues.
  • Fixed several docs and translation.

Kuiper 1.0.2 Released

Overview

Kuiper 1.0.2 optimized the SQL execution plan and runtime, it greatly improves performance for some scenarios. Also Kuiper support the binary type in this release, and it can be used for processing binary streams, such as image.

Features & fixes

• SQL plan optimization
  • Optimized the SQL plan, add PushDownPredicate support and it can filter some records in advance according to the condition specified in WHERE clause. It can reduce memory usage for long time window processing.
  • More optimization will be introduced in future releases.
• Binary data type support
  • Now the binary data type is supported, and user can process binary streams, such as processing images.
  • An image sink plugin for saving images files into local disks.
• Change streams & rules storage from file to SQLite.
• Ping API support, which is used for Kuiper service alive detection.
• Fixed one memory leak problem - some of memory is not released when the rule is deleted.
• No way to limit client and REST APIs to localhost
• File logging error message when server fileLog=false
• Added a retry count for all sinks
• Logrotate support for log files - rotate time is 24 hrs, and max age is 3 days
• Added several SQL validation during compilation stage
• Enhanced document compilation.
  • Fixed several document problem.
  • Fixed several issues for httppull source
  • Fixed issue when loading array types in management-ui

Kuiper 1.0.1 Released

Overview

Kuiper 1.0.1 fixed several problems and enhanced Kuiper manager.

Features & fixes

• Kuiper manager

  • Support for stream and rule update.
  • Add rule status visualization support.

• Add support for writing to Linux syslog.

• TDengine ARM64 sink plugin support.

• Improved EdgeX using tutorial.

• Fixed issues

  • The Kuiper crash issue for ARM64 environment.
  • Change to a smaller time unit for elapsed time (rule status)
  • Kuiper manager UI
    • The list value cannot be displayed in sink editor
    • Incorrect UI display for the complex streaming definition

Kuiper 1.0.0 Released

Overview

Kuiper 1.0.0 is the 1st major stable version. This version is fully integrated with other EMQ softwares for edge, including Neuron, Edge and edge manager. Readers can refer to edge-stack project for how to use EMQ softwares for edge.

Features & fixes

• A break change! If any program or scripts that references to Kuiper binary file name bin/server & bin/cli, you'll have to update correspondingly as in below.
  • Renamed bin/server to bin/kuiperd
  • Renamed bin/cli to bin/kuiper
• EdgeX enhancements & fixes
  • Gives error when configured to use redisstreams for EdgeX MessageBus
  • Updated EdgeX docker-compose file, and upgrade to Kuiper 0.9.1 version
  • Go 15 upgrading
  • Message bus SDK upgrading
  • EdgeX Kuiper document & tutorial update
  • Containerized Kuiper fails when started with read-only file system
• Updated the documents
  • Kuiper docker images usage document.
  • Wrote and translated the document for using Kuiper manager console.
• Enhanced the TDengine plugins
  • Renamed the plugin name from taos to TDengine
  • Allow user to control the timestamp for data saving into TDengine
  • Set default values for TDengine sink plugin
• Allowed user to pass parameters for plugin setup scripts, which make the plugins setup more flexible.
• Returns i18N message with rest services.
• Fixed below issues
  • The line number is not correct in log file

Thanks

• @TateDeng provides fix for DynamicChannelBuffer setting

Kuiper 0.9.1 version

Overview

Kuiper 0.9.1 provides a management console, which can be used for Kuiper node management, stream, rule and plugin visualize edit. It greatly improves the using experience.

Features & fixes

• Visualize management: A separated new Docker image was released with this new version. The image is a web based management console, and it provides streams, rules and plugins management. Please refer to doc fore more detailed information.
• Binary plugins: From this version, all of the plugins that submit to Github main repository will be compiled and published automatically when a new version is released. User can install & use the plugins through management console. When developer create PR for plugin code, the plugin metadata information can be provided, and then the related information will be displayed in the management console. It can greatly reduce the effort for using the plugins.
• User can control to cache the data that failed to publish in the sink by set disableCache of kuiper.yaml.
• Kubernetes configuration file dispatch tool multiple CPU arch support.
• Add Collect function support, which can be used for return all of data in the window.
• Add Deduplicate function support, which can deduplicate the data of window.
• Fixed below issues
  • Edgex-ui cannot post cross-domain requests
  • Kuiper process exits in log debug mode
  • Rule is created even with error reported during rule creation
  • Set sendSingle as true but no data template is specified, the process is quit
  • Use describe plugin command error
  • Nested struct definition reports syntax error.

Thanks

• @soyoo provides several log print fixes