Kuiper 0.9.0 version

Overview

Kuiper 0.9.0 is a major version that includes the stream state management, KubeEdge device model adoption, EdgeX array type support and TDengine database sink support.

Features & fixes

• State management function is supported. This feature enables Kuiper to implement stream with state:
  • Fault tolerant processing is supported. If an unexpected interruption occurs during stream processing, it can be resumed after the rule is restarted;
  • Checkpointing is supported, which allows users to implement QoS settings during stream processing, including At-most-once(0), At-least-once(1) and Exactly-once(2);
  • By consuming data from the specified offset, users can extend the relevant interface to realize the data source that can re-consume the data, so that it can be recovered offline or when there is an error in stream processing;
  • Configurable state persistent storage. The system stores the state in the file system by default, and also supports storing state data in a third-party system, such as Redis;
  • The user is supported to call the API provided by Kuiper to realize the storage of custom state data when extending the source, sink and function;
• Access support for KubeEdge data model is provided, with a tool for automatically issuing configuration files through Kubernetes configmap. Users can use Kuiper to directly support the analysis of device data from KubeEdge;
• TDengine plug-in is added, which can support saving analysis results to TDengine time series database;
• All Chinese documents are translated and synchronized;
• The execution process of FVT in Github Action is optimized, and some unnecessary test processes are deleted;
• filter in the window with support for filtering data can realize the data filtering and window grouping; this function is more important for counting windows: the result of filtering through the WHERE statement and then window grouping will be different;

Thanks

• @chensheng0 provides a fix for Kubernetes configmap, which can be integrated with Baidu Baetyl framework.
• @GZJ provides a fix that cleans up the site when Kuiper exits.
• @smart33690 provided a fix for the Influxdb sink sample plug-in.

Kuiper 0.5.1 version

This release fixed several issues.

• Refactor the kuiper.yaml loading
• Fixed rest sink header settings
• Fixed rest sink cache problems - the cache entries cannot be deleted even they are sent successfully
• Add insecureSkipVerify in rest sink, which allows user to set in-secured verification flag

Kuiper 0.5.0 version

Overview

Kuiper 0.5.0 adds some significant features and also fix problems requested from community.

Features & fixes

• Support for using Kuiper keywords in SQL statements.
• Support for count window, which allows user do count based window analysis.
• More JSON functions are supported, including json_path_exists, json_path_query, json_path_query_first.
• Updated Github action and add go fmt in continuously integration pipeline.
• Added contributing guide.
• Add Influxdb sink sample plugin.
• A document for reserved keywords in Kuiper.
• Update plugin development tutorial document.
• Fixed rule order issue.
• Fixed the problem with column name with '.' will have an error log.
• Fixed aggregation functions with nil value.
• Fixed aarch64 binary package issue.

Thanks

• @worldmaomao provided rule order issue.

• @smart33690 provided Influxdb sink sample plugin.

Kuiper 0.4.2 version

• Kuiper 0.4.2 fixes a problem for aarch64 - it throws Segmentation Fault, will not use upx to compress the binary file.
• Support to set Kuiper Docker environment with more generic way.
• Fixed EdgeX optional setting problem.

Kuiper 0.4.1 version

Overview

Kuiper 0.4.1 version fixed some problems reported from community.

Features

• Support multi-instances creation for plugins. In previous versions, plugin only supports one single instance, now multiple plugin instances can be supported by returning a construction function.
• Fixed dataTemplate usage in rule document.
• Fixed EdgeX floating data can not be processed problem in some cases.
• Support for updating EdgeX MQTT message bus configurations by Docker environment variables.
• Support for using alias for aggregation functions, and then use alias name in WHERE or HAVING clause, it can greatly improve the performance.
• Returns 404 response code when deleting not existed streams or rules, while previously returns 400.
• Fix sink infinite resend after rule stopped problem.
• When using SELECT * , now it can correctly process the field name case by reading stream schema definition or actual field name sent in streams.
• Fixed problem of multiple same type of sink cannot work correctly.
• http_pull source support, it can pull data from HTTP rest interface by specified interval time.

Thanks

@worldmaomao provided EdgeX floating processing fixes.

Kuiper 0.4.0 version

Overview

• The capability of supporting more complex plugin;
• Support template feature for all sinks, so users can customize complex output data.

Features

• Supported template feature for all of sinks, now user can leverage template to customize complex output data in all sinks.
• When user invoke API for creating a plugin, user can provide a install.sh script in zip file, so that the complex library dependencies installation can be supported.
• Added FVT scenario for plugin.
• Updated document structure.
• Added Docker image based on Debian, user can select this image if want to have more library dependencies.
• Add Kuiper plugin development tutorial.

Kuiper 0.3.2 version

Overview

Kuiper 0.3.2 is the candidate release that ship with EdgeX Foundry Geneva. User can follow this tutorial for starting use Kuiper rule engine in EdgeX Foundry Geneva release.

Features

• EdgeX integration

  • Add float decoding support in EdgeX source

• Return rule status when getting list of rules.

• Unify all of Golang compile version to 1.13.

Bug fixes

• Fixes some document and bug issues.

Kuiper 0.3.1 version

Overview

The Kuiper & EdgeX integration work is going to be released, Kuiper team fixed several problems during integration with EdgeX . Kuiper 0.3.1 or 0.3.2 will be released with EdgeX officially.

Features

• EdgeX integration

  • EdgeX MQTT message bus support. Kuiper source & sink now can support message bus over MQTT broker .
  • Benchmark result for EdgeX: EdgeX Kuiper rule engine supports 11.4k message throughput per second in AWS t2.micro( 1 Core * 1 GB). Refer to this doc for detailed info.
  • Expose Kuiper Rest API port in Docker environment variable.

• Add a Kuiper plugin development tutorial (Chinese).

Bug fixes

• Fixes some document issues.

Kuiper 0.3.0 version

Overview

The Kuiper & EdgeX integration work is almost completed, Kuiper will be integrated with EdgeX nightly Docker composer files soon. Kuiper 0.3.1will be released with EdgeX officially.

Features

• EdgeX integration

  • Fixed some issues based on user's feedback, and updated the tutorial doc .
  • New implementation EdgeX sink support based on the feedback from community, the analysis result can be published with the required format of EdgeX Message bus.
  • Translated EdgeX document to Chinese.

• Plugin management

  • Kuiper now supports plugin management, but it was not provided in old versions. In this version, Kuiper provides REST API & command line tools. User can add, delete and view plugins through management tools.

Bug fixes

• null handling in pre-processor
• Rest sink exceptions
• Fixes some document issues.

Kuiper 0.2.1 version

Overview

The initial version of Kuiper & EdgeX integration work was just completed, users can start to try it. In the coming EdgeX Geneva release, Kuiper 0.3.0 or 0.4.0 will be released with EdgeX officially.

Features

• EdgeX integration

  • EdgeX source support, now Kuiper can consume data from EdgeX Message Bus directly. You can refer to tutorial doc for learning how to use Kuiper to analyze data from EdgeX.
  • EdgeX sink support, the analysis result can be published to EdgeX Message Bus directly.

• Schemaless stream definition

  In previous Kuiper releases, user must create a stream with schema, but we found that in some user cases, the data schema could be very complex, and it will be difficult to create data schema for it. Now Kuiper supports to create a stream that does not have any fields, in this mode, Kuiper framework can't validate data types during data processing. So user need to very clear about the data types when writing rules, otherwise, it could probably have problems. EgeX tutorial doc is a good example of using schemaless stream.

• FVT test schenarios enhancements

  • Added 4 EdgeX testcases
  • Added testcases for Docker image

Bug fixes

• Fixed several problems of Github Actions