You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: configuration-resources/connectors/camel.md
+4-8
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
-
# Using Apache Camel connectors
1
+
# Apache Camel
2
2
3
3
Configuration for LangStream Sources using Apache Camel.
4
4
5
-
### Deplying the component into your LangStream application
5
+
### Deploying the component into your LangStream application
6
6
7
7
LangStream doesn't bundle all the Camel connectors, but you can easily deploy them into your application.
8
8
@@ -23,12 +23,10 @@ configuration:
23
23
24
24
You have to bundle all the jars needed by the connector - in this example the Apache Camel GitHub component requires the org.eclipse.egit.github.core jar file.
25
25
26
-
The jar files are downloaded by the LangStream CLI when you are deploying the application and then copied to the java/lib directory.
27
-
You are not required to use this mechanism - you can manually copy the jar files if you prefer - but if you use the dependency mechanism, the LangStream CLI will check the sha512sum of the files to make sure that they are not corrupted.
26
+
The jar files are downloaded by the LangStream CLI when you are deploying the application and then copied to the java/lib directory. You are not required to use this mechanism - you can manually copy the jar files if you prefer - but if you use the dependency mechanism, the LangStream CLI will check the sha512sum of the files to make sure that they are not corrupted.
28
27
29
28
We recommend adding a .gitignore file into your application so you don't commit the jar file into your git repository.
30
29
31
-
32
30
### Apache Camel Sources
33
31
34
32
Once you have your connector deployed into your application, you can use it in your pipeline.
@@ -60,6 +58,4 @@ pipeline:
60
58
61
59
You must provide the component-uri and the component-options in the "configuration" section.
62
60
63
-
All component-options are passed to the component as additional parameters in the query string, appended to the component-uri.
64
-
This mechanism helps you in defining each property in a separate secret, so that you can easily rotate the secrets without changing the pipeline configuration.
65
-
Additionally, the camel-source agent will automatically URL-encode the values of the parameters passed to the query string.
61
+
All component-options are passed to the component as additional parameters in the query string, appended to the component-uri. This mechanism helps you in defining each property in a separate secret, so that you can easily rotate the secrets without changing the pipeline configuration. Additionally, the camel-source agent will automatically URL-encode the values of the parameters passed to the query string.
Copy file name to clipboardexpand all lines: configuration-resources/connectors/kafka-connect.md
+7-13
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
-
# Using Apache Kafka Connect connectors
1
+
# Apache Kafka Connect
2
2
3
3
Configuration for LangStream Sources and Sinks using Kafka Connect.
4
4
5
-
### Deplying the connector into your LangStream application
5
+
### Deploying the connector into your LangStream application
6
6
7
7
LangStream doesn't bundle all the Kafka Connect connectors, but you can easily deploy them into your application.
8
8
@@ -17,12 +17,10 @@ configuration:
17
17
type: "java-library"
18
18
```
19
19
20
-
The jar file is downloaded by the LangStream CLI when you are deploying the application and it is copied to the java/lib directory.
21
-
You are not required to use this mechanism - you can manually copy the jar files if you prefer - but if you use the dependency mechanism, the LangStream CLI will check the sha512sum of the files to make sure that they are not corrupted.
20
+
The jar file is downloaded by the LangStream CLI when you are deploying the application and it is copied to the java/lib directory. You are not required to use this mechanism - you can manually copy the jar files if you prefer - but if you use the dependency mechanism, the LangStream CLI will check the sha512sum of the files to make sure that they are not corrupted.
22
21
23
22
We recommend adding a .gitignore file into your application so you don't commit the jar file into your git repository.
24
23
25
-
26
24
### Kafka Connect Sinks
27
25
28
26
Once you have your connector deployed into your application, you can use it in your pipeline.
Provide the configuration for the connector in the "configuration" section of the yaml file.
56
-
You must provide `connector.class`, `name`, `key.converter` and `value.converter`, as you would for any other Kafka Connect connector.
57
-
Check the reference documentation of the connector you are using for more details about its configuration properties.
53
+
Provide the configuration for the connector in the "configuration" section of the yaml file. You must provide `connector.class`, `name`, `key.converter` and `value.converter`, as you would for any other Kafka Connect connector. Check the reference documentation of the connector you are using for more details about its configuration properties.
58
54
59
55
### Kafka Connect Sources
60
56
61
-
Kafka Connect source connectors work the same way as sinks, but you have to provide an additional system topic that Kafka Connect uses to store the state of the source connector.
62
-
In this example the system topic is named "offset-topic".
57
+
Kafka Connect source connectors work the same way as sinks, but you have to provide an additional system topic that Kafka Connect uses to store the state of the source connector. In this example the system topic is named "offset-topic".
63
58
64
-
To make the Kafka Connect runtime happy you have to set the `cleanup.policy` config value to "compact" for the offset-topic.
65
-
And you have to configure it in the configuration of the agent with the offset.storage.topic property.
59
+
To make the Kafka Connect runtime happy you have to set the `cleanup.policy` config value to "compact" for the offset-topic. And you have to configure it in the configuration of the agent with the offset.storage.topic property.
66
60
67
61
This is not a special requirement of LangStream, please checkout the Kafka Connect documentation for more details.
68
62
@@ -87,4 +81,4 @@ This is not a special requirement of LangStream, please checkout the Kafka Conne
0 commit comments