Skip to content

Commit 3a5cc90

Browse files
nicoloboschimendonk
andauthoredNov 2, 2023
Document HTTP gateway (#131)
Co-authored-by: Mendon Kissling <[email protected]>
1 parent f0dffb9 commit 3a5cc90

File tree

5 files changed

+143
-15
lines changed

5 files changed

+143
-15
lines changed
 

‎SUMMARY.md

+1
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
* [Expression Language](building-applications/expression-language.md)
4343
* [API Gateways](building-applications/api-gateways/README.md)
4444
* [Websocket](building-applications/api-gateways/websocket.md)
45+
* [HTTP](building-applications/api-gateways/http.md)
4546
* [Message filtering](building-applications/api-gateways/message-filtering.md)
4647
* [Gateway authentication](building-applications/api-gateways/gateway-authentication.md)
4748
* [API Reference](building-applications/api-reference/README.md)

‎building-applications/api-gateways/README.md

+21-6
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,20 @@
11
# API Gateways
22

3-
API gateways in LangStream are WebSocket endpoints that allow applications to interact with agents via message topics.
3+
API gateways in LangStream are HTTP/WebSocket endpoints that allow applications to interact with agents via message topics.
44

55
Messages contain a key, a value, and headers for metadata. This metadata is used to connect clients to topics and can be used to [filter messages.](message-filtering.md)
66

77
LangStream applications can have three different gateway types:
88

99
**Producer** gateways are clients that create a message and write it to a topic.&#x20;
1010

11-
**Consumer** gateways are clients that watch a topic for new messages.&#x20;
11+
**Consumer** gateways are clients that watch a topic for messages from a given position. This gateway can only be accessed via WebSocket. &#x20;
1212

13-
**Chat** gateways create a producer client ("questions-topic") and a consumer client ("answers-topic") across one connection, optimized for "chat" style interactions between client and server.
13+
**Chat** gateways create a producer client (`questions-topic`) and a consumer client (`answers-topic`) across one connection, optimized for "chat" style interactions between client and server. This gateway can only be accessed via WebSocket.&#x20;
14+
15+
**Service (topics)** gateways create a producer client (`input-topic`) and a consumer client (`output-topic`), writing a single message and awaiting for the receivement of another message. This gateway can only be accessed via HTTP.&#x20;
16+
17+
**Service (agents)** gateways send requests directly to a `service` agent and return the response to the client. This gateway can only be accessed via HTTP.
1418

1519
### Example gateways
1620

@@ -29,8 +33,14 @@ gateways:
2933
- id: "chat"
3034
type: chat
3135
chat-options:
32-
answers-topic: output-topic
33-
questions-topic: input-topic
36+
questions-topic: questions-topic
37+
answers-topic: answers-topic
38+
39+
- id: "service"
40+
type: service
41+
service-options:
42+
input-topic: questions-topic
43+
output-topic: answers-topic
3444
```
3545
3646
The corresponding URLs to the gateways would be:
@@ -41,7 +51,12 @@ The corresponding URLs to the gateways would be:
4151
4252
**Chat gateway:** ws://localhost:8091/consume/my-super-cool-tenant/some-application/chat
4353
44-
The URL structure is ws://\<control-plane-domain>:\<api-gateway-port>/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id>
54+
**Service gateway:** http://localhost:8091/api/gateways/service/my-super-cool-tenant/some-application/service
55+
56+
The URL structure is ws://\<control-plane-domain>:\<api-gateway-port>/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id> for WebSocket.
57+
58+
The URL structure is http://\<control-plane-domain>:\<api-gateway-port>/api/gateways/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id> for HTTP.
59+
4560
4661
### Gateways configuration
4762

‎building-applications/api-gateways/gateway-authentication.md

+8-8
Original file line numberDiff line numberDiff line change
@@ -36,12 +36,12 @@ gateways:
3636
provider: google
3737
configuration:
3838
clientId: "{{ secrets.google.client-id }}"
39-
produceOptions:
39+
produce-options:
4040
headers:
4141
- key: langstream-client-user-id
42-
valueFromAuthentication: subject
42+
value-from-authentication: subject
4343
- key: langstream-client-session-id
44-
valueFromParameters: sessionId
44+
value-from-parameters: sessionId
4545

4646
- id: "bot-output"
4747
type: consume
@@ -52,17 +52,17 @@ gateways:
5252
provider: google
5353
configuration:
5454
clientId: "{{ secrets.google.client-id }}"
55-
produceOptions:
55+
produce-options:
5656
headers:
5757
- key: langstream-client-user-id
58-
valueFromAuthentication: subject
58+
value-from-authentication: subject
5959
- key: langstream-client-session-id
60-
valueFromParameters: sessionId
60+
value-from-parameters: sessionId
6161
```
6262
6363
#### Google authentication
6464
65-
Set `provider:google` to use a Google client ID to authenticate LangStream gateway connections.
65+
Set `provider: google` to use a Google client ID to authenticate LangStream gateway connections.
6666

6767
The Google field that is exposed for authentication is "subject".
6868

@@ -82,7 +82,7 @@ export GOOGLE_CLIENT_ID=99840107278-4363876v0hker43roujaubqom5g07or8.apps.google
8282
8383
#### Github authentication
8484
85-
Set `provider:github` to use a GitHub login to authenticate LangStream gateway connections.
85+
Set `provider: github` to use a GitHub login to authenticate LangStream gateway connections.
8686
8787
The Github field that is exposed for authentication is "login".
8888
+112
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# HTTP
2+
3+
You connect to a gateway via standard HTTP 1.1. While connecting to a gateway, the path holds pointers to the type of gateway, the tenant, the application id, and the gateway name.
4+
5+
The URL structure is http://\<control-plane-domain>:\<api-gateway-port>/api/gateways/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id>.
6+
7+
Authentication (`credentials,test-credentials`), options (`option:`) and parameters (`param:`) are expected to be in the query string. HTTP headers are ignored by the API Gateway service.
8+
9+
10+
## Produce messages
11+
12+
To produce messages via HTTP, configure a `produce` gateway:
13+
```yaml
14+
15+
gateways:
16+
- id: "user-input"
17+
type: produce
18+
topic: "questions-topic"
19+
parameters:
20+
- sessionId
21+
```
22+
23+
Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
24+
25+
```bash
26+
curl -X POST --body '{"value": "hello"}' "http://localhost:8091/api/gateways/produce/my-tenant/my-app/user-input?param:sessionId=12543yusi1"
27+
```
28+
29+
You can also use the LangStream CLI:
30+
31+
```bash
32+
langstream gateway produce my-app user-input --protocol http -v '{"value": "hello"}' -p sessionId=12543yusi1
33+
```
34+
35+
36+
## Produce messages and wait for a message
37+
38+
To produce messages via HTTP and wait for a message, configure a `service` gateway:
39+
```yaml
40+
41+
gateways:
42+
- id: "user-input-await"
43+
type: service
44+
parameters:
45+
- sessionId
46+
service-options:
47+
input-topic: inputs
48+
output-topic: results
49+
```
50+
51+
Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
52+
53+
```bash
54+
curl -X POST --body '{"value": "hello"}' "http://localhost:8091/api/gateways/service/my-tenant/my-app/user-input-await"
55+
```
56+
57+
The timeout of the wait is the TCP timeout of the connection, which is usually 30 seconds (may vary depending on the http client).
58+
59+
## Proxy service agent requests
60+
61+
To proxy requests to a specific `service` agent via HTTP, configure a `service` gateway:
62+
63+
```yaml
64+
gateways:
65+
- id: "my-service"
66+
type: service
67+
parameters:
68+
- sessionId
69+
service-options:
70+
agent-id: my-service-agent
71+
72+
```
73+
74+
A service agent might look like this in the pipeline configuration:
75+
```yaml
76+
pipeline:
77+
- name: "Start my service"
78+
id: my-service-agent
79+
type: "python-service"
80+
configuration:
81+
className: example.ChatBotService
82+
```
83+
84+
85+
Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
86+
87+
```bash
88+
curl -X POST \
89+
--body '{"value": "hello"}' \
90+
--H 'Authorization: Bearer XXX' \
91+
"http://localhost:8091/api/gateways/service/my-tenant/my-app/my-service/the/custom/path?service-param-1=yes"
92+
```
93+
94+
The final part of the URL, query string, HTTP method, and headers will be sent to the service agent.
95+
96+
97+
In the above case, the agent service will receive an equivalent request of:
98+
99+
```
100+
curl -X POST \
101+
--body '{"value": "hello"}' \
102+
--H 'Authorization: Bearer XXX' \
103+
"http://localhost:8000/the/custom/path?service-param-1=yes"
104+
```
105+
106+
107+
The `credentials`, `test-credentials`, `option:xx`, `param:xx` are stripped out from the routed requests.
108+
If the gateway has authentication enabled, it will be performed as for other gateways.
109+
110+
POST, GET, DELETE and PUT are all supported.
111+
112+
Leveraging the API gateway to expose your service solves authentication, HTTPS, high-availability, and scalability out of the box.

‎building-applications/api-gateways/message-filtering.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ Chat gateways behave a little differently, because they open producers and consu
5555
5656
### headers
5757
58-
<table><thead><tr><th width="244">Label</th><th width="113.33333333333331">Type</th><th>Description</th></tr></thead><tbody><tr><td>key</td><td>string</td><td>An identifier of the value. This can be thought of as the name. Allowed characters are a-zA-Z0-9._-</td></tr><tr><td>valueFromParameters</td><td>string</td><td>The mapped name to connect a querystring value in the Gateway URL to message headers. This must match a value set in the parameters collection.</td></tr><tr><td>valueFromAuthentication</td><td>string</td><td>The mapped name to connect a provided querysting value with the identity provider’s handshake value. See <a href="gateway-authentication.md">Gateway authentication.</a></td></tr></tbody></table>
58+
<table><thead><tr><th width="244">Label</th><th width="113.33333333333331">Type</th><th>Description</th></tr></thead><tbody><tr><td>key</td><td>string</td><td>An identifier of the value. This can be thought of as the name. Allowed characters are a-zA-Z0-9._-</td></tr><tr><td>value-from-parameters</td><td>string</td><td>The mapped name to connect a querystring value in the Gateway URL to message headers. This must match a value set in the parameters collection.</td></tr><tr><td>value-from-authentication</td><td>string</td><td>The mapped name to connect a provided querystring value with the identity provider’s handshake value. See <a href="gateway-authentication.md">Gateway authentication.</a></td></tr></tbody></table>
5959
6060
### Message offset
6161

0 commit comments

Comments
 (0)
Please sign in to comment.