You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|[v1.14](https://github.com/dapr/docs) (primary) |https://docs.dapr.io| Latest Dapr release documentation. Typo fixes, clarifications, and most documentation goes here. |
20
-
|[v1.15](https://github.com/dapr/docs/tree/v1.15) (pre-release) |https://v1-15.docs.dapr.io/| Pre-release documentation. Doc updates that are only applicable to v1.15+ go here. |
19
+
|[v1.15](https://github.com/dapr/docs) (primary) |https://docs.dapr.io| Latest Dapr release documentation. Typo fixes, clarifications, and most documentation goes here. |
20
+
|[v1.16](https://github.com/dapr/docs/tree/v1.16) (pre-release) |https://v1-16.docs.dapr.io/| Pre-release documentation. Doc updates that are only applicable to v1.15+ go here. |
21
21
22
22
For more information visit the [Dapr branch structure](https://docs.dapr.io/contributing/docs-contrib/contributing-docs/#branch-guidance) document.
Copy file name to clipboardexpand all lines: daprdocs/content/en/concepts/building-blocks-concept.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ Dapr provides the following building blocks:
22
22
|----------------|----------|-------------|
23
23
| [**Service-to-service invocation**]({{< ref "service-invocation-overview.md" >}}) | `/v1.0/invoke` | Service invocation enables applications to communicate with each other through well-known endpoints in the form of http or gRPC messages. Dapr provides an endpoint that acts as a combination of a reverse proxy with built-in service discovery, while leveraging built-in distributed tracing and error handling.
24
24
| [**Publish and subscribe**]({{< ref "pubsub-overview.md" >}}) | `/v1.0/publish``/v1.0/subscribe`| Pub/Sub is a loosely coupled messaging pattern where senders (or publishers) publish messages to a topic, to which subscribers subscribe. Dapr supports the pub/sub pattern between applications.
25
-
| [**Workflows**]({{< ref "workflow-overview.md" >}}) | `/v1.0/workflow` | The Workflow API enables you to define long running, persistent processes or data flows that span multiple microservices using Dapr workflows or workflow components. The Workflow API can be combined with other Dapr API building blocks. For example, a workflow can call another service with service invocation or retrieve secrets, providing flexibility and portability.
25
+
| [**Workflows**]({{< ref "workflow-overview.md" >}}) | `/v1.0/workflow` | The Workflow API enables you to define long running, persistent processes or data flows that span multiple microservices using Dapr workflows. The Workflow API can be combined with other Dapr API building blocks. For example, a workflow can call another service with service invocation or retrieve secrets, providing flexibility and portability.
26
26
| [**State management**]({{< ref "state-management-overview.md" >}}) | `/v1.0/state` | Application state is anything an application wants to preserve beyond a single session. Dapr provides a key/value-based state and query APIs with pluggable state stores for persistence.
27
27
| [**Bindings**]({{< ref "bindings-overview.md" >}}) | `/v1.0/bindings` | A binding provides a bi-directional connection to an external cloud/on-premise service or system. Dapr allows you to invoke the external service through the Dapr binding API, and it allows your application to be triggered by events sent by the connected service.
28
28
| [**Actors**]({{< ref "actors-overview.md" >}}) | `/v1.0/actors` | An actor is an isolated, independent unit of compute and state with single-threaded execution. Dapr provides an actor implementation based on the virtual actor pattern which provides a single-threaded programming model and where actors are garbage collected when not in use.
A [workflow]({{< ref workflow-overview.md >}}) is custom application logic that defines a reliable business process or data flow. Workflow components are workflow runtimes (or engines) that run the business logic written for that workflow and store their state into a state store.
State store components are data stores (databases, files, memory) that store key-value pairs as part of the [state management]({{< ref "state-management-overview.md" >}}) building block.
Copy file name to clipboardexpand all lines: daprdocs/content/en/concepts/dapr-services/scheduler.md
+114-6
Original file line number
Diff line number
Diff line change
@@ -5,28 +5,136 @@ linkTitle: "Scheduler"
5
5
description: "Overview of the Dapr scheduler service"
6
6
---
7
7
8
-
The Dapr Scheduler service is used to schedule jobs, running in [self-hosted mode]({{< ref self-hosted >}}) or on [Kubernetes]({{< ref kubernetes >}}).
8
+
The Dapr Scheduler service is used to schedule different types of jobs, running in [self-hosted mode]({{< ref self-hosted >}}) or on [Kubernetes]({{< ref kubernetes >}}).
9
+
- Jobs created through the Jobs API
10
+
- Actor reminder jobs (used by the actor reminders)
11
+
- Actor reminder jobs created by the Workflow API (which uses actor reminders)
9
12
10
-
The diagram below shows how the Scheduler service is used via the jobs API when called from your application. All the jobs that are tracked by the Scheduler service are stored in an embedded Etcd database.
13
+
From Dapr v1.15, the Scheduler service is used by default to schedule actor reminders as well as actor reminders for the Workflow API.
14
+
15
+
There is no concept of a leader Scheduler instance. All Scheduler service replicas are considered peers. All receive jobs to be scheduled for execution and the jobs are allocated between the available Scheduler service replicas for load balancing of the trigger events.
16
+
17
+
The diagram below shows how the Scheduler service is used via the jobs API when called from your application. All the jobs that are tracked by the Scheduler service are stored in an embedded etcd database.
11
18
12
19
<imgsrc="/images/scheduler/scheduler-architecture.png"alt="Diagram showing the Scheduler control plane service and the jobs API">
13
20
14
-
## Actor reminders
21
+
## Actor Reminders
15
22
16
23
Prior to Dapr v1.15, [actor reminders]({{< ref "actors-timers-reminders.md#actor-reminders" >}}) were run using the Placement service. Now, by default, the [`SchedulerReminders` feature flag]({{< ref "support-preview-features.md#current-preview-features" >}}) is set to `true`, and all new actor reminders you create are run using the Scheduler service to make them more scalable.
17
24
18
-
When you deploy Dapr v1.15, any _existing_ actor reminders are migrated from the Placement service to the Scheduler service as a one time operation for each actor type. You can prevent this migration by setting the `SchedulerReminders` flag to `false` in application configuration file for the actor type.
25
+
When you deploy Dapr v1.15, any _existing_ actor reminders are automatically migrated from the Actor State Store to the Scheduler service as a one time operation for each actor type. Each replica will only migrate the reminders whose actor type and id are associated with that host. This means that only when all replicas implementing an actor type are upgraded to 1.15, will all the reminders associated with that type be migrated. There will be _no_ loss of reminder triggers during the migration. However, you can prevent this migration and keep the existing actor reminders running using the Actor State Store by setting the `SchedulerReminders` flag to `false` in the application configuration file for the actor type.
26
+
27
+
To confirm that the migration was successful, check the Dapr sidecar logs for the following:
28
+
29
+
```sh
30
+
Running actor reminder migration from state store to scheduler
31
+
```
32
+
coupled with
33
+
```sh
34
+
Migrated X reminders from state store to scheduler successfully
35
+
```
36
+
or
37
+
```sh
38
+
Skipping migration, no missing scheduler reminders found
39
+
```
40
+
41
+
## Job Locality
42
+
43
+
### Default Job Behavior
44
+
45
+
By default, when the Scheduler service triggers jobs, they are sent back to a single replica for the same app ID that scheduled the job in a randomly load balanced manner. This provides basic load balancing across your application's replicas, which is suitable for most use cases where strict locality isn't required.
46
+
47
+
### Using Actor Reminders for Perfect Locality
48
+
49
+
For users who require perfect job locality (having jobs triggered on the exact same host that created them), actor reminders provide a solution. To enforce perfect locality for a job:
50
+
51
+
1. Create an actor type with a random UUID that is unique to the specific replica
52
+
2. Use this actor type to create an actor reminder
53
+
54
+
This approach ensures that the job will always be triggered on the same host which created it, rather than being randomly distributed among replicas.
55
+
56
+
## Job Triggering
57
+
58
+
### Job Failure Policy and Staging Queue
59
+
60
+
When the Scheduler service triggers a job and it has a client side error, the job is retried by default with a 1s interval and 3 maximum retries.
61
+
62
+
For non-client side errors, for example, when a job cannot be sent to an available Dapr sidecar at trigger time, it is placed in a staging queue within the Scheduler service. Jobs remain in this queue until a suitable sidecar instance becomes available, at which point they are automatically sent to the appropriate Dapr sidecar instance.
19
63
20
64
## Self-hosted mode
21
65
22
66
The Scheduler service Docker container is started automatically as part of `dapr init`. It can also be run manually as a process if you are running in [slim-init mode]({{< ref self-hosted-no-docker.md >}}).
23
67
68
+
The Scheduler can be run in both high availability (HA) and non-HA modes in self-hosted deployments. However, non-HA mode is not recommended for production use. If switching between non-HA and HA modes, the existing data directory must be removed, which results in loss of jobs and actor reminders. [Run a back-up]({{< ref "#back-up-and-restore-scheduler-data" >}}) before making this change to avoid losing data.
69
+
24
70
## Kubernetes mode
25
71
26
-
The Scheduler service is deployed as part of `dapr init -k`, or via the Dapr Helm charts. You can run Scheduler in high availability (HA) mode. [Learn more about setting HA mode in your Kubernetes service.]({{< ref "kubernetes-production.md#individual-service-ha-helm-configuration" >}})
72
+
The Scheduler service is deployed as part of `dapr init -k`, or via the Dapr Helm charts. Scheduler always runs in high availability (HA) mode in Kubernetes deployments. Scaling the Scheduler service replicas up or down is not possible without incurring data loss due to the nature of the embedded data store. [Learn more about setting HA mode in your Kubernetes service.]({{< ref "kubernetes-production.md#individual-service-ha-helm-configuration" >}})
73
+
74
+
When a Kubernetes namespace is deleted, all the Job and Actor Reminders corresponding to that namespace are deleted.
75
+
76
+
## Back Up and Restore Scheduler Data
77
+
78
+
In production environments, it's recommended to perform periodic backups of this data at an interval that aligns with your recovery point objectives.
79
+
80
+
### Port Forward for Backup Operations
81
+
82
+
To perform backup and restore operations, you'll need to access the embedded etcd instance. This requires port forwarding to expose the etcd ports (port 2379).
83
+
84
+
#### Kubernetes Example
85
+
86
+
Here's how to port forward and connect to the etcd instance:
When running in HA mode, you only need to expose the ports for one scheduler instance to perform backup operations.
117
+
118
+
### Performing Backup and Restore
119
+
120
+
Once you have access to the etcd ports, you can follow the [official etcd backup and restore documentation](https://etcd.io/docs/v3.5/op-guide/recovery/) to perform backup and restore operations. The process involves using standard etcd commands to create snapshots and restore from them.
121
+
122
+
## Monitoring Scheduler's etcd Metrics
123
+
124
+
Port forward the Scheduler instance and view etcd's metrics with the following:
125
+
126
+
```shell
127
+
curl -s http://localhost:2379/metrics
128
+
```
129
+
130
+
Fine tune the embedded etcd to your needs by [reviewing and configuring the Scheduler's etcd flags as needed](https://github.com/dapr/dapr/blob/master/charts/dapr/README.md#dapr-scheduler-options).
131
+
132
+
## Disabling the Scheduler service
133
+
134
+
If you are not using any features that require the Scheduler service (Jobs API, Actor Reminders, or Workflows), you can disable it by setting `global.scheduler.enabled=false`.
27
135
28
136
For more information on running Dapr on Kubernetes, visit the [Kubernetes hosting page]({{< ref kubernetes >}}).
29
137
30
138
## Related links
31
139
32
-
[Learn more about the Jobs API.]({{< ref jobs_api.md >}})
140
+
[Learn more about the Jobs API.]({{< ref jobs_api.md >}})
Copy file name to clipboardexpand all lines: daprdocs/content/en/concepts/faq/faq.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -27,11 +27,11 @@ Creating a new actor follows a local call like `http://localhost:3500/v1.0/actor
27
27
28
28
The Dapr runtime SDKs have language-specific actor frameworks. For example, the .NET SDK has C# actors. The goal is for all the Dapr language SDKs to have an actor framework. Currently .NET, Java, Go and Python SDK have actor frameworks.
29
29
30
-
###Does Dapr have any SDKs I can use if I want to work with a particular programming language or framework?
30
+
## Does Dapr have any SDKs I can use if I want to work with a particular programming language or framework?
31
31
32
32
To make using Dapr more natural for different languages, it includes [language specific SDKs]({{<refsdks>}}) for Go, Java, JavaScript, .NET, Python, PHP, Rust and C++. These SDKs expose the functionality in the Dapr building blocks, such as saving state, publishing an event or creating an actor, through a typed language API rather than calling the http/gRPC API. This enables you to write a combination of stateless and stateful functions and actors all in the language of your choice. And because these SDKs share the Dapr runtime, you get cross-language actor and functions support.
33
33
34
-
###What frameworks does Dapr integrate with?
34
+
## What frameworks does Dapr integrate with?
35
35
Dapr can be integrated with any developer framework. For example, in the Dapr .NET SDK you can find ASP.NET Core integration, which brings stateful routing controllers that respond to pub/sub events from other services.
Copy file name to clipboardexpand all lines: daprdocs/content/en/concepts/overview.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -46,7 +46,7 @@ Each of these building block APIs is independent, meaning that you can use any n
46
46
|----------------|-------------|
47
47
| [**Service-to-service invocation**]({{< ref "service-invocation-overview.md" >}}) | Resilient service-to-service invocation enables method calls, including retries, on remote services, wherever they are located in the supported hosting environment.
48
48
| [**Publish and subscribe**]({{< ref "pubsub-overview.md" >}}) | Publishing events and subscribing to topics between services enables event-driven architectures to simplify horizontal scalability and make them resilient to failure. Dapr provides at-least-once message delivery guarantee, message TTL, consumer groups and other advance features.
49
-
| [**Workflows**]({{< ref "workflow-overview.md" >}}) | The workflow API can be combined with other Dapr building blocks to define long running, persistent processes or data flows that span multiple microservices using Dapr workflows or workflow components.
49
+
| [**Workflows**]({{< ref "workflow-overview.md" >}}) | The workflow API can be combined with other Dapr building blocks to define long running, persistent processes or data flows that span multiple microservices using Dapr workflows.
50
50
| [**State management**]({{< ref "state-management-overview.md" >}}) | With state management for storing and querying key/value pairs, long-running, highly available, stateful services can be easily written alongside stateless services in your application. The state store is pluggable and examples include AWS DynamoDB, Azure Cosmos DB, Azure SQL Server, GCP Firebase, PostgreSQL or Redis, among others.
51
51
| [**Resource bindings**]({{< ref "bindings-overview.md" >}}) | Resource bindings with triggers builds further on event-driven architectures for scale and resiliency by receiving and sending events to and from any external source such as databases, queues, file systems, etc.
52
52
| [**Actors**]({{< ref "actors-overview.md" >}}) | A pattern for stateful and stateless objects that makes concurrency simple, with method and state encapsulation. Dapr provides many capabilities in its actor runtime, including concurrency, state, and life-cycle management for actor activation/deactivation, and timers and reminders to wake up actors.
@@ -76,7 +76,7 @@ Dapr exposes its HTTP and gRPC APIs as a sidecar architecture, either as a conta
76
76
## Hosting environments
77
77
78
78
Dapr can be hosted in multiple environments, including:
79
-
- Self-hosted on a Windows/Linux/macOS machine for local development
79
+
- Self-hosted on a Windows/Linux/macOS machine for local development and in production
80
80
- On Kubernetes or clusters of physical or virtual machines in production
0 commit comments