Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support concurrency limit #542

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 35 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ online example: https://upload.react-component.vercel.app/

## Feature

* support IE11+, Chrome, Firefox, Safari
- support IE11+, Chrome, Firefox, Safari

## install

Expand All @@ -54,29 +54,30 @@ React.render(<Upload />, container);

### props

|name|type|default| description|
|-----|---|--------|----|
|name | string | file| file param post to server |
|style | object | {}| root component inline style |
|className | string | - | root component className |
|disabled | boolean | false | whether disabled |
|component | "div"|"span" | "span"| wrap component name |
|action| string &#124; function(file): string &#124; Promise&lt;string&gt; | | form action url |
|method | string | post | request method |
|directory| boolean | false | support upload whole directory |
|data| object/function(file) | | other data object to post or a function which returns a data object(a promise object which resolve a data object) |
|headers| object | {} | http headers to post, available in modern browsers |
|accept | string | | input accept attribute |
|capture | string | | input capture attribute |
|multiple | boolean | false | only support ie10+|
|onStart | function| | start upload file |
|onError| function| | error callback |
|onSuccess | function | | success callback |
|onProgress | function || progress callback, only for modern browsers|
|beforeUpload| function |null| before upload check, return false or a rejected Promise will stop upload, only for modern browsers|
|customRequest | function | null | provide an override for the default xhr behavior for additional customization|
|withCredentials | boolean | false | ajax upload with cookie send |
|openFileDialogOnClick | boolean | true | useful for drag only upload as it does not trigger on enter key or click event |
| name | type | default | description |
| --- | --- | --- | --- |
| name | string | file | file param post to server |
| style | object | {} | root component inline style |
| className | string | - | root component className |
| disabled | boolean | false | whether disabled |
| component | "div" &#124; "span" | "span" | wrap component name |
| action | string &#124; function(file): string &#124; Promise&lt;string&gt; | | form action url |
| method | string | post | request method |
| directory | boolean | false | support upload whole directory |
| data | object/function(file) | | other data object to post or a function which returns a data object(a promise object which resolve a data object) |
| headers | object | {} | http headers to post, available in modern browsers |
| accept | string | | input accept attribute |
| capture | string | | input capture attribute |
| multiple | boolean | false | only support ie10+ |
| concurrencyLimit | number &#124; undefined | undefined | asynchronously posts files with the concurrency limit |
| onStart | function | | start upload file |
| onError | function | | error callback |
| onSuccess | function | | success callback |
| onProgress | function | | progress callback, only for modern browsers |
| beforeUpload | function | null | before upload check, return false or a rejected Promise will stop upload, only for modern browsers |
| customRequest | function | null | provide an override for the default xhr behavior for additional customization |
| withCredentials | boolean | false | ajax upload with cookie send |
| openFileDialogOnClick | boolean | true | useful for drag only upload as it does not trigger on enter key or click event |

#### onError arguments

Expand All @@ -88,26 +89,23 @@ React.render(<Upload />, container);

1. `result`: response body
2. `file`: upload file
3. `xhr`: xhr header, only for modern browsers which support AJAX upload. since
2.4.0

3. `xhr`: xhr header, only for modern browsers which support AJAX upload. since 2.4.0

### customRequest

Allows for advanced customization by overriding default behavior in AjaxUploader. Provide your own XMLHttpRequest calls to interface with custom backend processes or interact with AWS S3 service through the aws-sdk-js package.

customRequest callback is passed an object with:

* `onProgress: (event: { percent: number }): void`
* `onError: (event: Error, body?: Object): void`
* `onSuccess: (body: Object): void`
* `data: Object`
* `filename: String`
* `file: File`
* `withCredentials: Boolean`
* `action: String`
* `headers: Object`

- `onProgress: (event: { percent: number }): void`
- `onError: (event: Error, body?: Object): void`
- `onSuccess: (body: Object): void`
- `data: Object`
- `filename: String`
- `file: File`
- `withCredentials: Boolean`
- `action: String`
- `headers: Object`

### methods

Expand Down
22 changes: 19 additions & 3 deletions src/AjaxUploader.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import defaultRequest from './request';
import traverseFileTree from './traverseFileTree';
import getUid from './uid';
import ConcurrencyRequester from './concurrencyRequest';

interface ParsedFileInfo {
origin: RcFile;
Expand All @@ -26,6 +27,8 @@

reqs: any = {};

private concurrencyRequester?: ConcurrencyRequester<any>;

private fileInput: HTMLInputElement;

private _isMounted: boolean;
Expand Down Expand Up @@ -111,17 +114,26 @@
return this.processFile(file, originFiles);
});

const { onBatchStart, concurrencyLimit } = this.props;

if (concurrencyLimit) {
this.concurrencyRequester = new ConcurrencyRequester(concurrencyLimit);

Check warning on line 120 in src/AjaxUploader.tsx

View check run for this annotation

Codecov / codecov/patch

src/AjaxUploader.tsx#L120

Added line #L120 was not covered by tests
}

// Batch upload files
Promise.all(postFiles).then(fileList => {
const { onBatchStart } = this.props;

onBatchStart?.(fileList.map(({ origin, parsedFile }) => ({ file: origin, parsedFile })));

fileList
.filter(file => file.parsedFile !== null)
.forEach(file => {
this.post(file);
});

// Asynchronously posts files with the concurrency limit.
if (this.concurrencyRequester) {
this.concurrencyRequester.send();

Check warning on line 135 in src/AjaxUploader.tsx

View check run for this annotation

Codecov / codecov/patch

src/AjaxUploader.tsx#L135

Added line #L135 was not covered by tests
}
});
};

Expand Down Expand Up @@ -230,7 +242,11 @@
};

onStart(origin);
this.reqs[uid] = request(requestOption);
if (this.concurrencyRequester) {
this.reqs[uid] = this.concurrencyRequester.append(requestOption);

Check warning on line 246 in src/AjaxUploader.tsx

View check run for this annotation

Codecov / codecov/patch

src/AjaxUploader.tsx#L246

Added line #L246 was not covered by tests
} else {
this.reqs[uid] = request(requestOption);
}
}

reset() {
Expand Down
155 changes: 155 additions & 0 deletions src/concurrencyRequest.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
import type { ConcurrencyRequestTask, UploadRequestOption } from './interface';
import { onXHRLoad, prepareData, prepareXHR } from './request';

/**
* Asynchronously processes an array of items with a concurrency limit.
*
* @template T - Type of the input items.
* @template U - Type of the result of the asynchronous task.
*
* @param {number} concurrencyLimit - The maximum number of asynchronous tasks to execute concurrently.
* @param {T[]} items - The array of items to process asynchronously.
* @param {(item: T) => Promise<U>} asyncTask - The asynchronous task to be performed on each item.
*
* @returns {Promise<U[]>} - A promise that resolves to an array of results from the asynchronous tasks.
*/
async function asyncPool<T, U>(
concurrencyLimit: number,
items: T[],
asyncTask: (item: T) => Promise<U>,
): Promise<U[]> {
const tasks: Promise<U>[] = [];
const pendings: Promise<U>[] = [];

Check warning on line 22 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L20-L22

Added lines #L20 - L22 were not covered by tests

for (const item of items) {
const task = asyncTask(item);
tasks.push(task);

Check warning on line 26 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L24-L26

Added lines #L24 - L26 were not covered by tests

if (concurrencyLimit <= items.length) {
task.then(() => {
pendings.splice(pendings.indexOf(task), 1);

Check warning on line 30 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L29-L30

Added lines #L29 - L30 were not covered by tests
});
pendings.push(task);

Check warning on line 32 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L32

Added line #L32 was not covered by tests

if (pendings.length >= concurrencyLimit) {
await Promise.race(pendings);

Check warning on line 35 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L35

Added line #L35 was not covered by tests
}
}
}

return Promise.all(tasks);

Check warning on line 40 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L40

Added line #L40 was not covered by tests
}

type DataType = 'form' | 'blob' | 'string';

/**
* Represents a class for handling concurrent requests with a specified concurrency limit.
*
* @template T - The type of data to be uploaded.
*/
export default class ConcurrencyRequester<T> {
/**
* The concurrency limit for handling requests simultaneously.
*/
private concurrencyLimit: number;

/**
* An array to store the tasks for concurrent requests.
*/
private tasks: ConcurrencyRequestTask[] = [];

Check warning on line 59 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L59

Added line #L59 was not covered by tests

/**
* The type of data to be sent in the request ('form', 'blob', or 'string').
*/
private dataType: DataType;

/**
* Creates an instance of ConcurrencyRequester.
*
* @param {number} concurrencyLimit - The concurrency limit for handling requests simultaneously.
* @param {DataType} [dataType='form'] - The type of data to be sent in the request ('form', 'blob', or 'string').
*/
constructor(concurrencyLimit: number, dataType: DataType = 'form') {
this.concurrencyLimit = concurrencyLimit;
this.dataType = dataType;

Check warning on line 74 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L73-L74

Added lines #L73 - L74 were not covered by tests
}

/**
* Prepares data based on the specified data type.
*
* @param {UploadRequestOption<T>} option - The upload request option.
* @returns {string | Blob | FormData} - The prepared data based on the specified data type.
* @private
*/
private prepareData = (option: UploadRequestOption<T>): string | Blob | FormData => {

Check warning on line 84 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L84

Added line #L84 was not covered by tests
if (this.dataType === 'form') {
return prepareData(option);

Check warning on line 86 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L86

Added line #L86 was not covered by tests
}

return option.file;

Check warning on line 89 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L89

Added line #L89 was not covered by tests
};

/**
* Prepares a task for a concurrent request.
*
* @param {UploadRequestOption<T>} option - The upload request option.
* @returns {ConcurrencyRequestTask} - The prepared task for the concurrent request.
* @private
*/
private prepare = (option: UploadRequestOption<T>): ConcurrencyRequestTask => {
const xhr = prepareXHR(option);

Check warning on line 100 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L99-L100

Added lines #L99 - L100 were not covered by tests

const data = this.prepareData(option);

Check warning on line 102 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L102

Added line #L102 was not covered by tests

const task: ConcurrencyRequestTask = { xhr, data };

Check warning on line 104 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L104

Added line #L104 was not covered by tests

xhr.onerror = function error(e) {
task.done?.();
option.onError(e);

Check warning on line 108 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L106-L108

Added lines #L106 - L108 were not covered by tests
};

xhr.onload = function onload() {
task.done?.();

Check warning on line 112 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L111-L112

Added lines #L111 - L112 were not covered by tests

onXHRLoad(this, option);

Check warning on line 114 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L114

Added line #L114 was not covered by tests
};

return task;

Check warning on line 117 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L117

Added line #L117 was not covered by tests
};

/**
* Appends a new upload request to the tasks array.
*
* @param {UploadRequestOption<T>} option - The upload request option.
* @returns {{ abort: () => void }} - An object with an `abort` function to cancel the request.
*/
append = (option: UploadRequestOption<T>): { abort: () => void } => {
const task = this.prepare(option);

Check warning on line 127 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L126-L127

Added lines #L126 - L127 were not covered by tests

this.tasks.push(task);

Check warning on line 129 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L129

Added line #L129 was not covered by tests

return {
abort() {
task.xhr.abort();

Check warning on line 133 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L131-L133

Added lines #L131 - L133 were not covered by tests
},
};
};

/**
* Sends all the appended requests concurrently.
*/
send = (): void => {
asyncPool(

Check warning on line 142 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L141-L142

Added lines #L141 - L142 were not covered by tests
this.concurrencyLimit,
this.tasks,
item =>
new Promise<void>(resolve => {
const xhr = item.xhr;

Check warning on line 147 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L146-L147

Added lines #L146 - L147 were not covered by tests

item.done = resolve;

Check warning on line 149 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L149

Added line #L149 was not covered by tests

xhr.send(item.data);

Check warning on line 151 in src/concurrencyRequest.ts

View check run for this annotation

Codecov / codecov/patch

src/concurrencyRequest.ts#L151

Added line #L151 was not covered by tests
}),
);
};
}
7 changes: 7 additions & 0 deletions src/interface.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ export interface UploadProps
input?: React.CSSProperties;
};
hasControlInside?: boolean;
concurrencyLimit?: number;
}

export interface UploadProgressEvent extends Partial<ProgressEvent> {
Expand Down Expand Up @@ -76,3 +77,9 @@ export interface UploadRequestOption<T = any> {
export interface RcFile extends File {
uid: string;
}

export interface ConcurrencyRequestTask {
xhr: XMLHttpRequest;
data: File | FormData | string | Blob;
done?: () => void;
}
Loading
Loading