Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local server stress tests #23701

Merged
merged 71 commits into from
Feb 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
dfce207
package scafolding
anthony-murphy Jan 28, 2025
9f4da13
a bunch of compile fixes
anthony-murphy Jan 28, 2025
81d1c22
format
anthony-murphy Jan 28, 2025
ebe34bd
format
anthony-murphy Jan 28, 2025
905f13b
it builds and runs
anthony-murphy Jan 28, 2025
a3485fc
clean up naming
anthony-murphy Jan 28, 2025
9caeda6
cleanup exports
anthony-murphy Jan 29, 2025
7a87d9d
build out the defaultStressDataObjectFactory
anthony-murphy Jan 29, 2025
f30352e
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Jan 29, 2025
71ea73c
add some workloads
anthony-murphy Jan 29, 2025
b0ece92
integrate map model
anthony-murphy Jan 29, 2025
81d8e3a
working with some hacks
anthony-murphy Jan 30, 2025
203f02b
add reconnect
anthony-murphy Jan 30, 2025
1b914de
add validate model consistency
anthony-murphy Jan 30, 2025
c6be0f9
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Jan 30, 2025
3835077
rename map types
anthony-murphy Jan 30, 2025
dd394f8
move map model to utils
anthony-murphy Jan 30, 2025
f369dfa
rename dir fuzz types
anthony-murphy Jan 30, 2025
6610ad2
export map and dir models
anthony-murphy Jan 30, 2025
e2f82b1
add comments
anthony-murphy Jan 30, 2025
162b6c8
export sequence fuzz models
anthony-murphy Jan 30, 2025
0652c31
integrate multiple dds models
anthony-murphy Jan 30, 2025
2d81fdf
break files apart
anthony-murphy Jan 30, 2025
ca6d121
move validation
anthony-murphy Jan 30, 2025
741dbf8
fix dir fuzz
anthony-murphy Jan 30, 2025
547d066
add create channel
anthony-murphy Jan 30, 2025
1e2920d
child datastores and disposal
anthony-murphy Jan 30, 2025
3a926c8
remove client
anthony-murphy Jan 30, 2025
14be150
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Jan 30, 2025
8356bab
format fixes
anthony-murphy Jan 31, 2025
4d08036
allow id generation override
anthony-murphy Jan 31, 2025
ad03d25
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Jan 31, 2025
857f458
undo id changes
anthony-murphy Jan 31, 2025
3c77a32
move to custom id tracking
anthony-murphy Jan 31, 2025
442b236
channel handles
anthony-murphy Jan 31, 2025
8579ed0
round trip handles
anthony-murphy Feb 1, 2025
f75a048
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 3, 2025
7bda962
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 3, 2025
a909d69
fix generation issues
anthony-murphy Feb 3, 2025
b75f336
register objects
anthony-murphy Feb 3, 2025
f2bbc56
move channel and datastore selection to harness
anthony-murphy Feb 3, 2025
b74b48a
small fixes
anthony-murphy Feb 3, 2025
cfc8f93
fix sequence validation
anthony-murphy Feb 3, 2025
2efdae9
fix global object registration
anthony-murphy Feb 4, 2025
d80f757
fix options
anthony-murphy Feb 4, 2025
4d456d2
clean up
anthony-murphy Feb 4, 2025
421eb69
reverts
anthony-murphy Feb 4, 2025
1cc9574
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 4, 2025
833aad6
handle closed containers
anthony-murphy Feb 4, 2025
82613c2
move from ids to tags
anthony-murphy Feb 4, 2025
d866256
add tagging to state
anthony-murphy Feb 4, 2025
80475fd
remove results
anthony-murphy Feb 4, 2025
e08c871
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 5, 2025
f6838a0
add comments
anthony-murphy Feb 5, 2025
b7f9bb8
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 5, 2025
317913b
comments and lints
anthony-murphy Feb 5, 2025
7ca572c
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 18, 2025
853e616
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 18, 2025
2376ca4
some PR feedback
anthony-murphy Feb 18, 2025
3c94a51
clean up initialization
anthony-murphy Feb 19, 2025
bd3aebb
move reusable parts to stochastic-test-utils
anthony-murphy Feb 20, 2025
a82f245
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 20, 2025
010bcd0
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 20, 2025
8b2a984
Merge branch 'main' into local-sever-stress-tests
anthony-murphy Feb 21, 2025
1aef146
Update packages/test/local-server-stress-tests/src/utils.ts
anthony-murphy Feb 21, 2025
611121e
Merge branch 'main' into local-sever-stress-tests
anthony-murphy Feb 24, 2025
c582039
fix merge
anthony-murphy Feb 24, 2025
9a9e3e1
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Feb 26, 2025
08d39d9
Update packages/test/local-server-stress-tests/package.json
anthony-murphy Feb 26, 2025
e4c1efd
Merge branch 'local-sever-stress-tests' of https://github.com/anthony…
anthony-murphy Feb 26, 2025
08aa9f0
clean up unused options and code
anthony-murphy Feb 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions packages/test/local-server-stress-tests/.eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/

module.exports = {
extends: [
require.resolve("@fluidframework/eslint-config-fluid/minimal-deprecated"),
"prettier",
],
rules: {
"import/no-nodejs-modules": "off",
"@fluid-internal/fluid/no-unchecked-record-access": "warn",
},
parserOptions: {
project: ["./src/test/tsconfig.json"],
},
};
52 changes: 52 additions & 0 deletions packages/test/local-server-stress-tests/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Compiled TypeScript and CSS
dist
lib

# Babel
public/scripts/es5

# Logs
logs
*.log

# Runtime data
pids
*.pid
*.seed

# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov

# Coverage directory used by tools like istanbul
coverage

# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
.cache-loader

# node-waf configuration
.lock-wscript

# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release

# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git-
node_modules

# Typings
typings

# Debug log from npm
npm-debug.log

# Code coverage
nyc
.nyc_output/

# Chart dependencies
**/charts/*.tgz

# Generated modules
intel_modules/
temp_modules/
12 changes: 12 additions & 0 deletions packages/test/local-server-stress-tests/.mocharc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/

"use strict";

const getFluidTestMochaConfig = require("@fluid-internal/mocha-test-setup/mocharc-common");

const packageDir = __dirname;
const config = getFluidTestMochaConfig(packageDir);
module.exports = config;
7 changes: 7 additions & 0 deletions packages/test/local-server-stress-tests/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
nyc
*.log
**/*.tsbuildinfo
src/test
dist/test
lib/test
**/_api-extractor-temp/**
21 changes: 21 additions & 0 deletions packages/test/local-server-stress-tests/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Copyright (c) Microsoft Corporation and contributors. All rights reserved.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
7 changes: 7 additions & 0 deletions packages/test/local-server-stress-tests/biome.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"$schema": "./node_modules/@biomejs/biome/configuration_schema.json",
"extends": ["../../../biome.jsonc"],
"files": {
"ignore": ["results/**"]
}
}
120 changes: 120 additions & 0 deletions packages/test/local-server-stress-tests/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
{
"name": "@fluid-internal/local-server-stress-tests",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package seems like a better fit for @fluid-private

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought about this, and will change it, but i also wasn't sure if we could run fluid private tests in our long running out stress runs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't expect issues with that

"version": "2.23.0",
"private": true,
"description": "Stress tests that can only run against the local server",
"homepage": "https://fluidframework.com",
"repository": {
"type": "git",
"url": "https://github.com/microsoft/FluidFramework.git",
"directory": "packages/test/local-server-stress-tests"
},
"license": "MIT",
"author": "Microsoft and contributors",
"sideEffects": false,
"type": "commonjs",
"scripts": {
"build": "fluid-build . --task build",
"build:compile": "fluid-build . --task compile",
"build:test": "tsc --project ./src/test/tsconfig.json",
"check:biome": "biome check .",
"check:format": "npm run check:biome",
"check:prettier": "prettier --check . --cache --ignore-path ../../../.prettierignore",
"clean": "rimraf --glob dist lib \"**/*.tsbuildinfo\" \"**/*.build.log\" nyc",
"eslint": "eslint --format stylish src",
"eslint:fix": "eslint --format stylish src --fix --fix-type problem,suggestion,layout",
"format": "npm run format:biome",
"format:biome": "biome check . --write",
"format:prettier": "prettier --write . --cache --ignore-path ../../../.prettierignore",
"lint": "fluid-build . --task lint",
"lint:fix": "fluid-build . --task eslint:fix --task format",
"test": "npm run test:mocha",
"test:coverage": "c8 npm test",
"test:mocha": "mocha \"lib/test/**/*.spec.*js\" --exit",
"test:mocha:verbose": "cross-env FLUID_TEST_VERBOSE=1 npm run test:mocha"
},
"c8": {
"all": true,
"cache-dir": "nyc/.cache",
"exclude": [
"src/test/**/*.*ts",
"dist/test/**/*.*js",
"lib/test/**/*.*js"
],
"exclude-after-remap": false,
"include": [
"src/**/*.*ts",
"dist/**/*.*js",
"lib/**/*.*js"
],
"report-dir": "nyc/report",
"reporter": [
"cobertura",
"html",
"text"
],
"temp-directory": "nyc/.nyc_output"
},
"dependencies": {
"@fluid-experimental/tree": "workspace:~",
"@fluid-internal/client-utils": "workspace:~",
"@fluid-internal/mocha-test-setup": "workspace:~",
"@fluid-private/stochastic-test-utils": "workspace:~",
"@fluid-private/test-dds-utils": "workspace:~",
"@fluidframework/aqueduct": "workspace:~",
"@fluidframework/build-common": "^2.0.3",
"@fluidframework/build-tools": "^0.53.0",
"@fluidframework/container-definitions": "workspace:~",
"@fluidframework/container-loader": "workspace:~",
"@fluidframework/container-runtime": "workspace:~",
"@fluidframework/container-runtime-definitions": "workspace:~",
"@fluidframework/core-interfaces": "workspace:~",
"@fluidframework/core-utils": "workspace:~",
"@fluidframework/datastore": "workspace:~",
"@fluidframework/datastore-definitions": "workspace:~",
"@fluidframework/driver-definitions": "workspace:~",
"@fluidframework/driver-utils": "workspace:~",
"@fluidframework/eslint-config-fluid": "^5.7.3",
"@fluidframework/id-compressor": "workspace:~",
"@fluidframework/local-driver": "workspace:~",
"@fluidframework/map": "workspace:~",
"@fluidframework/runtime-definitions": "workspace:~",
"@fluidframework/runtime-utils": "workspace:~",
"@fluidframework/sequence": "workspace:~",
"@fluidframework/server-local-server": "^5.0.0",
"@fluidframework/telemetry-utils": "workspace:~",
"@fluidframework/test-utils": "workspace:~",
"@fluidframework/tree": "workspace:~",
"uuid": "^9.0.0"
},
"devDependencies": {
"@biomejs/biome": "~1.9.3",
"@types/mocha": "^10.0.10",
"@types/node": "^18.19.0",
"@types/uuid": "^9.0.2",
"c8": "^8.0.1",
"cross-env": "^7.0.3",
"eslint": "~8.55.0",
"mocha": "^10.2.0",
"mocha-multi-reporters": "^1.5.1",
"prettier": "~3.0.3",
"rimraf": "^4.4.0",
"ts-loader": "^9.5.1",
"typescript": "~5.4.5"
},
"fluidBuild": {
"tasks": {
"build:test": [
"^tsc",
"^api-extractor:commonjs",
"@fluidframework/sequence#build:test",
"@fluidframework/map#build:test"
]
}
},
"typeValidation": {
"disabled": true,
"broken": {},
"entrypoint": "internal"
}
}
8 changes: 8 additions & 0 deletions packages/test/local-server-stress-tests/prettier.config.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/

module.exports = {
...require("@fluidframework/build-common/prettier.config.cjs"),
};
68 changes: 68 additions & 0 deletions packages/test/local-server-stress-tests/src/ddsModels.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/

import { done, type AsyncGenerator } from "@fluid-private/stochastic-test-utils";
import { DDSFuzzModel, DDSFuzzTestState } from "@fluid-private/test-dds-utils";
import type { IChannelFactory } from "@fluidframework/datastore-definitions/internal";
// eslint-disable-next-line import/no-internal-modules
import { baseMapModel, baseDirModel } from "@fluidframework/map/internal/test";
import {
baseSharedStringModel,
baseIntervalModel,
// eslint-disable-next-line import/no-internal-modules
} from "@fluidframework/sequence/internal/test";

function repeatFactoryAsync<T, TState = void>(
factory: () => AsyncGenerator<T, TState>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL we have our own non-standard version of AsyncGenerator that isn't even iterable :( Is there a world where we can get away from this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I've lamented this choice I made for a while now. We don't expose any of this publicly anywhere, so we can change it whenever we want with some refactoring. The one thing to emphasize is that the programming model for fuzz tests gets a little funky: when writing generators, one generally would need to capture the state of the underlying model somehow, and that state can change 'from beneath you' whenever you yield. This is pretty natural for async generators (the exact same concept is generally applicable to async code), but weirder for synchronous ones. As long as that's emphasized though in the appropriate places (not too many), a lot of things get more natural.

I've been meaning to try hacking away at this in an FHL sometime. Maybe the upcoming one :)

): AsyncGenerator<T, TState> {
let generator = factory();
return async (state: TState) => {
const next = await generator(state);
if (next !== done) {
return next;
}
generator = factory();
return generator(state);
};
}

const generateSubModelMap = (
...models: Omit<DDSFuzzModel<IChannelFactory, any>, "workloadName">[]
) => {
const modelMap = new Map<
string,
{
factory: IChannelFactory;
generator: AsyncGenerator<any, DDSFuzzTestState<IChannelFactory>>;
reducer: DDSFuzzModel<IChannelFactory, any>["reducer"];
validateConsistency: DDSFuzzModel<IChannelFactory, any>["validateConsistency"];
minimizationTransforms?: DDSFuzzModel<IChannelFactory, any>["minimizationTransforms"];
}
>();
for (const model of models) {
const { reducer, generatorFactory, factory, validateConsistency, minimizationTransforms } =
model;
const generator = repeatFactoryAsync(generatorFactory);
modelMap.set(factory.attributes.type, {
generator,
reducer,
factory,
validateConsistency,
minimizationTransforms,
});
}

return modelMap;
};

/**
* here we import the dds models, and do some minor changes to make this easier to nest in the local server stress model.
*/
export const ddsModelMap = generateSubModelMap(
baseMapModel,
baseDirModel,
baseSharedStringModel,
baseIntervalModel,
);
Loading
Loading