DontReview Garnitin/add gke load testing/v1 #2225

gargnitingoogle · 2024-07-26T21:27:18Z

Description

Link to the issue in case of a bug fix.

NA

Testing details

Manual - NA
Unit tests - NA
Integration tests - NA

codecov · 2024-07-26T21:32:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.52%. Comparing base (5ae03ff) to head (d2952ab).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2225      +/-   ##
==========================================
+ Coverage   77.47%   77.52%   +0.05%     
==========================================
  Files         110      110              
  Lines       15705    15705              
==========================================
+ Hits        12167    12175       +8     
+ Misses       3016     3010       -6     
+ Partials      522      520       -2

Flag	Coverage Δ
unittests	`77.52% <ø> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

gargnitingoogle · 2024-07-26T21:29:20Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+fi
+
+function validateMachineConfig() {
+  echo "Validiting input parameters ..."


gargnitingoogle · 2024-07-26T21:29:52Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  local cluster_name=${1}
+  local zone=${2}
+  local node_pool=${3}
+  if [ $(gcloud container node-pools list --cluster=${cluster_name} --zone=${zone} | grep -ow ${node_pool} | wc -l) -gt 0 ] ; then


simplify this with grep -q

gargnitingoogle · 2024-07-26T21:30:59Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  if ClusterExists ${cluster_name} ; then
+    gcloud container clusters update ${cluster_name} --location=${zone} --workload-pool=${project_id}.svc.id.goog
+  else
+    # gcloud container --project "${project_id}" clusters create ${cluster_name} --zone "${zone}" --cluster-version "${cluster_version}" --workload-pool=${project_id}.svc.id.goog


remove commented code

gargnitingoogle · 2024-07-26T21:32:10Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  echo "Enabling/disabling csi add-on ..."
+  # By default, disable the managed csi driver.
+  if ${useCustomCsiDriver}; then
+    # gcloud -q container clusters update ${cluster_name} \


re-enable this

gargnitingoogle · 2024-07-26T21:32:29Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  gcloud container clusters get-credentials ${cluster_name} --location=${zone}
+  kubectl create namespace ${appnamespace}
+  kubectl create serviceaccount ${ksa} --namespace ${appnamespace}
+  for workload_bucket in ${buckets} ; do


add code to get buckets from somewhere.

gargnitingoogle · 2024-07-26T21:35:00Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+}
+
+# validateMachineConfig ${machine_type} ${num_nodes} ${num_ssd}
+# installDependencies


re-enable all these disabled steps.

gargnitingoogle · 2024-07-26T21:36:51Z

perfmetrics/scripts/testing_on_gke/utils/utils.py

+def get_cpu(pod_name: str, start: str, end: str) -> Tuple[float, float]:
+    # for some reason, the mash filter does not always work, so we fetch all the metrics for all the pods and filter later.
+    result = subprocess.run(["mash", "--namespace=cloud_prod", "--output=csv", 
+                             f"Query(Fetch(Raw('cloud.kubernetes.K8sContainer', 'kubernetes.io/container/cpu/core_usage_time'), {{'project': '927584127901'}})| Window(Rate('10m'))| GroupBy(['pod_name', 'container_name'], Max()), TimeInterval('{start}', '{end}'), '5s')"], 


put the project-number back to the original and change it by code during runtime and then revert it back when done during runtime.

gargnitingoogle · 2024-07-26T21:37:31Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+function updateMachineTypeInPodConfigs() {
+  for file in ${gke_testing_dir}/examples/fio/loading-test/values.yaml ${gke_testing_dir}/examples/dlio/unet3d-loading-test/values.yaml ; do
+    test -f ${file}
+    sed -i -E "s/nodeType: [0-9a-z_-]+$/nodeType: ${machine_type}/g" ${file}


add code to revert this back

gargnitingoogle · 2024-07-26T21:37:42Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  for file in ${gke_testing_dir}/examples/fio/loading-test/values.yaml ${gke_testing_dir}/examples/dlio/unet3d-loading-test/values.yaml ; do
+    test -f ${file}
+    # sed -i -E "s/mountOptions: [0-9a-zA-Z,\:\"-_]+$/mountOptions: \"${gcsfuse_mount_options}\"/g" ${file}
+    sed -i -E "s/mountOptions:[ \t]*\"?[a-zA-Z0-9,:_-]+\"? *$/mountOptions: \"${gcsfuse_mount_options}\"/g" "${file}"


add code to revert this back when done

gargnitingoogle · 2024-07-27T04:27:38Z

perfmetrics/scripts/testing_on_gke/utils/utils.py

+    return utc_timestamp_string
+
+def standard_timestamp(timestamp: int) -> str:
+    return timestamp.split('.')[0].replace('T', ' ') + " UTC"


insert newline after this line

gargnitingoogle · 2024-07-27T04:28:16Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

@@ -0,0 +1,425 @@
+#!/bin/bash
+


Add a header and a help option at the top.

Utilities: 1. Append given tabular data to the given gsheet id and worksheet name. 2. Return url for a gsheet given its ID. 3. Adds unit tests for the above utilities.

add generic utility to append to a gsheet Add utility for gsheet improve gsheet utility export fio output to gsheet encapsulate cpu/memory calculation in fio disable repeat operations for quick testing add dlio output export to gsheet fix a bug in dlio output parsing fix a column-name in fio csv output Revert "disable repeat operations for quick testing" This reverts commit 04bf834. add log of successful addition to gsheet clean-up code changes added some error-checking wip wip fix calls to download_gcs_objects support key-file on gcs in gsheet put back cpu/memory metrics fix couple of logs fix couple of help messages put back accidentally deleted command

Adds row with "ERROR" for all values rather than crashing during printing in CSV file.

Purposes. * Consistent behavior across all machines * Monitoring API has faster runtime than mash. * Monitoring API is supported on GCE VM too.

gargnitingoogle commented Jul 26, 2024

View reviewed changes

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 2958a3c to ad97f82 Compare July 29, 2024 10:47

gargnitingoogle commented Jul 29, 2024

View reviewed changes

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 7 times, most recently from ff7ee55 to 478921c Compare August 6, 2024 09:53

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 9 times, most recently from 1e4353a to b6a0e76 Compare August 16, 2024 14:45

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 9 times, most recently from b62b517 to d008a84 Compare August 22, 2024 11:53

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 0daf723 to 6dbb552 Compare August 26, 2024 09:05

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 4 times, most recently from 1a8d5ee to 00192b9 Compare October 22, 2024 05:09

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 4 times, most recently from ba27f43 to 6c7f199 Compare October 30, 2024 10:09

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 5 times, most recently from c914b7d to 6721f53 Compare November 9, 2024 10:26

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 6721f53 to 12704ce Compare November 12, 2024 10:26

gargnitingoogle added 15 commits November 13, 2024 10:26

Add gsheet utilities

3f5b2ee

Utilities: 1. Append given tabular data to the given gsheet id and worksheet name. 2. Return url for a gsheet given its ID. 3. Adds unit tests for the above utilities.

fix gsheet unit tests

d30599a

address self-review comments

fcc8257

address some more self-review comments

6dc12e1

address a self-review comment

d76b26f

fix bug in gsheet utility code

9be1283

rename some utilities

d5dd213

address some self-review comments

ccad78f

Add try-catch for exception during parsing

0c21f4a

Adds row with "ERROR" for all values rather than crashing during printing in CSV file.

add some useful comments

1f69ff6

Replace mash with monitoring-api everywhere

8e4591f

Purposes. * Consistent behavior across all machines * Monitoring API has faster runtime than mash. * Monitoring API is supported on GCE VM too.

fix some var names

57379c8

expose force_update_gcsfuse_code

fbd46ae

empty commit

d2952ab

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 12704ce to d2952ab Compare November 13, 2024 10:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DontReview Garnitin/add gke load testing/v1 #2225

DontReview Garnitin/add gke load testing/v1 #2225

gargnitingoogle commented Jul 26, 2024

codecov bot commented Jul 26, 2024 •

edited

Loading

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 27, 2024

gargnitingoogle Jul 27, 2024

DontReview Garnitin/add gke load testing/v1 #2225

Are you sure you want to change the base?

DontReview Garnitin/add gke load testing/v1 #2225

Conversation

gargnitingoogle commented Jul 26, 2024

Description

Link to the issue in case of a bug fix.

Testing details

codecov bot commented Jul 26, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jul 26, 2024 •

edited

Loading