Merge pull request #98 from GunashekarKM/master

Generic Wheel file & Broken URLs

README.md

@@ -41,7 +41,7 @@ Swarm Learning nodes works in collaboration with other Swarm Learning nodes in t
 
 ## User ML components
 
-User can transform any Keras or PyTorch based ML program that is written using Python3 into a Swarm Learning ML program by [making a few simple changes](./docs/User/How_to_Swarm_enable_an_ML_algorithm.md) to the model training code by including the `SwarmCallback` API. For more information, see the [examples](./docs/User/Examples.md) included with the Swarm Learning package for a sample code.
+User can transform any Keras or PyTorch based ML program that is written using Python3 into a Swarm Learning ML program by [making a few simple changes](./docs/User/How_to_Swarm_enable_an_ML_algorithm.md) to the model training code by including the `SwarmCallback` API. For more information, see any of the [examples](/examples/README.md) included with the Swarm Learning package for a sample code.
 
 The transformed user Machine Learning \(user ML node\) program can be run on the host or user can build it as a Docker container.
 
@@ -63,7 +63,7 @@ NOTE: All the ML nodes must use the same ML platform either Keras (based on Tens
     **Caution**: Users are recommended not to save their model related artifacts under this folder, as future version upgrade of Swarm Learning would delete these folders.
   - [Upgrading from earlier evaluation versions](/docs/Install/Versioning_and_upgrade.md)
   - [Download and setup Swarm Learning](/docs/Install/HPE_Swarm_Learning_installation.md) using the Web UI installer 
-  - Execute [MNIST example](/docs/User/MNIST.md) 
+  - Execute [MNIST example](/examples/mnist/MNIST.md) 
   - [Frequently Asked Questions](/docs/User/Frequently_asked_questions.md)
   - [Troubleshooting](/docs/User/Troubleshooting.md)
 

docs/User/Task_for_building_the_user_container.md

@@ -18,7 +18,7 @@ Body:
     - '   keras matplotlib opencv-python pandas protobuf==3.15.6 sklearn'
     - ' '
     - RUN mkdir -p /tmp/hpe-swarmcli-pkg
-    - COPY swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
-    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
+    - COPY swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
+    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
 ```
 

examples/README.md

@@ -1,6 +1,14 @@
 ## Prerequisite for all examples
-Start license server and install valid license before running any of the examples. Refer [Installing licenses and starting license server](/docs/Install/HPE_Swarm_Learning_installation.md).
+1. Start license server and install valid license before running any of the examples. Refer [Installing licenses and starting license server](/docs/Install/HPE_Swarm_Learning_installation.md).
 
+2. Install the Swarm Learning product using the Web UI installer.  Refer [Web UI installation](/docs/Install/HPE_Swarm_Learning_installation.md)
+
+3. All the examples refer to a generic Swarm Learning client python wheel file. Create a symbolic link to the actual wheel file version, before running the examples
+ ```
+ cd swarm-learning/lib
+ ln -fs swarmlearning-1.0.1-py3-none-manylinux_2_24_x86_64.whl swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
+ ```
+
 ## Swarm Learning Examples
 
 Several examples of using Swarm Learning are provided with the package. 
@@ -16,5 +24,5 @@ For details of running each example, see the below:
 
 -   [MNIST](/examples/mnist/MNIST.md)
 -   [MNIST-PYT](/examples/mnist-pyt/MNIST-PYT.md)
--   [CIFAR-10](/examples/cifar/CIFAR-10.md)
+-   [CIFAR-10](/examples/cifar10/CIFAR-10.md)
 -   [Credit card fraud detection](/examples/fraud-detection/Credit_card_fraud_detection.md)

examples/cifar10/CIFAR-10.md

@@ -10,7 +10,7 @@ This example shows the Swarm training of Keras based CIFAR-10 model using two ML
 
 The following image illustrates a cluster setup for the CIFAR-10 example:
 
-![CIFAR-10 Cluster Setup](GUID-4D303DEC-8E71-43F4-BDCB-04B0C1AE79D8-high.png)
+![CIFAR-10 Cluster Setup](/docs/User/GUID-4D303DEC-8E71-43F4-BDCB-04B0C1AE79D8-high.png)
 
 -   This example uses one SN node. SN1 is the name of the Docker container that runs on host 172.1.1.1.
 
@@ -70,7 +70,7 @@ cp -r examples/utils/gen-cert workspace/cifar10/
 5.  On both host-1 and host-2, copy Swarm Learning wheel file inside build context and build Docker image for ML that contains environment to run Swarm training of user models.
 
 ```
-cp lib/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl workspace/cifar10/ml-context/
+cp -L lib/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl workspace/cifar10/ml-context/
 docker build -t user-ml-env-tf2.7.0 workspace/cifar10/ml-context
 ```
 You may need to specify the correct https_proxy for the docker build if you are behind a firewall. For eg,

examples/cifar10/ml-context/Dockerfile

@@ -5,10 +5,10 @@
 
 FROM tensorflow/tensorflow:2.7.0
 
-COPY requirements.txt swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl /tmp/swarmlib/
+COPY requirements.txt swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl /tmp/swarmlib/
 
 # Install ml environment packages
 RUN pip3 install --upgrade pip && pip3 install -r /tmp/swarmlib/requirements.txt
 
 # Install SwarmLearning package
-RUN pip3 install /tmp/swarmlib/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
+RUN pip3 install /tmp/swarmlib/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl

examples/fraud-detection/Credit_card_fraud_detection.md

@@ -17,7 +17,7 @@ This example shows the Swarm training of the credit card fraud detection model u
 
 The following image illustrates a cluster setup that uses only one host:
 
-![Credit Card Fraud Detection](GUID-BE2185B8-5C3B-4BD3-91FF-9ABC77D0720C-high.png)
+![Credit Card Fraud Detection](/docs/User/GUID-BE2185B8-5C3B-4BD3-91FF-9ABC77D0720C-high.png)
 
 -   This example uses one SN node. The names of the docker containers representing this node is SN1. SN1 is also the Sentinel Node. SN1 runs on the host 172.1.1.1.
 
@@ -74,7 +74,7 @@ mv workspace/fraud-detection/data-and-scratch workspace/fraud-detection/user4/
 ```
 docker volume create sl-cli-lib
 docker container create --name helper -v sl-cli-lib:/data hub.myenterpriselicense.hpe.com/hpe_eval/swarm-learning/sn:1.0.0
-docker cp lib/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl helper:/data
+docker cp -L lib/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl helper:/data
 docker rm helper
 
 ```

examples/fraud-detection/swci/taskdefs/user_env_tf_build_task.yaml

@@ -16,6 +16,6 @@ Body:
     - '   keras matplotlib opencv-python pandas protobuf==3.15.6 '
     - ' '
     - RUN mkdir -p /tmp/hpe-swarmcli-pkg
-    - COPY swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
-    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
+    - COPY swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
+    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
 

examples/mnist-pyt/MNIST-PYT.md

@@ -14,7 +14,7 @@ The Machine Learning program, after conversion to Swarm Learning for the PyTorch
 
 This example shows the Swarm training of MNIST model using four ML nodes. ML nodes along with SL nodes are automatically spawned by SWOP nodes running on two different hosts. Swarm training is initiated by SWCI node and orchestrated by two SN nodes running in different hosts. This example also shows how private data, private scratch area and shared model can be mounted to ML nodes for Swarm training.
 
-The following image illustrates a cluster setup for the MNIST example:![Four cluster setup](GUID-25587679-1F3A-43DC-8D02-48E6BEFF7DA6-high.png)
+The following image illustrates a cluster setup for the MNIST example:![Four cluster setup](/docs/User/GUID-25587679-1F3A-43DC-8D02-48E6BEFF7DA6-high.png)
 
 -   This example uses two SN nodes. The names of the docker containers representing these two nodes are SN1 and SN2. SN1 is the Sentinel Node. SN1 runs on host 172.1.1.1 and SN2 runs on host 172.2.2.2.
 
@@ -109,7 +109,7 @@ sed -i "s+<CURRENT-PATH>+$(pwd)+g" workspace/mnist-pyt/swop/swop*_profile.yaml w
 ```
 docker volume create sl-cli-lib
 docker container create --name helper -v sl-cli-lib:/data hub.myenterpriselicense.hpe.com/hpe_eval/swarm-learning/sn:1.0.0
-docker cp lib/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl helper:/data
+docker cp -L lib/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl helper:/data
 docker rm helper
 ```
 

examples/mnist-pyt/swci/taskdefs/user_env_pyt_build_task.yaml

@@ -22,5 +22,5 @@ Body:
     - '   matplotlib opencv-python pandas sklearn'
     - ' '
     - RUN mkdir -p /tmp/hpe-swarmcli-pkg
-    - COPY swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
-    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
+    - COPY swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
+    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl

examples/mnist/MNIST.md

@@ -14,7 +14,7 @@ The Machine Learning program, after conversion to Swarm Learning for the TensorF
 
 This example shows the Swarm training of MNIST model using two ML nodes. ML nodes are automatically spawned by SWOP nodes running on two different hosts. Swarm training is initiated by SWCI node and orchestrated by two SN nodes running in different hosts. This example also shows how private data and shared model can be mounted to ML nodes for Swarm training.
 
-The following image illustrates a cluster setup for the MNIST example:![Two cluster setup](GUID-66B1F78E-34E6-475D-AE8D-966BCF93539B-high.png)
+The following image illustrates a cluster setup for the MNIST example:![Two cluster setup](/docs/User/GUID-66B1F78E-34E6-475D-AE8D-966BCF93539B-high.png)
 
 -   This example uses two SN nodes. The names of the Docker containers representing these two nodes are SN1 and SN2. SN1 is the Sentinel Node. SN1 runs on host 172.1.1.1 and SN2 runs on host 172.2.2.2.
 
@@ -88,7 +88,7 @@ sed -i "s+<CURRENT-PATH>+$(pwd)+g" workspace/mnist/swop/swop*_profile.yaml works
 ```
 docker volume create sl-cli-lib
 docker container create --name helper -v sl-cli-lib:/data hub.myenterpriselicense.hpe.com/hpe_eval/swarm-learning/sn:1.0.0
-docker cp lib/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl helper:/data
+docker cp -L lib/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl helper:/data
 docker rm helper
 ```
 

examples/mnist/swci/taskdefs/user_env_tf_build_task.yaml

@@ -16,6 +16,6 @@ Body:
     - '   keras matplotlib opencv-python pandas protobuf==3.15.6 sklearn'
     - ' '
     - RUN mkdir -p /tmp/hpe-swarmcli-pkg
-    - COPY swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
-    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-1.0.0-py3-none-manylinux_2_24_x86_64.whl
+    - COPY swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl
+    - RUN pip3 install /tmp/hpe-swarmcli-pkg/swarmlearning-client-py3-none-manylinux_2_24_x86_64.whl