This utility automates migrating Virtual Machine instances within GCP. You can migrate VMs from one zone to another zone/region within the same or different projects.
To be able to execute the utility, you should meet the following prerequisites:
- Have access to this repository
- Have at least Python 3.7 installed
- Ideally have a Python3 environment
- The service account with which you are running this script should be able to:
- Create/Delete/Shutdown GCE VMs
- Create/Delete VPC subnets
- Create/Delete machine images
- You should be able to create files on your filesystem to store the inventory data
- Clone this repository
- Change directory to this directoy (tools/vm-migrator)
- Optional if using a virtual environment:
python3 -m venv venv - Optional if using a virtual environment:
source venv/bin/activate - Run
make install-requirements - Run
make install-tools - Run
make install - Configure (see below)
Run make test
The Makefile has different variables which are passed to the subnet_region_migratory.py.
These variables can be overridden from the command line:
make VARIABLE=VALUE migrate-subnet
The variables NOT marked as "[Optional]" are mandatory.
| Variable | Value |
|---|---|
| STEP | The name of the step to run |
| MACHINE_IMAGE_REGION | The region where you want to store your machine images, e.g "us-central1" |
| SOURCE_PROJECT | Project ID where the VMs to be migrated are located |
| SOURCE_SUBNET | The source subnetwork URI, for example, "projects/my-host-project/regions/europe-west3/subnetworks/source-subnet" |
| SOURCE_ZONE | The zone in which the source subnet is present, this is used to fetch the inventory details, for example, "europe-west3-c" |
| SOURCE_ZONE_2 | [Optional] The second zone in which the source subnet is present, this is used to fetch the inventory details |
| SOURCE_ZONE_3 | [Optional] The third zone in which the source subnet is present, this is used to fetch the inventory details |
| TARGET_PROJECT | [Optional] If different from the source project, the Project ID where the new subnet will exist |
| TARGET_PROJECT_SA | Required if the target project is different than the source project, otherwise [Optional]. Used as Service Account on the target VMs. |
| TARGET_PROJECT_SA_SCOPES | [Optional] If the target project is different than the source project, scopes to assign to the Service Account on the VMs |
| TARGET_SUBNET | Leave this argument empty if you're just migrating VMs within the same subnet (and region) into a different service project. Otherwise the target subnetwork URI, for example, "projects/my-host-project/regions/us-central1/subnetworks/target-subnet" |
| BACKUP_SUBNET | [Optional] If you want to back up your instances to a subnet, specify the subnet's URI, for example, "projects/my-host-project/regions/us-central1/subnetworks/backup-subnet" |
| SOURCE_CSV | [Optional] (by default source.csv) The filename used by prepare_inventory |
| FILTER_CSV | [Optional] (by default filter.csv) File that contains the instance names that will be included by filter_inventory |
| INPUT_CSV | [Optional] (by default export.csv) The filtered list of machines created by filter_inventory used as input for the migration. |
| LOG_LEVEL | [Optional] Debugging level |
The FILTER_CSV variable from above has the name of the file that contains the instance names that will be included by the filter_inventory step. It is basically the list of the instances you actually want to migrate. Please either edit the filter.csv file or provide your file and change the variable value.
The file src/migrator/zone_mapping.py contains the mapping of the zones - which will be migrated to which. If your zones are not there, please edit it and add them there.
VM Migrator supports multiple functionality, each feature is triggered via the step name which is passed as a variable to the the script (see above).
While you can move all the machines in a subnet to the destination subnet, you have the flexibility to go with two approaches:
- Bulk move approach, where you export all the machines running in one zone and subnet to destination zone and subnet. Since you are moving all the machines, you can drain the entire subnet, clone and recreate the subnet in the destination region.
- This approach lets you keep the same IP ( Internal + Alias IPs ) of the source VM in the destination subnet
- Individual machine move, in this approach you can move only some of the machines to the destination subnet. Since you are not draining the entire subnet, the cloned machines created will have a different IP from the source machine.
| STEP | Description |
|---|---|
| prepare_inventory | This will dump the entire inventory in the specified subnet and zones in CSV format into the SOURCE_CSV (by default source.csv). |
| filter_inventory | Executes prepare_inventory and copies SOURCE_CSV to INPUT_CSV (by default export.csv) only selecting machines whose instance names match any row in FILTER_CSV (by default filter.csv). |
| shutdown_instances | This will shutdown all the machines whose details are found in the INPUT_CSV file, this is particularly useful when you want to shutdown the instances before taking their machine image |
| start_instances | This will start all the machines whose details are found in the INPUT_CSV file, this is particularly useful when you want to rollback the shutdown step |
| create_machine_images | This will create the machine image of the instances which are specifed in the INPUT_CSV file |
| backup_instances | The backup functionality moves the original instances to a backup subnet you specify in the BACKUP_SUBNET variable. Please see the section "Backup" below for more information |
| prepare_rollback | This populates the rollback.csv file with instances to be moved from the backup subnet to the original one. Please see the section "Rollback" below for more information |
| rollback_instances | This moves the instances listed in rollback.csv from BACKUP_SUBNET to SOURCE_SUBNET. Please see the section "Rollback" below for more information |
| disable_deletionprotection_instances | This will disable the deletion protection on all instances in the INPUT_CSV file. In case it's enabled on any instances, this should be run before delete_instances. |
| delete_instances | This will delete the instances which are specifed in the INPUT_CSV file |
| release_ip | This will release all the internal static IP addresses of the instances which are specifed in the INPUT_CSV file |
| release_ip_for_subnet | This will release all the internal static IP addresses of the source subnet |
| clone_subnet | This will delete and re-create the subnet in the destination region with the same config as the source subnet, this is required when you want to drain the entire subnet and create it in the destination region. Keep in mind that subnets can't be deleted from VPCs with auto subnet mode. |
| add_machineimage_iampolicies | This will provide the target project service account with the right permissions to access the source project machine images |
| create_instances | This will create the instances from the machine images in the destination subnet/region, it requires the destination subnet to be in place (if you are cloning the subnet, it will automatically be created) |
| create_instances_without_ip | This is similar to the create_instances step, just that it will not preserve the source IPs, this is useful in a situation when you are moving some of the machines to the destination subnet which has a different CIDR range |
Before running, please ensure that
- the latest code is pulled from repository
- the prerequisites are met
- your setup and configuration are complete
Run make STEP=<step name> migrate-subnet
The tool will log to the file "migrator.log", located in the same directory.
The file node_group_mapping.py has the mapping of source and destination node groups. If you are moving machines which are running on sole tenant nodes, then the create_instances/create_instances_without_ip step will look for mapping of node groups, and it finds the mapping, it will use the specified node group in the destination subnet/region to run the VM on automatically.
In the case of migrating VMs in a Shared VPC, the source and target projects are the service projects where the VMs live. The host project is implicitly indicated in the URI of the source and target subnet.
In the process of migration you can optimize the machine types based on the usage. You can do that by providing the source and destination machine type mappings in machine_type_mapping.py, e.g n1-standard-1: n1-standard-4 would upgrade all the machines running with n1-standard-1 to n1-standard-4.
The optional backup functionality (backup_instances step) does the following:
- It moves the original instances to a backup subnet you specify in the
BACKUP_SUBNETvariable. You have to create a subnet for this manually since you need to specify a CIDR block that will work for you. This subnet has to be in the same region as the source subnet! - Renames the instances: adds a '0' symbol at the end of each instance's name. This is done so that you can still migrate your original instances with the same names. Please make sure that your instances' names are not too long and one more symbol can be added! This is not being checked in the script.
For usage see the examples below.
Notes:
- Current limitations:
- work only within one project
- takes into account only one zone (no variables ZONE_2 and so on)
- only one internal IP of the instances is taken into account
- instances should only have one nic
- In order to work the function needs a "fingerprint" - a unique identifier of the network interface of an instance. This is collected during the
prepare_inventorystep. Thebackup_instancesstep checks for those fingerprints and aborts if any are not found. - Please make sure you keep the
export.csvfile intact, since if you will need to roll back, this file will be needed (see next section). - Important: remember that for any operation requiring moving the instance between subnets it has to be stopped.
-
If you did not use the backup functionality, in order to rollback the changes please change the parameters of the source file and interchange the source and destination parameters. Please note that some steps like
create_machine_imagewould not be required to run, as the machine images have already been created. -
Otherwise, with backup, you first have to prepare the rollback with the
prepare_rollbackstep, which populates therollback.csvfile with instances to be moved. This will also check theexport.csvfile to find the previous IPs of the instances and match them to the instances. If at least one is not found, the function aborts. Once the rollback file is populated, please check it to ensure the necessary instances get rolled back and that the right IP addresses get assigned.Then you execute the
rollback_instancesstep, which moves the instances listed inrollback.csvfromBACKUP_SUBNETtoSOURCE_SUBNETand renames them to their original names.Important: remember that for any operation requiring moving the instance between subnets it has to be stopped.
See the examples below.
If you backed up your instances and everything went well - you can just delete the backed up instances (this does NOT delete the backup subnet). But first you should list them to rollback.csv with the prepare_rollback step, so that you can check what will be deleted. Then:
make STEP=delete_instances INPUT_CSV=rollback.csv migrate-subnet
Be sure to specify the rollback.csv as input.
There are certain common scenarios you might be interested in. All of them require you to fill the proper required parameters in the Makefile and the proper zone mapping in zone_mapping.py. Afterwards, you can execute the following recipes:
This one requires a TARGET_PROJECT. By default, TARGET_PROJECT_SA points to the target project's compute engine service account.
# standard
make STEP=prepare_inventory migrate-subnet
make STEP=filter_inventory migrate-subnet
make STEP=shutdown_instances migrate-subnet
make STEP=create_machine_images migrate-subnet
make STEP=delete_instances migrate-subnet
# release VM static IPs
make STEP=release_ip migrate-subnet
# target service account requires access to the source machine images
make STEP=add_machineimage_iampolicies migrate-subnet
# create instances in the target project
make STEP=create_instances_without_ip migrate-subnet
This one requires the TARGET_SUBNET that indicates target host project, region and subnet name for the clone_subnet step.
# standard
make STEP=prepare_inventory migrate-subnet
make STEP=filter_inventory migrate-subnet
make STEP=shutdown_instances migrate-subnet
make STEP=create_machine_images migrate-subnet
make STEP=delete_instances migrate-subnet
# remove reserved static ips for this subnetwork
make STEP=release_ip_for_subnet migrate-subnet
# remove and recreate subnet in the same VPC but in the target region
make STEP=clone_subnet migrate-subnet
# recreate instances with the same IP in the recreated subnet
make STEP=create_instances migrate-subnet
Note: This case is particularly interesting if VMs need to stay in the same VPC. Otherwise, use the next example (move VMs to an existing subnet is a better fit).
This step also requires TARGET_SUBNET to be defined (which indicates target host project and region implicitly). In this case it is also expected that this subnet already exists with the given definition.
# standard
make STEP=prepare_inventory migrate-subnet
make STEP=filter_inventory migrate-subnet
make STEP=shutdown_instances migrate-subnet
make STEP=create_machine_images migrate-subnet
make STEP=delete_instances migrate-subnet
# release VM static IPs
make STEP=release_ip migrate-subnet
# create instances in the target subnet
make STEP=create_instances_without_ip migrate-subnet
This recipe is fully compatible with moving across projects. In this case, please specify the TARGET_PROJECT* variables additionally and execute make STEP=add_machineimage_iampolicies migrate-subnet before creating the instances as well.
The backup functionality moves the original instances to a backup subnet you specify in the BACKUP_SUBNET variable.
Only backup itself:
make STEP=prepare_inventory migrate-subnet
make STEP=filter_inventory migrate-subnet
make STEP=shutdown_instances migrate-subnet
make STEP=backup_instances migrate-subnet
Backup as part of the example above - "Move VMs across regions (recreate subnet)".
# standard
make STEP=prepare_inventory migrate-subnet
make STEP=filter_inventory migrate-subnet
make STEP=shutdown_instances migrate-subnet
make STEP=create_machine_images migrate-subnet
# back up instances - move them to the backup subnet
make STEP=backup_instances migrate-subnet
make STEP=delete_instances migrate-subnet
# remove reserved static ips for this subnetwork
make STEP=release_ip_for_subnet migrate-subnet
# remove and recreate subnet in the same VPC but in the target region
make STEP=clone_subnet migrate-subnet
# recreate instances with the same IP in the recreated subnet
make STEP=create_instances migrate-subnet
If anything went wrong - rollback:
make STEP=prepare_rollback migrate-subnet
make STEP=rollback_instances migrate-subnet
# optionally start the instances
make STEP=start_instances INPUT_CSV=rollback.csv migrate-subnet
If everything went well - delete backed up instances (this does NOT delete the backup subnet):
make STEP=delete_instances INPUT_CSV=rollback.csv migrate-subnet