Ubuntu 14.04 and up: Rename a volume group containing root

When checking my cloned Ubuntu 14.04 LTS VM, I noticed that it still contained the volume group name from the original, which is of course not desired.

It seemed fairly simple, but I kept receiving an error with update-grub, reporting the old Volume Group name:

Steps to be done as root: