1&1’s documentation is pretty awful but to be fair they only expect real system administrators to be using their dedicated servers. They give you a copy of Plesk 10 but it doesn’t have any capabilities to alter the disk or volume group layouts. 1&1 layout their default image in a way that only gives you 4GB on each of the major partitions and it leaves the other 212GB as unallocated. So in order to use the server properly you have to get interested in the Logical Volume Manager.
It is a daunting prospect because it can so easily go pear shaped if you are not careful. I haven’t done any volume management since my Macro 4 days sorting out HP-UX boxes. So consider this a beginners guide to 1&1 servers as well as Logical Volume Management.
You don’t get any information about the dedicated servers layout when you start the 1&1 contract. All you get is a user name and a password, you have to do a full investigation of the hardware, disks, partitions, volume groups and all the other scary stuff you don’t really want to be touching.
The article will take you through the platform hardware investigation and volume layout. Once we know what we have to deal with I will look at extending a logical volume and creating a new one.
I’ll start with a few terms that you need to understand if you haven’t done much work with logical volumes before. I’ll talk in terms of disks and partitions because that’s what we had to deal with before LVM.
Physical disk is an actual hard-drive with spinning platters and everything.
Partition is a space allocated on a physical disk that can hold a file system. You can’t put a file system straight on to the disk you need to have a container for it to sit in.
Physical volume same as a partition (an area on a physical disk drive).
Volume group is one or more physical volumes glued together to make one virtual drive.
Logical volume is a part of a volume group.
In the olden days you split your disks into partitions and used each partition as a chunk of storage. Nowadays we glue all the disk drives together to make an uber virtual disk that we can split up into lots of different pieces. It works by splitting each volume group into pieces called
extends and then allocating groups of extends into logical partitions - a bit like the way a file system works with sectors and blocks.
Ok, we are now sitting in front of our new 1&1 dedicated server with the blank sheet of instructions they gave us!
First we need to know the state of the system. So for this we need to check the display messages. These are the messages that appear on the screen during boot up and can be seem by typing:
dmesg. The bit we are interested in begins with SCSI device. Here is an example:
SCSI device sda: 488397168 512-byte hdwr sectors ( 250059 MB )
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3
The first line describes the size of the physical hard drive. Don’t worry about the other stuff because it is the final line we are interested in. It shows that the
sda drive has 3 partitions:
sda1,
sda2 and
sda3. If we look a bit further down the file we can see there is a similar set up
sdb which also has 3 partitions:
sdb1,
sdb2 and
sdb3.
The next section in
dmesg output talks about setting up the disk partitions to mirror each other. md3 is made of:
md: created md3
md: bind<sda3>
md: bind<sdb3>
md: running: <sdb3><sda3>
and md1 is made of:
md: created md1
md: bind<sda1>
md: bind<sdb1>
md: running: <sdb1><sda1>
Further down we can see:
2. Priority:-1 extents:1 across:1959920k
Adding 1959920k swap on /dev/sdb2. Priority:-2 extents:1 across:1959920k
which shows that our 2 remaining partitions are used as swap space.
The 1&1 default image set up contains the following pre-configured partitions:
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 3.7G 332M 3.4G 9% /
/dev/mapper/vg00-usr 4.0G 1.1G 3.0G 26% /usr
/dev/mapper/vg00-var 4.0G 2.1G 2.0G 52% /var
/dev/mapper/vg00-home 4.0G 2.1G 2.0G 51% /home
/dev/md1 is accounted for as the root partition so
/dev/md3 must be the drive which is backing our logical volume. As you can see only 12GB is being used so there is still about 215GB which is unallocated.
We can confirm this with the
vgs command, which shows that there is 215GB free. Even though they are mirrored 250GB drives we lose about 20GB to the file system indexes. :(
# vgs
VG #PV #LV #SN Attr VSize VFree
vg00 1 3 0 wz–n- 227.28G 215.28G
Now that we have finished all the checks, we now know that there aren’t any more physical drives or logical volumes that are unaccounted for.
On my system there are quite a lot of databases so I need to either increase the
/var partition or create a new partition that I can place under
/var/lib/mysql.
Having read all the documentation it’s pretty easy to do either. My unix administration background is erring me on the side of a new logical partition. The nice thing about LVM is that you can always change your mind later. If I create a new partition then it will free up space in my
/var partition so I won’t have to extend it for a while.
I will start by showing you how to resize an existing logical volume. I’m using CentOS 5 which has symbolic links to the lvm command making it somewhat simpler.
# ls -l /usr/sbin/vgdisplay
lrwxrwxrwx 1 root root 3 Jan 31 22:00 /usr/sbin/vgdisplay -> lvm
This means I can type
vgdisplay instead of
lvm vgdisplay so in the instructions below I’ll be using the short versions of the commands.
The logical volumes can be extended (or reduced) in units of
Extends which are defined when the volume group is set up. We must see how much we can extend the logical volume by, so let’s check the volume group and see how many we have left.
# vgdisplay
— Volume group —
VG Name vg00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1
Act PV 1
VG Size 227.28 GB
PE Size 4.00 MB
Total PE 58184
Alloc PE / Size 3072 / 12.00 GB
Free PE / Size 55112 / 215.28 GB
VG UUID t5aC2D-DWv9-Dh5s-Hz2c-dzIH-oSxX-4jMjwo
Here we can see that a Physical Extend is 4MB and we have 55112 of them which are unallocated. Next we need to look at what I have in my current /var logical partition:
# lvdisplay /dev/mapper/vg00-var
— Logical volume —
LV Name /dev/vg00/var
VG Name vg00
LV UUID xRq5Oj-Y41X-M6VK-s2es-nCOT-dDFF-X0Sznp
LV Write Access read/write
LV Status available
# open 1
LV Size 4.00 GB
Current LE 1024
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1
The Logical Extend and the Physical Extend are the same size, but I’m sure there is a reason why they have different names (anyone?). From the read out above you can see that we have 1024 extends. So if I wanted to extend the partition by 6GB making it 10GB in size I would have to increase the extends to 10240. To extend the logical partition I issue the command:
# lvm lvresize -l 10240 /dev/mapper/vg00-var
Extending logical volume vg00 to 10.0 GB
Logical volume vg00 successfully resized
The logical volume is now bigger but the file system that lives in it is still the same size, so we need to update it. Under CentOS 5 we can resize the filing system without unmounting it, which is really handy because there are loads of programs running which have open files on the
/var partition. The command is simple:
# resize2fs /dev/mapper/vg00-var
Note: I just tried this and it didn’t work. The CentOS documentations says the the code for ext2online (which was the old method of extending ext2/ext3 file systems) had been merged into resize2fs. My system uses xfs and not ext2/ext3 so I had to use:
xfs_growfs /dev/vg00/var
The command gives you the following output, which is very similar to what you get when you are creating the xfs filesystem.
meta-data=/dev/vg00/var isize=256 agcount=4, agsize=262144 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=1048576, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 1048576 to 2097152
I have decided to create a brand new logical volume that I can resize independently. The logical volume group already exists so I don’t need to create that.
Even though I have 2 physical disks they are joined together as 1 mirror and used as 1 entity in the volume group so I can’t specify the
-i option to have the volume group manager organise writes to both disks. I don’t think it matters it you do the mirroring up in the volume group level or on the disk partition level.
We issue the logical volume create command and set it to 10GB
# lvcreate -n mysql -L10G vg00
Logical volume “mysql” created
A quick check shows that everything is ok:
# ls -l /dev/mapper/
total 0
crw——- 1 root root 10, 63 Jan 31 22:30 control
brw-rw—- 1 root disk 253, 2 Jan 31 22:31 vg00-home
brw-rw—- 1 root disk 253, 3 Feb 7 21:11 vg00-mysql
brw-rw—- 1 root disk 253, 0 Jan 31 22:31 vg00-usr
brw-rw—- 1 root disk 253, 1 Jan 31 22:31 vg00-var
# lvdisplay /dev/mapper/vg00-mysql
— Logical volume —
LV Name /dev/vg00/mysql
VG Name vg00
LV UUID LQpgnU-Of2d-y3LP-Mj2n-0FH1-CaGd-1OW0kQ
LV Write Access read/write
LV Status available
# open 0
LV Size 10.00 GB
Current LE 2560
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3
Note: I tried to create a logical volume called “scratch-backup", but when I checked the
/dev/mapper it listed the group name as
vg00-scratch--backup. In my opinion anything unexpected is bad so I want to delete this logical volume and pick a different name.
lvremove /dev/mapper/vg00-scratch–backup
This new logical volume is blank so I need to create a file system in it. Not sure of all the magic options so I’ll let the system use its defaults:
# mkfs -t xfs /dev/vg00/mysql
meta-data=/dev/vg00/mysql isize=256 agcount=16, agsize=163840 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=2621440, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2560, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
The final disk test is to make sure we can mount it into the file system.
# mount /dev/vg00/mysql /mnt
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 3.7G 332M 3.4G 9% /
/dev/mapper/vg00-usr 4.0G 1.1G 3.0G 26% /usr
/dev/mapper/vg00-var 4.0G 2.1G 2.0G 52% /var
/dev/mapper/vg00-home 4.0G 2.1G 2.0G 51% /home
/dev/mapper/vg00-mysql 10G 4.6M 10G 1% /mnt
I want this file system to be mounted when the machine boot’s up so I must add it into the
/etc/fstab file. Now I can start the process of copying my existing databases into the new partition and we are done.
Primary documentation site for LVM
http://tldp.org/HOWTO/LVM-HOWTO/commontask.html
Resize instructions
http://wiki.centos.org/TipsAndTricks/ExpandLV
Clear(ish) description of concepts but mainly talks about the management via the system GUI.
http://www.linuxtopia.org/online_books/centos5/centos5_administration_guide/centos5_ch-lvm.html
Some 1&1 documentation has finally materialised!
http://faq.1and1.co.uk/server/root_server/linux_admin_help/7.html
