dmesg. The bit we are interested in begins with SCSI device. Here is an example:
SCSI device sda: 488397168 512-byte hdwr sectors ( 250059 MB ) sda: Write Protect is off sda: Mode Sense: 00 3a 00 00 SCSI device sda: drive cache: write back sda: sda1 sda2 sda3The first line describes the size of the physical hard drive. Don’t worry about the other stuff because it is the final line we are interested in. It shows that the
sdadrive has 3 partitions:
sda3. If we look a bit further down the file we can see there is a similar set up
sdbwhich also has 3 partitions:
sdb3. The next section in
dmesgoutput talks about setting up the disk partitions to mirror each other. md3 is made of:
md: created md3 md: bind<sda3> md: bind<sdb3> md: running: <sdb3><sda3>and md1 is made of:
md: created md1 md: bind<sda1> md: bind<sdb1> md: running: <sdb1><sda1>Further down we can see:
2. Priority:-1 extents:1 across:1959920k Adding 1959920k swap on /dev/sdb2. Priority:-2 extents:1 across:1959920kwhich shows that our 2 remaining partitions are used as swap space. The 1&1 default image set up contains the following pre-configured partitions:
# df -h Filesystem Size Used Avail Use% Mounted on /dev/md1 3.7G 332M 3.4G 9% / /dev/mapper/vg00-usr 4.0G 1.1G 3.0G 26% /usr /dev/mapper/vg00-var 4.0G 2.1G 2.0G 52% /var /dev/mapper/vg00-home 4.0G 2.1G 2.0G 51% /home
/dev/md1is accounted for as the root partition so
/dev/md3must be the drive which is backing our logical volume. As you can see only 12GB is being used so there is still about 215GB which is unallocated. We can confirm this with the
vgscommand, which shows that there is 215GB free. Even though they are mirrored 250GB drives we lose about 20GB to the file system indexes. :(
# vgs VG #PV #LV #SN Attr VSize VFree vg00 1 3 0 wz–n- 227.28G 215.28GNow that we have finished all the checks, we now know that there aren’t any more physical drives or logical volumes that are unaccounted for. On my system there are quite a lot of databases so I need to either increase the
/varpartition or create a new partition that I can place under
/var/lib/mysql. Having read all the documentation it’s pretty easy to do either. My unix administration background is erring me on the side of a new logical partition. The nice thing about LVM is that you can always change your mind later. If I create a new partition then it will free up space in my
/varpartition so I won’t have to extend it for a while. I will start by showing you how to resize an existing logical volume. I’m using CentOS 5 which has symbolic links to the lvm command making it somewhat simpler.
# ls -l /usr/sbin/vgdisplay lrwxrwxrwx 1 root root 3 Jan 31 22:00 /usr/sbin/vgdisplay -> lvmThis means I can type vgdisplay instead of lvm vgdisplay so in the instructions below I’ll be using the short versions of the commands. The logical volumes can be extended (or reduced) in units of Extends which are defined when the volume group is set up. We must see how much we can extend the logical volume by, so let’s check the volume group and see how many we have left.
# vgdisplay — Volume group — VG Name vg00 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size 227.28 GB PE Size 4.00 MB Total PE 58184 Alloc PE / Size 3072 / 12.00 GB Free PE / Size 55112 / 215.28 GB VG UUID t5aC2D-DWv9-Dh5s-Hz2c-dzIH-oSxX-4jMjwoHere we can see that a Physical Extend is 4MB and we have 55112 of them which are unallocated. Next we need to look at what I have in my current /var logical partition:
# lvdisplay /dev/mapper/vg00-var — Logical volume — LV Name /dev/vg00/var VG Name vg00 LV UUID xRq5Oj-Y41X-M6VK-s2es-nCOT-dDFF-X0Sznp LV Write Access read/write LV Status available # open 1 LV Size 4.00 GB Current LE 1024 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1The Logical Extend and the Physical Extend are the same size, but I’m sure there is a reason why they have different names (anyone?). From the read out above you can see that we have 1024 extends. So if I wanted to extend the partition by 6GB making it 10GB in size I would have to increase the extends to 10240. To extend the logical partition I issue the command:
# lvm lvresize -l 10240 /dev/mapper/vg00-var Extending logical volume vg00 to 10.0 GB Logical volume vg00 successfully resizedThe logical volume is now bigger but the file system that lives in it is still the same size, so we need to update it. Under CentOS 5 we can resize the filing system without unmounting it, which is really handy because there are loads of programs running which have open files on the
/varpartition. The command is simple:
# resize2fs /dev/mapper/vg00-varNote: I just tried this and it didn’t work. The CentOS documentations says the the code for ext2online (which was the old method of extending ext2/ext3 file systems) had been merged into resize2fs. My system uses xfs and not ext2/ext3 so I had to use:
xfs_growfs /dev/vg00/varThe command gives you the following output, which is very similar to what you get when you are creating the xfs filesystem.
meta-data=/dev/vg00/var isize=256 agcount=4, agsize=262144 blks = sectsz=512 attr=2 data = bsize=4096 blocks=1048576, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 1048576 to 2097152I have decided to create a brand new logical volume that I can resize independently. The logical volume group already exists so I don’t need to create that. Even though I have 2 physical disks they are joined together as 1 mirror and used as 1 entity in the volume group so I can’t specify the
-ioption to have the volume group manager organise writes to both disks. I don’t think it matters it you do the mirroring up in the volume group level or on the disk partition level. We issue the logical volume create command and set it to 10GB
# lvcreate -n mysql -L10G vg00 Logical volume “mysql” createdA quick check shows that everything is ok:
# ls -l /dev/mapper/ total 0 crw——- 1 root root 10, 63 Jan 31 22:30 control brw-rw—- 1 root disk 253, 2 Jan 31 22:31 vg00-home brw-rw—- 1 root disk 253, 3 Feb 7 21:11 vg00-mysql brw-rw—- 1 root disk 253, 0 Jan 31 22:31 vg00-usr brw-rw—- 1 root disk 253, 1 Jan 31 22:31 vg00-var # lvdisplay /dev/mapper/vg00-mysql — Logical volume — LV Name /dev/vg00/mysql VG Name vg00 LV UUID LQpgnU-Of2d-y3LP-Mj2n-0FH1-CaGd-1OW0kQ LV Write Access read/write LV Status available # open 0 LV Size 10.00 GB Current LE 2560 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:3Note: I tried to create a logical volume called “scratch-backup", but when I checked the
/dev/mapperit listed the group name as
vg00-scratch--backup. In my opinion anything unexpected is bad so I want to delete this logical volume and pick a different name.
lvremove /dev/mapper/vg00-scratch–backupThis new logical volume is blank so I need to create a file system in it. Not sure of all the magic options so I’ll let the system use its defaults:
# mkfs -t xfs /dev/vg00/mysql meta-data=/dev/vg00/mysql isize=256 agcount=16, agsize=163840 blks = sectsz=512 attr=0 data = bsize=4096 blocks=2621440, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=2560, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0The final disk test is to make sure we can mount it into the file system.
# mount /dev/vg00/mysql /mnt # df -h Filesystem Size Used Avail Use% Mounted on /dev/md1 3.7G 332M 3.4G 9% / /dev/mapper/vg00-usr 4.0G 1.1G 3.0G 26% /usr /dev/mapper/vg00-var 4.0G 2.1G 2.0G 52% /var /dev/mapper/vg00-home 4.0G 2.1G 2.0G 51% /home /dev/mapper/vg00-mysql 10G 4.6M 10G 1% /mntI want this file system to be mounted when the machine boot’s up so I must add it into the
/etc/fstabfile. Now I can start the process of copying my existing databases into the new partition and we are done. Primary documentation site for LVM http://tldp.org/HOWTO/LVM-HOWTO/commontask.html Resize instructions http://wiki.centos.org/TipsAndTricks/ExpandLV Clear(ish) description of concepts but mainly talks about the management via the system GUI. http://www.linuxtopia.org/online_books/centos5/centos5_administration_guide/centos5_ch-lvm.html Some 1&1 documentation has finally materialised! http://faq.1and1.co.uk/server/root_server/linux_admin_help/7.html
Comment from: tlp [Visitor]
Hey Mr N., your blog post is exactly what I was googling for !
I “digg” the post (first digg of my life, wow) to thank you :-)
Cheers from Mexico !
I second and third the other two comments - having scratched my head for a whole day as to how to do this on an unfamiliar OS (with a ludicrous 1.3 TB of unused filesystem!) we now have 1&1 server behaving properly…
Thanks for taking the time to explain things, too, as it really helps.
Comment from: Dan [Visitor]
Thank you very much for the article!
I just got a 1and1 server and was wondering whether they messed something up again (as they frequently do). My previous server from 1and1, which I had about 2 years ago, had all the space allocated (most of it to
/var), so I didn’t have to change anything…
Comment from: Chris T [Visitor]
You guys do know that they provide a step by step guide to do this very easily, credit to taking the time to create the guide above.
Comment from: [Member]
1&1’s documentation has been appalling in recent years and at the time I wrote this I couldn’t find any articles written by 1&1, so I had to write my own. There are no dates in their documentation so I can’t tell if the article you are referring to was written last year or yesterday. 1&1’s documentation is getting better.
I hope that my article offers something extra. It also gives you information on how to discover what the system comes with, practical advice on whether to extend a volume or create a new one and help with the maths you need to get the correct number of extends.
Thanks for the link though, I’ll add it to the article in the extra help section.
Comment from: chubba [Visitor]
Thanks for the wicked article! Saved me a lot of time. Any thoughts as to why they are set up like this? Is there less disk activity if the volumes are kept small? Is it worth gradually increasing the size as data grows in size or is it best to just utilise all the disk space available?
I have one of the L4 Core i machines with 1TB and have opted to just put 700GB on the /var/ volume, and 100 each on the /home and /usr partitions and this will still give some room should I need to bump up root or tmp or just add more to the already-increased ones.
I do monitor my machines with NAGIOS but really, do I want to have to do this every few months or is any performance increase worth adding space gradually?
Comment from: [Member]
I think they are set up like this to give you the maximum flexibility and give you the most amount of unallocated disk space. There is a bit of breathing room to get you up and running, but if you’re running a busy production server then you’ll need to do a bit of logical volume management within the first few months.
The main reason these days for using partitions is that of reliability and fault tolerance. Each partition has it’s own file system root with super-block structure so if there is a media failure or system crash then you have limited the amount of damage that can be done to the file systems. There is an overhead to having more partitions as the kernel needs to keep all these structures in memory but in my opinion it is worth it. Disk caching works on a file system bases too, so that could help with performance at the cost of more RAM.
If you run a busy server system to house something like Oracle Financials, SAP or even resellers web server it’s not easy to predict how the system will grow so having an amount of unallocated disk space has a real benefit. A good example of this is home directories
/home would be a single partition. Half the people at my company seem to be called David (which is a great name btw!) so you may want to create a new partition for
/home/d to put all the Davids into e.g.
/home/d/davidc. There will come a day when
/home/d has consumed all the physical disk space and you’ll need to buy another. All the allocated
/home/d can go back in the pot and you can mount the new disk under
/home/d. Or you could add the new disk to the logical volume pool and pimp it’s extends to create a new partition but there are pros and cons with this solution too.
Volume managers are pretty clever these days, so it’s less important to “grow” the system as it used to be. I’m a bit old school so when I needed an extra 10GB as a scratch working area to hold mysqldump files during the backup or wanted to add a bit of extra swap space to speed up the system then I had it immediately available and I could place it anywhere I wanted without having to add symbolic links. If you needed the extra web space under an Apache document route then Apache’s configuration has to change to traverse symbolic links and everything can get a bit more complicated.
In answer to your question, yes you can just split it all up because you can reduce the size of a partition later. I like to see where the space is going so that there aren’t any surprises and I have some extra space in my back pocket for emergencies.
Comment from: JC [Visitor]
Comment from: Frederique [Visitor]
Thanks for your help : it worked just fine for my /usr which was 100% used :)
Just a comment though : the command : “lvm lvresize -l 10240 /dev/mapper/vg00-var” extended the volume to 10x the initial size (which was 4Go) making it 40 Go and not extended the volume up to 10 Go as explained in your article… but the command is the same to reduce the volume so… no problem after all :p Cheers from France.
Form is loading...