Linux LVM Cookbook with Examples

I often manage LVM volumes on Linux servers. LVM is the best way to manage on growing disk storage demands on Linux servers running on virtual infrastructure platform; the beauty is – the whole disk administration process is ONLINE without server reboot or any outage to running services. Here is below my LVM cookbook with examples –

Before working with LVM – make sure you are experienced with disk administration utilities such as “fdisk” and Linux file systems.

Before moving forward – let’s get familiar with LVM terminologies –

LVM Terminology Descriptions
Physical Volume (PV) A PV is the actual physical disk or a physical disk partition to be used in LVM. The disk partition ID for LVM PV is “8e”; whereas standard Linux partition ID is “83”. Example :/dev/sdc – the whole disk cab be a LVM PV/dev/sda6 – the logical partition sda6 can be a LVM PV
Volume Group (VG) A VG is a pool of Physical Volumes (PV); a VG can have a single PV only or bunch of PVs bundle together. Member PV disk size can be same or different in a VG. VGs are identified by unique names within a Linux server. Example:VolGrp00 – actual location is /dev/VolGrp00VolGrp01 – actual location is /dev/VolGrp01
Logical Volume (LV) LVs are the logical/virtual partitions in Linux systems. Linux file system mount points are assigned against LVs. A single LV or multiple LVs can be created on a single Volume Group (VG). A LV name is unique within a VG. Example:/dev/VolGrp00/LogicalVol00 can be mount as /usr/dev/VolGrp00/LogicalVol01 can be mount as /var/dev/VolGrp01/LogicalVol00 can be mount as /mydb
Physical Extents (PE) PEs are physical disk blocks in LVM. The size of PE is often in megabytes. Larger PE size allow large Volume Group (VG); i.e. PE size 32M support a VG up to 2TB, 64MB PE size support a VG up to 4TB, 256MB PE size support a VG up to 16TB. Once a VG is created, the PE size cannot be changed; that’s why it is important to decide PE size before creating LVM.
Logical Extents (LE) LEs are logical disk blocks in LVM. LEs have the same size as PEs.

1. Let’s start with the “creation” of LVM-

The LVM creation process involves five tasks –

  • create PV on physical disk or on a free disk partition
  • then create VG on PV(s); specify PE size if required
  • create LV(s) on VG(s)
  • create Linux fie system (ext3 ot ext4 or other) on LV
  • mount the LV disk to a Linux directory/mount point.

Following table describe the creation tasks Linux commands –

LVM Terminology Linux Commands Description
Physical Volume #pvcreate /dev/sdbor#pvcreate /dev/sdc3 The first command will create PV on the whole disk /dev/sdb.The second command will create PV on the disk partition /dev/sdc3. Other partitions on the same disk /dev/sdc might not be LVM PV – and it’s ok.
Logical Volume #vgcreate MyVolGrp00 /dev/sdbor#vgcreate MyVolGrp00 /dev/sdb –s 32Mor#vgcreate VolGrp01 /dev/sdb /dev/sdc3 /dev/sda7 –s 64M The first command will create VG named MyVolGrp00 on PV /dev/sda. The PE size is based on Linux system default – if you don’t specify.The second command will create VG named MyVolGrp00 on PV /dev/sdb with PE size of 32MB. This VG can grow upto 2TB.The third command will create VG named VolGrp01 on multiple PVs with PE size of 64MB. This VG can grow upto 4TB.
Logical Volume #lvcreate -l +100%Free MyVolGrp01 –n LVol00or#lvcreate –L 10G VolGrp00 –n LVol01 The first command will create LV LVol00 on MyVolGrp01 VG – and will consume the whole size of the VG.The second command will create LV LVol01 on VolGrp00 – the LV size is 10GB; if any free space available you can create more LVs on the same VG VolGrp00.
Fie System #mkfs.ext3 /dev/VolGrp00/LVol01#mkfs.ext3 /dev/VolGrp02/LVol00 This will create Ext3 file system on LVol01 & LVol00.
Mount Point #mount /dev/VolGrp00/LVol00 /mymountpoint For auto mount – add mount commands onto > /etc/fstab.

2. LVM – Display Details

Following are Linux commands to display detailed information about LVM,

LVM Term Linux Commands Description
PV #pvdisplay#pvdisplay /dev/sdc The first command will display all available PVs.The second command will display PV on disk /dev/sdc.
VG #vgdisplay#vgdisplay VolGrp01 The first command will display details of all available VGs. You can find the PE size here.The second command will display details of VolGrp01 only.
LV #lvdisplay#lvdisplay /dev/VolGrp00/LVol00 The first command will display details of all available LVs.The second command will display details of /dev/VolGrp00/LVol00 only.

3. LVM – Disk Expansion

LVM disk expansion process works as following –

-you can expand a logical volume (LV) only if there available free space on underlying volume group (VG).

-if no free space available on volume group (VG) – then (i)add new physical disk, convert it to PV then add to VG or (ii)extend existing PV physical disk; this is very common on virtual environment.

-finally tell Linux file system (ext3/ext4) to recognize and work with the new size.

The whole operation can be done ONLINE. Following table describe Linux commands for LVM disk expansion operation –

LVM Terminology Linux Commands Description
Physical Volume (PV) #pvresize /dev/sdc The first command will grow PV to all available free space on /dev/sdc physical disk – this is useful when we expand an existing hard disk on virtual machine.For “pvresize” nothing need to done on VG – VG will automatically recognize new disk size after the “pvresize” command.
#pvcreate /dev/sda6or#pvcreate /dev/sdd These commands will create a new PV to be added to existing VG; these commands are same as create a new PV for a new VG.New PV need to be added to VG as a part of VG extend.
Volume Group (VG) #vgextend VolGrp02 /dev/sdeor#vgextend VolGrp02 /dev/sde  /dev/sdf1 The first command will extend the VG VolGrp02 to new PV /dev/sde.The second command will extend the VG VolGrp02 to new multiple PVs /dev/sde & /dev/sdf1.
Logical Volume (LV) #lvextend –l +100%Free /dev/VolGrp02/Vol00or#lvextend –size +20G /dev/VolGrp02/Vol00or#lvextend –size 800GB /dev/VolGrp02/Vol00 The first command will extend the LV to all available free space on the VG /dev/VolGrp02.The second command will add additional 20GB space to the LV /dev/VolGrp02/Vol00 – make sure you have 20GB free space on the VG.The third command will make the LV /dev/VolGrp02/Vol00 size to 800GB (new final size) – make sure you have available space on the VG to have this 800GB LV.
File System #resize2fs /dev/VolGrp02/Vol00 You need to tell the Linux file system to recognize the new size ONLINE – this command will do that for LV Vol00. No system reboot or file system re-mount required for this.

4. LVM – Delete Operations

Before delete a logical volume LV make sure it is not mounted. Also delete operation will destroy all the data – so be careful before delete.

Command examples are following –

#lvremove /dev/VolGrp03/Vol01 ; this will remove LV “/dev/VolGrp03/Vol01”

#vgremove VolGrp03 ; this will remove VG “VolGrp03”

#pvremove /dev/sdc ; this will remove PV “/dev/sdc”

5. LVM – Disk Shrink

Disk size shrink of a LV can be done without deleting & recreating it – you might experience data loss while shrinking if not planned well (I experienced data loss and restored data from backup). Make sure you have data backup and the LV is not mounted before you move ahead. The new shrinked size must not be below the volume used size. Command examples are following –

Unmount the LV first,

#umount /dev/VolGrp02/Vol00   ; this will unmount “/dev/VolGrp02/Vol00”

Run file system check and make sure the file system is OK before moving ahead,

#e2fsck –f /dev/VolGrp02/Vol00

Reduce the file system size first (without losing data),

#resize2fs /dev/VolGrp02/Vol00 30G; this will reduce the current file system size to 30GB (let’s say the LV size is 50GB – and it has less than 30GB of data) – make sure volume usage is less than <30GB otherwise you will lose data.

Reduce the Logical Volume now,

#lvreduce –L 35G /dev/VolGrp02/Vol00; this will shrink the LV to 35GB (around 30GB of data + 5GB free space)

Finally resize the file system again to 35GB,

#resize2fs /dev/VolGrp02/Vol00

In this above example – our target is reducing a LV from 50GB to 35GB. The first “resize2fs” we shrink the file system from 50GB to 30GB (where data usage is less than 30GB out of total 50GB disk size) and then resize the Logical Volume to 35GB. This estimation helps eliminate data loss.

6. LVM – “scan” Commands

There are three scan commands – #pvscan, #vgscan & #lvscan.

LVM scan commands scan all the available PV/VG/LV in the system and build the LVM cache file “/etc/lvm/.cache” to maintain a listing of current LVM devices.

A Linux system auto execute “scan” commands every time we restart/power-on server and also during LVM create/expand/resize/reduce operations.

Windows NTP Server – Windows NTP Cookbook

When we talk about standard NTP server that could provide time sync to cross-platform infrastructure objects such as Windows, Linux, Unix, Cisco, HP, F5, VMware etc – a lot of us prefer Linux based NTP server. However, a Windows based NTP server is also equally capable of providing cross-platform time sync just like a Linux NTP and very easy to configure. I assume when people think of cross-platform – they think Windows NTP is only good for Microsoft environment which actually not true.

I often do install and configure centralized NTP sync for customers. Previously I was always stick to Linux based NTP – however, few Windows shop customers pushed me to find a suitable solution for the same on Windows server, hence I come across this; I found this is working perfectly and very stable. There is no 3rd party software required to get this done on Windows 2003/2008/2012 Servers.

Here is below step-by-step configuration – most of these configurations are based on registry settings – to edit registry use “regedit” utility.

In this doco I have discussed the following –

1. How to configure a stand-alone NTP server on a Windows 2008/2012 Server
2. How to configure NTP service on Window AD DS environment
3. How to configure NTP sync to a non-domain joined Windows computer acting as  a NTP client.

1. Configure a stand-alone NTP server on a Windows 2008/2012 Server

Following configuration will enable “NTP server” service on a stand-alone Windows 2008/20012 Server (this is equivalent as a stand-alone Linux based NTP server) –

i. Enable “NTP server” service on the machine-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer

Set the value data to: 1Default value is 0 – which means NTP server service is not enabled.

ii. Change the server type to NTP on this machine-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Type

Set the value data to: NTPThis is “standard” NTP server that can provide time sync to cross-platform. Other value for this is “NT5DS” – which depends on active directory.

iii. Set the announce flag-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\AnnounceFlags

Set the data value to: 5; value 5 means – sync time to an external time source. Default value is 10 – this tells the server to sync time to local CMOS clock.

iv. Specify external time sync peers-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\NtpServer

Set the value to: 0.pool.ntp.org,0x01 1.pool.ntp.org,0x01 2.pool.ntp.org,0x02 (these are free public NTP servers on the Internet) or your preferred external NTP servers. Make sure you maintain a white space between servers.

The “0x01” flag indicate sync time with external server in special interval configured in “SpecialPollInterval” registry value.

Value “0x08” means – use client mode association while sync time to external time source.

Value “0x09” means – use special interval + client mode association to external time source. This is a good value when your machine sync time to an external time source.

Value “0x02” means – use this as UseAsFallbackOnly time source – if primary is not available then sync to this server.

Value “0xa” means – UseAsFallbackOnly + client mode association.

v. Set time sync pool interval (special interval) –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpClient\SpecialPollInterval

Set the value to: 900; Microsoft TechNet & lots of other reference documents recommend a value of 900 seconds (every 15 min).

vi. Set the time correction settings-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxPosPhaseCorrection

Set the value to: 3600; Microsoft recommends a value of 3600 or 1800 seconds. Default value is 54000.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxNegPhaseCorrection

Set the value to: 3600Microsoft recommends a value of 3600 or 1800 seconds.

Original description for this time correction is > “Specifies the largest positive time correction, in seconds, that the Windows Time service is allowed to make. If the service determines that a change larger than this is required, then the service logs an event instead”.

vii. Make sure to restart windows time server called “w32time”-

>net stop w32time

>net start w32time

viii. Make sure to start Windows Time service to start automatically with operating system reboot.

Services.msc > Windows Time > Startup Type > Automatic.

At this stage all the required registry settings are DONE – this server is ready to serve as a standard NTP server that can provide time sync to cross-platform.

Let’s verify NTP server configurations are entered correctly and the NTP server is syncing time to external source correctly –

>w32tm /query /status /verbose; this will display last sync status or any error

w32tm-query

>w32tm /query /peers; this will display NTP external peers

>w32tm /query /source; this will display current NTP time source

>w32tm /query /configuration; this will display current configurations

>w32tm /resync; this will force immediate time resync

2. Configure NTP Service on Window AD DS Environment

Windows Active Directory time sync works a bit differently – not all the domain controllers are responsible to sync time to external time sources. Only the domain controller have the PDC emulator role sync time to external time sources. NTP server service is enabled by default on PDC emulator.

Active Directory Domain Service time sync flow is following –

External time sources>> Domain Controller with PDC emulator>> all other Domain Controllers>> all domain members.

The external time sources can be the standalone NTP server just been described in the above section 1 – or this can be NTP servers sitting on the Internet. Make sure you have outbound internet connection allowed for the DC with PDC role – also windows firewall not blocking NTP in/out on this DC.

To find out which domain controller is PDC emulator – execute “Netdom Query FSMO” command –

netdom-pdc

Following settings will make the PDC emulator DC to sync time to external time source –

>w32tm /config /update /syncfromflags:manual /manualpeerlist:myntp01.test.local,0x09

Or edit the registry value “NtpServer” and enter NTP servers DNS address.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\NtpServer

The above command tells the server to sync time to myntp01.test.local.

“0x09” flag tells the server to use a client-mode association with special interval.

>w32tm /config /reliable:yes /update

This command tells the PDC emulator server to mark itself as reliable time source to domain member computers.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpClient\SpecialPollInterval

Set the value to: 900; Microsoft & lot of other documents recommend a value of 900 seconds (every 15 min).

Set the time correction settings –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxPosPhaseCorrection

Set the value to: 3600; Microsoft recommends a value of 3600 or 1800 seconds. Default value is 54000.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxNegPhaseCorrection

Set the value to: 3600; Microsoft recommends a value of 3600 or 1800 seconds.

Finally restart windows time service.

>net stop w32time

>net start w32time

Configure all other domain controllers to use time sync from Active Directory Hierarchy automatically. The following commands need to be executed on every non-PDC emulator domain controllers –

>w32tm /config /syncfromflags:domhier /update

At this stage ALL AD DS domain controllers are ready to serve time sync to all domain members.

Verify configuration by using “w32tm /query /status /verbose” and also check “system logs” in the “event viewer” for any w32time warning or error.

No configuration need to be done on domain members – if you execute “w32tm /query /source” on a domain member – this should return FQDN of a domain controller.

3. Configure NTP Sync to a Non-Domain Joined Windows Computer as  NTP Client

Sometimes you might find non domain join Windows computers that need be to configure time sync to NTP server(s). The NTP server can be the one configured at section 1 or can be NTP servers sitting on Internet – make sure windows firewall configuration allow NTP sync.

Change the following registry value to configure time sync  to an external server –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\AnnounceFlags

Set the data value to: 5; Default value is 10.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpClient\SpecialPollInterval

Set the value to: 900; Microsoft recommends a value of 900 seconds (every 15 min).

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\NtpServer

Enter myntp01.test.local,0x09 or your preferred external NTP server address; this can be IP address instead of DNS name as well.

You can also enter “time correction” registry values described in the above sections.

Now restart windows time service –

>net stop w32time

>net start w32time

Check Windows event logs and “w32tm /query” commands to make sure time sync is working fine.

4. Reset NTP Registry Settings to the Default

There might be some situation when you need to reset NTP related registry settings to Windows default values; following are the commands –

>net stop w32time

>w32tm /unregister

>w32tm /register

>net start w32time