Archive for the ‘Linux’ Category

wget behind proxy

Friday, March 26th, 2010

One of the ways to get wget to work behind proxy is to create .wgetrc file in the user’s home directory. In it put this:

http_proxy = http://proxy_server_yourdomain.com:8080/

if your proxy is using authentication, add also the following lines to your .wgetrc file

proxy_user=user
proxy_password=password

Fix failed LVM partition

Monday, November 9th, 2009

These instructions are made by using Knoppix Live CD or other Knoppix boot media. During recent reboot, I got the following message on the screen:

No devices found
Setting up Logical Volume Management: No Volume groups found
[ OK ]
Checking filesystems
fsck.ext3: No such file or directory while trying to open /dev/VolGroup00/LogVol00
/dev/VolGroup00/LogVol00:
The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem ( and not swap or ufs or something else), then the superblock is corrupt, and you might thy running e2fsck with and alternate superblock:
e2fsck -b 8193

[FAILED]

*** An error orrured during the file system check.
*** Droping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D to continue):

And this does not looks good, but it may just mean that your system did not shut down cleanly. On the computer with the lvm partition that has a problem, boot with Knoppix boot media and become root (sudo su -)

Then run vgscan to find LVM volumes:

root@Microknoppix:~# vgscan
Reading all physical volumes. This may take a while…
Found volume group “VolGroup00″ using metadata type lvm2

and vgchange -a y to make LVM volumes active:

root@Microknoppix:~# vgchange -a y
2 logical volume(s) in volume group “VolGroup00″ now active

List the LVM volumes:

root@Microknoppix:~# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup00 -wi-a- 72.44G
LogVol01 VolGroup00 -wi-a- 1.94G

Now we can run fsck on the LVM volume that had a problem:

root@Microknoppix:/# fsck /dev/mapper/VolGroup00-LogVol00
fsck 1.41.3 (12-Oct-2008)
e2fsck 1.41.3 (12-Oct-2008)
Adding dirhash hint to filesystem.

/dev/mapper/VolGroup00-LogVol00 primary superblock features different from backup, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/mapper/VolGroup00-LogVol00: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/VolGroup00-LogVol00: 204004/19005440 files (1.5% non-contiguous), 3271409/18989056 blocks

another fsck to verify that everything is good this time:

root@Microknoppix:/# fsck /dev/mapper/VolGroup00-LogVol00
fsck 1.41.3 (12-Oct-2008)
e2fsck 1.41.3 (12-Oct-2008)
/dev/mapper/VolGroup00-LogVol00: clean, 204004/19005440 files, 3271409/18989056 blocks

check VG:

root@Microknoppix:/# vgdisplay
— Volume group —
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 74.38 GB
PE Size 32.00 MB
Total PE 2380
Alloc PE / Size 2380 / 74.38 GB
Free PE / Size 0 / 0
VG UUID o8wP8s-605P-cP85-EnQh-r73x-bElI-27Ldub

mount the volume:

root@Microknoppix:~# mount /dev/mapper/VolGroup00-LogVol00 /mnt
root@Microknoppix:~# ls -la /mnt
total 224
drwxr-xr-x 26 root root 4096 Nov 6 20:21 .
drwxr-xr-x 15 root root 0 Nov 9 16:29 ..
-rw-r–r– 1 root root 0 Feb 5 2009 .autorelabel
drwxr-xr-x 3 root root 4096 Nov 5 23:04 app
drwxr-xr-x 2 root root 4096 Jun 18 08:11 bin
drwxr-xr-x 2 root root 4096 Feb 5 2009 boot
drwxr-xr-x 2 root root 4096 Feb 6 2009 cstr
drwxr-xr-x 4 root root 4096 Feb 5 2009 dev
drwxr-xr-x 109 root root 12288 Nov 6 20:21 etc
-rw-r–r– 1 root root 0 Nov 6 20:20 halt
drwxr-xr-x 4 root root 4096 Nov 5 23:03 home
drwxr-xr-x 12 root root 4096 Sep 30 19:53 lib
drwxr-xr-x 7 root root 12288 Oct 1 08:03 lib64
drwx—— 2 root root 16384 Feb 5 2009 lost+found
drwxr-xr-x 2 root root 4096 Mar 11 2009 media
drwxr-xr-x 2 root root 4096 May 25 2008 misc
drwxr-xr-x 2 root root 4096 Mar 11 2009 mnt
dr-xr-xr-x 2 root root 4096 Feb 6 2009 net
drwxr-xr-x 3 root root 4096 Mar 11 2009 opt
drwxr-xr-x 2 root root 4096 Feb 5 2009 proc
drwxr-x— 7 root root 4096 Nov 6 20:20 root
drwxr-xr-x 2 root root 12288 Oct 1 08:03 sbin
drwxr-xr-x 2 root root 4096 Mar 11 2009 selinux
drwxr-xr-x 2 root root 4096 Mar 11 2009 srv
drwxr-xr-x 2 root root 4096 Feb 5 2009 sys
drwxrwxrwt 11 root root 4096 Nov 6 20:20 tmp
drwxr-xr-x 16 root root 4096 Nov 5 22:56 usr
drwxr-xr-x 26 root root 4096 Jun 17 17:04 var
-rw-r–r– 1 root root 1658 Oct 5 17:02 xfs_8p_4k.wks

everything looks good so far, so I unmounted /mnt partition, run sync couple of times and rebooted without Knoppix media

Create bootable Linux USB drive

Monday, November 9th, 2009

It is very easy, thanks to Knoppix. On the http://www.knopper.net/knoppix/knoppix60-en.html page is described how to create bootable Knoppix USB (or other flash media) drive (and it works).

Booting from Flash

In order to create a bootable USB-medium (memory flashdisk, SD-card, digital camera with USB connector, cellphone with microSD, …), the program flash-knoppix can be started from a running Knoppix system. This program installs all needed Knoppix files onto the FAT-formatted flashdisk, and creates a boot record for it. Other files on the medium stay intact, since no repartitioning or formatting is done by flash-knoppix.

After having copied the system to flash, using the persistent Knoppix image (overlay feature), it is possible to also store files permanently in live mode. That way, personal settings and additionally installed programs survive a reboot.

flash-knoppix is supposed to be able to install Knoppix on existing FAT partition, that didn’t work for me. I had to partition USB flash drive and install vfat file system on it.
So, to do that, with fdisk -l you can see what is device name that is assigned to your USB flash drive (/dev/sdb for example), then you need to partition it (fdisk /dev/sdb (if /dev/sdb is device name for your USB drive)) use the menu to remove all partitions, and then create at least 1 partition in fdisk, save partition table and exit fdisk. Then create vfat partition by running mkfs.vfat /dev/sdb1 and then run ‘flash-knoppix /dev/sdb1′.
On the 2GB USB flash drive, I have created 2 x 1GB partitions, one for Knoppix and one saving my data. When you boot from Knoppix bootable USB flash drive, you’ll be asked to specify amount of space you want to dedicate for saving your data.

Disk partition aligment

Monday, November 2nd, 2009

Misaligned partitions can affect disk I/O peformance by 30% or more for random I/O. Test on this page http://sqlblog.com/blogs/linchi_shea/archive/2007/02/01/performance-impact-of-disk-misalignment.aspx confirm that.

The article below is from “EMC Host Connectivity Guide for Linux” and is described for CLARiiONs, but it equally applies to LUNs provided by other storage arrays:

File-system misalignment affects performance in two ways:

? Misalignment causes disk crossing: an I/O broken across two
drives (where normally one would service it).
? Misalignment makes it hard to stripe-align large uncached writes.

The first case is more commonly encountered. Even if the disk
operations are buffered by cache, the effect can be detrimental, as it
will slow flushing from the cache. Random reads, which by nature
require disk access, are also affected, both directly (waiting for two
drives in order to return data) and indirectly (making the disks busier
than they need to be).
Alignment issues only affect striped LUNs, so RAID 1 is not affected.
However, two disk RAID 1/0 groups are now allowed. They will use
striping once expanded to four or more disks and should be aligned
from the outset.
For greater detail on this and other performance issues regarding the
CLARiiON array, please refer to Powerlink and search for EMC
CLARiiON Best Practices for Fibre Channel Storage.
Aligning the partition
In Linux, align the partition table before data is written to the LUN as
the partition map will be rewritten and all data on the LUN
destroyed. In the following example, the LUN is mapped to
/dev/emcpowerah, and the LUN stripe element size is 128 blocks.
Arguments for the fdisk(8) utility are as follows:
root #> fdisk /dev/emcpowerah

enter x expert mode
enter b adjust starting block number
enter 1 choose partition 1
enter 128 set it to 128, our stripe element size
enter w write the new partition

This method is preferable to the Navisphere LUN alignment offset
method for LUNs that will have a snapshot, BCV, or MirrorView
image made of them. It is preferred for SANCopy sources and targets
as well.

Here is process for partition alignment for windows (from http://technet.microsoft.com/en-us/library/aa995867%28EXCHG.65%29.aspx):

To align Exchange I/O with storage track boundaries using Diskpart.exe

1.

If the disk you are aligning is already blank (raw), proceed to Step 3. If the disk contains data, backup the disk before proceeding.
2.

Delete all partitions on the disk.
3.

Open a command prompt, and execute Diskpart.exe.
4.

At the Diskpart command prompt, type List Disk and press Enter. If the disk you want to align does not appear in the list make sure that it exists and is accessible using the Disk Management snap-in.
5.

At the Diskpart command prompt, type Select Disk X, where X is the number of the disk as shown in the output of the List Disk command. Diskpart should return a message that indicates that Disk X is the selected disk.
6.

At the Diskpart command prompt, type Create Partition Primary Align=X, where X is either 32 or 64, depending on the recommendation from your storage vendor. If your storage vendor does not have any specific recommendations, it is recommended that you use 64.
7.

At the Diskpart command prompt, type Assign Letter=. For example, to assign letter Z to the disk, type Assign Letter=Z.
8.

Once the drive letter is assigned, type exit to exit out of the Diskpart utility.
9.

Use the Disk Management snap-in or the Windows Format command to format the partition as an NTFS-formatted partition.

Vista seems to be aligning its disk partitions while XP 32-bit and 64-bit are not.

Going to Lisa '09

Monday, October 26th, 2009

For those who didn’t know, Lisa ’09 is in Baltimore, MD from Nov. 2nd to Nov, 6th. Nov. 4th (Data Storage Day) is free http://www.usenix.org/events/lisa09/dsdl09.html
I'm going to LISA '09

Flush DNS resolver cache on OS X, Linux, Solaris and Windows

Wednesday, October 21st, 2009

OS X: dscacheutil -flushcache

Linux (CentOS, RHEL): /sbin/service nscd restart

Solaris: ps -eaf|grep nscd

kill -9 PID   ### PID is process ID from the previous step

/usr/sbin/nscd

Windows (from command line of course): ipconfig /flushdns (run ipconfig /displaydns to view the DNS resolver cache)

how to install ext4 on CentOS 5.3 (and RHEL 5.3)

Wednesday, October 21st, 2009

Download ext4 utilities

# yum install e4fsprogs

ext4 utilities are renamed so they don’t conflict with the existing utilities:  e4fsck   e4image  e4label  e4undo

Create ext4 file system

# mkfs.ext4 /dev/mapper/testvg-testvol
mke4fs 1.41.1 (01-Sep-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
6520832 inodes, 26076160 blocks
1303808 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
796 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 21 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Once the filesystem is created, it can be mounted as follows:

# mount -t ext4 /dev/mapper/testvg-testvol /testfs

verify that it is mounted properly:

# mount
/dev/mapper/vg0-root on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/mapper/testvg-testvol on /testfs type ext4 (rw)

enjoy ext4

rsync over ssh

Wednesday, October 21st, 2009

This example show how to rsync files over ssh running the rsync on the destination host:

rsync -avz -e ssh remote_user@remote_host:/remote/dir /local/dir/

in this case you’d have to enter remote_user password. If you are running rsync on a regular basis, create ssh keys with no password and copy public key to the .ssh/authorized_keys file on the remote user you are connecting as on the remote host.

You can also copy multiple files or directories from the destination host:

rsync -avz -e ssh  remote_host:/remote_dir/file1 remote_dir2/file2 /local/dir

Meaning of options used (from rsync man page):

-a, –archive               archive mode; same as -rlptgoD (no -H)

-z, –compress              compress file data during the transfer

-v, –verbose               increase verbosity

-e, –rsh=COMMAND           specify the remote shell to use

installing php and its modules on CentOS 5.3

Wednesday, October 21st, 2009

First update PHP:

# yum -y update php
Then install PHP modules, I use the following ones:

yum -y install php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc

and it is done. Restart apache. Create a page called info.php or whatever you want that ends with .php and put this in it:

<?php

phpinfo();

phpinfo(INFO_MODULES);
?>

save it and access this page from your web browsers, it will show you php config on your server and all php modules installed. If for some reason you get txt output of the file in your browser, your Apache server may not be configured to handle .php files, and in that case edit your httpd.conf  (on CentOS 5.3 default location is: /etc/httpd/conf/httpd.conf) and make DirectoryIndex  line to look like this:

DirectoryIndex index.html index.cgi index.pl index.php index.xhtml index.htm

that should take care of most of the file types you’ll use in your web server

Sending emails with subject line by using sendmail

Monday, October 12th, 2009

(echo “Subject: e-mail subject here” ; cat your_file ) | sendmail admin@xyz.com

or if you want text to be inside command line:

(echo “Subject: e-mail subject here” ; echo “test”) | sendmail admin@xyz.com