The all powerful find command

find <starting point> <search criteria> <action>

The starting point is the name of the directory where find should start
looking for files. The find command examines all files in this
directory (and any subdirectories) to see if they meet the specified
search criteria. If any do, find performs the specified action on each
found file. Here are some of the most useful search criteria options:

-name pattern Find files with names that match the pattern.
-size [+|-] n Find files larger or smaller than a certain size.
-atime [+|-] n Find files accessed before or after a certain date.
-mtime [+|-] n Find files modified before or after a certain date.
-type filetype Find only regular files or only directories.

And here are the actions that can be applied to found files:

-print Print just the names of matching files.
-ls Print the names, dates, sizes, and so on of matching files.
-exec command Execute a command with the file name as input.
-ok command Same as -exec, but asks for confirmation first.

That all might look a bit confusing, so here are some examples to bring
things down to earth. To find files (starting in the current directory)
with names ending with .data and to print their names, try this:

find . -name ‘*.data’ -print
company.data
donor.data
grades.data
sorted.data
words.data

To find files larger than 40K and print the file names and details (use
a minus sign instead of a plus sign to find files smaller than a
certain size), issue this command:

find . -size +40k -ls

To find all files modified within the last 5 days:

find / -mtime -5 -print

The – in front of the 5 modifies the meaning of the time as “less than five days.” The command

find / -mtime +5 -print

To find all files with zero length and ask if they should be deleted:

find / -size 0 -ok rm {} ;

Another Linux Web MySQL Backup Script

I found this on on the nixcraft craft. Looks pretty good to me. Similar
in functionality to my other script posted here. I’m going to poach
some concepts from here to make a script that will auto-optimize all
tables in all databases on the mysql server.

#!/bin/sh
##################
## jcz 17-feb-2007
## copied from http://www.cyberciti.biz/tips/
## how-to-backup-mysql-databases-web-server-
## files-to-a-ftp-server-automatically.html
##############################################
## System + MySQL backup script
## Full backup day – Sun (rest of the day do incremental backup)
## Copyright (c) 2005-2006 nixCraft
## This script is licensed under GNU GPL version 2.0 or above
## Automatically generated by http://bash.cyberciti.biz/backup/wizard-ftp-script.php
## Make full backup every Sunday night
## i.e. backup everything every Sunday
## Next backup only those files that
## have been modified since the full
## backup (incremental backup)
##################################################
## * This is a seven-day backup cycle.
## * It will store data as follows:
## * /home/nixcraft/full/mm-dd-yy/files – Full backup
## * /home/nixcraft/incremental/mm-dd-yy/files – Incremental backup
## * 1. First script will collect all data from both
## * MySQL database server and from file system to
## * temporary directory called /backup using tar command
## * 2. Next, script will login to ftp server and
## * create a directory structure as discussed above
## * 3. Script will dump all files from /backup to ftp server
## * 4. Script will remove temporary backup from /backup
## * 5. Script will send you an email notification
## * if ftp backups failed due to any reason.
##
## * You must have following command installed:
##
## * ncftp ftp client
## * mysqldump command
## * GNU tar command
###########################################
# ———————————————————————
### System Setup ###
DIRS=”/home /etc /var/www”
BACKUP=/tmp/backup.$$
NOW=$(date +”%d-%m-%Y”)
INCFILE=”/root/tar-inc-backup.dat”
DAY=$(date +”%a”)
FULLBACKUP=”Sun”
### MySQL Setup ###
MUSER=”admin”
MPASS=”mysqladminpassword”
MHOST=”localhost”
MYSQL=”$(which mysql)”
MYSQLDUMP=”$(which mysqldump)”
GZIP=”$(which gzip)”
### FTP server Setup ###
FTPD=”/home/vivek/incremental”
FTPU=”vivek”
FTPP=”ftppassword”
FTPS=”208.111.11.2?
NCFTP=”$(which ncftpput)”
### Other stuff ###
EMAILID=”admin@theos.in”
### Start Backup for file system ###
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :
### See if we want to make a full backup ###
if [ “$DAY” == “$FULLBACKUP” ]; then
FTPD=”/home/vivek/full”
FILE=”fs-full-$NOW.tar.gz”
tar -zcvf $BACKUP/$FILE $DIRS
else
i=$(date +”%Hh%Mm%Ss”)
FILE=”fs-i-$NOW-$i.tar.gz”
tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
fi
### Start MySQL Backup ###
# Get all databases name
DBS=”$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse ’show databases’)”
for db in $DBS
do
FILE=$BACKUP/mysql-$db.$NOW-$(date +”%T”).gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done
### Dump backup using FTP ###
#Start FTP backup using ncftp
ncftp -u”$FTPU” -p”$FTPP” $FTPS<
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF
### Find out if ftp backup failed or not ###
if [ "$?" == "0" ]; then
rm -f $BACKUP/*
else
T=/tmp/backup.fail
echo “Date: $(date)”>$T
echo “Hostname: $(hostname)” >>$T
echo “Backup failed” >>$T
mail -s “BACKUP FAILED” “$EMAILID” <$T
rm -f $T
fi

gzip all directories in a directory

I don’t know why I keep having to do this, but I do. I always to create
separate archives of all the directories in a directory. So, here is
script from David (aka Matir aka EmptyCinema) from linuxquestions.org http://www.linuxquestions.org/questions/showthread.php?s=&postid=1839513#post1839513

to do just that (along with a sample sessions using it).

#!/bin/bash
for dir in */
do dir=`echo $dir | tr -d ‘/’`
echo $dir
tar czf $dir.tar.gz $dir
done

[jcz@actinella ~]$ ./zipdir.sh
cdcatalogs
ddclient-3.6.6
Desktop

[jcz@actinella ~]$ ls -lht *.gz
-rw-rw-r– 1 jcz jcz 17M Sep 6 23:31 Desktop.tar.gz
-rw-rw-r– 1 jcz jcz 13K Sep 6 23:31 cdcatalogs.tar.gz
-rw-rw-r– 1 jcz jcz 74K Sep 6 23:31 ddclient-3.6.6.tar.gz

Here is another more advanced version:

#!/bin/bash
# jcz 13-nov-05
# zips (or tar.gz) all directories
# in the directory in which it is run
##################################
echo ” ******************************* “
echo ” this app zips all directories in this directory,”
echo ” tests the created zips for integrity, then “
echo ” copies them to some directory.”
echo ” ******************************* “

# user enters the directory that they want the zips copied to
echo -n “Destination directory for zips e.g. /cygdrive/f/BACKUPS ( . =
here): “
read dest

for dir in */
do dir=`echo $dir | tr -d ‘/’`
echo $dir

# for zipping, -r recurse into directories
# and -u update new or changed files
# echo “zip -r” $dir.zip $dir’/*’
zip -ru $dir.zip $dir
# for gzipping
# tar czf $dir.tar.gz $dir
done

# to test zip file integrity
for zipf in ls *.zip
do zip -T $zipf
done

# move the zips to some directory
for zipf in ls *.zip
do mv -v $zipf $dest
done

Postgis data loader from shapefile bash shell script

Run this script in a directory of shp files to create STDOUT that will load them all into postgis

——————————————————
#!/bin/sh
# jcz aug 24, 2005
# clip off the “.shp” file extensions before use
# drops existing shapes if they are the same name

for z in `ls *.shp
do
echo “shp2pgsql $z $z > $z.sql”
echo “psql -d -f $z.sql”
done

Bash shell script for creating a poor man’s CD-ROM (removable media) catalog for linux

#!/bin/sh
# jcz 2004-jan-12

# assumes iso9660 CD-ROM
mount -t iso9660 -r /dev/cdrom /mnt/cdrom

echo “Disc Mounted. Run this program, then grep keywords in the “
echo “cdcatalogs directory to find which CD-ROM some file “
echo “is on. “

# makes the directory to store the catalog files
mkdir cdcatalogs

# runs volname (part of the eject program) to extract the volume label information
cd=$(volname /dev/cdrom)

# enter user defined CD label (something written on the CD itself)
echo -n “Enter written CD-ROM label and any notes from the disc itself: “
read labler

# trims white space after the name always written out by volname
cdshort=$(echo $cd | sed -e ‘s/[ntr ]*$//’)
echo $cdshort
echo $cdshort”_catalog.txt”
disk=$cdshort”_catalog.txt”
echo $disk
echo $labler
echo “Disk Volume Label: “$cdshort > $disk
echo “Label and Notes on Disc: ” $labler >> $disk
echo ” ——————————————” >> $disk
echo “——– <<<<END DISC ENTRY>>> ————” >> $disk
echo ” ——————————————” >> $disk
echo ” ” >> $disk

# keeps only relevant columns from ls, and date is in a fixed length format which is understandable
# by M$ Office products if needed
ls -ghGR –full-time /mnt/cdrom | awk ‘{ print $1 “t” $3 “t”$4 ” ” $5 “t”$7 $8 $9 $10 $11 }’>> $disk

# fixes the line endings for windows if you want read the catalogs directly in Notepad
unix2dos $disk

# moves file to consistent directory
mv $disk cdcatalogs/

ls -lht cdcatalogs/

umount /mnt/cdrom
# ejects the disk when done to prepare for next disk
eject

Paths on Linux

Today I’m going to do a test install of the J2EE mapserver-like
facilities provided by geoserver version 1.3 Rc2. I recently installed
java and the JDK on this machine, so I still need to set JAVA_HOME in
the path. I do this so rarely everytime I need to do it I have to look
it up.

I use the BASH shell, so all of this applies to BASH on linux.

Type env to see a listing of your curent environment, including your path:

actinella:/opt/java# env
SSH_AGENT_PID=5592
HZ=100
KDE_MULTIHEAD=false
TERM=xterm
SHELL=/bin/bash
GTK2_RC_FILES=/etc/gtk-2.0/gtkrc:/home/jcz/.gtkrc-2.0:/home/jcz/.kde/share/config/gtkrc
GS_LIB=/home/jcz/.fonts
GTK_RC_FILES=/etc/gtk/gtkrc:/home/jcz/.gtkrc:/home/jcz/.kde/share/config/gtkrc
HUSHLOGIN=FALSE
WINDOWID=4194309
KDE_FULL_SESSION=true
USER=root
XCURSOR_SIZE=
SSH_AUTH_SOCK=/tmp/ssh-vBaaqD5569/agent.5569
SESSION_MANAGER=local/actinella.homelinux.org:/tmp/.ICE-unix/5643
KONSOLE_DCOP=DCOPRef(konsole-10017,konsole)
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/usr/local/sbin:/usr/local/bin:/usr/games
MAIL=/var/mail/jcz
PWD=/opt/java
KONSOLE_DCOP_SESSION=DCOPRef(konsole-10017,session-1)
LANG=C
HOME=/root
SHLVL=5
LANGUAGE=us
XCURSOR_THEME=default
LOGNAME=jcz
DISPLAY=:0.0
XAUTHORITY=/home/jcz/.Xauthority
COLORTERM=
_=/usr/bin/env
OLDPWD=/opt

I set JAVA_HOME to point to the directory the extraction just created.
Ie, if I was in /opt/java when I ran the extraction, it would have
created a directory named java, so I would set JAVA_HOME to /opt/java.

export JAVA_HOME=”/opt/java”

You may want to edit your path to set this enironment variable
permanently. To do that (again, assuming the Bash shell), edit
.bash_profile in your home directory, and add the JAVA_HOME setting
there. On my debian system I stuck this in /etc/profile . Ie:

JAVA_HOME=/opt/java

Then change the PATH statement to include the JAVA_HOME setting. Ie, from:
PATH=$PATH:$HOME/bin:/usr/local/iperf

to
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:/usr/local/iperf

Then “source” your copy of .bash_profile to invoke the changes:
source .bash_profile

or in my case

source /etc/profile

then type env to see the changes.

Tips for using the ls command to list files in Cygwin or Linux

Classify

ls –classify or ls -F will append characters to files to show their type:

* / directory
* * executable

Code: ls -F
directory/ me.jpeg script.sh*

ls –color=tty

Will color the ‘ls’ output. Directories are blue, regular files stay black (or white) and executable files are green.

Make an Alias of your prefered method.

Example:

alias ls=’ls –color=tty –classify’

List only directories

ls -d */

Will list only dentries ended by a “/”, and with the “-d” option, will not descend into the next level of directory.

Upgrade your debian sources.list

apt-spy is a program that benchmarks debian apt-get sources. Run this
script when you need to find your bets local mirror for grabbing debian
packages.

#!/bin/sh
# jcz 2005-july-13
#
# This script will add the best debian servers to your sources.list file.
# You should prune the file by hand when it’s done.

# Date and other variables pretty self explanatory, S is seconds
# date format is currently YYYYMMDD_HHMMSS
datearc=$(hostname)_sourcesbackup_$(date +%Y%m%d_%H%M%S).txt

cp /etc/apt/sources.list /etc/apt/sources.list.$datearc

# benchmarks 3 north american servers for downloading the testing debian packages
# and will overwrite sources.list with the results
apt-spy -d testing -a north-america -e 3

# the overwrite makes us copy back our favorites
cat /etc/apt/sources.list.$datearc >> /etc/apt/sources.list

echo “Don’t forget to prune /etc/apt/sources.list !!!”

Changing run levels at boot time

As with most stories on this site, I use my stories to store notes on
things that I keep needing to lookup and that might help others.

This one is how to change the default startup runlevel of a debian (via
knoppix distribution). In this case I want it to stop booting into
graphical mode, or boot into runlevel 3.

The “/etc/inittab” file tells init which runlevel to start the system at and describes the processes to be run at each runlevel.

So, according to

# Default runlevel. The runlevels used by RHS are:
# 0 – halt (Do NOT set initdefault to this)
# 1 – Single user mode
# 2 – Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 – Full multiuser mode
# 4 – unused
# 5 – X11
# 6 – reboot (Do NOT set initdefault to this)

the entry

id:3:initdefault:

would boot into multiuser mode, without X windows starting which is what I want.

Simple grep and search & replace

grep -Hn -e ” int” *.c* *.h

searches for the string “int” files ending in .c* or .h in the the current directory directory

Returns:

! P8.CPP:52: cerr << “cannot allocate int *p1″ << endl ;
! P8.CPP:59: } //format => int *p = new int[100];
! P9.CPP:9:inline int sumup( int x, int y)
! P9.CPP:17: int i1 = 10, i2 = 20, sum = 0;
! functions.h:3:int doTotal(int x1, int x2)
! functions.h:12:float doAverage(int x1, int x2)
! functions.h:19:int doDifference(int x1, int x2)

In case you want to search through some text files in a series of
directories replacing one set of text for another in each of the files,
try this shell script.

#!/bin/sh
for file in `grep -liR “someword” ./*`;
do
sed ‘s/someword/someother_word/g’ $file > tmp/$$ && mv tmp/$$ $file
done