Site search

Recent Posts

Meta

Categories

Posts

January 2009
M T W T F S S
« Dec    
 1234
567891011
12131415161718
19202122232425
262728293031  

Open Source County - Impressive Geospatial Portal

I keep having to fish around for this URL so I’m just going to blog into this online note space that I maintain. One stop shopping.

http://maps.co.mecklenburg.nc.us/gp/

This is the Mecklenburg County GIS data portal to both environmental and cadastral-type information. I ran across it some time ago as an avid reader of Tobin Bradley’s excellent blog . I can’t say enough about Tobin’s blog and the work that he writes about. I think he employs some well thought out, no nonsense, light-weight-yet-effective approaches to solving common problems and the the projects he writes about demonstrate clean implementations. The mapping site above and the framework behind it embodies this ethic in my mind.

Putting your own content into a website with Google Maps

So I’m doing a little research tonight on some requirements for a client:

1. Embed a map in the client’s home page (non-database driven ASP.Net which is essentially HTML) that depicts the county’s watersheds and some number of additional layers
2. Let users click each watershed polygon to trigger an event that leads to them learning more about the conditions within the polygon (think old school image maps). This could be as simple as take the user to a static HTML page about the watershed.

The client currently supports ArcIMS (v9.1) and Google Maps. There is no immediate plan to move to ArcGIS server, though we will learn more next week.

I think that embedding ESRI in their pages for such a simple need is overkill and the client has already won awards for their use of Google Maps. So I’m researching options to use the Google Maps API and Google Gadgets to meet their needs

Cross Compare SQL Server 2008 Spatial, PostgreSQL/PostGIS 1.3-1.4, MySQL 5-6

http://www.bostongis.com/PrinterFriendly.aspx?content_name =sqlserver2008_postgis_mysql_compare

From the PostGIS Listserv comes a very useful comparison table that has been partially vetted by the members of the list. I just wish they included Oracle Spatial and Oracle Locator.

Paragon wrote:
/> As part of our due diligence work, we are formulating a comparison of the
/> spatial functionality offered by SQL Server 2008 Spatial, PostGIS and MySQL.
/> Its currently in draft, but we hope to release within the next week.
/>
/> We would appreciate if people could take a quick look at it and let us know
/> if you see any gross errors or important things we left out. Most of this
/> information we curled from reading the help docs of each product and from
/> our general familiarity with the databases and GIS products out there. We
/> tried to be as fair as possible.
/>
/>
/> http://www.bostongis.com/PrinterFriendly.aspx?content_name=sqlserver2008_pos
/> tgis_mysql_compare
/>
/> Thanks,
/> Regina and Leo

Google Earth/ ArcExplorer Killer?

“TITAN takes spatial data sharing, viewing, and publishing to a new level. It seems to magically ingest almost any spatial data format, read it, use it, and publish it back out in any format — and do so quickly. It does for spatial data what the Babel fish did for language and speech. A universal translator designed for sharing and the sharing environment is completely controllable via permissions, so you don’t lose data ownership. TITAN delivers data via geospatial Web services, such as Web Map Services (WMS), permitting you to view spatial data without actually getting access to the source dataset. ”

ERDAS Titan

But what if I want to keep the data?

Gotta love interop

I love it when the big boys actually play nice together and support standards. Autodesk and Bentley team up on AEC Interoperability.

The GIS industry demonstrates this harmony so much better than main stream IT businesses. But even they are learning.

Changing a light bulb

Bought one of these Bulb Changer Kit/Pole tonight after months of dragging my feet about hauling the ladder up stairs to change the ceiling bulb in the bathroom. Totally worth money. If you have tall ceilings with light bulbs that hang down. Get one of these things. Period.

Optimize all tables in a database, Part 2

Ok, not being satisfied with my first exploration of a global “do something” mysql script I asked the community for help.

The result is posted below, and here are the links that got me here.
http://www.linuxquestions.org/questions/showthread.php?p=2668261#post2668261
http://www.linuxforums.org/forum/linux-programming-scripting/85836-loop-within-loop-mysql-ops.html#post445403

Here is the final example script that optimizes (or replace optimize
with your favorite command like backup, alter table to InnoDB, etc.)
all tables in all databases on a server except the core mysql databases
or others that you exclude.

#!/bin/sh
MUSER=”USER”
MPASS=”PASSWORD”
MHOST=”localhost”
MYSQL=”$(which mysql)”
# the Bs makes the output appear without the formatting
# and header row.
# Step 1: list all databases EXCEPT core mysql tables and others that can be added
DBS=”$($MYSQL -u$MUSER -p$MPASS -Bse ’show databases’ | egrep -v ‘information_schema|mysql|test’)”

for db in ${DBS[@]}
do

# Step 2: list all tables in the databases
echo “$MYSQL -u$MUSER -p$MPASS $db -Bse ’show tables’”
TABLENAMES=”$($MYSQL -u$MUSER -p$MPASS $db -Bse ’show tables’)”
echo “[START DATABASE]“
echo “Database: “$db
echo ${TABLENAMES[@]}

# Step 3: perform an optimize (or other op) for all tables returned

for TABLENAME in ${TABLENAMES[@]}
do
echo $TABLENAME
$MYSQL -u$MUSER -p$MPASS $db -Bse “optimize TABLE $TABLENAME;”
done
echo “[END DATABASE]“
done

The all powerful find command

find <starting point> <search criteria> <action>

The starting point is the name of the directory where find should start
looking for files. The find command examines all files in this
directory (and any subdirectories) to see if they meet the specified
search criteria. If any do, find performs the specified action on each
found file. Here are some of the most useful search criteria options:

-name pattern Find files with names that match the pattern.
-size [+|-] n Find files larger or smaller than a certain size.
-atime [+|-] n Find files accessed before or after a certain date.
-mtime [+|-] n Find files modified before or after a certain date.
-type filetype Find only regular files or only directories.

And here are the actions that can be applied to found files:

-print Print just the names of matching files.
-ls Print the names, dates, sizes, and so on of matching files.
-exec command Execute a command with the file name as input.
-ok command Same as -exec, but asks for confirmation first.

That all might look a bit confusing, so here are some examples to bring
things down to earth. To find files (starting in the current directory)
with names ending with .data and to print their names, try this:

find . -name ‘*.data’ -print
company.data
donor.data
grades.data
sorted.data
words.data

To find files larger than 40K and print the file names and details (use
a minus sign instead of a plus sign to find files smaller than a
certain size), issue this command:

find . -size +40k -ls

To find all files modified within the last 5 days:

find / -mtime -5 -print

The - in front of the 5 modifies the meaning of the time as “less than five days.” The command

find / -mtime +5 -print

To find all files with zero length and ask if they should be deleted:

find / -size 0 -ok rm {} ;

Another Linux Web MySQL Backup Script

I found this on on the nixcraft craft. Looks pretty good to me. Similar
in functionality to my other script posted here. I’m going to poach
some concepts from here to make a script that will auto-optimize all
tables in all databases on the mysql server.

#!/bin/sh
##################
## jcz 17-feb-2007
## copied from http://www.cyberciti.biz/tips/
## how-to-backup-mysql-databases-web-server-
## files-to-a-ftp-server-automatically.html
##############################################
## System + MySQL backup script
## Full backup day - Sun (rest of the day do incremental backup)
## Copyright (c) 2005-2006 nixCraft
## This script is licensed under GNU GPL version 2.0 or above
## Automatically generated by http://bash.cyberciti.biz/backup/wizard-ftp-script.php
## Make full backup every Sunday night
## i.e. backup everything every Sunday
## Next backup only those files that
## have been modified since the full
## backup (incremental backup)
##################################################
## * This is a seven-day backup cycle.
## * It will store data as follows:
## * /home/nixcraft/full/mm-dd-yy/files - Full backup
## * /home/nixcraft/incremental/mm-dd-yy/files - Incremental backup
## * 1. First script will collect all data from both
## * MySQL database server and from file system to
## * temporary directory called /backup using tar command
## * 2. Next, script will login to ftp server and
## * create a directory structure as discussed above
## * 3. Script will dump all files from /backup to ftp server
## * 4. Script will remove temporary backup from /backup
## * 5. Script will send you an email notification
## * if ftp backups failed due to any reason.
##
## * You must have following command installed:
##
## * ncftp ftp client
## * mysqldump command
## * GNU tar command
###########################################
# ———————————————————————
### System Setup ###
DIRS=”/home /etc /var/www”
BACKUP=/tmp/backup.$$
NOW=$(date +”%d-%m-%Y”)
INCFILE=”/root/tar-inc-backup.dat”
DAY=$(date +”%a”)
FULLBACKUP=”Sun”
### MySQL Setup ###
MUSER=”admin”
MPASS=”mysqladminpassword”
MHOST=”localhost”
MYSQL=”$(which mysql)”
MYSQLDUMP=”$(which mysqldump)”
GZIP=”$(which gzip)”
### FTP server Setup ###
FTPD=”/home/vivek/incremental”
FTPU=”vivek”
FTPP=”ftppassword”
FTPS=”208.111.11.2?
NCFTP=”$(which ncftpput)”
### Other stuff ###
EMAILID=”admin@theos.in”
### Start Backup for file system ###
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :
### See if we want to make a full backup ###
if [ “$DAY” == “$FULLBACKUP” ]; then
FTPD=”/home/vivek/full”
FILE=”fs-full-$NOW.tar.gz”
tar -zcvf $BACKUP/$FILE $DIRS
else
i=$(date +”%Hh%Mm%Ss”)
FILE=”fs-i-$NOW-$i.tar.gz”
tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
fi
### Start MySQL Backup ###
# Get all databases name
DBS=”$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse ’show databases’)”
for db in $DBS
do
FILE=$BACKUP/mysql-$db.$NOW-$(date +”%T”).gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done
### Dump backup using FTP ###
#Start FTP backup using ncftp
ncftp -u”$FTPU” -p”$FTPP” $FTPS<
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF
### Find out if ftp backup failed or not ###
if [ "$?" == "0" ]; then
rm -f $BACKUP/*
else
T=/tmp/backup.fail
echo “Date: $(date)”>$T
echo “Hostname: $(hostname)” >>$T
echo “Backup failed” >>$T
mail -s “BACKUP FAILED” “$EMAILID” <$T
rm -f $T
fi

gzip all directories in a directory

I don’t know why I keep having to do this, but I do. I always to create
separate archives of all the directories in a directory. So, here is
script from David (aka Matir aka EmptyCinema) from linuxquestions.org http://www.linuxquestions.org/questions/showthread.php?s=&postid=1839513#post1839513

to do just that (along with a sample sessions using it).

#!/bin/bash
for dir in */
do dir=`echo $dir | tr -d ‘/’`
echo $dir
tar czf $dir.tar.gz $dir
done

[jcz@actinella ~]$ ./zipdir.sh
cdcatalogs
ddclient-3.6.6
Desktop

[jcz@actinella ~]$ ls -lht *.gz
-rw-rw-r– 1 jcz jcz 17M Sep 6 23:31 Desktop.tar.gz
-rw-rw-r– 1 jcz jcz 13K Sep 6 23:31 cdcatalogs.tar.gz
-rw-rw-r– 1 jcz jcz 74K Sep 6 23:31 ddclient-3.6.6.tar.gz

Here is another more advanced version:

#!/bin/bash
# jcz 13-nov-05
# zips (or tar.gz) all directories
# in the directory in which it is run
##################################
echo ” ******************************* “
echo ” this app zips all directories in this directory,”
echo ” tests the created zips for integrity, then “
echo ” copies them to some directory.”
echo ” ******************************* “

# user enters the directory that they want the zips copied to
echo -n “Destination directory for zips e.g. /cygdrive/f/BACKUPS ( . =
here): “
read dest

for dir in */
do dir=`echo $dir | tr -d ‘/’`
echo $dir

# for zipping, -r recurse into directories
# and -u update new or changed files
# echo “zip -r” $dir.zip $dir’/*’
zip -ru $dir.zip $dir
# for gzipping
# tar czf $dir.tar.gz $dir
done

# to test zip file integrity
for zipf in ls *.zip
do zip -T $zipf
done

# move the zips to some directory
for zipf in ls *.zip
do mv -v $zipf $dest
done