The Unix Clubhouse Blog

The past few days I've had to work with firmware configurable raid 1 (mirror) arrays.

I administer a number of HP-UX enterprise Itanium based Integrity servers which each have SAS (serial attached SCSI) interfaces model (HP 8 Internal Port SAS Controller w/ raid). This controller supports 2 raid 1 mirror arrays with a spare, but I've decided to break the mirrors (oh no, 7 years bad luck) and experiment with the controller prior to the boot disk upgrades (replacing 72 gig. dual-port 15k SAS drives with 146 gig. versions). This may not seem like much space, but all the data is stored on a SAN (Storage Area Network) via fiber channel interfaces.
Here is what I found...

When you add a single new drive (any location) a new raid 1 logical drive is created in degraded status as there is only 1 disk of a 2 disk mirror configuration. When you remove this disk, the logical drive is NOT removed which is very important as
each controller only supports 2 raid 1 mirrors, so if you have a boot mirror and alternate disk (spare) as I do, the new disk created the second raid (logical drive) and even though it was removed the controller is maxed out. Attempting to add another disk without deleting the last logical drive doesn't work, the disk will not be recognized.
Here is some information on deleting logical drives...

Checking your raid status: #> sasmgr -N get_info -D /dev/sasd0 -q raid

All changes to the raid logical drives is accomplished via firmware so you need to boot to the EFI shell> drvcfg -s

If you delete a logical drive that contains a single or pair of disks which is a destructive process all references to the disks are lost so HP-UX no longer sees these devices, however they can be recovered with data intact. Boot to the EFI shell> map

You should see the drive(s) as FS0 or FS2, etc... EFI shell> fs0:

EFI shell> cd EFI

EFI shell> cd HPUX

EFI shell> hpux.efi (boots the kernel file)

Once the kernel is booted #> ioscan -fnNC disk

#> setboot -p (primary) or -a (alternate) /dev/disk/diskX (X should be what you got back from ioscan)

Once you have used setboot the references should be fixed, so you can reboot normally.

In addition, I normally make a clone (dd) of the boot disk and set that to alternate boot.

#> dd if=/dev/rdisk/disk1 bs=1024k of=/dev/rdisk/disk2

Just a little tip on working with the SAS controller. After EFI shell> drvcfg -s

firmware options such as, creating and deleting arrays is menu driven and user friendly.

ADDITIONAL NOTE:

You probably want to clean up your lvmtab file as it still contains the old array info and if vg00 is your volume group then...

#> mv /etc/lvmtab /etc/lvmtab.bak

#> mv /etc/lvmtab_p /etc/lvmtab_p.bak

#> vgscan -k -f /dev/vg00

#> lvlnboot -R /dev/vg00

#> lvlnboot -v (to verify the new settings)

Life with 3 dogs can be a heck of a ride... Awesome most of the time, but every now and then...

OK, about 2 weeks ago I decided to grow a new lawn from seed (most of the front and some of the back). Not sure what I was thinking at the time, but charged ahead with 6 yards of nice dark topsoil, 3 bags of tall fescue grass seed and all day shoveling, raking, tree pruning, watering, etc... After finishing the back, I let the dogs out and kept working around the front. My better half comes home a few hours later and she lets the dogs in... Of course all 3 dogs had been rolling and running around in the the recently watered topsoil so they proceeded to track mud all over the house, muddy paw prints were everywhere, which of course was MY fault.

In order for the seed to sprout, I had to water the "dirt" each day and of course those lovable dogs would roll around in it and track it all over the house. If I wasn't in the dog house over the topsoil thing, I was most certainly in it over the new dog food thing. You see, I'm an avid "low-carber" and (in my infinite wisdom) decided to get the lowest carb dog food I could find (EVO http://www.evopet.com/ ). We started mixing the new food in at about 60 or 70%, they absolutely loved it, but I realize now we should have mixed it in at about 20 to 30% because one of them got sick all over the rug and of course got sick again (all over the rug) on the second day. We're up to 100% on the new food now and they have fully adjusted, but the rug may never be the same and of course, that was MY fault also. 

So what did I get for caring about dog food and trying to improve the yard? A sore back from shoveling topsoil and hours of cleaning rugs, but I wouldn't have it any other way. Now to clean the guinea pig cage and turtle aquarium. 

#!/bin/bash

The first line of this script is generally called a shebang and indicates that this script will be passed to the BASH (Bourne Again Shell) interpreter. Once the script has been passed to bash, the shebang is ignored as it starts with a hash (#) symbol typically used for comments and is not parsed.

echo -n "Enter a valid 5-digit zipcode: "
read zipcode

The (read) command reads a line from standard input. The (zipcode) is simply the name of the variable where the input will be saved. Once the read command is executed, the script stops until it receives input (which ends with the <enter> key), so in this example, the script is waiting for a zipcode which will be saved into a variable called zipcode. Once the input is saved, it can be retrieved by using the ($) and variable name, ($zipcode).

var_url="http://www.google.com/ig/api?weather=$zipcode&hl=en"

Here we are assigning a line of text between the double quote marks to the variable name (var_url). NOTE: when assigning a variable, do not leave a space before the equal sign. In this example, the type of quotes used makes a difference. Single quotes sometimes called literal quotes means just that, exactly as is, but double quotes allows variables to be parsed within the quotes. So lets say we previously entered a zipcode of 12345, if we used single quotes the contents of  the var_url variable looks like this...
http://www.google.com/ig/api?weather=$zipcode&hl=en

if we use double quotes, the contents of the var_url variable looks like this...
http://www.google.com/ig/api?weather=12345&hl=en

As I'm sure you have noticed, the actual text IS the Google weather API. This could be used in a browser url address to see what xml data google replies with.  

var_weather_wget=`wget -q $var_url -O -`

Again we are assigning a variable here, but take notice of the (`) backticks which contain a function. The variable (var_weather_wget) will contain the output of the function (wget -q $var_url -O -). So lets break down this command...

wget -q $var_url -O -

Here we are downloading the XML (Extensible Markup Language) response from Google via wget. The (-q) option indicates quiet mode which  avoids all the status outputs. In addition I'm using the (-O) option which outputs the downloaded data to a file, but I'm also using the (-) instead of a file name so the output is dumped to standard output and in turn assigned to the (var_weather_wget) variable.

var_weather_xml=`echo "$var_weather_wget" | sed 's/<forecast_conditions>.*//'`

This section may look a little confusing, so lets take it apart...
Like before, we are assigning the output of a function contained within (`) backticks to a variable (var_weather_xml). The function contains a few different elements, so lets break them down...

echo "$var_weather_wget" | sed 's/<forecast_conditions>.*//'

First I'm extracting the contents of ($var_weather_get) with the (echo) command. Normally the contents of the variable would be displayed in the shell or standard out but in this case I'm piping that output into sed (stream editor). The pipe (|) is used to connect the output of one command and the input of another. In this case, the output of the echo command and the input of the sed. Multiple pipes can be used to connect many commands and this is generally called a pipeline.

sed 's/<forecast_conditions>.*//'

Now lets look at the sed command. SED is a very powerful tool with many options, but in this case I'm using it modify the output "on the fly". Notice the (') single quotes and (/) slashes and (s), sed receives the contents of the variable and substitutes (thats what the (s) means). When you substitute with sed the target is between the first two slashes (/target/) and the substitute or replacement is between the last two slashes, so putting it together...
s/target/replacement/
regular expressions or patterns can be used and I make use of them in the target. I look for (<forecast_condition>) and add a (.) meaning match any single character and a (*) meaning repeat and by leaving it at that, I will replace <forcast_condition> and everything after it with the replacement which in this case is nothing //.

What's really going on here, is I'm using forcast_information and current_conditions tags, but getting rid of the forcast_condition tags (a forcast for the next few days).

 svar_weather=`echo "$var_weather_xml" | sed 's/></>\n</g'`

More of the same going on here, except I'm substituting or replacing (><) with (>\n<) which add a (\n) newline character and the trailing (g) means to do it globally or for every occurrence within the contents that was piped from (echo).
(var_weather_xml) is basically one long line containing everything, but after running the above command  (var_weather) contains all the same data but each piece is on a different line. This allows everything to be parsed easily using grep.

var_date=`echo "$var_weather" | grep -e '<forecast_date' | \
sed -e 's/<forecast_date data="//' -e 's/"\/>//'`

var_city=`echo "$var_weather" | grep -e '<city' | \
sed -e 's/<city data="//' -e 's/"\/>//'`

var_condition=`echo "$var_weather" | grep -e '<condition' | \
sed -e 's/<condition data="//' -e 's/"\/>//'`

var_temp_f=`echo "$var_weather" | grep -e '<temp_f' | \
sed -e 's/<temp_f data="//' -e 's/"\/>//'`

var_temp_c=`echo "$var_weather" | grep -e '<temp_c' | \
sed -e 's/<temp_c data="//' -e 's/"\/>//'`

var_humidity=`echo "$var_weather" | grep -e '<humidity' | \
sed -e 's/<humidity data="//' -e 's/"\/>//'`

var_wind=`echo "$var_weather" | grep -e '<wind' | \
sed -e 's/<wind_condition data="//' -e 's/"\/>//'`

Each command above does the same exact thing, so lets look at all of them. It should look pretty familiar with assigning a variable, sed substitution and piping so the only thing different here is grep.
The variable (var_weather) is echoed out and piped into grep. Grep simply extracts lines that match the pattern given. For instance, in the last example above, any line containing (<wind) will be piped into sed. Notice the (\) at the end of the line which only extends the command to the next line for easier reading. The sed portion should look OK now, the (-e) is used because I have more than one sed input and this next part might need some explaining ('s/"\/>//'). It is a sed substitution with an escape character (\) which is needed because my target is ("/>) and replacement is nothing, but the target contains a (/) slash which will be interpreted by sed as a border or container so an escape character (\) is used telling sed to look for the slash. NOTE: quite often thes escape characters are used for good reason, but the result may be a confusing bunch of slashes associated with regular expressions.
Simply put, grep extracts the line of data I'm looking for and sed strips off the xml tags and the resulting data is assigned to a variable.

echo "Date: $var_date"
echo "City: $var_city"
echo "Condition: $var_condition"
echo "Temp: $var_temp_f Deg. Fahrenheit / $var_temp_c Deg. Celsius"
echo "$var_humidity"
echo "$var_wind"

This last section outputs the variables to the shell standard output using echo.

Welcome to the Unix Clubhouse Blog.

Initially the unixclubhouse.com website was completely hand coded with the exception of a Simple Machines Forum, but maintenance became time consuming and what once was an enjoyable experiment soon became a
dreaded chore. I initially had podcast, tutorial and misc. other pages as well as the forum to maintain. About the time I started loosing interest, I learned about OPA a strongly typed, compiled web language with a built in web server and database/key value store. WoW! how
awesome, so I proceeded to re-code the website using OPA, I dropped the forum and custom coded a blog. Long story short (sorry too late) after a few months the logs ate all the VPS resources and the site went offline.

After a friend of mine noticed and sent me an email (thanks), I decided to start over.

This time I chose to take advantage of a few open source software projects and use their infrastructure to easily maintain this site. My initial requirements were a simple blog and forum which could both use a SQLite
backend as MySQL is now an Oracle product which I try to stay away from. After a few days, the choices were obvious...

Serendipity CMS for the blog as well as the initial landing site and FUDforum for the site forum. Not only did these projects both support a SQLite backend, but they offer excellent customizing capabilities. I was
especially impressed with how easy it was to create a new theme and edit the templates within FUDforum, so I picked a cool (IMHO) Serendipity theme and created a FUDforum theme to match all within a day or so.

This is only the first blog post, so time will tell if I chose wisely or poorly, but I'm already enjoying the reduction in website maintenance time... I might even crunch out a podcast in the near future and get
back to using the site instead of maintaining it.

UPDATE: 5/9/2012 -- I've decide to drop the forum and replace it with something like a custom tag cloud, not sure yet...