parolski.com

See for yourself!

Turning Solaris to goo with destructive commands is something I’ve been enjoying for a while now, so its great to see someone add cash to the equation and take the next step in “squashing” bugs.. and hard drives.

This is an interesting video of a story of stuff. I’d like to see the sources for the figures that get mentioned. If the figures are accurate, it would be a fairly good summary of the failings of Capitalism. Or maybe we aren’t “externalizing our costs” enough.

Having ZFS and SAMBA on the home machine is great. It was simple to set up a share and offer it out to my house mates for them to backup their dissertation work on. With ZFS compressing the files, some of my file systems are getting a 1.6x ratio (60% disk space for free compared with a non-compressing file system)

The interesting bit comes with file permissions and Windows file attributes which aren’t quite the same, in fact, not at all. The attributes get mapped onto the unix file permissions in an interesting way, and you’ll need to keep that in mind when ls -l ing around a shared directory.

Meanwhile, there was a directory named “—-FILES—-” on the share. Trying to use

cd "----FILES----"

doesn’t quite work! Despite quoting the directory name, cd still tries to take some of those dashes as a switch. I’m not sure if theres a way around this using the relative path, but there is a different workaround: simply use the absolute path, instead of a relative one to get you there, eg:

cd /mydir/----FILES..../

To make a directory with leading dashes, you’ll need to specify the full pathname, or use something like this to make the process less painful:

mkdir `pwd`/--mydir--

My final year dissertation project is a software tool for asset tracking in a data center environment. The basic idea is that you run this tool after install your Solaris machine, and it reports back information about the hardware to a database for later inspection. Its also has hooks for being run as a service, so you can probe the machine at any time to see what its got. Simple enough, right?

Well, Sun has just announced that their Service Tag technology has just been open sourced. They fit in as a part to their Sun Connection infrastructure management platform. Basically, the service tags are the little bits of software used to identify you hardware and report it back to you. This can be implemented for hardware and software, and should mean that ISVs and all hardware vendors can make their products that little bit easier to manage! To quote the site:

Open Service Tag is the reference implementation of a standards initiative that will seek the involvement of OEMs (Original Equipment Manufacturer) and ISVs (Independent Software Vendor) to define a standard definition for product identification and discovery.

This is very good news for System Administrators! I can speak from experience that its not nice having to open up hundreds of machines looking for a particular piece of hardware, because there isn’t a command to find it. I truly hope that Sun go through all their back catalog of hardware to make sure absolutely everything is possible to find! This should also include bits of hardware that carry other interesting things built in, like motherboard network interfaces.

Its also quite entertaining that if a Sun product isn’t called a “Sun Java Web enabled, scalable ..” something or other, it still has important sounding features like “Sun Update Satellite Console”.

Its also surprising that Sun has been working on this since May 2007 and I’ve missed it till now! A high level overview of how this might help you manage your enterprise can be found here.

Today we finally moved our computing equipment into a room that has been provided for us by the faculty (BIG THANK YOU TO THE FACULTY TEAM, MIKE PHILIPS AND IAN ROBINSON! ). Its in the Surrey Club, just across the road from the Penryhn road campus:

Hopefully we’ll be getting a network connection soon, its been difficult to find a place on the University network where we can have complete autonomy without breaking anything else!

Theres plenty of room, with about five tables and twice as many chairs, as well as about 14 power sockets in the walls!

Its a bit messy in these shots, maybe we should have taken an after shot! The tables are now lined against the walls, and the beastly Dell servers are under the table to the left behind the door. We were initially rejected by the uni marketing team for compsoc.kingston.ac.uk because we might not represent the views of the uni, much like the case of The River who also applied for a subdomain of kingston, but were denied. However, Ian is going to restate our case, given that we are affiliated to the Student Union and so by default only represent our own views anyway!

Progress!

In my last post, the files were being written to an IDE hard disks. Now lets see what happens if we write to /tmp instead. Will Solaris cope with ten million files in /tmp? First, if we want to make use of the compression, we need to make a file system:

We make the files (we can use files instead of real disks…):

anton@solaris-devx ~ $ mkfile 100M /tmp/file1
anton@solaris-devx ~ $ mkfile 100M /tmp/file2
and then su to root to make the ZFS file system (mirrored):

# zpool create crazedPool mirror /tmp/file1 /tmp/file2

I should note that for some reason ZFS didn’t make use of the entire file size:

# zfs list crazedPool
NAME USED AVAIL REFER MOUNTPOINT
crazedPool 110K 63.4M 20K /crazedPool

And now the real test. How about a big file? Lets say, 100G?:

anton@solaris-devx dir1 $ time mkfile 100G woot
real 1m21.995s
user 0m0.191s
sys 0m30.308s
And what about 10000 files, each 10M in size?:
anton@solaris-devx dir1 $ i="0"
anton@solaris-devx dir1 $ time while [ $i -lt 10000 ]
> do
> mkfile 10M la0$i
> i=$[$i+1]
> done
real 1m46.789s
user 0m4.665s
sys 0m43.492s

So far, so good. So now lets push the envelope off the desk. Or maybe off a cliff. Lets see what happens when we make a 100TB file with ZFS!

anton@solaris-devx dir1 $ ls -l megaFile
-rw------- 1 anton staff 107374182400000 Mar 15 18:05 megaFile

and the compression ratio?:

anton@solaris-devx tmp $ zfs get compressratio crazedPool
NAME PROPERTY VALUE SOURCE
crazedPool compressratio 1.00x -

hmm, not quite what I was expecting!

I’m sure my readers at some time or another have had the pleasure of installing Windows XP. They will also probably had the pleasure of having to wait 3/4 of an hour for it to finish! Well, there is a solution!

Furthering the idea of putting the disk install image in RAM before installing, why not write to a Virtual hard drive, also in RAM (if your using VirtualBox)! Specify /tmp as the prefix to your disks name in the “Virtual Disk Location and Size” dialogue window. The result? An installation of windows that will reboot in about 15 seconds. Of course, this is good for some tasks, but not all! Your millage will vary!

NOTE01: I would recommend your have at least 3GB of RAM before trying this!

NOTE02: If you want to keep the install for good, you will need to copy it off /tmp before you reboot, otherwise your virtual drive will be gone!

I must admit though, it still wont solve any post-install issues!

So last time we pulled out an IDE hard disk, and Solaris lived. It got me thinking…What else can we do to a hard disk that Solaris might not like? What about making a really big file? What about a 9TB file on an 80GB hard disk?:

anton@solaris-devx ~ $ time mkfile 9000G deleteThis

real 120m42.953s
user 0m16.448s
sys 46m55.092s

anton@solaris-devx ~ $ ls -l deleteThis
-rw------- 1 anton staff 9663676416000 Mar 9 02:22 deleteThis

And the machine didn’t die! But what about the compression ratio?

anton@solaris-devx ~ $ zfs get compressratio tank/home
NAME PROPERTY VALUE SOURCE
tank/home compressratio 1.22x

Looks like the ratio is an average for all the files on that file system. So lets try to artificially inflate this value. We can run a little script like this:

#/bin/sh
#A little script to make ten million files each 1 megabyte in size
i="0"
while [ $i -lt 10000000 ]
do
mkfile 10M la$i
i=$[$i+1]
done

The result (after about an hour of making files) wasn’t good:

bash: fork: Not enough space

This looks more like an issue bash had rather than ZFS. So did we push up that compression ratio?:

anton@solaris-devx mess $ zfs get compressratio tank/home
NAME PROPERTY VALUE SOURCE
tank/home compressratio 1.22x -

Seemingly not. To compensate, we’ll do something more crazy for PART2

I really do wonder whether or not this was a real advert that went out, or if it was just an office joke. Either way, thumbs up!