Comments on: Abusing Solaris attempt #2: stressing out ZFS, PART2 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/ Faith, Solaris, and Chicken Korma, by Anton Parol Mon, 20 Jun 2011 20:37:35 +0000 hourly 1 http://wordpress.org/?v=3.3 By: parolski.com » Blog Archive » Abusing Solaris with style http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-67 parolski.com » Blog Archive » Abusing Solaris with style Sat, 19 Apr 2008 01:20:16 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-67 [...] Solaris to goo with destructive commands is something I’ve been enjoying for a while now, so its great to see someone add cash to the equation and take the next step in [...] [...] Solaris to goo with destructive commands is something I’ve been enjoying for a while now, so its great to see someone add cash to the equation and take the next step in [...]

]]> By: Anton Parol http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-41 Anton Parol Mon, 24 Mar 2008 21:32:30 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-41 Mark, is that a minimum overhead? i.e. is that the least that ZFS has to take in order to work? Something that gets me interested is how much space do null block pointers take up, i.e. the pointer itself! Mark, is that a minimum overhead? i.e. is that the least that ZFS has to take in order to work?
Something that gets me interested is how much space do null block pointers take up, i.e. the pointer itself!

]]> By: Mark J Musante http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-40 Mark J Musante Mon, 24 Mar 2008 02:46:57 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-40 Of course you'll get a better ratio compressing 1's than compressing 0's, because obviously 1's are bigger than 0's! Seriously, though, the reason you only got around 64mb in your mirror pool of 100mb files is that ZFS uses around 32mb of overhead. In proper pools, you won't miss it. But in such a tiny pool as crazedPool, it's a bit more obvious. Of course you’ll get a better ratio compressing 1′s than compressing 0′s, because obviously 1′s are bigger than 0′s!

Seriously, though, the reason you only got around 64mb in your mirror pool of 100mb files is that ZFS uses around 32mb of overhead. In proper pools, you won’t miss it. But in such a tiny pool as crazedPool, it’s a bit more obvious.

]]> By: Jeff Bonwick http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-33 Jeff Bonwick Tue, 18 Mar 2008 10:19:53 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-33 Simple: when you ask to read some block of a file, if the pointer to that block is a null pointer, that means it's a hole, so we just return a buffer full of zeroes. Any filesystem that supports holey files does more or less the same thing; the only thing that's different about ZFS is that when compression is enabled, we deliberately convert runs of zeroes into holes. Simple: when you ask to read some block of a file, if the pointer to that block is a null pointer, that means it’s a hole, so we just return a buffer full of zeroes. Any filesystem that supports holey files does more or less the same thing; the only thing that’s different about ZFS is that when compression is enabled, we deliberately convert runs of zeroes into holes.

]]> By: Anton Parol http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-29 Anton Parol Mon, 17 Mar 2008 21:37:34 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-29 Ah ha! So the next test should fill the files with all 1's instead, and the compression ratio will go up!! Makes alot of sense. But Jeff, when going from a ZFS filesystem to UFS (i.e. not compressed), how does it know to make a block pointer to re-create all those zeros? /me open up opensolaris.org and tries to make sense of c code! Ant Ah ha! So the next test should fill the files with all 1′s instead, and the compression ratio will go up!! Makes alot of sense. But Jeff, when going from a ZFS filesystem to UFS (i.e. not compressed), how does it know to make a block pointer to re-create all those zeros?

/me open up opensolaris.org and tries to make sense of c code!
Ant

]]> By: Rishi http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-28 Rishi Sun, 16 Mar 2008 12:03:02 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-28 Keep on going, and the chances are that you will stumble on something, perhaps when you are least expecting it. I never heard of anyone ever stumbling on something sitting down ;-) Keep on going, and the chances are that you will stumble on something, perhaps when you are least expecting it. I never heard of anyone ever stumbling on something sitting down ;-)

]]> By: Jeff Bonwick http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/comment-page-1/#comment-27 Jeff Bonwick Sun, 16 Mar 2008 08:06:22 +0000 http://www.parolski.com/2008/03/15/abusing-solaris-attempt-2-stressing-out-zfs-part2/#comment-27 Yep, this is an artifact of the implementation. When you enable compression, the first thing we do is scan the block to see if it's all zeroes. If so, we simply discard the block and mark it as a hole in the block tree. So a compressed block of zeroes consumes no space. However, because it has no associated block pointer, it doesn't enter either the numerator or the denominator of the compression ratio. You will see, however, if you run du -sh on the file, that it consumes zero bytes. We considered addressing this by distinguishing between a true hole and a block of zeroes by using a special block pointer value, but doing so would just complicate the code without solving any real problem... so we erred on the side of simplicity. Yep, this is an artifact of the implementation. When you enable compression, the first thing we do is scan the block to see if it’s all zeroes. If so, we simply discard the block and mark it as a hole in the block tree. So a compressed block of zeroes consumes no space. However, because it has no associated block pointer, it doesn’t enter either the numerator or the denominator of the compression ratio. You will see, however, if you run du -sh on the file, that it consumes zero bytes.

We considered addressing this by distinguishing between a true hole and a block of zeroes by using a special block pointer value, but doing so would just complicate the code without solving any real problem… so we erred on the side of simplicity.

]]>