As you might have heard and likely read in the back-and-forth blogging of Dave Hitz (a NetApp founder) and Jonathan Schwartz (CEO of Sun Microsystems), the two are at each other’s throats. Well not really at each other’s throats: NetApp went nuclear and Sun hit back even harder.
Basically NetApp says that Sun’s ZFS steals it’s existence from NetApp’s WAFL, an earlier Copy-On-Write (COW) file system. Sun in return wants to ensure that NetApp cannot sell its filers and pull NetApp’s ability to legally use NFS.
For those of you that need reminding, Sun invented NFS (the wikipedia page has a sufficient history) but NetApp did the best commercial appliance implementation. I forget if NetApp handed Sun their asses in the marketplace with a superior product, or if Sun just never delivered a filer-like appliance (I say “I forget” because while I can recall using NetApp, I can’t remember using a single offering from Sun beyond just a workstation).
NetApp filers do happen to be one of the best NFS appliances out there, and they happen to combine that with iSCSI and Fiber Channel in one rig. Pretty flexible, and if you look around you’ll see that it’s pretty unique. For example, an EMC Clariion does iSCSI or Fiber Channel, and if you want it to do NFS, then you basically buy a “gateway” (just a server attached by fiber).
We at Joyent are in a bit of a odd and unique position (I think). We happen to use what I call The Holy Z2D Trinity (ZFS, Zones and DTrace), and not just that we use ZFS on top of block storage from NetApp filers. And the team here from myself, to Mark, to Ben have written tools for filers, have managed many petabytes worth of them (often petabytes all at one time), and have been around them since they came out.
Hmm.
In fact, besides my common boast that we’re one of the larger or the largest OpenSolaris/Solaris Nevada installations in the world, I’d venture to guess that we have more ZFS on top of NetApp filers via iSCSI than just about anyone else.
Now let’s think how we even got here in the first place.
- NetApp developed a nice little filesystem they call WAFL.
- NetApp contributes to FreeBSD and FreeBSD is the core OS in some of their products (yes please don’t try and “educate” me on a rig I’ve used for a decade, I said “some”, I know they have their own OS).
- WAFL or something like it has presumably been ported to FreeBSD.
- When pulling up block storage LUNs from our filers, we still need to a filesystem on it. (Got that? Not everything is NFS, There still needs to be a filesystem on the server.)
- Nothing like WAFL or a nice consistent COW filesystem was ever contributed back to FreeBSD.
- FreeBSD instead just has UFS2, and while it has soft updates, you’ll still to need to run a fsck in single user mode for hours once you’re in the 100s of GBs, and ironically the busier the system is, the worse that is (again, yes I know there’s a background fsck, try running that on a busy mail server, it’ll crash, guaranteed).
So larger drives (or LUNs) plus a shitty, non-resilient file system meant that all of FreeBSD had to go. Despite everything great about it and a lot of time invested by the team over a decade in using it. We had to leave FreeBSD and go to wherever we could to get a good, easy to use, resilient, modern file system.
That file system happened to be ZFS and the operating system was OpenSolaris. The additions of DTrace and Zones (we used Jails before) formed our three pillars.
But stop and imagine for a minute that NetApp had been kind enough to give WAFL or a similar filesystem back to the FreeBSD project. Imagine that we had something like WAFL to put on top of our block storage LUNs that were coming up from the NetApp filers.
Got that?
We didn’t want or need WAFL on a real operating system to develop a competitive storage product, we needed WAFL-like on a real operating system in order to simply use our NetApps.
Then we wouldn’t have ever made our first step to OpenSolaris. We’d currently have a business based on FreeBSD 7 with WAFL, Jails and DTrace (hooray for that port!), and believe me leaving FreeBSD was painful in many ways, with the salve being ZFS and DTrace.
While I think there is a good degree of posturing going on betwee the two companies, and it’s fascinating to see it going on in blogs, both parties are full of it and don’t quite get it.
NetApp Dave:
You should have given WAFL or a WAFL-lite file system to FreeBSD and then all of us could happily use it on top of iSCSI or fibre-channel block storage. You would have made it the best operating system to put on top of any networked block storage, being smart guys, you could have figured out how to do it while making NetApp even more money, and without spawning a bunch of FreeBSD storage appliance clones. The fact that you didn’t do this is why Joyent uses Solaris, and you’re responsible for the new market need that’s out there for ZFS and via ZFS, Solaris itself.
Stop being so shortsighted and give a little, think FreeBSD 7.5 + WAFL.
We’re the poster child for OpenSolaris adoption, and that fact that we’re using it … you could have prevented it.
Sun Jonathan:
It’s great that ZFS was developed, magnificent that it was open-sourced, and I know that it was valid and true creation of Jeff Bonwick (who we count as a good friend). You make hardware, you created NFS, you’ve always touted the importance of the network in computing, yet I’ve had to always use NetApps to get fast and reliable NFS and iSCSI block storage (even to put my ZFS filesystems on).
Ship a decent piece of hardware + software capable of fast and reliable NFS + iSCSI. NetApp only exists because of Sun’s failures in actual product development.
“What Ifs”
If NetApp wins, there goes our beloved ZFS (yes I understand indemnity, but I care more about continued development) and NetApp, you don’t have an alternative for us. Thanks for nothin’.
If Sun wins, there’s goes our NetApps, and Sun, we can’t get an alternative from you. Thanks for nothin’.
And finally
I think we’ve been a good customer of both of you, when you fight, it hurts us more than anyone else.
14 responses to “NetApp versus Sun, Sun versus NetApp, and Both versus Common Sense”
Well said. Many of us are in the same boat: environments mixing netapp with zfs, zones and dtrace.
Netapp’s products and support (especially) are superior to Sun. It’s just that you have to pay up the you know what for it.
Because of the expense of the product and this law suit, I hope Sun comes out with Thumper junior!
ZFS has been ported to FreeBSD and will be included in FreeBSD 7. That probably doesn’t help you guys much since you’ve needed it for quite awhile, but I’m looking forward to using something more modern and featureful than UFS2 without having to switch to Solaris. I’m sure ZFS works better on Solaris than it will in the first FreeBSD release with it, but enough of the FreeBSD community wants ZFS that I think it will get better as time goes on.
If anyone needs an ass kicking, let me know.
(oh, and you guys should put a ‘required’ note next to the email field on this blog. and the error message you get if you don’t type in an email address is pretty hard to read. or not require that comments have an email address)
Time someone knocked heads together.
@Nick: Netapp will come for FreeBSD if that ever happens – it’s ZFS as competition against their filers that they are attacking, not Sun per se.
Definitely time for some knocked heads. Fact is, I bet both companies have patents the other infringes upon. Sensible course of action would be for them to cross-license, kiss and make up.
And yes, I agree: NetApp wouldn’t exist if Sun had a better clue about what its customers wanted, historically, and I’ve always been surprised that Sun didn’t try harder to acquire NetApp.
“Sun in return wants to ensure that NetApp cannot sell it’s filers”
That should be “its” – drop the apostrophe. What you are saying here is “NetApp cannot sell it is filers,” which does not make any sense.
So does this article imply that Thumpers are over-hyped?
Sun may be following “the Chicago Way” in all this, but what would you have done differently in their shoes?
>For example, an EMC Clariion does iSCSI or Fiber Channel, and if you want it to do NFS, then you basically buy a “gateway” (just a server attached by fiber).
While the CX are dual protocol systems the NS series are multi-protocol systems. (NAS/ISCSI/FC) The gateway products serve a different customer segment.
@Chad, thumpers are a piece of hardware, they lack a few things to be able to really really provide storage up to other servers. As far as running things directly on all those spindles, they’re great.
@Dick Davies, at this point? Well NetApp went nuclear on Sun, and if they couldn’t work it out before Sun’s response, then Sun had no choice really but to go nuclear back. ZFS is that important. However, they both need to stop their respective superiority things that’s been going for years and years, and realize that neither provide nor have provided a full solution. Going back and forth like this doesn’t help customers.
@Dick Davies: If [ZFS] ever happens in FreeBSD? It’s in FBSD 7 beta which is available now, and the final version should be out shortly. I’ve been using it in -CURRENT for a while and it works great.
I agree with most of your points except this bit:
“You should have given WAFL or a WAFL-lite file system to FreeBSD and then all of us could happily use it on top of iSCSI or fibre-channel block storage.”
I don’t think this is fair.
NetApp is WAFL. It’s the source of all their power — just about every unique feature owes its existence to this layer of block indirection. It was invented over a decade ago, and was truly unique. Innovation is surprisingly rare in computing — most companies trumpet it, but few deliver. WAFL is a true innovation — why should the company give it away?
You say that NetApp “could have figured out how to do it while making … even more money, and without spawning a bunch of FreeBSD storage appliance clones.”
This is a hollow compliment. How would one actually achieve this? If the WAFL innovation is the source of NetApp’s success, why should they give it away, and why do you assume they would prosper without it?
I agree that both companies should “kiss and make up”, and I think that’s the inevitable conclusion, but there’s no reasonable argument to say that NetApp somehow owed it to the world to give away its core technology.
@Tyler, what isn’t fair is spending millions on NetApp filers and having to put UFS on top their iSCSI storage.
Their filesystem is in fact not the reason for their commercial success.
Their commercial success is a result of relatively high prices, great margins, great sales people, great support and great hardware.
Pretty simple.
How about they make it available as binary? A loadable kernel module for their own customers (that works fine with FreeBSD)? It doesn’t require the distribution of source code. How about doing what they’re doing now and suing people that use it to compete with them, but make it fine for customers to use?
How about simply doing what someone like Nividia does when they ship a driver for a video card on FreeBSD?
Are those source code distributions?
Nope.
Guess where they sit?
/kernel/
I didn’t say that NetApp owed it to the world to give away its core technology.
I didn’t say that they even owe it to their block storage customer.
I said that considering the lack of a decent filesystem to put on top of their block storage, they shouldn’t be surprised that we flocked to ZFS. And now if they’re actually going to sue and end up being successful in taking ZFS away from us (that’s their goal right? Why else do it?), then yes, they would owe us WAFL.
I am vendor agnostic so I don’t like comments that are a misstatement of fact.
NetApp makes a great NAS Appliance, but their advantage was based upon their user interface, not their Hardware. You made a misstatement, in that with the EMC Clarion is a block level SAN device (where NETAPP is a file level system) but your claim that you need a Gateway (or an NSXXX-G) is not a true statement. You can run NFS without the use of a NAS gateway. The Gateway is only a diskless NAS head, that allows you to utilize existing disk in your SAN via a the NS HEAD and provide separate management and control. But you can do NFS directly to the SAN infrastructure…
From a basic level you cannot compare the NetApp to the Clariion, you have to compare it to the Celeara (EMCs NAS Product). You have to remember that NetApp is a file level device, and as a file level device it is pretty good, mostly because of its user interface and ease of use. If you look into the their method of data read/write, you will find that the random write anywhere process (even though fast and attractive in the beginning) presents a real problem as the device space fills up, and the data is continually being changed. It is a simple matter of Physics.
You write in any open space, you do not reorganize files to continuous blocks, which the means you are writing fragmented files (or data segments), as the unit reaches 60-70% utilization of space, the problems start to become apparent. To read a file the head has to move to the appropriate space to read the next segment of the file. This means physical movement. As the files become more fragmented (because the NetApp) does not restructure the space or move data for efficiency, the need for the head to move to read the file becomes a burden. Thus NetApps are very ineffective after 60-70% space utilization.
The other point is to argue the proper place for data. Block data performs best on block level devices (SAN), file level data performs best on file level system (NAS) , with out getting into all the detailed reasons, as simple on is this. A file stored on a block level device can be rendered unreadable based upon a small amount of block level errors. Alternately a Block level application (such as a data base), operates inefficiently on a file level system (or obvious reasons), and the file level system does not have the protective features, etc. to handle a block level failure (that could be critical to a database).
Every manufacture has its benefits and disadvantages… In Storage, the all use the same spinning disk, it’s the software that make the product work, and for diversity it’s the offerings the company has…