SysAdmin1138 Events

DataProtector 6 has a problem, continued

2008-05-12T11:00:00.003-07:00

I posted last week about DataProtector and its Enhanced Incremental Backup. Remember that "enhincrdb" directory I spoke of? Take a look at this:

See? This is an in-progress count of one of these directories. 1.1 million files, 152MB of space consumed. That comes to an average file-size of 133 bytes. This is significantly under the 4kb block-size for this particular NTFS volume. On another server with a longer serving enhincrdb hive, the average file-size is 831 bytes. So it probably increases as the server gets older.

On the up side, these millions of weensy files won't actually consume more space for quite some time as they expand into the blocks the files are already assigned to. This means that fragmentation on this volume isn't going to be a problem for a while.

On the down side, it's going to park (in this case) 152MB of data on 4.56GB of disk space. It'll get better over time, but in the next 12 months or so it's still going to be horrendous.

This tells me two things:

When deciding where to host the enhincrdb hive on a Windows server, format that particular volume with a 1k block size.
If HP supported NetWare as an Enhanced Incremental Backup client, the 4kb block size of NSS would cause this hive to grow beyond all reasonable proportions.

Some file-systems have real problems dealing with huge numbers of files in a single directory. Ext3 is one of these, which is why the b-tree hashed indexes were introduced. Reiser does better in this case out of the box. NSS is pretty good about this, as all GroupWise installs before GW became available for non-NetWare platforms created this situation by the sheer design of GW. Unlike NSS, ext3 and reiser have the ability of being formatted with different block-sizes, which makes creating a formatted file-system to host the enhincrdb data easier to correctly engineer.

Since it is highly likely that I'll be using DataProtector for OES2 systems, this is something I need to keep in mind.

DataProtecter 6 has a problem

2008-05-07T12:44:00.004-07:00

We're moving our BackupExec environment to HP DataProtector. Don't ask why, it made sense at the time.

Once of the niiiice things about DP is what's called, "Enhanced Incremental Backup". This is a de-duplication strategy, that only backs up files that have changed, and only stores the changed blocks. From these incremental backups you can construct synthetic full backups, which are just pointer databases to the blocks for that specified point-in-time. In theory, you only need to do one full backup, keep that backup forever, do enhanced incrementals, then periodically construct synthetic full backups.

We've been using it for our BlackBoard content store. That's around... 250GB of file store. Rather than keep 5 full 275GB backup files for the duration of the backup rotation, I keep 2 and construct synthetic fulls for the other 3. In theory I could just go with 1, but I'm paranoid :). This greatly reduces the amount of disk-space the backups consume.

Unfortunately, there is a problem with how DP does this. The problem rests on the client side of it. In the "$InstallDir$\OmniBack\enhincrdb" directory it constructs a file hive. An extensive file hive. In this hive it keeps track of file state data for all the files backed up on that server. This hive is constructed as follows:

The first level is the mount point. Example: enhincrdb\F\
The 2nd level are directories named 00-FF which contain the file state data itself

On our BlackBoard content store, it had 2.7 million files in that hive, and consumed around 10.5GB of space. We noticed this behavior when C: ran out of space. Until this happened, we've never had a problem installing backup agents to C: before. Nor did we find any warnings in the documentation that this directory could get so big.

The last real full backup I took of the content store backed up just under 1.7 million objects (objects = directory entries in NetWare, or IIRC inodes in unix-land). Yet the enhincrdb hive had 2.7 million objects. Why the difference? I'm not sure, but I suspect it was keeping state data for 1 million objects that no longer were present in the backup. I have trouble believing that we managed to churn over 60% of the objects in the store in the time I have backups, so I further suspect that it isn't cleaning out state data from files that no longer have a presence in the backup system.

DataProtector doesn't support Enhanced Incrementals for NetWare servers, only Windows and possibly Linux. Due to how this is designed, were it to support NetWare it would create absolutely massive directory structures on my SYS: volumes. The FACSHARE volume has about 1.3TB of data in it, in about 3.3 directory entries. The average FacStaff User volume (we have 3) has about 1.3 million, and the average Student User volume has about 2.4 million. Due to how our data works, our Student user volumes have a high churn rate due to students coming and going. If FACSHARE were to share a cluster node with one Student user volume and one FacStaff user volume, they have a combined directory-entry count of 7.0 million directory entries. This would generate, at first, a \enhincrdb directory with 7.0 million files. Given our regular churn rate, within a year it could easily be over 9.0 million.

When you move a volume to another cluster node, it will create a hive for that volume in the \enhincrdb directory tree. We're seeing this on the BlackBoard Content cluster. So given some volumes moving around, and it is quite conceivable that each cluster node will have each cluster volume represented in its own \enhincrdb directory. Which will mean over 15 million directory-entries parked there on each SYS volume, steadily increasing as time goes on taking who knows how much space.

And as anyone who has EVER had to do a consistency check of a volume that size knows (be it vrepair, chkdsk, fsck,or nss /poolrebuild), it takes a whopper of a long time when you get a lot of objects on a file-system. The old Traditional File System on NetWare could only support 16 million directory entries, and DP would push me right up to that limit. Thank heavens NSS can support w-a-y more then that. You better hope that the file-system that the \enhincrdb hive is on never has any problems.

But, Enhanced Incrementals only apply to Windows so I don't have to worry about that. However.... if they really do support Linux (and I think they do), then when I migrate the cluster to OES2 next year this could become a very real problem for me.

DataProtector's "Enhanced Incremental Backup" feature is not designed for the size of file-store we deal with. For backing up the C: drive of application servers or the inetpub directory of IIS servers, it would be just fine. But for file-servers? Good gravy, no! Unfortunately, those are the servers in most need of de-dup technology.

Being annoyed by rug?

2008-05-06T14:57:00.004-07:00

Rug/zmd in SLES10-SP1 is still a headache maker. Novell knows this, but I strongly suspect that we'll have to wait until SLES11 before we get anything improved. OpenSUSE now has zypper which works pretty good, and I think you can do it in SLES if you want, but I haven't tried.

One of the chief annoyances of rug is that the zmd.db file kept in /var/lib/zmd/zmd.db gets corrupted far too easily. And when that happens, rug can take HOURS to return anything. If it returns anything at all.

The fix for it is easy, stop zmd, delete the zmd.db file, restart zmd. Since I'm doing this fairly often, I've whipped up a bash script to do it for me.

nukezmd

#!/bin/sh
#
# For killing ZMD when it is clearly hung. An all too often occurance.
#

declare PIDZMD

# First get the PID of ZMD

printf "Getting PID... "
let PIDZMD=`rczmd showpid`
printf "$PIDZMD\n"
# Then unconditionally kill it

printf "Killing zmd hard... \n"
kill -9 $PIDZMD

# Remove the old, inconsistent database

printf "Nuking old database... \n"
rm /var/lib/zmd/zmd.db

# Restart ZMD, which will build a new, consistent database

printf "Restarting ZMD\n"
rczmd start

Simple, to the point. Works.

Linux @ Home

2008-05-05T16:32:00.003-07:00

My laptop at home dual-boots between openSUSE and WinXP. There are a few reasons why I don't boot the Linux side very often, some of them work related. And, what the heck, here are the two reasons.

1: Wireless driver problems
I have an intel 3945 WLAN card. It works just fine in linux, well supported. What throws it for a loop, however, are sleep and hibernate states. It can go one, two, four, maybe five cycles through sleep before it will require a reboot in order to find the home wireless again. If it doesn't lock the laptop up hard. Since my usage patterns are heavily dependent upon Sleep mode, this is a major, major disincentive to keep the Linux side booted.

I understand the 2.6.25 kernel is a lot better about this particular driver. Thus, I wait with eager anticipation the release of openSUSE 11.0. This driver is currently the ipw3945 driver, and will eventually turn into iwl3945 driver once it comes down the pipe. What little I've read about it suggests that the iwl driver is more stable through power states.

2: NetWare remote console
I use rconip for remote console to NetWare. Back when Novell first created the IP-based rconsole, they also released rconj along side ConsoleOne to provide it. As this was written in Java, it was mind bogglingly slow. This little .exe file was vastly faster, and I've come to use it extensively. Unless I get Wine working, this tool will have to stay on my Windows XP partition. It works great, and I haven't found a good linux-based replacement yet.

Time has moved on. Hardware has gotten faster, and the 'java penalty' has reduced markedly. RconJ is actually usable, but I still don't use it. Plus, it would require me to install ConsoleOne onto my laptop. It's 32-bit, so that's actually possible, but I really don't want to do that.

The Remote Console through the Novell Remote Monitor (that service out on :8009) has a nice remote-console utility, but it also requires Java. I'm still biased against java, and java-on-linux still seems fairly unstable to me. I don't trust it yet. It also doesn't scale well. When I'm service-packing, it is a LOT nicer looking to have 6 rconip windows up than 6 browser-based NRM java-consoles open. Plus, rconip will allow me access to the server console if DS is locked, something that NRM can't do and is invaluable in an emergency.

Once the wireless driver problems are fixed, I'll boot the linux side much more often. Remote-X over SSH actually makes some of my remote management a touch easier than it is in WinXP. And if I really really need to use Windows, my work XP VM is accessible over RDesktop. There are a few other non-work reasons why I don't boot Linux very often, but I'll not go into those here.

So, oddly, NetWare is partly responsible for keeping me in Windows at home. But only partly.

Back-scatter spam

2008-05-05T12:56:00.003-07:00

There was a recent slashdot post on this. We've had a fair amount of this sort of spam. And the victims are at pretty high levels of our organization, too. Last week the person who is responsible for us even having a Blackberry Enterprise Server asked us to figure out a way to prevent these emails from being forwarded to their blackberry. When a spam campaign is rolling, that person can get a bounce-message every 5-15 minutes for up to 8 hours, into the wee hours of the night. And that's just the mails that get PAST our anti-spam appliance. We set up some forwarding filters, but we haven't heard back about how effective they are.

This is a hard thing to guard against. You can't use the reputation of the sender IP address, since they're all legitimate mailers being abused by the spam campaign and are returning delivery service notices per spec. So the spam filtering has to be by content, which is a bit less effective. In one case, of the 950-odd DSN's we received for a specific person during a specific spam campaign, only 15 made it to the inbox. But that 15 was enough above what they normally saw (about 3 a day) that they complained.

Backscatter is a problem. However, our affected users have so far been sophisticated enough users of email to realize that this was more likely forgery than something wrong with their computer. So, we haven't been asked to "track down those responsible." This is a relief for us, as we've been asked that in the past when forged spams have come to the attention of higher level executives.

If it becomes a more wide-spread problem, we will be told to Do Something by the powers that be. Unfortunately, there isn't a lot that can be done. Blocking these sorts of DSNs is doable, but that's an expensive thing to manage in terms of people time. In 6-12 months we can expect the big anti-spam vendors to include options to just block DSN's uniformly, but until that time comes (and we have the budget for the added expenses) we'd have to do it through dumb keyword filters. Not a good solution. And it would also cause legitimate bounce messages to fail to arrive.

Legal processes

2008-04-30T08:58:00.002-07:00

Yesterday we received a Litigation Hold request. For those of you who don't know, this is the order given as part of a lawsuit ordering us to take steps to preserve data that could be used as part of the Discovery process of the suit. This is something that is becoming more and more common these days.

Our department has been pretty lucky so far. Since I started here in late 2003 this is the first Litigation Hold request we've had to deal with. We've had a few "public records requests" come through which are handled similarly, but this is the first one involving data that may be introduced under sworn testimony.

This morning we had an article pointed out to us by the Office of Finance Management at the state. WWU is a State agency, so OFM is in our chain of bureaucracy.

Case Law/Rule Changes Thrust Electronic Document Discovery into the Spotlight.

It's an older PDF, but it does give a high level view of the sorts of things we should be doing when these requests come in. One of the things that we don't have any processes for are the sequestration of held data and chain of custody preservation. We are now building those.

Guideline #4 has the phrase, "Consultants are particularly useful in this role," referring to overseeing the holding process and standing up before a court to testify that the data was handled correctly. This is very true! Trained professionals are the kind of people to know the little nuances that hostile lawyers can use to invalidate gathered evidence. Someone who has done a lot of reading and been to a few SANS classes is not that person.

Just because it is possible to self represent yourself in court as your own lawyer, doesn't make it a good idea. In fact, it generally is a very bad idea. Same thing applies to the above phrase. You want someone who knows what the heck they're doing when they climb up there onto the witness stand.

This is going to be an interesting learning experience.

The GPL in a software-as-a-service world

2008-04-28T10:54:00.003-07:00

Just this last weekend I went to Linuxfest Northwest, which is held here in Bellingham. This is nice! It's just a short drive.

One of the talks I went to was held by Ted Haeger, currently of Bungee Labs. The topic of the talk was one he had just posted to his blog, "Sharing Source Code In The Cloud".

One point he brought up that I hadn't heard of before is that the GPL triggers when you 'convey' the software to someone else. And that the GPL specifically excludes where the software is hosted on a server and users just use the software there, so long as the software itself never leaves the company in question. This is exactly what Google did and still does. All of their search IP was built on an OSS platform, but is still held as the crown jewels of their company; all because they haven't given the software to anyone else.

Apparently, this 'loophole' is being exploited by a LOT of new companies trying to get in on the software-as-a-service market. Such as Bungee Labs, as it happens. What effect will this have on the state of GPLed software? Hard to say, the market is still in its early days.

It makes you think.

And a gripe

2008-04-17T11:32:00.002-07:00

2.5 hours is too freakin' long for "rug lu" to tell me which patches need application to this particular OES2 server. This needs fixing. I hope its fixed in SLES10 SP2.

NetWare and Novell, changing a company

2008-04-17T10:53:00.003-07:00

A couple days ago Richard Bliss had a long blog entry about, "Novell's Cash Cow - How NetWare almost killed the company". It had some very interesting points. Some we knew:

We are all familiar with NetWare, the dominate Network Operating system of the 1980s and 1990s. We are all familiar with Microsoft's tactics of penetrating the NOS market with Windows NT by focusing on using Windows as an application platform.

Apparently Richard worked for Novell around 2001. I find that interesting since my first BrainShare was 2001, and that was when they announced the release of NetWare 6.0. While there he saw what seemed to be an outright denial that NetWare had been passed up by Windows and something new needed to be done.

In 2001 I knew that Windows had for all intents and purposes won. The only place you ever really saw NetWare servers were as file-servers, or running GroupWise or the small handful of apps that used NetWare as an application server. The stalwart loyalists among us saw this as annoying, but not a major problem.

It was also good for Novell's bottom line. NetWare still accounted for a large percentage of their revenues. Even though the writing was on the wall, they were still making real money on it so didn't see a need to change. This is why NetWare 6.0 introduced the AMP stack to NetWare, as a way to better make NetWare an application server and to slow the loss of customers. At BrainShare 2001 there was open speculation about "NetWare 7.0" and what it would look like.

And there still was until 2005 when Novell announced what the next version of NetWare would be. This being after the SUSE and Ximian purchases, it would be based on Linux. This move had been rumored, and alternately derided and lauded, for some time. There was a great wailing and gnashing of teeth on the part of the stalwart NetWare loyalists. It also started an exodus of customers, as Novell's financial reports at the time point out.

Fortunately for the company, they started actively promoting (for certain values of 'active' that are higher than they were previously, but still in the theme of Novell Stealth Marketing) and developing their other products, like GroupWise, Novell Identity Management, ZenWorks, and most especially their Linux business. It took them until last quarter to turn in a quarter in the black, and NetWare revenues are under 20% of total now. So, they've turned the corner and are no longer dependent on the NetWare cash cow. They have a couple of them in the field now, which is a MUCH healthier place to be.

It's a funny thing, but one of the reasons why NetWare is such a kick-butt file-server compared to everything else is why it's a challenging environment to develop in. Had Novell seen the light earlier and bought SUSE (or rolled their own Linux distro) in... 1999 instead, right after the NW5.1 release, they still would have run into the fundamental architectural problems in 32-bit linux that make it an inferior file-serving platform for large environments. By 2008 their server could have been a LOT more mature, and perfectly poised to take advantage of 64-bit Linux.

Novell in the 1990's is not an example of a 'nimble' company. It is trying to get there now through diversification. Not many companies (especially tech companies) have survived the loss of their prime money earner; Apple has done it through OSX, which required a fanatically loyal fan base to survive the dark years. This is the prime reason people kept predicting the imminent demise or buyout of Novell. Now that they're earning profits again, and have diversified away from just the OS sector, they're not going to be going out of business any time soon.

Now if only they had better SMB packages and programs. I hear repeatedly from peers who support SMBs that Novell's packages and programs in that space are lacking or exploitative. Significant revenue, and more importantly mindshare, are in the SMB market. Plus, today's SMB is tomorrow's large or global enterprise.

Beta attitudes

2008-04-15T13:09:00.002-07:00

One thing I've noticed while working on this beta is a change in attitude. Specifically, attitude regarding problems. I've run into problems so far that would have had me throwing things across the room by now. Yet, instead I get that 'ahah!' feeling and proceed to figure out how it went poink exactly like that. And then report it. That feels good.

All of my prior bug-hunting has been post-release, when we ran into issues in production. Now, it's in pre-release and the bugs and issues I find now will be fixed by release (or at least documented so people know to expect it to break that way).

It's an interesting change in attitude.

On email, what comes in it

2008-04-11T10:30:00.002-07:00

A friend recently posted the following:

80-90% of ALL email is directory harvesting attacks. 60-70% of the rest is spam or phishing. 1-5% of email is legit. Really makes you think about the invisible hand of email security, doesn't it?

Those of us on the front lines of email security (which isn't quite me, I'm more of a field commander than a front line researcher) suspected as much. And yes, most people, nay, the vast majority, don't realize exactly what the signal-to-noise ratio is for email. Or even suspect the magnitude. I suspect that the statistic of, "80% of email is crap," is well known, but I don't think people even realize that the number is closer to, "95% of email is crap."

Looking at statistics on the mail filter in front of Exchange, it looks like 5.9% of incoming messages for the last 7 days are clean. That is a LOT of messages getting dropped on the floor. This comes to just shy of 40,000 legitimate mail messages a day. For comparison, the number of mail messages coming in from Titian (the student email system, and unpublished backup MTA) has a 'clean' rate of 42.5%, or 2800ish legit messages a day.

People expect their email to be legitimate. Directory-harvesting attacks do constitute the majority to discrete emails; these are the messages you receive that have weird subjects, come from people you don't know, but don't have anything in the body. They're looking to see which addresses result in 'no person by that name here' messages and those that seemingly deliver. This is also why people unfortunate enough to have usernames or emails like "fred@" or "cindy@" have the worst spam problems of any organization.

As I've mentioned many times, we're actively considering migrating student email to one of the free email services offered by Google or Microsoft. This is because historically student email has had a budget of "free", and our current strategy is not working. The way it is not working is because the email filters aren't robust enough to meet expectation. Couple that with the expectation of effectively unlimited mail quota (thank you Google) and student email is no longer a "free" service. We can either spend $30,000 or more on an effective commercial anti-spam product, or we can give our email to the free services in exchange for valuable demographic data.

It's very hard to argue with economics like that.

One thing that you haven't seen yet in this article are viruses. In the last 7 days, our border email filter saw that 0.108% of incoming messages contain viruses. This is a weensy bit misleading, since the filter will drop connections with bad reputations before even accepting mail and that may very well cut down the number of reported viruses. But the fact remains that viruses in email are not the threat they once were. All the action these days are on subverted and outright evil web-sites, and social engineering (a form of virus of the mind).

This is another example of how expectation and reality differ. After years of being told, and in many cases living through the after-effects of it, people know that viruses come in email. The fact that the threat is so much more based on social engineering hasn't penetrated as far, so products aimed at the consumer call themselves anti-virus when in fact most of the engineering in them was pointed at spam filtering.

Anti-virus for email is ubiquitous enough these days that it is clear that the malware authors out there don't bother with email vectors for self-propagating software any more. That's not where the money is. The threat had moved on from cleverly disguised .exe files to cunningly wrought (in their minds) emails enticing the gullible to hit a web site that will infest them through the browser. These are the emails that border filters try to keep out, and it is a fundamentally harder problem than .exe files were.

The big commercial vendors get the success rate they do for email cleaning in part because they deploy large networks of sensors all across the internet. Each device or software-install a customer turns on can potentially be a sensor. The sensors report back to the mother database, and proprietary and patented methods are used to distill out anti-spam recipes/definitions/modules for publishing to subscribed devices and software. There is nothing saying that an open-source product can't do this, but the mother-database is a big cost that someone has to pay for and is a very key part of this spam fighting strategy. Bayesian filtering only goes so far.

And yet, people expect email to just be clean. Especially at work. That is a heavy expectation to meet.

Generations

2008-04-10T09:58:00.002-07:00

My boss pointed us at an article this morning, about a topic near and dear to managers everywhere. Boomers are retiring, and for every 2 boomers leaving, 1.2 workers are entering the workforce. I know I've been watching a steady drum-beat of retirements the last few years.

In the article is this sentence:

Statistically, Millennials are the most pluralistic, integrated, high-tech generation in American history—traits that make them ideally suited to our increasingly demanding, diverse and dispersed global workplace.

I had to snort. Not 10 years ago you could replace the word "Millennials" with "GenX" and it would have been true. And before that the, "tweeners," the folk between GenX and the Boom, got the same treatment. And the boomers before them got it too. Each new generation is the most puralistic, integrated, high-tech generation in American history. Whatever the people being born right now get called will be the same and the Millennials will get to feel a bit fuddy duddy.

My boss is a boomer, and our chief Unix admin is a boomer. That's it for Technical Services, so it doesn't apply as much to us as other groups. We're all GenX here, with one Millennial shared with Telecom who is moving on to something else soon. It's a bit different across the hall in ADMCS, but not a lot.

Protecting against Cosmic Rays

2008-04-09T12:36:00.002-07:00

Apparently Intel filed a patent for a system to protect chips from cosmic rays.

This makes a lot of sense. I've explained to many people over the years just why it is that the computers that run the Space Shuttle are so much less capable than what they have on their desk. Part of that reason is due to cosmic rays. The smaller the transistor feature size, the more vulnerable the transistor is to charge flipping from things like cosmic rays. NASA has to deal with this any time it puts hardware in space.

The Cassini Probe around Saturn regularly goes into safe-modes due to Galactic Cosmic Rays that twiddle bits they aren't supposed to. Again, NASA expected these and engineered around them. Of scientific interest, they've run into different concentrations of these galactic cosmic rays during the cruise to Saturn and while in orbit around Saturn.

So why is Intel worrying about this here on the surface of the Earth? Because we also get cosmic rays down here too. Not nearly as many, but we get them. For years I've used the phrase, "Must have been a cosmic ray strike," when something computer-like breaks in truly weird ways. Only partially am I being flip about it.

In a more wider scope, these 35nm feature size chips they're now coming out with are designed to work in very low radiation environments. Such as the type humans can live in unsupported. So when NASA/ESA/JAXA/Proton send laptops to the ISS, they're probably running older CPU's that are more radiation tolerant. Space is not a good place for supercomputing clusters.

Stupid user tricks

2008-04-09T09:52:00.002-07:00

I had a case of this the other day. I was minding my own business, when suddenly one of my monitors starts going wonky. This is an LCD monitor, but an older one, so it isn't inconceivable that it could be going bad. How else would I explain the weird spots that were showing up on it? They looked like this:

Which looks like weird hot-spots in the screen. So I started to muttering. Plus, the screen was noticeably dimmer. Futzing with the brigthness and contrast settings didn't do a thing for it either. Plus it seemed to follow no matter which window I put on the hot spots.

Then, I realized what the problem was.

Compiz. Somehow, the rdesktop window that represents had been made slightly transparent, and the wall-paper was showing through. This screen shot is with the transparency fully down, you can barely make out the ConsoleOne icon in it.

So no, I didn't have a monitor going bad, I had a mouse mis-cue somewhere that caused that rdesktop window to go a bit transparent. No worries!

Aie.

From Slashdot: Should users manage their own PC's?

2008-04-02T22:07:00.001-07:00

Should IT Shops Let Users Manage Their Own PCs?

It's a very Web 2.0 concept. And there is some merit to it. Back in the day when workstation lock-downs were getting common in workplace settings (ZENworks was good for that), there was a debate about some of this. At my old job one thing we wanted to lock down was the wall-paper. That one thing would help reinforce the idea that this was a WORK Pc, not a home PC. The counter argument to this is that such user environment things are mostly harmless, so permitting them allows the lock-down to be less intrusive on the user.

This is another step in that direction. Workplaces have PC configuration standards for a variety of good reasons. You want all machines plugged into your network to not be festering hives of scum and malware, and these sorts of standards can prevent that. On the other end of the scale, high end users know the tools of their field better than your general IT desktop support person does and in theory can do more with the tools they know versus the tools forced upon them.

On the control end of the spectrum, you keep IT costs down by standardizing the configs in your enterprise. This keeps the Total Cost of Ownership down, a big thing for companies with the right internal costing controls (*nudge nudge*). One tech can support many more end users that way, since the range of things they support is kept to a minimum.

On the freedom end of the spectrum, the end user gets exactly the tools they want to do their job. They're happier that way. And since they support themselves, IT costs are controlled. One tech can support many more end users that way, since the bits they're supporting are significantly reduced.

The 'freedom' end of things runs smack into some standard industry practices, such as volume licensing and big-buy discounts. Dell, for instance, sells PCs cheaper if you buy them by the gross rather than in singles as users are onboarded. Specialized packages like AutoCAD also come cheaper if you buy them in packs of 10 rather than one at a time. Licenses all too often these days are timed and enforced, so you could have end users forgetting to renew the license on their Scrivener install and being non-productive for a few days while purchasing gets them a renewed license. The big 'endpoint management suites', what they seem to be calling the AntiVirus/Firewall package these days, all assume enterprise central control.

On the other hand, users liked being treated like reasoning, intelligent people who are capable of making choices about their work environment. This makes for happier workers.

Also working in this favor is the trend to webify everything in the workplace. The days when you have a whonking big file-server to store all the company data on are slowly going away, and being replaced with things like SharePoint (which can get just as big, don't get me wrong). The fights we've had in the past about how to roll out a new Novell Client to all our desktops would be moot in such an environment as the 'client' is called 'Firefox' (or Gnome, or Office 2007).

On the downside of the 'freedom' end of things is piracy. Tools like Zen Asset Management are there to make sure that the software in use is actually legal. In this freedom environment there is the significantly increased probability of someone bringing their 'backup' copy of something from home to install on their work machine and creating legal liability for the company if they get audited.

Another downside is interoperability problems. The Microsoft Office users create document-macros that the WordPerfect Office users can't run, and the OpenOffice users can't read the WordPerfect files. The Microsoft Office users publish things to SharePoint, where the OpenOffice users drop their stuff onto a handy WebDAV server somewhere. Office peer-pressure will still work on software selection to a point, even if you absolutely love Package Q for your day-to-day work you won't use it if the software everyone else in the office uses can't do a thing with it.

The trade-off here is balancing the chaos and increased direct costs 'freedom' will introduce to the IT environment versus the productivity bonuses and intangible benefits (morale). That will decidedly depend on the culture of the office, and what it is that they do. I know some people who would leave their current jobs just to get the freedom to order the machine they want and use the software they want to use, even if it means somewhat less benefits.

A friend of mine recently changed jobs. The old job was was Microsoft. Since Microsoft is a software development firm of some significant size, they try to dog-food their own stuff wherever possible; even if the tool is a poor fit for the task at hand. She spent a lot of time clubbing her software to do what it didn't really want to do, all the while knowing that there were two non-Microsoft packages that did exactly what she wanted. The new job is not with Microsoft, and the first day there they gave her an order sheet to order the software she wanted; they wanted results and trusted her to turn them in in an understandable format. Thus, the joys of freedom.

So, to answer the question, it depends. It depends on corporate culture to a significant degree, as well as the sector the company is in, as well as the work being done. In highly creative areas such as design, the benefits can be great. In highly regimented areas such as accounting, perhaps not so much or at least a high degree of freedom won't be worth the ultimate costs.

Slow blogging

2008-04-01T19:19:00.003-07:00

I found out at BrainShare that WWU has been accepted as a Novell Authorized Beta site for OES2 SP1. And that's what I've been doing for the better part of the past week. Due to the NDA required, I can't talk about it. So, not much bloggable stuff to bring forward.

We requested entry into the program in part because of what I learned at BrainShare 2007. Specifically, Novell doesn't test for our scales of users. Therefore, it is in our best interest to make sure that organizations like us are in the beta. We have the hardware to make a go of it right now (all those new ESX boxes are liberating some still-useful 3-5 year old servers), and I have the time. Unfortunately, the only 64-bit testing we'll be doing will be in VMWare, so the newest of the new code will have to be really tested by other people.

That's why I've been quiet.

IPv6 vs IPX

2008-03-25T13:57:00.002-07:00

In a session last week came the following comment from a presenter (paraphrased):

How may of you in the room have been at this long enough to do IPX? Ok, great. Now how many of you have done anything with IPv6? Doesn't that look JUST like IPX?

And he's right, to a point. IPX addresses are of the form network-number:node-number, such as:

00008021:0002a540d0e1

Where 'node number' is the MAC address of the network card in question. It's up to the routers to figure out where network-numbers live, and advertised services issue full-network broadcasts to advertise said service, which is the primary reason that IPX just doesn't scale if WAN links are in the mix. But that's by the by.

IPv6 addresses work similarly:

2001:0db8:85a3:08d3:1319:8a2e:0370:7334

The last 48 bits are the MAC address and the bits ahead of it constitute the network number. Except... the IPv6 designers knew about the failings of IPX and worked around them. The last 48 bits don't have to be the MAC address, though as I understand it that address has to exist for each physical interface. Unlike IPX, IPv6 has the ability to have 'secondary' addresses. The lack of this ability was the main reason that Novell Cluster Services only worked on IP networks, which caused its own wave of grief when clustering was introduced in the NetWare 5.1 era. Secondary IPv6 numbers don't have to follow the MAC format, which in my opinion is a good thing!

Yes, when I first read about IPv6 addressing I had that same, "wow, this is just like IPX," moment the BrainShare presenter had. Only, more scalable, and more flexible.

BrainShare Thursday

2008-03-20T16:05:00.002-07:00

Not a good day. My first course, "Advanced BASH," could more accurately be described as, "BASH scripting tips & tricks". I then proceeded to skip the other three sessions I had signed up for.

Novell Open Enterprise Server 2 Interoperability with Windows and AD. All about Domain Services for Windows and Samba. Neither of which we'll ever use. No idea why I wanted to be in this session.
Rapid Deployment of ZENworks Configuration Management. Other people around here have suggested that if we haven't moved yet, wait until at least SP3 before moving. If then. So, demotivated. Plus I was rather tired.
Configuring Samba on OES2. CIFS will do what we need, I don't need Samba. Don't need this one. Skipped.

DL236: Advanced BASH Course
BASH tips and tricks. I got a lot out of it, but the developers around me were quietly derisive.

ZEN Overview and Features
Not so much with the futures, but it did explain Novell's overall ZEN strategy. It isn't a coincidence that most of Novell's recent purchases have been for ZEN products.

TUT303: OES2 Clusters, from beginning to extremes
This was great. They had a full demo rig, and they showed quite a bit in it. Including using Novell Cluster Services to migrate Xen VM's around. They STRONGLY recommended using AutoYast to set up your cluster nodes to ensure they are simply identical except for the bits you explicitly want different (hostname, IP). And also something else I've heard before, you want one LUN for each NSS Pool. Really. Plus, the presenters were rather funny. A nice cap for the day.

And tonight, Meet the Experts!

BrainShare Wednesday

2008-03-20T15:54:00.002-07:00

The Wednesday keynote was, indeed, a bunch of demos. It was also mostly pointless as far as the technology I'm concerned with. Lots of GroupWise (don't care), lots and lots of PlateSpin (can't afford it), lots of Zen (not the bits I'd use).

That said, the new GroupWise WebAccess is gorgeous. I wish Exchange had their non-ActiveX pages look that good.

TUT175: RBAC: Avoiding the horror, getting past the hype
Mostly about IDM as it turned out. Only minimally interesting from an abstract viewpoint about roles in general.

TUT 277: Advanced eDirectory Configuration, new features, and tuning for performance
I learned a few things I didn't know, such as the fact that each object as an "AncestorList" attribute listing who their parent objects are. This apparently greatly speeds up searching. SP3, coming out this Summer, will have faster LDAP binds for a couple of reasons. Right now Novell is recommending 2 million objects as a reasonable maximum size for a partition for performance reasons.

And also they reiterated something I've heard before...

You know how back in the NetWare 4 days, we said to design your tree by geography at the first level, and then get to departments? Um, sorry about that. It was great back then, but for LDAP or IDM it really, really slows things down.

Yep. I took my first class for my CNA when 'Green River' was just coming out, or was just out. So I remember that.

TUT221: iPrint on Linux, what Novell Support wants you to know
A nice session from a mainline support guy about the ways people don't do iPrint on linux correctly. We're not going there until pcounter can run in linux, so this is still somewhat abstract. But, nice to know.

The reason that some print jobs render differently than direct-print jobs, is because of how Windows is designed. Direct-print jobs render with the 'local print provider', and iPrint jobs render with the 'network print provider'. This is a Microsoft thing, not an iPrint thing. You can duplicate it by setting up a microsoft IPP printer (assuming you're not mandating SSL like we are) and printing to the same printer with the same driver.
The Manager on Linux doesn't use a Broker, it uses a 'driver store'.
The Manager on NetWare doesn't always bind to the same broker. I didn't know that.
It is recommended to have only one Broker, or one driver store per tree.
Novell recommends using DNS rather than IP for your printer-agents, check your manager load scripts.

BrainShare Tuesday

2008-03-18T19:21:00.004-07:00

Today started off with a bit of panic, as I hadn't set my alarm. Me being a west-coaster, 7:20 (when I woke up) is an entirely reasonable time to get up as far as my body is concerned. Only, I needed to get dressed and breakfasted before my first session at 8:30. Aie! I had to eat quick, but I got there. Didn't get a chance to check work email, though.

ATT326: Advanced Linux Troubleshooting
An ATT, therefore hard to summarize. But I learned about a few new commands I didn't know about before. Like strace. And vimdiff.

TUT130: Challenges in Storage I/O in Virtualization
Another nice one, but an emergency at work (printing down in a dorm, during finals week) distracted me heavily during the first half of it. Which resulted in the following note in my notes:

NPIV looks really nifty. Look into it.

NPIV being how you can use fibre-channel zoning to zone off VM's, rather than HBA's. Highly useful. I also learned about a neat new thing called Virtual Fabrics. Virtual Fabrics work kind of like VLANS for fabrics. You can segregate your fabrics into fabrics that share hardware but nothing else. Handy if your, say, Solaris admins don't want you mucking about with their zoning, while saving money through consolidated hardware.

TUT216: OES2 SP1 Architectural Overview
There is a LOT of new stuff in SP1.

It will include eDir 8.8.4 (8.8.3 will ship this summer sometime)
NCP and eDir will be fully 64-bit
OES2 SP1 will be based on SLES SP2, which will be releasing about the same time
AFP Support

AFP 3.1
Uses Diffie-Helman 1 for password exchange, meaning the 8-character password problem is solved.
Fully SMP-safe
Has cross-protocol locking with NCP. CIFS doesn't have cross-protocol locking yet, but IIRC, Samba does
Does not need LUM enabled users

CIFS Support

NTLMv1, but v2 is a possibility if enough people ask, so file those enhancement requests!!
CIFS is separate from Samba, therefore can not be used in conjunction with Domain Services for Windows
As with AFP, fully SMP safe

EDir 8.8.4

LDAP auditing enhanced
"newer auth protocols", but they didn't say what.

I should also mention that they're still deploying Novell Integrated Samba, which is what you'll have to use to get Domain Services For Windows. Samba still doesn't scale as far as I'd like ('only' 700-800 concurrent users), so that may be an issue for higher ed types who want high concurrency CIFS and also DSFW on the same box.

TUT211: Enhanced Protocol Support in OES2 SP1
This is the session where they went into detail about the AFP and CIFS support. They said that netatalk, the existing AFP stack on Linux, gets really slow once you go over the 20 concurrent users. Whoa! I can soooo understand why Novell felt the need to make a new one.

The 8 character password limit has been fixed! They now support DH1 for passing passwords.
The 'afptcp' daemon can use one password protocol at a time, so you can only use DH1, or one of the other three I can't remember.
Support for OSX 10.1 and 10.2 is scanty, and 10.5 is limited but users may not notice anyway.
Passwords will be case sensitive.
Kerberos will be in a future release
Performance is faster than NetWare, partly due to the ability to multi-thread
Can register services by way of SLP
Only supports NSS for the time being, the other Linux file-systems will be a future feature.
Can support 500 concurrent users, and 1000+ in the future. This fits our current AFP loads.
We can configure more about how it works than we could on NetWare, such as how many worker threads to spawn.
Has meaninful debug logs!
Has a new command, 'afpstat' that works like 'netstat' for giving a snapshot of afp connections.

And then some CIFS stuff. We can't use it for political reasons so I didn't pay attention. Sorry.

Tonight was the night formerly known as 'Sponsor Night,' but has a new name now that everyone who gets a booth is no longer a 'sponsor'. Some are sponsors, some are exhibiters. I can't keep track. Anyway, today was their party. "World of Novellcraft!" Homage to vid-gaming.

Lots of Wii, lots of Rock Band, some Halo, lots of women dressed in Renaissance Festival gear getting their pictures taken by the 90%+ male audience. I've blogged before about my ambivalence about Sponsor Night. I lasted until about 7, when I came back to the hotel.

Tomorrow I have an actual LUNCH BREAK in my schedule! Ooo! And ~~Soul Asylum~~ ~~Soul Coughing~~ Collective Soul plays the concert! I've been listening to two of their CD's for the past two months so I think I may even know a few songs by now.

Today at Brainshare

2008-03-17T19:33:00.002-07:00

Monday. Opening day. I had trouble getting to sleep last night due to a poor choice of bed-time reading (don't read action, don't read action, don't read action). And had to get up at 6am body time in order to get breakfast before the morning keynote. There be zombies.

Breakfast was uninspired. As per usual, the hashbrowns had cooled to a gellid mass before I found everything and got a seat.

The Monday keynotes are always the CxO talks about strategy and where we're going. Today a mess of press releases from Novell give a good idea what the talks were about. Hovsepian was first, of course, and was actually funny. He gave some interesting tid-bits of knowledge.

Novell's group of partners is growing, adding a couple hundred new ones since last year. This shows the Novell 'ecosystem' is strong.
8700 new customers last year
Novell press mentions are now only 5% negative.

Jeff Jaffe came on to give the big wow-wow speech about Novell's "Fossa" project, which I'm too lazy to link to right now. The big concern is agility. He also identified several "megatrends" in the industry:

High Capacity Computing
Policy Engines
Orchestration
Convergence
Mobility

I'm not sure what 'Convergence' is, but the others I can take a stab at. Note the lack of 'virtualization' in this list. That's soooo 2007. The big problem is now managing the virtualization, thus Orchestration. And Policy Engines.

Another thing he mentioned several times in association with Fossa and agility, is mergers and acquisitions. This is not something us Higher Ed types ever have to deal with, but it is an area in .COM land that requires a certain amount of IT agility to accommodate successfully. He mentioned this several times, which suggests that this strategy is aimed squarely at for-profit industry.

Also, SAP has apparently selected SLES as their primary platform for the SMB products.

Pat Hume from SAP also spoke. But as we're on Banner, and it'll take a sub-megaton nuclear strike to get us off of it, I didn't pay attention and used the time to send some emails.

Oh, and Honeywell? They're here because they have hardware that works with IDM. That way the same ID you use for your desktop login can be tied to the RFID card in your pocket that gets you into the datacenter. Spiffy.

ATT375 Advanced Tips & Tricks for Troubleshooting eDir 8.8
A nice session. Hard to summarize. That said, they needed more time as the Laptops with VMWare weren't fast enough for us to get through many of the exercises. They also showed us some nifty iMonitor tricks. And where the high-yield shoot-your-foot-off weapons are kept.

BUS202 Migrating a NetWare Cluster to OES2
Not a good session. The presenter had a short slide deck, and didn't really present anything new to me other than areas where other people have made major mistakes. And to PLAN on having one of the linux migrations go all lost-data on you. He recommended SAN snapshots. It shortly digressed into "Migrating a NetWare Cluster to Linux HA", which is a different session all together. So I left.

TUT215 Integrating Macintosh with Novell
A very good session. The CIO of Novell Canada was presenting it, and he is a skilled speaker. Apparently Novell has written a new AFP stack from scratch for OES2 Sp1, since NETATALK is comparatively dog slow. And, it seems, the AFP stack is currently out performing the NCP stack on OES2 SP1. Whoa! Also, the Banzai GroupWise client for Mac is apparently gorgeous. He also spent quite a long time (18 minutes) on the Kanaka client from Condrey Consulting. The guy who wrote that client was in the back of the room and answered some questions.

Brainshare Sponsors

2008-03-13T09:36:00.002-07:00

In order to keep costs to us walking sales leads down, Novell solicits sponsors for BrainShare to help subsidize the whole event. There is nothing wrong with that, it means a lot of potential freebies for the people who are good at saying No politely ;).

So I'm offering this list of companies who have booths at BrainShare, what Novell product they're primarily interested in, and how it relates to me. The PDF I'm sucking this off of is this one of the Sponsor Hall.

SAP. The 'Cornerstone Sponsor'. I think everyone who reads my blog knows what they do. At a guess, their primary interest is in Identity Manager. SCT Banner is the ERP for the .EDU space, so we don't use 'em.
IBM. From last year, it's clear this is their Hardware division. So their primary interest is in SLES. We're on a different hardware platform, but... it's hardware. I'll still drop by to look at the pretty.
GWAVA. They make message filtering software for GroupWise. If you need anti-spam/virus for your GW installation, you're probably running GWAVA. We don't use GroupWise, so they have nothing I need.
GroupLink HelpDesk. A Helpdesk product that appears to be cross-platform. Their product is probably Linux, but it wouldn't surprise me to learn that they still have a lot of NetWare hiding back there. We use Magic Helpdesk for that function.
Microsoft. You know who they are. Officially their product is SLES but... who knows what they'll bring. We use a LOT of them around here, what with being an Exchange deployment and owning 96% of the desktops.
Messaging Architects. They are a more general email security and archiving provider. Their product is GroupWise, but they also sell some appliances that I could theoretically use in front of our Exchange servers. We've settled on a product from a much bigger vendor for that function, but still.
Novacoast IT. A consulting firm specializing in Novell. Their products are a wide gamut of Novell stuff, SLES, ZEN, IDM, and GroupWise. We're a poor .EDU, and can't afford consultants.
Honeywell. Honeywell is kind of like GE and IBM, they do a little of everything. I don't know what their Novell tie-in is.
Syncsort. They were one of the first backup products to fully support OES1. They are arguably the backup software that supports Novell stuff the best. Their products are SLES, OES, and NetWare. We looked at them when we were looking for a new backup vendor, but they didn't quite measure up for various reasons. I just might drop by.
Omni. Another consulting firm that specializes in Novell products, but they also have some discrete products. Their web-site says they do SLES, OES, NetWare, GroupWise, and NetMail (now a Messaging Architects product). We're a poor .EDU, and can't afford consultants.
HP. They do hardware. Their booth isn't as big as it was last year, so there will be less pretty to look at. Their product is SLES/OES. They're our hardware vendor, so I'll be talking real good with these folks.
Condrey Corporation. Another consulting company specializing in Novell products. They do IDM, Novell Storage Manager, NetWare, and probably OES/SLES. Poor .edu, can't afford 'em. yadda yadda. Also, we built our own IDM stuff so don't need no steeenkin other stuff.

And a bunch more vendors in smaller booths. Some big names (Blackberry), some not so big (idEngines).

There are exceedingly few (two, really) vendors there that can expect to see any of WWU's money any time soon. Nor is that at all likely to change. Our user head-count (21,000+) and FTE count (13,000+) combine to mean that anything that charges per-user is going to be out of our price-range pretty quickly, or will be subjected to a bidding process. We build our own solutions to problems a lot of the time because of this.

Which means that I'm a very poor sales lead.

It also means I feel a bit guilty trading my contact info for Shiny! during Vendor Night since those vendors are sooo going to strike out when they call me in April.

New novell.com web site

2008-03-11T10:03:00.002-07:00

Novell just updated their web-site.

As in, updated in the last 12 hours or so, so expect some broken links for a while.

Another thing I noticed is a very slight rendering difference between Linux and Windows.

The page as rendered in SeaMoneky from Linux

The page as rendered in Opera from Linux

The page as rendered in SeaMonkey from WinXP

It's a very simple lay-out thing, but it does indent the page that much. I kinda like it.

What I don't like is that the front page is very flash-heavy. I've had issues with flash on x86-64 machines, so I'm a bit burned by it. That said, I do realize that flash is about as prevalent as the ability to render .PNG files so it's a valid web technology.

More HP annoyances

2008-03-06T15:57:00.003-08:00

They've recently revised their alert emails to be even more badly formatted. The below slug of text contains a critical alert. Somewhere.


Your alerts 
Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DSeQ.1Lki.DKEbf000 Priority:
Critical; Products: All-in-One Storage Systems,Disk-to-disk Backup,HP Integrity
Entry-level Servers,HP Integrity High-end Servers,HP Integrity Mid-range Servers;
OS: not applicable; Release Date: Feb 26 2008; Description: Advisory: (Revision)
FIRMWARE UPGRADE or WORKAROUND REQUIRED to Prevent Rare Scenario of Potential
Logical Drive Failure on HP Smart Array Controller Attached to Multiple Drive
Arrays if Drive Failure or Incorrect Drive Replacement Occurs After Power Loss
(c01232270)  Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgW.1Lki.ccEcI000 Priority:
Recommended; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP
ProLiant ML Servers,MSA Disk Arrays,Server Controllers; OS: not applicable; Release
Date: Feb 28 2008; Description: Advisory: FIRMWARE UPGRADE RECOMMENDED for Certain
HP Smart Array Controllers to Avoid False SAS and SATA Hard Drive  (c01382041) 
Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTf8.1Lki.DeBcEbI0 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
ML Servers,HP ProLiant Packaged Cluster Servers,Server/Storage Infrastructure
Management Software; OS: not applicable; Release Date: Feb 20 2008; Description:
Advisory: HP Systems Insight Manager (HP SIM) Running in an Environment with a
Large Number of WBEM Managed Nodes May Experience Task Page Interface Slowdown or
Out of Memory Errors (c01371984)  Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgo.1Lki.DAMEdA00 Priority:
Routine; Products: HP ProLiant BL Server Blades,Server Management Software; OS: not
applicable; Release Date: Feb 28 2008; Description: Advisory: Virtual Connect
Enterprise Manager (VCEM) 1.0 May Not Be Able To Add Virtual Connect (VC) Domains
to a Virtual Connect Domain Group After Updating the VC Domain Group on a ProLiant
Server (c01382035)  Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgi.1Lki.CPQEca00 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
Packaged Cluster Servers; OS: not applicable; Release Date: Feb 28 2008;
Description: Advisory: ProLiant Essentials Virtual Machine Manager (VMM) Displays
Incorrect VMM Warning Message on FireFox Browser for ActiveX Controls (c01382044) 
Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgQ.1Lki.MAEcC000 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
ML Servers,HP ProLiant Packaged Cluster Servers; OS: not applicable; Release Date:
Feb 28 2008; Description: Advisory:  (c01382042)  Document: Customer Advisory;
Link: http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgg.1Lki.CJcEcY00
Priority: Routine; Products: HP ProLiant DL Servers,HP ProLiant ML Servers,HP
ProLiant Packaged Cluster Servers,Server Network Interface Cards; OS: not
applicable; Release Date: Feb 28 2008; Description: Advisory: Novell NetWare
Teaming Driver (QASM.LAN) May Fail to Load After Upgrading to ProLiant Support Pack
for Novell NetWare 7.80 (or later) (c01382039)  Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgU.1Lki.XIEcG000 Priority:
Routine; Products: All-in-One Storage Systems,HP Integrity Entry-level Servers,HP
Integrity High-end Servers,HP Integrity Mid-range Servers,HP ProLiant BL Server
Blades; OS: not applicable; Release Date: Feb 28 2008; Description: Advisory:
(Revision) HP ProLiant Smart Array SAS/SATA Event Notification Service Version
6.4.0.xx Does Not Log All Events to the Windows Registry (c01177411)  Document:
Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTgk.1Lki.CVEEcc00 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
ML Servers,HP ProLiant Packaged Cluster Servers,ProLiant Essentials Software; OS:
not applicable; Release Date: Feb 28 2008; Description: Advisory: SmartStart
Scripting Toolkit Reboot Utility May Not Respond Or May Display a Segmentation
Fault Error On a ProLiant Server Running SUSE LINUX Enterprise Server 10 Service
Pack 1 (SP1) (c01382031)  Document: Customer Notice; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DTh2.1Lki.DdWYEbE0 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
ML Servers; OS: not applicable; Release Date: Feb 28 2008; Description: Notice:
Linux System Health Application and Insight Management Agents (hpasm),
Lights-Out-Driver and Agents (hprsm), and NIC Agents (cmanic) Are Now Delivered as
a Single rpm Package for all Supported HP ProLiant Linux Servers (c01382040) 
Document: Customer Advisory; Link:
http://alerts.hp.com/r?2.1.3KT.2ZR.xl4lg.C0m3Bi..T.DU7I.1Lki.DbNQEaL0 Priority:
Routine; Products: HP ProLiant BL Server Blades,HP ProLiant DL Servers,HP ProLiant
ML Servers,HP ProLiant Packaged Cluster Servers; OS: not applicable; Release Date:
Feb 28 2008; Description: Advisory: Virtual Machine Manager (VMM) 3.1 May Cause a 
(c01383032)

This is a plain-text email, no HTML->Plain formatting weirdness. It COMES this glommed together. Time to send a cranky-gram.

Flash on openSUSE 10.3

2008-03-05T09:08:00.004-08:00

For the past few months the flash plugin hasn't been working for me. I didn't miss it much since I have a WinXP VM up all the time and it can play them, just not sound. I hadn't been using it much since the nspluginwrapper processes had a tendency to hang once in a while and consume 100% single-thread CPU. Annoying that. As I use the 'flashblock' plugin, it didn't bite me hard I just didn't click on flash unless I really wanted to view it. Since like 80% of non-text ads are now delivered as flash, this has greatly reduced the advertising I have to sit through and fly-ins like CNN is now doing are transparent.

But. No sound for my YouTube! So, I had to fix that.

Right now I'm running openSUSE 10.3, and using Seamonkey as my primary browser (though Firefox was broken for flash too). It took me close to two hours to figure out what the heck went wrong and how to fix it.

Everything I found said I should run the following command to just make it work:

nspluginwrapper -v -i /usr/lib/browser-plugins/libflashplayer.so

Except I kept getting this error:

nspluginwrapper: no appropriate viewer found for ./libflashplayer.so

What ultimately ended up fixing it is the following series of commands:

As root:

zypper rm nspluginwrapper nspluginwrapper-i386. This removed the existing nspluginwrapper install, which I suspect was borked.
zypper in nspluginwrapper nspluginwrapper-i386. This installed both packages. Both packages ARE required for this to work on x86-64 machines. Remember, nspluginwrapper allows you to run 32-bit plugins in a 64-bit browser, so it has to cross the boundaries.

As my primary user:

nspluginwrapper -v -i /usr/lib/browser-plugins/libflashplayer.so This gave the following output:

Install plugin /usr/lib/browser-plugins/libflashplayer.so
into /home/[user]/.mozilla/plugins/npwrapper.libflashplayer.so

Also of note, the resulting binary, npwrapper.libflashplayer.so, is about 90% larger than the old binary it replaced. I know that nspluginwrapper has had some updates since openSUSE 10.3 came out, and I suspect that a lot has changed. So I have high hopes that perhaps the hanging-plugin problem will go away. We shall see.

Also of note, I believe that running the nspluginwrapper -v -i process may have to be done every time nspluginwrapper gets updated. But, it would seem I have to explicitly upgrade it so remembering to do it shouldn't be an issue.