I’m not sure if my expectations are just too high for today’s world. When I buy a new vehicle, I don’t want to see surface rust forming two weeks after it leaves the lot. I don’t like the current presidential election and the horrible choice that voters have to make. And I actually expect that network vendors will put out decent code, or at least be very up front and open when significant faults are found.
You see, those significant faults have real-world consequences. They bring operations to a screeching halt, and diminish organizational credibility. And ill-conceived “work arounds” and cavalier vendor attitudes to the customer’s bug-induced plight just make matters worse.
Here’s a real-world example.
I had a carefully worked-out maintenance window to upgrade both ends of a site-to-site VPN topology that spans Syracuse to London, using my favorite cloud-managed vendor’s gear. I’ve done this procedure at least a half dozen times, and have installed at least 30 of this particular security appliance. My Syracuse work was coordinated with a gent on the other end, and we’d do one end at a time. But… we never got past my end.
I configured the new appliance with what few settings it needed: IP address, gateway, subnet mask, and DNS servers. I saved them, then I waited for the indications that the box had made contact with the cloud and pulled down it’s updates. But those indications never came.
Like many a networker would do, I went to verify that the settings that I entered were correct. Curiously, there were NO settings saved. OK- maybe I forgot to save… The second try yielded the exact same result as the first. It was time to open a support case- as my maintenance window ticked away and my partner in London waited patiently.
I opened the case, then immediately called the support line (for the sake of expedience). I was told that this particular appliance has a firmware bug straight from the factory and that I’d need to find a DHCP-served network to use because it won’t actually save anything you enter with out-of-box firmware. When I asked if this was documented anywhere, I was told very matter-of-factly “we don’t share that information with customers” and that it shouldn’t be a big deal to just use DHCP.
Most places I’ve installed these appliances don’t have DHCP services readily available, because ultimately the appliances use a static IP and eventually ARE the DHCP servers for inside clients. And, I don’t tend to lug around an extra SOHO router just on the off-chance I’ll have to jam something in that can act like a DHCP server to get around a code bug that my vendor doesn’t feel customers need to know about before they actually try to use the product.
Let’s skip to the end:
- I got to use some of my best “military” language after I realized the gravity of the situation
- The maintenance window was busted, and the scheduled change didn’t happen
- I probably lost credibility with my London partner as I was the Guy in Charge for this
- My vendor has absolutely lost my confidence given the bug, and the “you should just be okay with this” attitude. I’m just not sure I can trust them at this point
- This vendor had my respect and trust for years, and those have pretty much been undone with this one incident
So… I dragged the appliance off to where I could hook it up to a DHCP server and it could get a firmware upgrade. We’ll have to do the same on the London end, and then reschedule the outage and maintenance.
Sadly, the examples don’t end here. Same vendor- different hardware set. Also dealing with a long-running problem with a feature set that absolutely adds to the appliance’s stratospheric price tag. The work around? Don’t use the feature. The feature that I bought- to use. It’s insanity, and it’s way too frequent.
And I can just deal with that, because code bugs are pretty much a way of life anymore with certain vendors.