News





CodeSOD: A Type of Standard;

I’ve brushed up against the automotive industry in the past, and have gained a sense about how automotive companies and their suppliers develop custom software. That is to say, they hack at it until someone from the business side says, “Yes, that’s what we wanted.” 90% of the development time is spent doing re-work (because no one, including the customer, understood the requirements) and putting out fires (because no one, including the customer, understood the requirements well enough to tell you how to test it, so things are going wrong in production).

Mary is writing some software that needs to perform automated testing on automotive components. The good news is that the automotive industry has adopted a standard API for accomplishing this goal. The bad news is that the API was designed by the automotive industry. Developing standards, under ideal conditions, is hard. Developing standards in an industry that is still struggling with software quality and hasn’t quite fully adopted the idea of cross-vendor standardization in the first place?

You’re gonna have problems.

The specific problem that led Mary to send us this code was the way of defining data types. As you can guess, they used an XML schema to lay out the rules. That’s how enterprises do this sort of thing.

There are a bunch of “primitive” data types, like UIntVariable or BoolVariable. There are also collection types, like Vector or Map or Curve (3D plot). You might be tempted to think of the collection types in terms of generics, or you might be tempted to think about how XML schemas let you define new elements, and how these make sense as elements.

If you are thinking in those terms, you obviously aren’t ready for the fast-paced world of developing software for the automotive industry. The correct, enterprise-y way to define these types is just to list off combinations:

<xs:simpleType name="FrameworkVarType">
        <xs:annotation>
                <xs:documentation>This type is an enumeration of all available data types on Framework side.</xs:documentation>
        </xs:annotation>
        <xs:restriction base="xs:string">
                <xs:enumeration value="UIntVariable"/>
                <xs:enumeration value="IntVariable"/>
                <xs:enumeration value="FloatVariable"/>
                <xs:enumeration value="BoolVariable"/>
                <xs:enumeration value="StringVariable"/>
                <xs:enumeration value="UIntVectorVariable"/>
                <xs:enumeration value="IntVectorVariable"/>
                <xs:enumeration value="FloatVectorVariable"/>
                <xs:enumeration value="BoolVectorVariable"/>
                <xs:enumeration value="StringVectorVariable"/>
                <xs:enumeration value="UIntMatrixVariable"/>
                <xs:enumeration value="IntMatrixVariable"/>
                <xs:enumeration value="FloatMatrixVariable"/>
                <xs:enumeration value="BoolMatrixVariable"/>
                <xs:enumeration value="StringMatrixVariable"/>
                <xs:enumeration value="FloatIntCurveVariable"/>
                <xs:enumeration value="FloatFloatCurveVariable"/>
                <xs:enumeration value="FloatBoolCurveVariable"/>
                <xs:enumeration value="FloatStringCurveVariable"/>
                <xs:enumeration value="StringIntCurveVariable"/>
                <xs:enumeration value="StringFloatCurveVariable"/>
                <xs:enumeration value="StringBoolCurveVariable"/>
                <xs:enumeration value="StringStringCurveVariable"/>
                <xs:enumeration value="FloatFloatIntMapVariable"/>
                <xs:enumeration value="FloatFloatFloatMapVariable"/>
                <xs:enumeration value="FloatFloatBoolMapVariable"/>
                <xs:enumeration value="FloatFloatStringMapVariable"/>
                <xs:enumeration value="FloatStringIntMapVariable"/>
                <xs:enumeration value="FloatStringFloatMapVariable"/>
                <xs:enumeration value="FloatStringBoolMapVariable"/>
                <xs:enumeration value="FloatStringStringMapVariable"/>
                <xs:enumeration value="StringFloatIntMapVariable"/>
                <xs:enumeration value="StringFloatFloatMapVariable"/>
                <xs:enumeration value="StringFloatBoolMapVariable"/>
                <xs:enumeration value="StringFloatStringMapVariable"/>
                <xs:enumeration value="StringStringIntMapVariable"/>
                <xs:enumeration value="StringStringFloatMapVariable"/>
                <xs:enumeration value="StringStringBoolMapVariable"/>
                <xs:enumeration value="StringStringStringMapVariable"/>
        </xs:restriction>
</xs:simpleType>

So, not only is this just awkward, it’s not exhaustive. If you, for example, wanted a curve that plots integer values against integer values… you can’t have one. If you want a StringIntFloatMapVariable, your only recourse is to get the standard changed, and that requires years of politics, and agreement from all of the other automotive companies, who won’t want to change anything out of fear that their unreliable, hacky solutions will break.

[Advertisement] Atalasoft’s imaging SDKs come with APIs & pre-built controls for web viewing, browser scanning, annotating, & OCR/barcode capture. Try it for 30 days with included support.


Error'd: PIck an Object, Any Object;

"Who would have guessed Microsoft would have a hard time developing web apps?" writes Sam B.

 

Jerry O. writes, "So, if I eat my phone, I might get acid indigestion? That sounds reasonable."

 

"Got this when I typed into a SwaggerHub session I'd left open overnight and tried to save it," wrote Rupert, "The 'newer' draft was not, in fact, the newer version."

 

Antonio write, "It's nice to buy software from another planet, especially if year there is much longer."

 

"Either Meteorologist (http://heat-meteo.sourceforge.net/) is having some trouble with OpenWeatherMap data, or we're having an unusually hot November in Canada," writes Chris H.

 

"This is possibly one case where a Windows crash can result in a REAL crash," writes Ruben.

 

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!


Representative Line: A Case of File Handling;

Tim W caught a ticket. The PHP system he inherited allowed users to upload files, and then would process those files. It worked… most of the time. It seemed like a Heisenbug. Logging was non-existent, documentation was a fantasy, and to be honest, no one was exactly 100% certain what the processing feature was supposed to do- but whatever it was doing now was the right thing, except the times that it wasn’t right.

Specifically, some files got processed. Some files didn’t. They all were supposed to.

But other than that, it worked.

Tim worried that this was going to be difficult to replicate, especially after he tried it with a few files he had handy. Digging through the code though, made it perfectly clear what was going on. Buried on about line 1,200 in a 3,000 line file, he found this:

while (false !== ($file = readdir($handle))) {
    if ($file != "." && $file != ".." && ( $file == strtolower($file) ) ) {
        …
    }
}

For some reason, this code required that the name of the file contain no capital letters. Why? Well, again, no documentation, no comments, and the change predated the organization’s use of source control. Someone put in the effort to add the feature, but was it necessary?

Tim took the line out, and now it processes all files. Unfortunately, it’s still only working most of the time, but nobody can exactly agree on what it’s doing wrong.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!


News Roundup: Calculated;

A long time ago, in a galaxy right here, we ran a contest. The original OMGWTF contest was a challenge to build the worst calculator you possibly could.

We got some real treats, like the Universal Calculator, which, instead of being a calculator, was a framework for defining your own calculator, or Rube Goldberg’s Calculator, which eschewed cryptic values like “0.109375”, and instead output “seven sixty-fourths” (using inlined assembly for performance!). Or, the champion of the contest, the Buggy Four Function Calculator, which is a perfect simulation of a rotting, aging codebase.

The joke, of course, is that building a usable calculator app is easy. Why, it’s so easy, that we challenged our readers to come up with ways to make it hard. To find creative ways to fail at handling this simple task. To misinterpret and violate basic principles of how calculators should work.

Well, I bring this up, because just a few days ago, iOS 11.2 left beta and went public. And finally, finally, they fixed the calculator, which has been broken since iOS 11 launched. How broken? Let's try 1+2+3+4+5+6 shall we?

For those who can't, or don't wish to watch the video, according to the calculator, 1+2+3+4+5+6 is 75. I entered the values in quickly, but not super-speed.

I personally discovered the bug for myself while scoring at the end of a round of board games. I just ran down the score-sheet to sum things up, tapping away like one does with a calculator, and got downright insane results.

The underlying cause, near as anyone has been able to tell, is a combination of input lag and display updates, so rapidly typing “1+2+3” loses one of the “+”es and becomes “1+23”.

Now Apple’s been in the news a lot recently- in addition to shipping a completely broken calculator, they messed up character encoding, causing “I” to display a placeholder character, released a macOS update which allowed anyone to log in as root with no password, patched it, but with the problem that the patch broke filesharing, and if you didn’t apply it in the “right” order, the bug could come back.

The root cause of the root bug, by the way, was due to bad error handling in the login code.

Now, I’ll leave it to the pundits to wring their hands over the decline of Apple’s code quality, worry that “is this the future of Apple?!?!!11?”, or claim “this never would have happened under Jobs”. I’m not interested in the broad trends here, or prognosticating, or prognostibating (where you please only yourself by imagining alternate realities where Steve Jobs still lives).

What I am interested in is that calculator app. Some developer, I’m gonna assume a more junior one (right? you don’t need 15 years of experience to reimplement a calculator app), really jacked that up. And at no point in testing did anyone actually attempt to use the calculator. I’m sure they ran some automated UI tests, and when they saw odd results, they started chucking some sleep() calls in there until the errors went away.

It’s just amazing to me, that we ran a contest built around designing the worst calculator you could. A decade later, Apple comes sauntering in, vying for an honorable mention, in an application they actually shipped.

[Advertisement] High availability, Load-balanced or Basic – design your own Universal Package Manager, allow the enterprise to scale as you grow. Download and see for yourself!


Editor's Soapbox: Protect Yourself;
We lend the soapbox to snoofle today, to dispense a combination of career and financial advice. I've seen too many of my peers sell their lives for a handful of magic beans. Your time is too valuable to waste for no reward. -- Remy

There is a WTF that far too many people make with their retirement accounts at work. I've seen many many people get massively financially burned. A friend recently lost a huge amount of money from their retirement account when the company went under, which prompted me to write this to help you prevent it from happening to you.

A pile of money

The housing bubble that led up to the 2008 financial collapse was caused by overinflated housing values coming back down to reality. People had been given mortgages far beyond what they could afford using traditional financial norms, and when the value of their homes came back down to realistic values, they couldn't afford their mortgages and started missing payments, or worse, defaulted. This left the banks and brokerages that were holding the mortgage-backed-securities with billions in cash flow, but upside down on the balance sheet. When it crossed a standard threshold, they went under. Notably Bear Stearns and Lehman. Numerous companies (AIG, Citi, etc.) that invested in these MBS also nearly went under.

One person I knew of had worked at BS for many years and had a great deal of BS stock in their retirement account. When they left for Lehman, they left the account in-tact at BS. Then they spent many years at Lehman. When both melted down, that person not only lost their job, but the company stock in both retirement accounts was worth... a whole lot less.

As a general statement, if you work for a company, don't buy only stock of that company in your retirement account because if the place goes belly up, you lose twice: your job and your retirement account!

Another thing people do is accept stock options in lieu of pay. Startups are big on doing this as it limits the cash outflow when they are new. If they succeed, they'll have the cash to cover the options. If they go bust, you lose. Basically, you put in the long hours and take a large chunk of the financial risk on the hopes that the managers know what they're doing, and are one of the lucky unicorns that "makes it". But large companies also pay people (in part) in options. A friend worked their way up to Managing Director of a large firm. He was paid 20% cash and 80% company stock options, but had to hold the options for five years before he was allowed to exercise them - so that he'd be vested in the success of the company. By the time the sixth year had rolled by, he had forgotten about it and let-it-ride, with the options auto-exercising and being converted into regular shares. When he left the job, he left the account in-tact and in-place. When the market tanked, so did the value of the stock that he had earned and been awarded.

When you leave a job, either voluntarily or forcibly, roll the assets in your retirement account in-kind into a personal retirement account at any bank or brokerage that provides that (custodian) service. You won't pay taxes if you do a direct transfer, but if some company where you used to work goes under, you won't have to chase lawyers to get what belongs to you.

Remember, Bill Gates routinely divested huge blocks of MS stock as part of diversifying, even while it was still increasing in value. Your numbers will be smaller but the same principle applies to you too (e.g.: Don't put all your eggs in one basket).

While the 2008 fiasco and dot-com bust will hopefully never be repeated, in the current climate of deregulation, you never know. If you've heavily weighted your retirement account with company stock, or have a trail of retirement accounts at former employers, please go talk to a financial advisor about diversifying your holdings, and collect the past corporate retirement accounts in a single personal retirement brokerage account, where you can more easily control it and keep an eye on it.

Personally, I'm retired. My assets are split foreign/domestic, bonds/equities, large/medium/small-cap and growth/blend/value. a certain percentage is professionally managed, but I keep an eye on what they're doing and the costs. The rest is in mutual funds that cover the desired sectors, etc.

The amounts and percentages across investment types in which you invest will vary by your age, total assets and time horizon. Only you can know what's best for your family, but you should discuss it with an independent advisor (before they repeal the fiduciary rule, which states that they must put your interests ahead of what their firm is pushing).

For what it's worth, over my career, I've worked at five companies that went under, more than twenty years down the road after I moved on. I have always taken the cash value of the pension/401(k) and rolled it into a brokerage account where I manage it myself. Had I left those assets at the respective companies, I would have lost over $100,000 of money that I had earned and been awarded - for absolutely no reason!

Consider for a moment that the managers that we all too often read about in this space are often the same ones who set up and manage these workplace retirement plans. Do you really want them managing money that you've already earned? Especially after you've moved on to the next gig? When you're not there to hear office gossip about Bad Things™ that may be happening?

One final point. During the first few years of my career, there were no 401(k)'s. If you didn't have a pension, your savings account was your main investment vehicle. Unless the IRA and 401(k) plan rules are changed, you can start saving very early on. At first, it seems like it accumulates very slowly, but the rate of growth increases rapidly as you get nearer to the end of your career. The sooner you start saving for the big ticket items down the road, the quicker you'll be able to pay for them. Patience, persistence and diversification are key!

As someone who has spent the last quarter century working for these massive financial institutions, I've seen too many people lose far too much; please protect yourself!

[Advertisement] Otter, ProGet, BuildMaster – robust, powerful, scalable, and reliable additions to your existing DevOps toolchain.


CodeSOD: Pounding Away;

“Hey, Herbie, we need you to add code to our e-commerce package to send an email with order details in it,” was the requirement.

“You mean like a notification? Order confirmation?”

“Yes!”

So Herbie trotted off to write the code, only to learn that it was all wrong. They didn’t want a human-readable confirmation. The emails were going to a VB application, and they needed a machine-readable format. So Herbie revamped the email to have XML, and provided an XML schema.

This was also wrong. Herbie’s boss wrangled Herbie and the VB developer together on a conference call, and they tried to hammer out some sort of contract for how the data would move from system to system.

They didn’t want the data in any standard format. They had their own format. They didn’t have a clear idea about the email was supposed to contain, either, which meant Herbie got to play the game of trying his best to constantly revamp the code as they changed the requirements on the fly.

In the end, he produced this monster:

   private function getAdminMailString(){
        $adminMailString = '';
        $mediaBeans = $this->orders->getConfiguredImageBeans();
        $mediaBeansSize = count($mediaBeans);

        $adminMailString .= '###order-start###'."\n";
        $adminMailString .= '###order-size-start###' . $mediaBeansSize . "###order-size-end###\n";
        $adminMailString .= '###date-start###' . date('d.m.Y',strtotime($this->context->getStartDate())) . "###date-end###\n";
        $adminMailString .= '###business-context-start###' . $this->context->getBusinessContextName() . "###business-context-end###\n";

        if($this->customer->getIsMassOrderUser()){

            $customers = $this->customer->getSelectedMassOrderCustomers();
            $customersSize = count($this->customer->getSelectedMassOrderCustomers());

            $adminMailString .= '###is-mass-order-start###1###is-mass-order-end###'."\n";
            $adminMailString .= '###mass-order-size-start###'.$customersSize.'###mass-order-size-end###'."\n";

            $adminMailString .= '###mass-start###'."\n";
            for($i = 0; $i < $customersSize; $i++){

                $adminMailString .= '###mass-customer-' . $i . '-start###'."\n";
                $adminMailString .= '###customer-start###' . $customers[$i]->getCompanyName() . '###customer-end###'."\n";
                $adminMailString .= '###customer-number-start###' . $customers[$i]->getCustomerNumber() . '###customer-number-end###'."\n";
                $adminMailString .= '###contact-person-start###' . $customers[$i]->getContactPerson() . '###contact-person-end###'."\n";
                $adminMailString .= '###mass-customer-' . $i . '-end###'."\n";

            }
            $adminMailString .= '###mass-end###'."\n";

        } else {
            $adminMailString .= '###is-mass-order-start###0###is-mass-order-end###'."\n";
        }

        for($i = 0; $i < $mediaBeansSize; $i++){

            $adminMailString .= '###medium-' . $i . "-start###\n";

            if($mediaBeans[$i] instanceof ConfiguredImageBean){

                $adminMailString .= '###type-start###picture###type-end###' . "\n";
                $adminMailString .= '###name-start###' . $mediaBeans[$i]->getTitle() . "###name-end###\n";
                $adminMailString .= '###url-start###' . $mediaBeans[$i]->getConfiguredImageWebPath() . "###url-end###\n";

            } else if($mediaBeans[$i] instanceof MovieBean){

                $adminMailString .= '###type-start###movie###type-end###' . "\n";
                $adminMailString .= '###name-start###' . $mediaBeans[$i]->getTitle() . "###name-end###\n";
                $adminMailString .= '###url-start###' . $mediaBeans[$i]->getMoviePath() . "###url-end###\n";

            } else {
                throw new Exception('Bean is wether of type ConfiguredImageBean nor MovieBean!');
            }

            $adminMailString .= '###medium-' . $i . "-end###\n";
        }

        $adminMailString .= '###order-end###'."\n";

        return $adminMailString;
    }

Yes, that’s XML, if instead of tags you used ###some-field-start###value###some-field-end#### in place of traditional tags. Note how in many cases, the tag name itself is dynamic: $adminMailString .= '###medium-' . $i . "-start###\n";

It was bad enough to generate it, but Herbie was glad he wasn’t responsible for parsing it.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!


Error'd: Get Inspired;

"The great words of inspirationalAuthor.firstName inspirationalAuthor.lastName move me every time," wrote Geoff O.

 

Mark R. writes, "Some countries out there must have some crazy postal codes."

 

"I certainly hope the irony isn't lost on the person who absolutely failed sending out a boiler plate email on the subject of machine learning!" Mike C. wrote.

 

Adrian R. writes, "I'd love to postpone this update, but I feel like I'm playing button roulette."

 

"It's pretty cool that Magneto asks to do a backup before an upgrade," Andrea wrote, "It's only 14TB."

 

Ky W. writes, "I sure hope that these developers didn't write the avaionics software."

 

[Advertisement] High availability, Load-balanced or Basic – design your own Universal Package Manager, allow the enterprise to scale as you grow. Download and see for yourself!