The Daily WTF

Coded Smorgasbord: Archive This;

Michael W came into the office to a hair-on-fire freakout: the midnight jobs failed. The entire company ran on batch processes to integrate data across a dozen ERPs, mainframes, CRMs, PDQs, OMGWTFBBQs, etc.: each business unit ran its own choice of enterprise software, but then needed to share data. If they couldn’t share data, business ground to a halt.

Business had ground to a halt, and it was because the archiver job had failed to copy some files. Michael owned the archiver program, not by choice, but because he got the short end of that particular stick.

The original developer liked logging. Pretty much every method looked something like this:

public int execute(Map arg0, PrintWriter arg1) throws Exception {
    Logger=new Logger(Properties.getString("LOGGER_NAME"));
    Log=new Logger(arg1);
catch (Exception e) {
    Logger.error("Monitor: Incorrect arguments");
    Log.printError("Monitor: Incorrect arguments");
    arg1.write("In Correct Argument Passed to Method.Please Check the Arguments passed \n \r");
    System.out.println("Monitor: Incorrect arguments");

Sometimes, to make the logging even more thorough, the catch block might look more like this:

catch(Exception e){
    Logger.error("An exception happened during SFTP movement/import. " + (String)e.getMessage());

Java added Generics in 2004. This code was written in 2014. Does it use generics? Of course not. Every Hashtable is stringly-typed:

Hashtable attributes;
if (((String) attributes.get(key)).compareTo("1") == 0 | ((String) attributes.get(key)).compareTo("0") == 0) { … }

And since everything is stringly-typed, you have to worry about case-sensitive comparisons, but don’t worry, the previous developer makes sure everything’s case-insensitive, even when comparing numbers:

if (flag.equalsIgnoreCase("1") ) { … }

And don’t forget to handle Booleans…

public boolean convertToBoolean(String data) {
    if (data.compareToIgnoreCase("1") == 0)
        return true;
        return false;

And empty strings…

if(!TO.equalsIgnoreCase("") && TO !=null) { … }

Actually, since types are so confusing, let’s make sure we’re casting to know-safe types.

catch (Exception e) {
    Logger.error((Object)this, e.getStackTraceAsString(), null, null);

Yes, they really are casting this to Object.

Since everything is stringly typed, we need this code, which checks to see if a String parameter is really sure that it’s a string…

protected void moveFile(String strSourceFolder, Object strSourceObject,
                     String strDestFolder) {
    if (strSourceObject.getClass().getName().compareToIgnoreCase("java.lang.String") == 0) { … }

Now, that all was enough to get Michael’s blood pressure up, but none of that had anything to do with his actual problem. Why did the copy fail? The logs were useless, as they were spammed with messages with no particular organization. The code was bad, sure, so it wasn’t surprising that it crashed. For a little while, Michael thought it might be the getFiles method, which was supposed to identify which files needed to be copied. It did a recursive directory search (with no depth checking, so one symlink could send it into an infinite loop) nor did it actually filter files that it didn’t care about. It just made an ArrayList of every file in the directory structure and then decided which ones to copy.

He spent some time really investigating the copy method, to see if that would help him understand what went wrong:

sourceFileLength = sourceFile.length();
newPath = sourceFile.getCanonicalPath();
newPath = newPath.replace(".lock", "");
newFile = new File(newPath);
destFileLength = newFile.length();
    //Copy In Progress
//Remy: I didn't elide any code from the inside of that while loop- that is exactly how it's written, as an empty loop.

Hey, out of curiosity, what does the JavaDoc have to say about renameTo?

Many aspects of the behavior of this method are inherently platform-dependent: The rename operation might not be able to move a file from one filesystem to another, it might not be atomic, and it might not succeed if a file with the destination abstract pathname already exists. The return value should always be checked to make sure that the rename operation was successful.

It only throws exceptions if you don’t supply a destination, or if you don’t have permissions to the files. Otherwise, it just returns false on a failure.

So… if the renameTo operation fails, the archiver program will drop into an infinite loop. Unlogged. Undetected. That might seem like the root cause of the failure, but it wasn’t.

As it turned out, the root cause was that someone in ops hit “Ok” on a security update, which triggered a reboot, disrupting all the scheduled jobs.

Michael still wanted to fix the archiver program, but there was another problem with that. He owned the InventoryArchiver.jar. There was also OrdersArchiver.jar, and HRArchiver.jar, and so on. They had all been “written” by the same developer. They all did basically the same job. So they were all mostly copy-and-paste jobs with different hard-coded strings to specify where they ran. But they weren’t exactly copy-and-paste jobs, so each one had to be analyzed, line by line, to see where the logic differences might possibly crop up.

[Advertisement] Infrastructure as Code built from the start with first-class Windows functionality and an intuitive, visual user interface. Download Otter today!

Alien Code Reuse;

“Probably the best thing to do is try and reorganize the project some,” Tim, “Alien”’s new boss said. “It’s a bit of a mess, so a little refactoring will help you understand how the code all fits together.”

“Alien” grabbed the code from git, and started walking through the code. As promised, it was a bit of a mess, but partially that mess came from their business needs. There was a bunch of common functionality in a Common module, but for each region they did business in- Asia, North America, Europe, etc.- there was a region specific deployable, each in its own module. Each region had its own build target that would include the Common module as part of the build process.

The region-specific modules were vaguely organized into sub-modules, and that’s where “Alien” settled in to start reorganizing. Since Asia was the largest, most complicated module, they started there, on a sub-module called InventoryManagement. THey moved some files around, set up the module and sub-modules in Maven, and then rebuilt.

The Common library failed to build. This gave “Alien” some pause, as they hadn’t touched anything pertaining to the Common project. Specifically, Common failed to build because it was looking for some files in the Asia.InventoryManagement sub-module. Cue the dive into the error trace and the vagaries of the build process. Was there a dependency between Common and Asia that had gone unnoticed? No. Was there a build-order issue? No. Was Maven just being… well, Maven? Yes, but that wasn’t the problem.

After hunting around through all the obvious places, “Alien” eventually ran an ls -al.

~/messy-app/base/Common/src/com/mycompany > ls -al
lrwxrwxrwx 1 alien  alien    39 Jan  4 19:10 InventoryManagement -> ../../../../../Asia/InventoryManagement/src/com/mycompany/IM/
drwxr-x--- 3 alien  alien  4096 Jan  4 19:10 core/

Yes, that is a symbolic link. A long-ago predecessor discovered that the Asia.InventoryManagement sub-module contained some code that was useful across all modules. Acutally moving that code into Common would have involved refactoring Asia, which was the largest, most complicated module. Presumably, that sounded like work, so instead they just added a sym-link. The files actually lived in Asia, but were compiled into Common.

“Alien” writes, “This is the first time in my over–20-year working life I see people reuse source code like this.”

They fixed this, and then went hunting, only to find a dozen more examples of this kind of code “reuse”.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Error'd: Alphabetical Soup;

"I appreciate that TIAA doesn't want to fully recognize that the country once known as Burma now calls itself Myanmar, but I don't think that this is the way to handle it," Bruce R. writes.


"MSI Installed an update - but I wonder what else it decided to update in the process? The status bar just kept going and going..." writes Jon T.


Paul J. wrote, "Apparently my occupation could be 'All Other Persons' on this credit card application!"


Geoff wrote, "So I need to commit the changes I didn't make, and my options are 'don't commit' or 'don't commit'?"


David writes, "This was after a 15 minute period where I watched a timer spin frantically."


"It's as if DealeXtreme says 'three stars, I think you meant to say FIVE stars'," writes Henry N.


[Advertisement] Universal Package Manager – store all your Maven, NuGet, Chocolatey, npm, Bower, TFS, TeamCity, Jenkins packages in one central location. Learn more today!

CodeSOD: The Least of the Max;

Adding assertions and sanity checks to your code is important, especially when you’re working in a loosely-typed language like JavaScript. Never assume the input parameters are correct, assert what they must be. Done correctly, they not only make your code safer, but also easier to understand.

Matthias’s co-worker… doesn’t exactly do that.

      function checkPriceRangeTo(x, min, max) {
        if (max == 0) {
          max = valuesPriceRange.max
        min = Math.min(min, max);
        max = Math.max(min, max);
        x = parseInt(x)
        if (x == 0) {
          x = 50000

        //console.log(x, 'min:', min, 'max:', max);
        return x >= min && x <= max

This code isn’t bad, per se. I knew a kid, Marcus, in middle school that wore the same green sweatshirt every day, and had a musty 19th Century science textbook that discussed phlogiston in his backpack. Over lunch, he was happy to strike up a conversation with you about the superiority of phlogiston theory over Relativity. He wasn’t bad, but he was annoying and not half as smart as he thought he was.

This code is the same. Sure, x might not be a numeric value, so let’s parseInt first… which might return NaN. But we don’t check for NaN, we check for 0. If x is 0, then make it 50,000. Why? No idea.

The real treat, though, is the flipping of min/max. If the calling code did this wrong (min=6,max=1) then instead of swapping them, which is obviously the intent, it instead makes them both equal to the lowest of the two.

In the end, Matthias has one advantage in dealing with this pest, that I didn’t have in dealing with Marcus. He could actually make it go away. I just had to wait until the next year, when we didn’t have lunch at the same time.

[Advertisement] Otter enables DevOps best practices by providing a visual, dynamic, and intuitive UI that shows, at-a-glance, the configuration state of all your servers. Find out more and download today!

In $BANK We Trust;

During the few months after getting my BS and before starting my MS, I worked for a bank that held lots of securities - and gold - in trust for others. There was a massive vault with multiple layers of steel doors, iron door grates, security access cards, armed guards, and signature comparisons (live vs pre-registered). It was a bit unnerving to get in there, so deep below ground, but once in, it looked very much like the Fort Knox vault scene in Goldfinger.

Someone planning things on a whiteboard

At that point, PCs weren't yet available to the masses and I had very little exposure to mainframes. I had been hired as an assistant to one of their drones who had been assigned to find all of the paper-driven-changes that had gone awry and get their books up to date.

To this end, I spent about a month talking to everyone involved in taking a customer order to take or transfer ownership of something, and processing the ledger entries to reflect the transaction. From this, I drew a simple flow chart, listing each task, the person(s) responsible, and the possible decision tree at each point.

Then I went back to each person and asked them to list all the things that could and did go wrong with transaction processing at their junction in the flow.

What had been essentially straight-line processing with a few small decision branches, turned out to be enough to fill a 30 foot long by 8 foot high wall of undesirable branches. This became absolutely unmanageable on physical paper, and I didn't know of any charting programs on the mainframe at that time, so I wrote the whole thing up with an index card at each junction. The "good" path was in green marker, and everything else was yellow (one level of "wrong") or red (wtf-level of "wrong").

By the time it was fully documented, the wall-o-index-cards had become a running joke. I invited the people (who had given me all of the information) in to view their problems in the larger context, and verify that the problems were accurately documented.

Then management was called in to view the true scope of their problems. The reason that the books were so snafu'd was that there were simply too many manual tasks that were being done incorrectly, cascading to deeply nested levels of errors.

Once we knew where to look, it became much easier to track transactions backward through the diagram to the last known valid junction and push them forward until they were both correct and current. A rather large contingent of analysts were then put onto this task to fix all of the transactions for all of the customers of the bank.

It was about the time that I was to leave and go back to school that they were talking about taking the sub-processes off the mainframe and distributing detailed step-by-step instructions for people to follow manually at each junction to ensure that the work flow proceeded properly. Obviously, more manual steps would reduce the chance for errors to creep in!

A few years later when I got my MS, I ran into one of the people that was still working there and discovered that the more-manual procedures had not only not cured the problem, but that entirely new avenues of problems had cropped up as a result.

[Advertisement] Easily create complex server configurations and orchestrations using both the intuitive, drag-and-drop editor and the text/script editor.  Find out more and download today!

Why Medical Insurance Is So Expensive;

VA One AE Preliminary Project Timeline 2001-02

At the end of 2016, Ian S. accepted a contract position at a large medical conglomerate. He was joining a team of 6 developers on a project to automate what was normally a 10,000-hour manual process of cross-checking spreadsheets and data files. The end result would be a Django server offering a RESTful API and MySQL backend.

"You probably won't be doing anything much for the first week, maybe even the first month," Ian's interviewer informed him.

Ian ignored the red flag and accepted the offer. He needed the experience, and the job seemed reasonable enough. Besides, there were only 2 layers of management to deal with: his boss Daniel, who led the team, and his boss' boss Jim.

The office was in an lavish downtown location. The first thing Ian learned was that nobody had assigned desks. Each day, everyone had to clean out their desks and return their computers and peripherals to lockers. Because team members needed to work closely together, everyone claimed the same desk every day anyway. This policy only resulted in frustration and lost time.

As if that weren't bad enough, the computers were also heavily locked down. Ian had to go through the company's own "app store" to install anything. This was followed by an approval process that could take a few days based on how often Jim went through his pending approvals. The one exception was VMWare Workstation. Because this app cost money, it involved a 2-week approval process. In the middle of December, everyone was off on holiday, making it impossible for Ian's team to get approvals or talk to anyone helpful. Thus Ian's only contributions that month were a couple of Visio diagrams and a Django "hello world" that Daniel had requested. (It wasn't as if Daniel could check his work, though. He didn't know anything about Python, Django, REST, MySQL, MVC, or any other technology relevant to the project.)

The company provided Ian a copy of Agile for Dummies, which seemed ironic in retrospect, as the team was forced to the spend entire first week of January breaking the next 6 months into 2-week sprints. They weren't allowed to leave sprints empty, and had to allocate 36-40 hours each week. They could only make stories for features, so no time was penciled in for bug fixes or paying off technical debt. These stories were then chopped into meaningless pieces ("Part 1", "Part 2", etc.) so they'd fit into their arbitrary timelines.

"This is why medical insurance is so expensive", Daniel remarked at one point, either trying to lighten the mood or stave off his pending insanity.

Later in January, Ian arrived one morning to find the rest of his team standing around confused. Their project was now dead at the hands of a VP who'd had it in for Jim. The company had a tenure process, so the VP couldn't just fire Jim, but he could make his life miserable. He reassigned all of Jim's teams that he didn't outright terminate, exiled Jim to New Jersey, and gave him nothing to do but approve timesheets. Meanwhile, Daniel was told not to bother coming in again.

"Don't worry," the powers-that-be said. "We don't usually terminate people here."

Ian's gapingly empty schedule was filled with a completely different task: "shadowing" someone in another state by screen-sharing and watching them work. The main problem with this arrangement was that Ian's disciple was a systems analyst, not a programmer.

Come February, Ian's new team was also terminated.

"We don't have a culture of layoffs," the powers-that-be assured him.

They were still intent on shoving Ian into a systems analyst position despite his requisite lack of experience. It was at that point that he gave up and moved on. He later heard that within a few months, the entire division had been fired.

[Advertisement] Release! is a light card game about software and the people who make it. Play with 2-5 people, or up to 10 with two copies - only $9.95 shipped!

Representative Line: Tern Back;

In the process of resolving a ticket, Pedro C found this representative line, which has nothing to do with the bug he was fixing, but was just something he couldn’t leave un-fixed:

$categories = (isset($categoryMap[$product['department']]) ?
                                        : NULL) : NULL);

Yes, the venerable ternary expression, used once again to obfuscate and confuse.

It took Pedro a few readings before he even understood what it did, and then it took him a few more readings to wonder about why anyone would solve the problem this way. Then, he fixed it.

$department = $product['department'];
$classification = $product['classification'];
$categories = NULL;
//ED: isset never triggers as error with an undefined expression, but simply returns false, because PHP
if( isset($categoryMap[$department][$classification]) ) { 
    $categories = $categoryMap[$department][$classification];

He submitted the change for code-review, but it was kicked back. You see, Pedro had fixed the bug, which had a ticket associated with it. There were to be no code changes without a ticket from a business user, and since this change wasn’t strictly related to the bug, he couldn’t submit this change.

[Advertisement] Atalasoft’s imaging SDKs come with APIs & pre-built controls for web viewing, browser scanning, annotating, & OCR/barcode capture. Try it for 30 days with included support.