Category Archives: Rumblings from the Secret Labs

When Coding, Always use Descriptive Function Names

Posted on by
1

This evening I’ve been coding up something on a tight deadline. A few minutes ago I wrote a function named:

handlePluralTypeNameBecauseRESTWasInventedByIdiots()

REST is an acronym for… you can look it up if you care. It’s a way for code running in your browser to communicate with services off somewhere else. Some guy got his Phd making this nonsense up, and it has now become an industry standard. That guy is very lucky I was not one of the ones to review his dissertation, and the rest of the world is very unlucky that I wasn’t.

Like with HTML before it, someone came up with a half-assed solution to a real problem, and before the smart people in the room could say, “hey, that has some pretty serious flaws, but with only a little more effort we could fix most of them” the whole world went romping off with the flawed solution. And here we are.

Not only does REST violate previously-existing standards, it does so for no technical advantage. Servers and programming languages had to be updated to accommodate those violations. Maybe that should have been a red flag.

It would be SO DAMN EASY to fix most of the problems with REST. Use your head(ers)!

But here we are sweating over REST. And here’s a fun thing: for no technical advantage people who use this standard-violating standard have to understand the rules of pluralization in American English. At least in any implementation of REST I’ve had the pleasure of working with. Not only is that fucking annoying, it’s exclusionary. Sorry, kid in Senegal, we’re making a standard that disadvantages you.

Sure, you can get a code library to do plurals for you, and with any luck the rules in the browser code will match the rules on the server. Up until now, I’ve chosen the approach, “always name your data types in a way that just adds ‘s’ for the plural.” Tonight that wasn’t an option, so I a made a way for specific REST servers to keep only the rules relevant to them. More efficient and more reliable than someone else’s library.

And as I’ve often said, your code should express what it does without resorting to comments.

5

Thanks!

Optimization

Posted on by
Reply

Tonight it hit me, as I was giving advice, that I was dispensing wisdom I would do well to listen to myself. It was a question about optimization. Whenever you use that word, you have to know what you are optimizing for. Sometimes I forget.

One of the approximations of social media I engage in is a place called StackOverflow.com. That site and dozens of its sisters are an indispensable resource for programmers and tech folk in general. Some nights, like tonight, when I’m a little adrift, I will stop by over there and see if there are any questions pending I am qualified to answer.

When you’ve been around a while, you realize that there are almost no new questions, and in the major categories there are semi-pro question-answerers who can swiftly point the new question to the ancient answer. For php, you will see the name Barmar time and again, always helpful, always gentle.

But there is a category of question that in general is found to be annoying on Stack Overflow, but that I can answer well. These are questions that go beyond the nuts-and-bolts of a language to get to the heart of what programming really is. I will see the code they post, often a mess, and I can sort through it and figure out what is the actual question.

While others will downvote the question for being a mess, I find an opportunity to teach. I see someone who, with a little help, will see the mess also. My answers are long, and meticulous, but if I feel like I’m answering a homework question I leave out the key stones in the path.

A recent example: Someone had code that took the result of a database query and built an html table. But the result of the query didn’t directly match what was to go into the table. So while drawing the table, the code was also trying to figure shit out.

I suggested that while there was no way I was going to figure out what was wrong with the code as written, I had a simple concept to apply: Think, then Draw. Get all the information set up beforehand so that creating the html was just a brain-dead loop.

I have discovered that I have axioms. One is “Keep the guards close to the gate.” Another, apparently, is “Think, then draw.”

Tonight I came across a question asking for how to optimize a data manipulation operation. Massive data in, then a rearrangement of the data to be more useful later. The code to accomplish the data transform included nested loops, but was pretty tight and pretty clean. “How do I optimize this?” was the question.

I have come to realize that optimization of code is an economic judgement, not a technical one. The most obvious tradeoff: You can use more memory to make your process run faster.

But there is another, more subtle, economic tradeoff. Tonight I answered the optimization question with another economic argument: that Engineer Hours are worth more then CPU milliseconds. I told that person that if their code worked, and there wasn’t a specific problem caused by it, then it was perfect and they should move on to the next problem. I told that programmer to value their own time. Not just for personal reasons, but to better judge how their time could best help their endeavor.

Value your time, but also recognize the cost of your time.

My friends, this is something I do not do well. Where I work, I am moving our stuff from an environment where it was hosted for free, to a place where we will have to pay for the resources. I am, at heart, a cheap bastard. While that does me well at the poker table, it may not always serve the interests of my employer. I am constantly aware that all resources I request will now be billed.

So I spend time, hours of my time, trying to guess how small a footprint I can squeeze into on behalf of our group. I spend hours on elaborate schemes to do more with less. But I am not serving my clients well. I’m not including the cost of my time in the value equation. Things are too slow? I could spend a week doing clever stuff, at a cost to my company of thousands of dollars in my compensation while I defer projects that actually will make things run better, or I could just submit a form and get more RAM for the database, at an incremental cost for the department each month.

In my defense, I HATE filling out forms. I HATE the call that comes after while someone else re-enters the form into a different system, and invariably makes a mistake. I will take a week with the code over an afternoon of nonsense any time. But that is a weakness, not a strength. That is me allowing my own preferences to make decisions that are not properly optimized.

I am, at heart, a person who can take a problem and slice it and slice it and slice it until each part is a simple question, and I can take the answers to those questions and combine them to arrive at the solution. While I’m at it, I’ll make tests so that each part proves itself. When you see my code at the flea market, it will be on the shelf behind the table, where you will have to ask to get a closer look. I am proud of this. But it is not always a good thing.

There was a time I was VP of Software Engineering at a small company, and I did all right in that role. During the dot-com boom I held a pretty dang good team together. I didn’t have to fill out forms then, either. We were all just building something awesome. The calculus of the value of my time was different then; Engineer-hours — my hours — were a commodity; the value only to be realized if the venture succeeded. (Picture Facebook, before Facebook, only far more dynamic, with the fatal flaw of being private.)

Good times, good times. The sleep-optional days of youth. But if I’m going to justify that little walk in the past in this ramble, I have to tie it to the present.

Back then, I was paid well enough but there was never doubt that Engineer Hours were the currency we were paying to make our name. Our data center was a few racks in a room that had once been a bank vault (AWS was decades away yet). Everything, everything, was optimized for minimum CPU.

Tonight, while I was (very gracefully I’m sure) reminding a young coder about the relative value of CPU cycles relative to neuron flashes, I realized that I have not been very good at making that judgement myself. I need to do better. Not just for my team, but for myself. I need to fill out the goddam form, bluster my way to 4x the resources I actually think we need (magnificently tiny, in the scale of my company), and recognize that my time is better spent making new things, rather than helping the old things go.

4

Thanks!

Apple’s Little Monster

Posted on by
Reply

I used to post about techno-gear fairly frequently, but I’ve tailed off, largely because since I began to work at Apple it would be easy to accuse me of homerism.

Today, as I was preparing to leave work, I got a message from the Official Sweetie of Muddled Ramblings and Half-Baked Ideas, asking for a friend about the Mac Studio Cube. That was how I learned about the new hardware my company announced today. Maybe I should have watched that presentation. At any rate, do not waste your time coming to me for any inside information.

The word “cube” immediately conjured images of the original Mac Cube, one of which sits, in its metal-wrapped-in-polycarbonate elegance on a shelf in the Muddled HQ compound.

The new Cube not as pretty, even if it is more recycled (100% of rare earth metals are recycled), but that’s not what got my attention. This little computer is a fuckin’ beast.

Not that long ago, Apple bailed on Intel and started making their own chips. The first generation of the M1 processor was a ground-breaker. Fast and incredibly power-efficient. The reason turns out to be pretty simple: a lot of power and speed is lost to the places the disparate chips in your computer talk to each other. The answer: put everything on one chip. Processor cores, memory, GPU, I/O, and all the rest. The result is much faster computers that use a lot less power.

Apple has rapidly expanded the line of processors—literally. The name “M1 Max” is not just an empty marketing name, the chip is literally the biggest single chip that can be made in quantity with current technology. To extend this idea any further, you either have to improve the silicon die technology to make bigger chips (which obviously everyone is doing), or you have to incur the cost of having separate chips talking to each other.

But what if you built your chip so that you should just sorta… glue it to another one? Today we heard about the M1 Ultra, a gigantic chip that is literally two times maximum. And the connection between the chips is so efficient that the scaling is almost linear.

It has been a long time since I followed the chip-fabrication press; back around 1986 they were still talking about optical computing (using lasers for logic gates to get to extremely high clock rates), and gallium arsenide was still the next big thing. Remember gallium arsenide? Of course you don’t. It was not the next big thing.

Which means while I have an interest in this stuff, I have no idea whatsoever whether anyone at Intel or AMD did a spit-take when Apple announced the M1 Ultra today. Probably not, unless it was the speed at which Apple brought the idea to market.

I wonder, honestly, if Intel is even able to compete here. System-on-a-Chip (SoC) constrains versatility; if your chip has to work in many places, it can’t optimize for any.

But I raise my glass tonight to the silicon people at Apple. They are punching the rest of the industry in the face right now. Maybe I’m a homer, but what they have done speaks for itself.

4

Thanks!

php’s missing array_usearch

Posted on by
2

Pure geekery today, kids.

php has a bunch of functions that work on arrays. (In most modern languages arrays are defined by classes or prototypes and have methods built-in. php is not a modern language.) There are functions like array_sort that takes an array and puts the elements of that array in order. That’s fine if you have an array of numbers or strings, but for more complex things, how does the code decide which comes first?

For that use, there is another function, array_usort that takes an array, and you tell it what code to use to compare the two items.

php also has a method called array_search which finds whether an array has a particular item in it. As before this works fine for simple items, but becomes less useful as the items in the array grow in complexity, or you want to find something that you don’t already have a full example of. What if you have a list of books and you want to find the one titled Huckleberry Finn?

It seems logical that there would be a search function where, as for array_usort, you tell the code what defines a “match”, and then off it goes to see what comes up. Logical, but it’s not there (unless it’s tucked away with a terrible name that makes no sense, which is entirely possible in php).

So after about the eleventy-hundredth time writing a little loop to find something in an array I said, “dangit, I’m writing array_usearch.”

function array_usearch(array $array, Closure $test) {
    $found = false;
    $iterator = new ArrayIterator($array);
 
    while ($found === false && $iterator->valid()) {
        if ($test($iterator->current())) {
            $found = $iterator->key();
        }
        $iterator->next();
    }
 
    return $found;
}

All this does is try each element in the array against a function you provide until the function returns true, then it returns the key for that item in the array. If no match is found, it returns false, the same way array_search does. Simple! Using it would look something like this:

// define a type to put into a list
class Thing  {
    public $id;
    public $name;
 
    public function __construct($id, $name, $category) {
        $this->id = $id;
        $this->name = $name;
    }
}
 
// make a list of them, mixed up a bit
$listOfThings = [
    new Thing(1, 'one'),
    new Thing(2, 'two'),
    new Thing(4, 'four'),
    new Thing(3, 'three'),
];
 
// find the index of the item with id = 4
$id4Index = array_usearch($listOfThings, function($thing) {
    return $thing->id === 4;
});
// $id4Index will now be 2

The function will work on all php array types, whether with numeric indices or strings.

php purists might object to using the name array_usearch because all the other array_u* functions take a callable for defining the function, while this version uses a Closure. There are a couple of reasons: 1) Closures didn’t exist in php when the array_u* functions were defined, 2) it’s the 21st century now and other languages use closures in this manner for a reason, and 3) closures allow the function that gets passed to array_usearch to be reused with different values. With a little extra setup we can make searching super-clean:

// function that returns an anonymous function that captures the id to search for
$idClosure = function($id) {
    return function($item) use ($id) {
        return $item->id = $id;
    }
}
 
$id4Index = array_usearch($idClosure(4)); // value will be 2
$id2Index = array_usearch($idClosure(2)); // value will be 1

Now we can write code compactly that can search for matches of arbitrary complexity, and we can create little factories to produce the search functions themselves, so the complexity is tucked away out of sight. This variation takes an array of key/value pairs and searches for items that match all of those values:

function firstIndexMatching(array $array, array $criteria, bool $useStrict = true) {
 
    if (count($criteria) < 1) {
        return false;
    }
 
    // create a closure that has captured the search criteria
    $testWithCriteria = function($criteria, $useStrict) {
 
        return function($item) use ($criteria, $useStrict) {
 
            foreach($criteria as $key => $value) {
                if (!isset($item->$key)) {
                    return false;
                } else if ($useStrict && $item->$key !== $value) {
                    return false;
                } else if (!$useStrict && $item->$key != $value) {
                    return false;
                }
            }
 
            return true;
        };
    };
 
    return array_usearch($array, $testWithCriteria($criteria, $useStrict));
}

Now if you have an array of people, for instance, you can search for the first match with a given name:

$joeCoolIndex = firstIndexMatching($people, [
    'firstName' => 'Joe',
    'lastName' => 'Cool'
]);

The loop and the comparisons are moved out of the way and all the main part of your code need to do is supply the criteria for the search.

Ultimately after a search like this, you will want to have the item, not just its index. That’s easy enough, but don’t forget that if no match is found, array_usearch will return false, which php will often conflate with 0, so extra care has to be taken when using the returned index.

$joeCool = $joeCoolIndex !== false ? $people[$joeCoolIndex] ?? null : null;

Obviously this could be added to the firstIndexMatching function if one is never interested in the index itself.

And there you have it! A simple callback-based search function, ready to keep your main code clean and clear.

2

Thanks!

Bitcoin is Cool, but It’s not Money

Posted on by
3

The subject of blockchain technology in general, and cryptocurrency in particular, has come up a few times lately, and I’ve been doing some reading. When you look, you mostly find stuff that does a bad job describing what blockchain is, before jumping to some particular use for it – generally cryptocurrency, and why you should buy some.

But “blockchain” is the second-least important word in this discussion. “Cryptocurrency” is the least important. Blockchain is a way to achieve a utopian dream, and it’s the dream we will talk about today. The dream is the Distributed Ledger – a system where there isn’t some central institution who decides who owns what, instead that information is all kept in an encrypted ledger that we all share and maintain, and magically we can only read the parts of the ledger that are our business to read.

All the blocks and chains and whatnot are an implementation detail that is not really that important. But… later we will see that some implementation details matter a lot.

Let’s talk about the distributed ledger. Instead of some bank tracking how much money is in everybody’s account, there are thousands of copies of the ledger, spread around the world, immune to deprivations of institutions who use the ledgers to control us. It’s a pretty sweet idea. Better yet, ideally even when the ledger is spread around the world, only the right people can read the parts about you. For all the rest, the ledgers just have to agree.

To make this happen there are two key concepts: redundancy and consensus. Redundancy we just spoke of. Thousands, maybe millions of instances of the ledger, all verifying that they are the same, even if they can’t see the individual transactions.

But imagine if Ronald McDonald decides to give a Bitcoin to Mayor McCheese. He duly records the transaction and that information propagates through the network as all the instances of the ledger are updated. But at the same time, on another ledger it is recorded that in fact Ronald gave that same Bitcoin to the Hamburgler! I heard that gasp of horror, and it is well-placed!

With every distributed ledger, there has to be a way to resolve discrepancies that through sloppiness, bad timing, or malice will inevitably arise. Eventually all those ledgers have to concur about what actually happened. Therefore, the people who run the system need to make it difficult for the bad guys to overwhelm the honest transactions. They need to allocate deciding power based on some resource they control that makes the holders invested in the success of the platform.

In the case of Bitcoin, that resource is pure computing power. Solve math puzzles, get Bitcoins. Once you have Bitcoins, you will protect them. So to push false transactions onto Bitcoin, you would have to to solve those math puzzles faster than everyone else on the network combined.

That would not be easy. I read an estimate today that the current Bitcoin puzzle-solving economy, which uses extremely efficient hardware designed to solve these particular problems and nothing else, is currently chewing through the amount of electricity consumed by the entire country of Austria – at the low end. So to fool Bitcoin, you’d need about 1.1 Austrias (at the low end) of power. That’s pretty impractical, and that’s what keeps your Bitcoins safe.

Or, to defraud the system you could find a different way to generate sha256 hashes (that’s the Bitcoin puzzle). If you came up with a new way to do that calculation that took 1% of the power, you could destroy Bitcoin. Quantum computing would trash Bitcoin, but the latter will be long gone before the former arrives on the scene. Yep, Bitcoin will be long gone.

There are other ways for distributed ledgers to form consensus that are far less carbon-awful. In fact, there’s a currency that was recently announced that awards blocks (coins) for each ton of CO2 sequestered. And away from cryptocurrency, the distributed ledger promises to transform some really complex problems like adaptive energy grids and a world filled with self-driving cars. All the new cryptocurrencies are finding less ecologically-disastrous ways to manage consensus. Etherium is launching a new less-eco-awful version of their currency, and leaving their old version to the winds of fate. The power bill will eventually destroy Bitcoin.

I mentioned above redundancy and consensus. We have seen that consensus can be extremely expensive. New distributed ledgers are working to reduce that cost. But redundancy also has a cost.

All the ledgers have to share information, constantly updating each other. For the blockchain implementation, each update itself requires a great deal of computation to ensure security — digital signatures, hashes, more signatures. Recording a single transaction in thousands of ledgers eats up CPU time, to the point where processing a single Bitcoin transaction takes the juice to run your house for a week. (Actually, a German house for a week, whatever that means.)

And this is where we get to “Bitcoin is not Money”. Despite demanding the power of a European nation to operate, Bitcoin can only process a few transactions per second. Like, less than ten. How many credit card transactions take place every second? A global-scale distributed ledger makes each transaction very expensive. It is simply impossible for Bitcoin to be a factor in everyday commerce.

EDIT: In fact, bitcoin intentionally adjusts the difficulty of adding a block to the chain so that one 1MB block is added every ten minutes, so that transactions can be “digested” and shenanigans rooted out. This puts a very hard limit on the number of transactions that can be added to the chain, and as computing power increases, the difficulty of adding a block to the chain increases with it. Bitcoin by design cannot handle the transaction rate of an actual currency.

(Although I have to say that since you can know the entire history of each coin, you could, for instance, simply refuse to accept any coin ever touched by a company that dealt with blood diamonds, effectively making their money worth less. That is the true power of the distributed ledger. Someday it will be real.)

When it comes right down to it, our current attempts at the distributed ledger are way better at things that aren’t money – things where there is value in decentralizing, but they don’t move as fast as we need money to move. Or things that move fast but in a smaller context, like an office or a company.

Or, God help us all, Non-Fungible Tokens. A topic for another day.

When you hear about the ways blockchain technology will change the world, quietly, to yourself, substitute the term “distributed ledger”. That is the idea that has the power to change so many things for the better, and it’s a lot easier to fit in your head. Blockchain is an implementation of that idea, but it’s got warts big enough to mostly obscure the magical toad underneath. Moore’s Law may finally get us to the promised land, but computers will literally have to be a million times faster than they are now to turn blockchain-based cryptocurrency into actual money. My bet is now that we have seen the value of the distributed ledger, we will find a better way to accomplish it. And that’s pretty exciting.

5

Thanks!

Late-Night Puzzle Solving

Posted on by
1

First, a warning: this may be my geekiest post ever. If you want to give it a pass, you won’t hurt my feelings. In fact, I found a bunch of fluffy cats for you if you would prefer.

Anywhoo, I frequent a Web site called FiveThirtyEight.com that is about statistics and math, and applying them to sports and politics. On Fridays, they pose little (and not-so-little) math challenges for readers. A couple of weeks ago, they posed a question about numbers that were the difference between two perfect squares. As I was reading the question an ad came up to the side, pointing out that 42 = 1 + 3 + 5 + 7.

The mandate was clear: solve the puzzle, using the information in the ad.

I noodled on the problem idly for a while, and came up with some interesting observations, but it wasn’t until I really, really couldn’t sleep the night before last that I lay in the darkness and chewed on the puzzle (long after the submission deadline to receive accolades on the site, but that’s not what matters).

The question is here, but I’ll copy the relevant chunk for you:

Benjamin likes numbers that can be written as the difference between two perfect squares. He thinks they’re hip. For example, the number 40 is hip, since it equals 72−32, or 49−9. But hold the phone, 40 is doubly hip, because it also equals 112−92, or 121−81.

With apologies to Douglas Adams, 42 is not particularly hip. Go ahead and try finding two perfect squares whose difference is 42. I’ll wait.

Now, Benjamin is upping the stakes. He wants to know just how hip 1,400 might be. Can you do him a favor, and figure out how many ways 1,400 can be written as the difference of two perfect squares? Benjamin will really appreciate it.

Let’s do this! First we need to dig a little deeper into the information in the advertisement: 42 = 1 + 3 + 5 + 7. It turns out you can make this into a rule: a2 = the sum of the first a odd integers. 122 is the sum of the first 12 odd integers, and so forth.

That’s pretty interesting, but the question was about the difference between two perfect squares, and that’s actually where the fun begins (for certain values of fun).

Consider 52 – 32. It’s the first five odd integers, minus the first three odd integers, leaving us with 7 + 9 = 16. The subtracted square cancels out part of the series of odd integers, and the difference is the sum of the ones left over.

So now we know that the difference between two squares can always be expressed as the sum of consecutive odd integers. And we also know that every series of consecutive odd integers sums to the difference between two squares.

Fun fact: Every odd number can be expressed as the difference between two squares: There will always be a value of a where a2 – (a-1)2 = n, where n is our odd number. Crazy!

A little side trip here to button things down: 5+7+9 adds up to a difference between what two perfect squares, a2 – b2? Knowing how to figure this will come in handy later to check assumptions. 9 is the fifth odd integer, so we know a is 5. We can solve that for any series that ends with n to say that a = (n+1)/2. The series we’re working on here is three numbers long, so we can quickly surmise that b = a – 3, or more generally, b = a – l, where l is the length of the series. In this case, 5 + 7 + 9 = 52 – 22 = 21.

So now with that info in hand, we can turn to the actual question, but rephrase it “how many different series of consecutive odd integers add up to 1400?”

This is how far I’d gotten on the problem before the long, terrible, sleepless night. A computational solution would be easy at that point, just walking the numbers and testing the results. I wanted to find an analytical solution, but I kind of assumed it would be beyond me, or that series of odd numbers wouldn’t lend itself to such a generalization.

Wide awake in the darkness at 2am, I started to think about the problem from a programming standpoint, trying to optimize the algorithm. What else do you do when you can’t sleep, amirite?

First Optimization: know when to stop. There’s no point in testing the sum of a series after any term goes past half the total.

Second Optimization: The target number is even, so there’s no point testing series with an odd number of terms.

In fact… somewhere around 3:00 am I found the twist from a computational approach to an analytical one, merely by using the optimizations and discarding the code.

If the series of consecutive odd integers has two terms that add up to 1400, then they must be centered on 1400/2. If there is a series of four consecutive integers that add up to 1400, those numbers must be centered on 1400/4.

Let’s look a little deeper at the simplest case to deconstruct what that all means. 1400/2 = 700. the series of odd integers that centers on 700 is (699, 701). Just for giggles we can confirm that ((701+1)/2)2 – (((701+1)/2) – 2)2 = 1400. And it does!

By 3:30, doodling number lines in my head, I had observed that what I was doing was factoring 1400. But there was a hitch – I considered the number 10. There aren’t two consecutive odd numbers centered around five. I realized that both factors have to be even. For an even number to be the difference of two squares, it has to be a multiple of four. That’s why 42 is not hip.

So now we can finally get to the answer to the Riddler puzzle, by answering, “how many unique pairs of even factors does 1400 have?” To answer that, we can reduce 1400 to its prime factors, and count the different ways to arrange them into two buckets. 1400 is 23 x 52 x 7. Since both factors must be even, there must be a 2 in each bucket. That means there are two ways to distribute the remaining 2 (either in one bucket or the other), three ways to distribute the three 5’s (two in one bucket, one in each, or two in the other bucket), and two ways to distribute the 7. That means 2 x 3 x 2 ways to allocate the remaining factors. But there’s one final hitch, because that method will yield both 2 x 700 and 700 x 2. So the final answer is half of that, or 6.

There are six pairs of integers, a and b, such that a2 – b2 = 1400.

  • 2 x 700 = 699 + 701 = 3512 – 3492 = 1400
  • 4 x 350 = 347 + 349 + 351 + 353 = 1772 – 1732 = 1400
  • 10 x 140 = 131 + 133 + 135 + 137 + 139 + 141 + 143 + 145 + 147 + 149 = 752 – 652 = 1400
  • and three other examples that are too long to fit here.

I didn’t do the actual factoring that night; by then it was 4:30. But I knew how to get the answer, for that or any number. To find the solution for odd numbers the process is similar, but the length of the number series will always be odd, and obviously there will be no even factors.

There are simpler ways to solve this problem, but I’m pleased that I could put a mostly-useless factoid from an advertisement to good use, right on the Web page it was displayed on. And yesterday involved a whole lot of caffeine.

3

Thanks!

Sweet-o-Meter Fixed!

Posted on by
Reply

There is a lot of shit code in this world. This blog uses some of it. I have fixed one aspect of the shittiness, so now you can once again voice your appreciation for words well-spoken. I will look into the problem with the comment upvote thingie… later. It could also use a face-lift, I think, so people even notice it’s there.

3

Thanks!

A Pair of Coding Aphorisms

Posted on by
5

I write software for a living, and I take great pleasure when fixing a problem means reducing the number of lines of code in the system. In the last two days, I have come up with a couple of observations:

Every line of code is a pre-cancerous cell in the body of your application.

Now, “line of code” can be a deceptive measurement, as cramming a whole bunch of logic into a single line will certainly not make the application more robust. There are even robots that can comb through your code and sniff out overly-complex bits. But just as in humans weight is a proxy for a host of more meaningful health measurements, lines of code is the proxy for a host of complexity measurements.

But the point stands. I recently had to fix a bug where someone had copy/pasted code from one place to another. Then the original was modified, but not the copy. All those apparently-safe lines of code (already tested and everything!) were a liability, where instead a function call so everyone used the same code would have been more compact, easier to read, and much easier to maintain. There’s even an acronym for this type of practice: DRY — Don’t Repeat Yourself.

While that’s one of the more flagrant ways code bloat happens, there are plenty of others, mostly symptoms of not thinking the problem through carefully at the get-go, or not stopping to reconsider an approach as the problem is better-understood. Stopping and thinking will almost always get the project done sooner — and smaller.

One important thing to keep in mind is that programming languages are for the benefit of humans, not the machines that will eventually execute the program. If the purpose of your code is not obvious from reading it, go back and do it again. Comments explaining the code are generally an indication that the code itself is poorly written.

No software is so well-written that it ages gracefully.

I work on a lot of old code written by others, and I know people who work on old code written by me. In some cases, the code was shit to start with, but in others time has simply moved on, requirements have changed, and the code has been fiddled and futzed until the pristine original is lost to a host of semi-documented tweaks.

Naturally no code I have ever written falls into the “shit to start with” category (how could you even think that?), but that doesn’t mean the people who have to maintain that old stuff won’t be cursing my name now and then, as some clever optimization I did back in the day now completely breaks with a new requirement I didn’t have to deal with at the time.

And sometimes even if the code itself is still just fine, the platform it runs on will change, and break stuff. Jer’s Novel Writer was pretty elegant back in the day, but now when I compile I get literally hundreds of warnings about “That’s not how we do things anymore.” Some parts of JersNW are simply broken now. When I no longer work where I do, I will likely rebuild the whole thing from scratch.

Speaking of work, I am very fortunate to work in an environment that allows us to trash applications and rebuild them from scratch every now and then. Having a tiny user base helps in this regard. And as we build the new apps, we can apply what we’ve learned and maybe the next system will age a little better than the one before. Maybe. But sure as the sun rises at the end of a long day of coding, someone will be cursing the new system before too long.

4

Thanks!

Forward to the Past

Posted on by
4

I’ve spent the last few days learning Ember, which is a software framework for making apps that run in your browser. It’s fun to learn new things, and it has been fun to learn Ember, which to me is a less-awful-than-most javascript framework.

(Things are going to be technical for a bit; please stand by for the rant that is the foundation of this episode — which is also technical.)

The good news: I have never taken a tutorial for any framework on any platform that put testing right up front the way Ember’s did. That is magnificent. The testing facilities are extensive, and to showcase them in the training can only help the new adopters understand their value. Put the robots to work finding bugs!

Also good news: Efficient route handling. Nested routes that efficiently know which parts of the page need to be redrawn, while providing bookmarkable URLs for any given state is pretty nice.

But… I’m still writing html and css shit. WTF?

Yeah, baby, it’s ranting time.

Let’s just start with this: HTML is awful. It is a collection of woefully-shortsighted and often random decisions that made developing useful Web applications problematic. But if your app is to work in a browser, it must generate HTML. Fine. But that shouldn’t be my problem anymore.

When I write an application that will run on your computer or your phone or whatever, I DON’T CARE how the application draws its stuff on the screen. It’s not important to me. I say, at a very high level, that I expect text in a particular location, it will have a certain appearance based on its role in my application, and that if something changes the text will update. That’s all.

I don’t want to hear about html tags. Tags are an implementation detail that the framework should take care of if my application is running in a browser. Tags are the HOW of my text appearing where I want it. I DON’T CARE HOW. Just do it!

When I came to work in my current organization, the Web clients of all our applications were built with a homegrown library called Maelstrom. It was flawed in many ways, being the product of two programmers who also had to get their projects done, and neither of whom were well-suited for the task of ground-up framework design (in their defense, the people who invented HTML were even less qualified). But Maelstom had that one thing. It had the “you don’t have to know how browsers work, just build your dang application” ethos.

There was work to be done. But with more love and a general overhaul of the interfaces of the components, it could have been pretty awesome.

Why don’t other javascript libraries adopt that approach? I think it’s because they are made by Web developers who, to get where they are, have learned all the HTML bullshit until it is second nature. The HOW has been part of their life since they were script-kiddies. It’s simply never occurred to them that the HOW should not be something the app developer has to think about.

There have been exceptions — SproutCore comes to mind — but I have to recognize that I am a minority voice. Dealing with presentation minutiae is Just Part of the Job for most Web client developers. They haven’t been spoiled by the frameworks available on every other platform that take care of that shit.

My merry little band of engineers has moved on from Maelstrom, mainly because something like that is a commitment, and we are few, and we wanted to be able to leverage the efforts of other people in the company. So our tiny group has embraced Ember, and on top of that a huge library of UI elements that fit the corporate standards.

It’s good mostly, and the testing facilities are great. Nothing like that in Maelstrom! But here I am, back to dealing with fucking HTML and CSS.

1

Thanks!

A Thing I said to a Friend Recently

Posted on by
Reply

“Dude, you’ve crossed the threshold into big data now. It’s a moment where you have to swallow hard and denormalize.”

1

Thanks!

I’m Doing it Wrong

Posted on by
Reply

It is a lovely evening, and I’m enjoying patio life. My employer had a beer bash today, but The Killers are playing and I didn’t reserve a spot in time. So I came home instead, and after proper family greetings I repaired to the patio to do creative stuff. It’s blogtober, after all.

So what creative stuff have I been up to?

Creating a class that extends Event Service Sessions to add calendar server capabilities. (php is about the worst language on the planet for injecting new context-related capabilities into an existing class definition. In other words, php is not friendly to duck punching, or “Monkey Patching” as the kids call it these days.

The linked Wikipedia article completely misses the most common use-case for this practice, in which I want to get a thing from some service and then augment it. But php doesn’t flex that way, so I just have to deal with it.

Which is to say, I’m doing Friday evening wrong. It is lovely out, my co-workers are chugging down the last of their beers as The Killers wrap up. I am on my patio with my dogs, the air finally starting to cool after an unusually warm day. It is nice. You’d think I could find a better use of this time than wrangling with a programming language.

But apparently you’d think wrong.

3

Thanks!

Sometimes it’s not so Good when People are Happy to see You

Posted on by
Reply

“There he is!” both my fellow engineers said as I walked into the office this morning. I knew right away that this was not going to be the Monday I expected it to be.

“The database is down,” my boss said before I reached my chair. “Totally dead.”

“Ah, shit,” I replied, dropping my backpack and logging in. I typed a few commands so my tools were pointing at the right place, then started to check the database cluster.

It looked fine, humming away quietly to itself.

I checked the services that allow our software running elsewhere to connect to the database. All the stuff we controlled looked perfectly normal. That wasn’t a surprise; it would take active intervention to break them.

Yet, from the outside, those services were not responding. Something was definitely wrong, but it was not something our group had any control over.

As I was poking and prodding at the system, my boss was speaking behind me. “Maybe you could document some troubleshooting tips?” he asked. “I got as far as the init command, then I had no idea what to do.”

“I can write down some basics,” I said, too wrapped up in my own troubleshooting world to be polite, “but it’s going to assume you have a basic working knowledge of the tools.” I’d been hinting for a while that maybe he should get up to speed on the stuff. Fortunately my boss finds politeness to be inferior to directness. He’s an engineer.

After several minutes confirming that there was nothing I could do, I sent an urgent email to the list where the keepers of the infrastructure communicate. “All our systems are broken!” I said.

Someone else jumped in with more info, including the fact that he had detected the problem long before, while I was commuting. Apparently it had not occurred to him to notify anyone.

The problem affected a lot of people. There was much hurt. I can’t believe that while I was traveling to work and the system went to hell, no one else had bothered to mention it in the regular communication channels, from either the consumer or provider side.

After a while, things worked again, then stopped working, and finally started working for realsies. Eight hours after the problem started, and seven hours after it was formally recognized, the “it’s fixed” message came out, but by then we had been operating normally for several hours.

By moving our stuff to systems run by others, we made an assumption that those others are experts at running systems, and they could run things well enough that we could turn our own efforts into making new services. It’s an economically parsimonious idea.

But those systems have to work. When they don’t, I’m the one that gets the stink-eye in my department. Or the all-too-happy greeting.

2

Thanks!

WordPress Geekery: Reversing the Order of Posts in Specific Categories

Posted on by
1

Most blogs show their most recent posts at the top. It’s a sensible thing to do; readers want to see what’s new. Muddled Ramblings and Half-Baked Ideas mostly works that way, but there are a couple of exceptions. There are two bits of serial fiction that should be read beginning-to-hypothetical-end.

A while back I set up custom page templates for those two stories. I made a WordPress category for each one, and essentially copied the default template and added query_posts($query_string.'&order=ASC'); and hey-presto the two fiction-based archive pages looked just right.

Until infinite scrolling came along. Infinite scrolling is a neat little feature that means that more episodes load up automagically as you scroll down the page. Unfortunately, the infinite scrolling code knew noting about the custom page template with its tweaked query.

The infinite scrolling is provided by a big, “official” package of WordPress enhancements called Jetpack. The module did have a setting to reverse the order for the entire site, but that was definitely a non-starter. I dug through the code and the Jetpack documentation and found just the place to apply my custom code, a filter called infinite_scroll_query_args. The example even showed changing the sort order.

I dug some more to figure out how to tell which requests needed the order changed, and implemented my code.

It didn’t work. I could demonstrate that my code was being called and that I was changing the query argument properly, but it had no effect on the return data. Grrr.

There is another hook, in WordPress itself, that is called before any query to get posts is executed. It’s a blunt instrument for a case like this, where I only need to change behavior in very specific circumstances (ajax call from the infinite scrolling javascript for specific category archive pages), but I decided to give it a try.

Success! It is now possible to read all episodes of Feeding the Eels and Allison in Animeland from top to bottom, the way God intended.

I could put some sort of of settings UI on this and share it with the world, but I’m not going to. But if you came here with the same problem I had, here’s some code, free for your use:

/**
 * reverses the order of posts for listed categories, specifically for infinite Jetpack scrolling
 */
function muddled_reverse_category_post_order_pre_get_posts( $query ) {
	if (isset($_REQUEST['query_args']['category_name'])) {
 
		$ascCategories = [
			'feeding-the-eels',
			'allison-in-animeland',
		];
 
		if (in_array($_REQUEST['query_args']['category_name'], $ascCategories)) {
			$query->set( 'order', 'ASC' );
		}
	}
}
add_filter( 'pre_get_posts', 'muddled_reverse_category_post_order_pre_get_posts' );

In the meantime, feel free to try the silly serial fiction! It will NOT change your life!
Feeding the Eels
Allison in Animeland

2

Thanks!

Gimme Swift

Posted on by
3

As a computer programmer, I live in a familiar cycle: Write some code, then run it repeatedly to work out all the kinks. There is a moment when you hit “run” for the first time, already anticipating what the errors might be, thinking about next steps when the error inevitably presents itself.

It’s been weird writing server-side Swift. I do my hacking, adding a feature or refactoring or whatever, I make the compiler happy, then it’s time to get to the nitty-gritty. I roll up my sleeves, start the program… and it works. Just like that. I run the tests against the other systems. It works.

It’s like you’re all ready for a fight and the other guy doesn’t show up. NOW what are you going to do?

Swift can be annoying with how hard-assed it is about certain things, but that picky compiler that sometimes forces long-winded syntax is like that really picky English teacher who you realize after the fact gave you a command of words you didn’t have before. If you have a null pointer in Swift, you went out of your way to create it.

Programming languages exist for the convenience of humans, not machines. So if you can make a language that makes it harder for humans to make a mistake, why wouldn’t you?

Man I enjoy writing code in Swift. Of the four languages I use regularly, Swift is hands-down the one I’m most productive with, even though I’ve been using the others for far longer. And just today I remembered that functions could return tuples, and I was like, “Damn!” all over again, thinking how I can shrink my interfaces.

That and a performance profile comparable to C (each is better for certain sorts of operations), and you have a language with some mojo. This ain’t JavaScript, homey.

Most of my days are consumed writing code in other languages (at least for now), and what strikes me every day is that the mistakes I make would not have been possible in Swift. Think of that!

3

Thanks!

Facebook, Continuous Integration, and Fucking Up

Posted on by
Reply

If you ask the engineers at Facebook (I have), they are experts at continuously evolving their platform almost invisibly to the users. If you ask the users, Facebook is really fucking annoying because shit is breaking all the time and the button that was there yesterday is nowhere to be found.

Continuous Integration is a development practice that means that each little tweak to the software goes through the tests and then goes live. It’s a powerful idea, and can massively decrease the risk of publishing updates — rather than push out the work of several geek-years all at once, with all the risk of something going terribly wrong, you push out the result of a couple of geek-weeks of effort on a regular basis, taking baby-steps to the promised land. Tick, tick, tick, with an army of robots making sure no old bugs sneak back in again.

I fully embrace this idea.

Never has a company been more proud of accomplishing this than Facebook. They crow about it around here. Also, never has a company been so bad at actually doing it. What Facebook has managed to do is annoy users with endless changes that affect how people work, while still publishing bugs.

The key is that a continuous, minor set of tweaks to software is good, but endless tweaks to how people experience the software is bad. People don’t want to be constantly adjusting to improvements. So in continuous integration, you can enhance the user experience, but you can’t lightly take away something that was there before. You can’t move things around every couple of weeks.

Back in the day when I went on Facebook more frequently, I was constantly bemused by a user interface that felt like quicksand. Meanwhile, frequent users reported a never-ending stream of bugs.

Facebook, you are the champion of Continuous Integration, and the poster child for CI Gone Wrong.

1

Thanks!