I came across CrashPlan, the backup solution my employer uses. Turns out their software is free to chumps like me; they make their cash providing a place for you to put that valuable information.

There are two parts to any backup plan: you must gather your data together and you must put it somewhere safe that you can get to later. The CrashPlan software handles the gathering part, making it easy, for instance, to save all my stuff to the external hard drive sitting on my desk, but if the house burns down that won’t do me much good.

Happily CrashPlan also makes it easy to talk to remote computers, provided they have the software installed. I put CrashPlan on my server in a bunker somewhere in Nevada, and now this site and a couple of others are saved automatically to my drive in California as well. Easy peasy! Any computer signed up under my account can make backups to any other.

But wait! There’s more! The cool idea CrashPlan came up with was letting friends back each other up. I give you a special code and you can put backups of your stuff on my system. I can’t see what you saved, it’s all encrypted. But unless both our houses burn down at the same time, there’s always a safe copy.

Sure, if you pay you get more features and they will store your stuff in a safe place where you don’t have to wait if I happen to be on vacation, but for free that’s not bad at all. The idea of friends getting together and forming a backup community appeals to me as well. It’s a great way for geeks to look out for one another.

I started the Muddled Media Empire using a tool called iBlog, because it was free and worked with Apple’s hosting service, which I was already paying for. iBlog’s claim to fame was that it didn’t require a database – every time you made a change it went through and regenerated all pages that were affected. Toward the end, that was getting to be thousands of pages in some cases, each of which had to be uploaded individually. When iBlog’s support and development faltered, it was already past time for me to move on.

WordPress is an enormously popular Web-publishing platform. It comes in two flavors: you can host your blog on their super-duper servers and accept their terms of service and the slightly limited customization options, or you can install the code on your own server and go nuts. I chose the latter, mainly because I wanted to be able to touch the code. I’m a tinkerer.

So I signed up for a cheap Web host and set to work building what you see now. At first things were great, but after a while the host started having issues, and the once-great customer service withered up and vanished. So much for LiveRack. I think they just didn’t want to be in the hosting business anymore. I moved to iPage.

iPage was cheap, but I was crammed onto a server with a bunch of other people and sometimes my blog would take an agonizing time to load. Like, almost a minute. Then there was the time a very popular Geek site linked to my CSS border-radius table and iPage shut me down because the demand on the server was too much. Ouch! My moment in the sun became my moment at the bottom of a well.

I set out to find ways to make this blog more server-friendly and more user-friendly at the same time. Step 1: caching. WordPress doesn’t store Web pages, it stores data and the instructions on how to build a Web page. So, every time you ask to load a page here, WordPress fires up a program that reads from the database and assembles all the parts to the page. The thing is, that takes longer than just finding the requested file and sending it back, the way iBlog did. Caching is a way for the server to say, “hey, wait a minute – I just did this page and nothing’s changed. I’ll just send the same thing I did last time.” That can lead to big savings, both in time and server load.

I looked at a few WordPress cacheing programs and eventually chose W3 Total Cache, because it does far more than just cache data. For instance, it will minify scripts and css files (remove extra spaces and crunch them down) and combine the files together so the browser only has to make one request. It will zip the data, meaning fewer 1′s and 0′s moving down the pipe, and it does a few other things as well, one of which I will get to shortly.

I installed W3 Total Cache, and although some settings broke a couple of javascripts (for reasons I have yet to figure out – I’ll get to that someday), the features I could turn on definitely made a difference. Hooray!

But Muddled Ramblings and Half-Baked Ideas was still way too slow. I continued my search for ways to speed things up. I also began a search for a host that sucked less than iPage. (iPage was also starting to have outages that lasted a day or more. Not acceptable.) I decided I was willing to pay extra to be sure I wasn’t on an overwhelmed machine.

I’m not sure which came first – new server or Amazon Simple Storage Service. S3 is a pretty basic concept – you put your stuff on their super-duper servers, and when people need it they will get it really quickly. Things that don’t change, like images and even some scripts, can live there and your server doesn’t have to worry about them.

This is where W3 Total Cache earned my donation to their cause. You see, you can sign up for Amazon S3, and then put your account info into the proper W3TC panel and Bob’s Your Uncle. W3TC goes through your site, finds images and whatnot, puts them in your S3 bucket, and automatically changes all the links in your Web pages to point to your bucket instead of your own server. (Sometimes I find I have to copy the image to my S3 bucket manually, but that’s a small price to pay.)

Now a lot of the stuff on my blog, like the picture of me with the Utahraptors the other day, sits on a different, high-performance server out there somewhere, and no matter how overwhelmed my server happens to be at the moment those parts will arrive to you lickety-split. Amazon S3 is not free, however – each month I get an invoice for two or three cents. Should Muddled Ramblings suddenly become wildly popular, that number would increase.

About that server – the next stop on my quest for a good host was a place called Green Geeks. I wanted to upgrade to a VPS, which means I get a dedicated slice of a server that acted just like it was my very own machine. There is a lot to like about those, but my blog just wouldn’t run in the base level of RAM they offered. I upgraded and reorganized so that different requests would not take up more ram than they needed. Still, I had outages. Sometimes the server would just stop freeing up memory and eventually choke and die. Since it was a virtual server in a standard configuration, logic says it was caused by something I was doing, but all my efforts to figure it out were fruitless, and Green Geeks ran out of patience trying to help me figure it out.

The server software itself is Apache. At this point I considered using nginx (rhymes with ‘bingin’ ex’) instead. It’s supposedly faster, lighter, and easier to configure. But, I already know Apache. I may move to nginx in the future, but it’s not urgent anymore.

During the GreenGeeks era I came across another service that improves the performance of Web sites while reducing the load on the servers. I recently wrote glowingly about CloudFlare, but I will repeat myself a bit here for completeness. CloudFlare is a service that has a network of servers all over the world, and they stand between you the viewer and my server. They stash bits of my site all around the world, and much of the time they will have a copy of what you need on hand, and won’t even need to trouble my server with a request. About half of all requests to muddledramblings.com are magically and speedily taken care of without troubling my server at all. They also block a couple thousand bogus requests to my server each day, so I don’t have to deal with them (or pay for the bandwidth). It’s sweet, and the base service is free.

Unfortunately, it was not enough to keep my GreenGeeks server from crashing. Once more I began a search for a new host. I found through word of mouth a place called macminicolo. Apple employees get a discount, but I wasn’t an Apple employee yet. It was still a bargain. For what turned out to be the same monthly cost of sharing part of a machine at GreenGeeks, I get an entire server, all to myself, with plenty of RAM. I’ve set up several servers on Mac using MacPorts, and I knew just how to get things up and running well. It costs less than half what a co-located server costs anywhere else I have found (Mac, Windows, or Linux). (Co-location has up-front costs, but in the long term saves money.) So I have that going for me.

The only thing missing is that at GreenGeeks I had a fancy control panel that made it much simpler to share the machine with my friends. I do miss that, but I’m ready now to host friend and family sites at a very reasonable cost.

So there you have it! This is just your typical Apache/WordPress/W3 Total Cache/Amazon S3/CloudFlare site run off a Mac mini located somewhere in Nevada. Load times are less than 5% of what they were a year ago. Five percent! Conservatively. Typically it’s more like 1/50th of the load time. Traffic is up. Life is good.

Now I have no incentive at all to learn more about optimization.

As I perused my options I came upon this table:

Note that, depending on how I reach Los Angeles, the Southwest Chief departs at different times. The back end of the train catches up with the front end over the course of the journey; the arrival time is almost the same.

Sometimes a movie maker will see a shot in a film and have to ask, “how did they do that?” Most of the time, a question like that is a compliment. But here I am, a Web/database guy, asking, “how did they do that?” and it’s with a disbelieving shake of the head. Who on this planet would design a system that allowed such inconsistency? Trust me, it takes extra work to get system behavior like that.

Don’t tell the people signing my time sheets every week, but this stuff is not that hard.

Think of it this way: You’re on a cruise ship, and you have a priceless diamond tiara (don’t we all?). You know it’ll be much safer in the ship’s vault than in your cabin. The ship’s purser is only too happy to watch over your valuables in their very strong safe. Now you can rest easy.

Except… there’s someone else who can open the vault. What if the government serves the purser with a warrant (or some other constitutionally-questionable writ) and takes your tiara? What if someone fools the purser into handing over your tiara? For most things, trusting the purser is fine, but that tiara is really something special. What you need, then, is a special box with a really strong lock. You give the purser the box and neither he nor anyone else can even see what’s inside, and you can make it a really strong box, so even if the purser hands over the keys to his vault, your stuff is still safe.

The same principle applies with Dropbox. It’s really convenient and pretty darn secure, but someone else is holding the keys. For most things, like my writing, no further security is necessary. Yet I have a few files that I don’t want to leave to someone else to protect, but I still want the convenience and data backup Dropbox provides. On my mac I’ve set up a very simple system that allows me to see my most secret files whenever I need to on any of my machines, but protects them from prying eyes. It’s actually pretty simple, and there’s almost certainly a direct analog on Windows.

The disk utility that comes with Macs can create an encrypted disk image using pretty dang strong encryption. If you put that image file in your dropbox, then any files you add to that virtual disk will encrypted and saved to your Dropbox when you unmount the disk. Here are the steps:

1. Fire up Disk Utility (it’s in the Utilities folder).
2. Click New Image
3. Decisions, decisions….
   • name your new disk. If you name it “secret stuff” that will just make people curious.
   • Size: For reasons I’ll go into shortly, I’d advise not making this any bigger than you really need. If you’re protecting text files, it can be pretty small. The 100MB setting is probably more than enough for most people.
   • Format: Just use the default
   • Encryption: I say, what the heck. Go for the maximum unless you’ll be using a really old machine.
   • Partition: just use the default.
   • Image Format: sparse disk image – this will keep the size of the actual disk file down.
4. Save. You will be asked for a password. You’ll not need to remember it, so make it good and strong, nothing like any password you’ve used anywhere else. Keep the “save in keychain” option selected. (If you need it later, you can find it with Keychain Access.) – Remember: this is the secret that protects all your other secrets.
5. Voila! Put the disk image in your Dropbox folder. When you open the image file, a new hard drive will appear in finder. Anything you put on the drive will be added to the .dmg file you created.
6. “Eject” the drive on that machine and open the .dmg on any other machines you want to share the information. While you remember your crazy password, get it saved in the keychains of your various machines.

A couple of notes:

• The .dmg file will only update when you “eject” the drive. So I advise you not keep it mounted most of the time. Open it, add/access the files inside, and close it again. If you open it on two machines at the same time, you will end up with two versions in your Dropbox folder.
• I advised saving your password on your keychain, but remember that anyone who can access your computer can also access your secrets. That’s pretty much true whatever you do, but you might want to consider not putting the password on your laptop, for instance, if you think it might fall into the wrong hands.
• Since your secret files are saved as a single blob of data, you won’t have automatic backups of individual files. If you need to recover one, you’ll have to find the right version of the image file.
• Since your information is saved as a big ol’ blob, if you make a huge .dmg file it will eat up space in your Dropbox and burn up unnecessary bandwidth each time your save.
• It’s possible to set things up to protect individual files, but it’s complicated. Hopefully it won’t always be.
• Important! If you only store the password on one machine – Save it somewhere else also!. If you lose that password (if your hard drive crashes or your computer is stolen, for instance), you’re not getting into your strongbox. Ever. That was the whole point, after all.

Microsoft Excel uses a method to represent time that is both smart and frustrating. How do they manage this? They take a good engineering solution, then fiddle with it. First some background:

Long before Y2K people who knew what they were doing had already abandoned the practice of using strings of text to represent dates in a computer. Using strings like “3/10/2011″ to represent a date has plenty of drawbacks, from cultural (is that March 10th or October 3rd?) to performance-related (sort 3/10/2011, 4/2/1902, 3/8/2012). Therefore long ago people who were smarter than I am came up with other ways to represent time. Happliy, time is nice and linear. All you really need is a number line. Remember them? A number line stretches from zero to infinity in both directions. To measure time all you need to do is decide on a zero point and then any point in the history of the universe can be represented by some number of time units from that instant.

My first exposure to a more rational way to measure time was in the old MacOS. I don’t remember anymore exactly when the zero point was, or even what the units chosen were. I do remember that the number gets too big for the computer to handle some time in 2014. Ancient Macs will have a problem then. I blame the Aztecs.

The UNIX boys count seconds from January 1, 1970 at 00:00. You get special Geek Cred if you went to a party to celebrate second 1234567890 of the Unix epoch. 32-bit computers that use the Unix epoch will break in 2038, when the number of seconds gets too big to fit in 32 bits. (Note also that you can only go back a finite distance before the negative number goes out of the range the processor can handle.

That’s all well and good, but I’m here to discuss Microsoft Excel today, and in particular Microsoft Excel for Mac. Excel counts in days, but allows fractional values. 12.5 represents noon twelve days after the zero point. I haven’t checked, but I think this system works for dates far, far into the future. So good on Microsoft for coming up with it. (As long as you don’t need dates before the zero time. As far as I can tell, Excel doesn’t handle them).

Of course, there are a couple of caveats. First: the historical oddity. In Excel, the day February 29, 1900 exists. Alas, there never was such a day. Microsoft included this error because they wanted to be compatible with Lotus, who simply messed up. To change it now would cause problems, because the zero point for the Microsoft time is January 1, 1900. Every date in every spreadsheet would suddenly be off by one. A thousand years from now we may still be calculating time based on the insertion of a bogus day.

Oh, except that Microsoft time doesn’t always start in 1900, and here’s where things start to get squirrely. If you’re using Excel for Mac, the default day zero is January 1, 1904. Except when it isn’t. More on that in a bit.

I descended into Excel recently to write a macro that does fancy formatting based on data I dump from a Web-based tracking tool I’m building. The dates in my data are based on the UNIX epoch, so I need to convert them. I dump the raw data into one sheet and then display it correctly converted and formatted on the main sheet that people actually look at. Here’s the code I use in a cell of the spreadsheet that needs to show a converted date:

    =DATE(1970,1,1)+import!Z3/(60*60*24)

where the unix time is in cell Z3 of the ‘import’ sheet. This divides the unix time by the number of seconds in a day, which gives me the number of days that have passed since the UNIX zero time. The formula then adds on the number of days from the spreadsheet zero to the UNIX zero time. (I could just say 86,400 instead of 60*60*24, but this way I can tell at a glance I’m dealing with days, and speed will not be an issue.) Happily, this formula will work on both Mac and Windows versions of Excel, because the DATE function will return the right value for the start of the unix epoch based on whichever version of Excel is running.

So, no problem, right?

Well… except. I also have some more fancy work to do that requires scripting. The good news: Mac Excel 2011 uses Visual Basic for Applications (VBA), which while imperfect is a zillion times better than AppleScript. So away I went, coding with a twinkle in my eye and a song in my heart. To convert dates, I naturally followed the same plan I did in the sheet’s cells: get the value for 1/1/1970, then add the unix epoch days.

And the dates came out different. Yep, when scripting, Excel always uses the Windows zero time, even when the spreadsheet in question uses the Mac zero time. Dates calculated in cells in the sheet are four years different than dates calculated using the same method in a script.

Aargh. Of course, once I realized what the problem was, it was not too difficult to work around it. I just lost some of the portability of my code, because now it has to be tweaked based on what the zero date of the spreadsheet is.

An aside here: If you’re here to resolve some date confusion in your own Mac spreadsheet, I strongly recommend you start by going to Preferences->Calculation->Workbook Options and uncheck the “use 1904 date” option. Unfortunately it will not recalculate the dates already entered in your sheets, so if that’s a problem then it’s too late for you, bucko. Read on.

Here’s some not-as-portable-as-it-could-be code. You need to choose one of the first two lines based on whether your sheet uses mac dates or windows dates:

'excelZeroDate = DateSerial(1904, 1, 1)              ' mac zero
excelZeroDate = 0      ' DateSerial(1900, 1, 1)     ' windows zero
unixZeroDate = DateDiff("d", excelZeroDate, DateSerial(1970, 1, 1))

Then if I have a date in the ‘import’ sheet to convert, I can write something like:

startDate = DateAdd("s", Worksheets("import").Cells(dateRow, dateColumn), unixZeroDate)

The nice part is that these functions handled converting seconds and days for me. Overall it’s not a bad system if you overlook the part where a single application gives two different answers to the same question.

I couldn’t even make a backup.

Things seem to be getting back to normal (though they are not there yet – the site is still quite slow). There’s even a chance that I’m running on a brand-new server that is not being shared with as many other people. Or at least a brand-new server. Unfortunately, however, while I have come to appreciate iPage the company, which was very helpful and patient getting me up and running, iPage the service has not been so great.

I have vowed that the next move I make will be to a server that I control completely, so I can choose who shares it with me. I’m looking at Co-location deals now, though I might wimp out ant take the middle road. A VPS (virtual private server) gives me all the control of having my own machine, but in fact it’s an illusion — I still share physical hardware with an unknown number of others.

I will illustrate with an example. Things will get geeky for a while as I set the stage, then mellow out as I focus on the human interactions between various groups.

From a technical standpoint, Drupal’s biggest flaw is that it sucks when it comes to many-to-many relationships. Imagine I have a data type called “shirt” and another called “color”. It is very easy for me to set up “shirt” so that it can have several colors. So, when I look at a specific shirt in my database I can see that it has red and yellow in it. That’s all pretty straightforward.

The catch comes when I want a list of all shirts with yellow in them. If I had direct control over the database, many-to-many relationships like this are trivial and do not diminish the performance of the server. Drupal has no built-in way to get a list of all shirts with yellow in them.

But wait! Drupal is open source, and better yet has been built to be easy to extend by outside programers. Into this glaring hole in Drupal several folks have stepped forward with modules that solve the problem in a variety of different ways. Some of these methods are clever (one uses the indexes built by the search engine, for instance), but all have trade-offs and weaknesses.

So, you’re a Drupal developer, and you want a list of shirts with yellow in them. Which module do you use? Each module works differently, each requires some installation and fiddling to get working. Then there are the two modules by the same guy that are for similar but different purposes, yet the actual differences are not spelled out very clearly. What would help a lot would be some concrete examples of when to use which.

Now we’re getting closer to the Drupal Attitude. Remember as I rant about this that all the modules I’m evaluating are free, posted by geeks who wanted to contribute to make Drupal better. So, some slack-cutting is in order. BUT…

I had already spent more time than I had available trying to figure out which module to use, when I found a question posted by a guy asking “can I use this module for x”, where x was very similar to what I needed. “Aha!” thought I, “Now we’ll get a definitive answer!” Except that the response to the question was, “In this discussion (the article was about the differences between two modules) we want to focus on generalities, not specific applications. You should download both modules and fiddle with them for a few hours to determine which is right for you.” Or something like that. Notably absent from the answer was a pointer to where specific questions would be answered.

The guy who asked the question responded a bit harshly, pretty much saying, “Would it kill you to just answer my question? I don’t want to spend hours learning something you already know and could tell me in fifteen seconds.”

Well, this is just the sort of uppity user that the Drupal community loves to hate. Several people piled on in defense of the developer who had refused to answer the question. “He’s doing this for free, he’s helping the community, you should be grateful, blah, blah, blah.” None of them deigned to answer the original question either. There is a real, entrenched cadre in the Drupal community that says, “we learned things the hard way, and you should too.” Who needs documentation when you can read the source code?

Let’s step back for a moment and ask ourselves, “Why did the developer give this code back to the Drupal community?” The obvious answer, the one everyone talks about, is that he wants to make things easier for other Drupal users. That is a noble motivation and one I wholeheartedly support. He wants to be useful. Perhaps he just isn’t aware that a huge part of utility of software lies in the documentation. Perhaps he isn’t aware that a few choice examples of what his modules are meant to accomplish would have cost him an hour of his time and improved the acceptance of his work dramatically. He’s a coder, after all, not a marketer or a technical writer.

Even with all that, however, when someone, in the form of a question, contributes to the documentation by providing a specific example, he didn’t answer the question. No light came on that even if that was not the place for the question, then spending five minutes creating an FAQ would have helped the community far more than adding a new feature to his software. So an opportunity to spend just a few seconds and make his contribution to the community better went completely ignored. His supporters congratulated him for not capitulating to the demands of his potential users for more clarity.

Any of them could have stepped up and helped the newbie, probably in ten words or less, but none did. None of them wanted improved documentation. “We had to learn it the hard way, so you should too,” with a side order of “we make lots of money because we’ve figured all this stuff out.” Ladies and gentlemen, the Drupal Attitude.

If the guy posted his module but doesn’t seem interested in making it useful, then why did he post it? Well, he’s certainly getting lots of love from the people who figured out his work the hard way. They can all feel good about how smart they are.

And in the end, should I be thankful this guy shared his work with the rest of us? Actually, no. In my case, the presence of his modules ultimately had negative value. They cost me time, and never getting an answer about which was appropriate for my task, I went with a module developed by someone else.

So, Drupal contributors: If you don’t want to document your module, and you don’t want to answer straightforward questions from people who need to get a job done in limited time, don’t bother posting your fucking module at all. I don’t have time for endless fiddling and I sure as hell don’t have time for the Drupal Attitude.

• Why my algebraic attempts to calculate the point on a sloped line where the two halves of the shape had the same area were turning out so complex
• How to make money off PeoplePost (ten years too late).
• What to call my next version of PeoplePost
• One of the reasons Tomcat won’t run as a daemon on my machine
• There was a WordPress thing, too. What was it?
• Sometimes a weasel with a hammer… um… maybe that wasn’t so productive.

Anyway, eventually I fell asleep. That was about two hours ago. I’m more convinced than ever that alarm clocks are the bane of our civilization.