Why You Should Be Controversial

I have to tell you a story before any of this makes sense, so bear with me for a minute.

Michael Murray, a good buddy of mine, IMed me today telling me he had been invited to write a blog post for the the main page of tech.lds.org. I was really happy for him; he deserves it as a tribute to his tireless contributions to the LDS tech community.

Unfortunately, for reasons that I can understand, Mike was having a hard time coming up with what to say. I mean, there's tons you could say, but you have to be somewhat careful... after all some may mistake criticism of the software the LDS Church employees write with criticism of the LDS Church period. And then there's also the fact that Mike really cares about the tech.lds community; he's an active moderator in it.

Unfortunately, the tech.lds community suffers from the same problem any extremely homegenous community does: everyone says the same freaking thing all of the freaking time! Everyone talks about how IT is a means to an end; everyone talks about how we, as IT folk, should seek for way to further the Kingdom and be more wise servants. Please don't get me wrong: I know that what the tech.lds community preaches is true, but haven't we heard enough of that already?

Why don't we talk about some of the real issues with IT at the Church?

Here's a few ideas off the top of my head:

• How come LDS software is not open sourced? I still remember when I helped build my chapel back in Ecuador; truly a memorable experience - it was great to be able to contribute beyond tithing money.
• How come phone numbers are tied to households in MLS? What's up with that?!? And even more important, how come that wasn't fixed 10 releases ago?
• How come the IT department used to pay terrible salaries to IT employees? I recognize this is changing, but what kind of people was the IT department expecting to hire?
• Why did it take 10 years for meetinghouses to get broadband internet access? Did IT managers really just learn of white lists?
  
• Why does it take days to transfer membership records into a ward? Isn't it just a simple look up? And if it's not just a simple look up, how come it's not?
• Why is the Church developing on the Java stack? When was the last time you saw MLS running on a Linux box?
  

I've been deliberately controversial on this post because I wanted to make point: if you're going to say something, you might as well say something that's worth thinking about. I suspect there are good answers to all of the questions I've listed above; furthermore, I realize in the grand scheme of things I know nothing compared to what the fine folks at the LDS Church do. But at least, you read this far and know you're thinking about software at the Church too.

Disclaimer: I cannot say it more clearly than this: I'm criticizing IT at the LDS Church, I am NOT criticizing the Church of Jesus Christ of Latter Day Saints. To some extent, I'm not trying to criticize the folks the work in IT at the Church; I realize that were I in charge of IT things would probably be even worse than they're now.

The Secret Behind LINQ To SQL

I finally get it! It's all about expression trees!

It all finally clicked for me when I saw the declaration of IQueryable:

public interface IQueryable : IEnumerable
{
  Type ElementType { get; }
  Expression Expression { get; }
  IQueryProvider Provider { get; }
}

See that? It's right there! How could I have been so blind for so long? But I'm getting ahead of myself. Let me tell you about expression trees.

What are expression trees? Well, they're a trees with expressions as nodes. What do they do? They provide a mechanism to convert code into data (expressions). Why is that useful? Because you may want to examine and/or modify you code before execution. In particular, it would be very useful, if we wanted to take say C# code and convert it to, oh... i dunno.., SQL statements.

Say then you created the following Expression:

Expression<Func<int, int, int>> expression = 
     (a,b) => a + b;

(There's a little magic going on in the above sample: the compiler knows how to take the lambda "(a, b) => a + b" and make a delegate out of it).

The above code would literally translate to a "+" plus node with two children nodes: "a" and "b". Obviously, I've simplified the tree for clarity. If you really want to see the tree structure, fire up VS2008 and take a look at the expression tree with the ExpressionTreeVisualizer plugin.

Anyhow, now that we have our C# code in a tree, we can easily parse that tree and convert the data to a string that SQL can understand. We can then take that string, send it across the wire to our SQL server and voila: we have LINQ to SQL. Cool, huh?

Now, here's something else to think about: IEnumerable offers most of the same methods IQueryable offers, yet the declaration is completely different:

public interface IEnumerable<T> : IEnumerable
{
   IEnumerator<T> GetEnumerator();
}

Notice how IEnumerable does not have an Expression tree? That's a fundamental difference between the two interfaces. This means that IEnumerable won't do anything with your code but execute it. Therefore, if you order, filter, or project an IEnumerable collection, the action will execute in the process where the collection lives; it will not be sent to a beefy SQL box that can handle ordering large sets easily (Yes, I've made that mistake. In fact, that's what inspired this post).

Turns out my CS professors were right: trees really are useful data structures.

Why Programmers Should Have Private Offices.

Non-programmers don't get it: interruptions are a huge deal; it takes a lot of mental effort to load the program you're working on onto your brain.

Once you're in "the zone", you're effective and you write good code. Naturally then, having some dude from marketing stop by to ask a "quick question" is a total disaster; you have to literally unload a bunch of important information from your brain, just to listen to the marketing clown ask you for the 23rd time when the product will ship. Not cool. Not cool at all.

What non-programmers don't get is how hard it is to resume what you were working on. They don't understand that, to some extent, you have to load all the classes, variables, functions, etc. in your code right onto your memory; you have to enter the freaking Matrix, ant that's no easy task.

I'm guessing non-programmers think going back to writing code is as easy as going back to writing an email: you just read the last sentence you wrote and pick up from there. Well, it's not that simple.

Joel Spolsky has been advocating for private offices for developers well for almost a decade now. Yet there's few companies out there that give their developers such luxury. So, if you ever come across such perk, there's a good chance that's a company you want to work for.

From Recursion To Dynamic Programming

Dynamic programming is a really useful technique. When used appropriately, it saves processor time and memory space.

Shoot! After writing all the code for this entry, I just noticed that the Wikipedia entry for dynamic programming uses the same example I thought about using in this post. Oh well, let me just show you some code and explain just a little.

private int recursiveFib(int n)
{
  if (n == 0)
   return 0;
  if (n == 1)
   return 1;
  return recursiveFib(n - 1) + recursiveFib(n - 2);
}

The above code calculates the Fibonacci number for any integer n. The code is easy to read and effective. However, there's one problem: we repeat calls to recursiveFib even though we may have already computed a value. For example for n=4, we'll call recursiveFib(2) twice; this may not seem like a big problem but for larger Ns the number of unique calls to recursiveFib is very small. What a waste of processor cycles!

Now that we've realized we're wasting resources because the number of unique calls to our recursive methods is small, we're ready to move on to dynamic programming. Here's the dynamic code:

private int dynamicFib(int n)
{
  int[] dynamicArray = new int[n];
  dynamicArray[0] = 0;
  dynamicArray[1] = 1;
  for (int i = 2; i < n; i++)
  {
   dynamicArray[i] = dynamicArray[i - 1] + dynamicArray[i - 2];
  }
  return dynamicArray[n];
}

Even though this code may not be as succinct as the recursive code, it saves resources. By taking a bottom up approach (rather than the top to bottom approach of the recursive method) and saving our previous calculations, we never have to recompute a value again! Although, with oil being as cheap as it is nowadays, you may not really care.

In summary:
If you want to solve a problem with dynamic programming, I recommend you first solve it recursively. This may seem like a waste of time, but you need the recursive solution to spot where the improvement to the algorithm needs to occur. Once you've figured that out, all you have to do is compute and store you partial solutions as you go.