usu's ezportal security assessment

about 3 weeks ago i finished a security assessment for a new application the university of utah is planning on rolling out so that students, teachers, and staff to can manage all of their usu data.

our analysis included looking at the overall application architecture, looking at the coding, and a test to try to exploit vulnerabilities.

i thought it was interesting that every major hole in the application, came from the developers trusting the libraries and subcomponents they were using. for example, the developers were using an open source rich text editor (so that users could upload nice looking content without having to know html), that could easily be exploited to upload malicious code, or render content from any other site (yes, as in a xss attack).

so, the mantra of "find the dependencies -- and eliminate them", turns out to be true for security problems too.

particle swarm optimization (pso) in c#

i was going to give a presentation at "code camp" on particle swarm optimization (pso), but unfortunately was not able to do so because of circumstances beyond my control.

so, i've decided to post my c# implementation here so you can take look at it and play with it. the code graphs the movement of the particle swarm, so it's cool to see how the different pso parameters affect the movement of the swarm.

the swarm is not divided into neighborhoods and each member only knows about their local best and the swarm's best value to date. if you want to change the code so that i does neighborhoods, be my guest.

the code is really rough, especially in the way the UI runs. also, there's no way to dynamically change anything; if you want to make any sort of change you have to find the pertinent code, recompile it and rerun it (ie. there are no config files, no input paramenters, nothing).

so, if you think my code sucks, please let me know!

biztalk database problems

when we started implementing biztalk at tgn there was hardly any documentation on it, and so we had to resolve all of our problems through trial and error, or through help from blogs.

we had two database problems early on that made us wish someone would have warned us about potential problems:

1. by default, the biztalk messagebox is set to autogrow by 1mb. this is a problem, because if you have to autogrow your db, it's likely that you'll need to grow it by more than 1mb and so the 1mb is consumed promplty and the db has to autogrow once again. we had problems with this twice: the first time we just saw the cpu usage spike every time we autogrew; the second time the effects were worst as we we were running out of space on the disk and the server became stuck autogrowing and rolling back once it realized there wasn't enough space. so, if you're gonna autogrow your BizTalkMessageBoxDb database, autogrow it in big enough chunks to avoid performance problmes.
   
2. the PurgeSubscriptionJob_BizTalkMsgBoxDb and TrackedMessages_Copy_BizTalkMsgBoxDB jobs need to run every minute to keep biztalk running in good shape, but unfortunately they are not setup to run by default. we only found out about this when our applications started running absurdly slow and all of our clients complained about time outs.

perhaps if we had had a dedicated dba, and admin for our system, we would have noticed these problems before they caused issues. hopefully, you won't have to suffer through these yourself.

refactoring code with too many conditional statements

i was once talking with the IS director of a certain company about the tell tale signs of bad code. i mentioned a few of the items that raise flags for me: code duplication, low cohesion in classes, etc.

the director, which happens to be a really good coder even though he's been in management for several years, mentioned that when he sees code with too many if statements (or any conditional branching for that matter), he knows the code could be cleaned up. after hearing his statement, i started considering on how to clean code that has too many if statements.

last week i had the opportunity to help a coworker refactor some code that looked something like this:

public class CreditCardService()
{
  public void DoWork()
  {
       if(creditCardCode = "VISA")
       {
            //about 10 or so lines of code here
       }
       else if (creditCardCode = "MC")
       {
            //about 10 or so lines of code here
            //the code is very similar in all branches
       }
       //a few more branches for the other credit cards we support
       //...
  }
}

the obvious problem with the code above is that it's not making good use of a basic programming principle: polymorphism.

we could clean up the above code by:

• writing a base class that implements the common functionality across all credit cards
• writing children credit card classes that inherit from the base class and implement what's different in each credit card
• writing a credit card factory that return the right implementation to the CreditCardService class

so, we should end up with code looking something like this:

public class BaseCreditCard
{
     //all common fields go here

     public void DoWork()
     {
          //all common functionality goes here
     }
}

public class VisaCreditCard : BaseCreditCard
{
     //all fields pertaining to VISA go here
     
    public void DoWork()
    {
         base.DoWork();
         //visa specific functionality goes here
    }
}

public class CrediCardFactory()
{
     public static BaseCreditCard GetCard(string cardType)
     {
          //return appropriate credit card child class
          if(cardType.Equals("VISA")
               return new VisaCreditCard();
          //more code like the one above
     }
}
public class CreditCardService()
{
     private CreditCard card;
     public CreditCardService(string cardType)
     {
          card = CreditCardFactory.GetCard(cardType)
     }

     public void DoWork()
     {
          card.DoWork();
     }    
}

ideally, our factory will be really smart about picking the right credit card type, so that all of the if statements necessary to pick the right credit card will all be contained in it.

using polymorphism in this case makes the code much easier to read, and way easier to maintain as we now have all decision logic in one place, all common logic in another place, and type specific logic in its own place.

simple dependency injection

the idea behind dependency injection has been around for a while, and i believe the term "poor man's dependency injection" is popular as well, but i thought i'd share the poor man's method anyhow.

it's common to find that n-tiered applications (especially applications with a persistence layer and a "business logic layer") although layered, are usually tightly coupled.

one nuisance that this coupling creates is the inability to write unit tests that only test the business layer. even though you can write tests for the business layer, these cannot run without calling the persistence layer, and thus you end up with slow tests (because of db calls) and redundant tests (assuming you have tests for your persistence layer).

a very simple method (the poor man's method, of course) to solve this coupling problem is to:

1. code to an interface (in the case of the service layer, make sure you're not calling a specific implementation of your persistence layer, but an interface).
2. overload the constructor(s) for your business logic class so that they also take a specific implementation of the interfaces it depends on.

if you do what i've described above, your business layer should end up looking something like this:

public interface IPersistenceLayer
{
    void DoWork();
}

public class BusinessLogicLayer
{
    IPersitenceLayer myDataStore;
    public BusinessLogicLayer(IPersistenceLayer someImplementation)
    {
         myDataStore = someImplementation;
    }
   
    public void DoWork()
    {
         //business logic here
         myDataStore.DoWork();
    }
}

with the above code you can easily write tests for your business layer that just take a dummy IPersistenceLayer and return mock objects. and so now we have fast tests that don't require any database setup and/or maintenance.

there is however, one obvious problem with the code presented: why should the clients to the business layer have to know about the layer's dependency? the answer is they shouldn't and that's why we keep all the default constructors, and just have those set the IPersistenceLayer reference to the commonly use implementation of the interface.

although almost trivial, this method of dependency injection is appropriate for simple scenarios and provides much flexibility.

linq and lambda expressions

i'm aware that i'm about a year (or more, probably) behind on this, but the new c# lambda and linq features are awesome!

i wrote my first dlinq query today, and i'm now of the opinion that linq will literally change the way we code. i'm aware that i don't recognize all of the repercussions that come from linq, but just the very basic queries i did today, literally changed some more of the fundamental programming paradigms i held.

as for lambda expressions, i must say that even though the concept is not new (in fact, anonymous delegates and types are almost as old as programming), there's something to be said for how elegant lambda expression are: what would be ugly (or sometimes even impossible constructs) in c# are extremely simple and easy to read statements thanks to lambda expressions.

i'll keep posting on linq and lambda expressions as i learn more. but if you haven't had a chance to learn about them, i strongly recommend you to.

a4host.net

i'm excited to announce that syscoders lc (the consulting company i started) has finally started a web hosting branch. please take a minute to visit our website at a4host.net!

some problems with biztalk's ftp adapter

there are two issues you need to be aware of when using the ftp as a receive location:

1. ftp does not have a way to lock files while you're reading them; under some conditions, you could have more than one machine pick up the file and you could end up with duplicate messages. obviously then, you cannot have the host running the ftp adapter live on more than one machine. the receive function of the ftp adapter MUST run under a singleton host.
   
2. the biztalk adapter closes the ftp connection after 3 minutes of inactivity. this means that you must process your files promptly. if you're processing a large file, and it takes more than 3 minutes for your process to commit the message(s) to the message box, you will not be able to delete the file from the ftp receive location once your process finishes. this means that next time your receive location runs again, you'll pick up the same file!

we've encountered both of these issues recently, and both are easy to solve:

1. like i've already mention, have your ftp receive location run under a singleton host.
2. just bring the file down (without processing it) through ftp, and put it on some file folder. have the process that would normally work on the file pick the file up from this new location.

the "guaranteed delivery" nature is a really nice feature, but you have to be aware of the problems it can create. hopefully this will help you in your development; i know i was scratching my head for a while trying to figure out why the ftp adapter wouldn't delete the files it had processed.

on throwing errors

not too long ago i spent quite a few days trying to solve what was a really easy bug.

after moving a class from one assembly to another, one of my biztalk orchestrations started throwing a runtime error saying that it couldn't serialize an instance of a XmlDocument.

because the runtime error did not return a stack trace, i was very confused: even though XmlDocument truly is not serialiazable, biztalk is able to dehydrate the contents of a XmlDocument instance by getting the XmlDocument's outerxml and storing it; upon rehydration, the XmlDocument is loaded with the saved string.

after a few days of thinking about this problem, I realized that the error was not coming from the orchestration's instance of XmlDocument, but instead from a class that held and instance of XmlDocument and which the orchestration was using.

i simply marked the XmlDocument variable declaration with [NonSerializable] and all the problems went away.

i mention all of this because if had just gotten the stack trace, all the hassle I went through could have been avoided.

i've noticed that the errors i throw are usually not very descriptive, because i understand the code and i know what's going on. however, i've also noticed that when co-workers get exception from my code, it's not clear to them what the error means.

thus the importance of throwing very specific and very clear error messages; even if they're a little verbose. and outsider needs to understand the context of what's going on in order to be able to understand and solve problems.

two important biztalk tips

without further ado, here are the two tips (in question form):

1. do you really need an orchestration?
2. do you really need an xsd/xml message?

most developers making the switch to biztalk are most comfortable with procedural code, and thus writing orchestrations feels natural and is a lot more intuitive. however, i've learned that messaging based solutions are extremely powerful, easier to maintain, and incredibly flexible. i'm not saying that you can't build a messaging based solution if you are using orhcestrations... all i'm saying is that most of the time, whatever you're doing in an orchestration, really doesn't need to be done in an orchestration; this is specially true if all you're doing is message manipulation.

also, most projects don't need and xsd/xml message. a .net class as a message is, for most solutions, the better option. there's one big con in using a .net class however. if you need to publish your message externally, the you have to go with xsd, because a .net class offers nothing as far as publication goes. nonetheless, if you don't need an externally published message, go with a .net class... it'll save you tons of times and headaches ;)

i'm on slashdot!!!

ok, so i'm really not on slashdot, but the project i'm working on was recently reviwed there.

since i don't particularly like touting my own horn (not to mention that no one reads this blog anyhow), i'll just say that i'm writing all of the back-end communication with sorenson genomics and internal systems.

i was excited about this project from the get-go, but the "national" attention it's gotten really makes this one something else.

communication!

i've always been acutely aware of the fact that a lot of problems in life arise out of the lack of proper communication - and IT is no exception to this rule.

the oracle guys at place i work, are all (with one exception) non-developers, and have decided to start writing some web-services in order to expose some features that haven't been available before.

one of the services they wrote gets the purchase order for any given sales order. as i tried to consume the service today, i noticed that the purchase order had a flag for whether the purchase order had been canceled - but here's the thing: the flag was not a boolean, it was a string and it returned either 'Yes' or null.

so i called them and explained that it might be better to use a boolean. they seemed a little reluctant, so i went over to their desks to try to persuade them. and when i got there i realized that i could not explain what i wanted to explain without technical terms, and in a way that would make sense to them. i guess that happens when you only communicate with other techies on a regular basis, but that is really no excuse.

everything ended up ok; i had to explain a few concepts and terms like coupling, cohesion, etc. but in a way that meant something to them and would show them the values of their code.

communication is hard, especially when we can't speak the same language. i'm looking forward to more opportunities to interact with the oracle guys, so i can get out of my comfort zone and expand my vocabulary.

how to be an expert

kathy sierra has an excellent article on how to become an expert. here's a quote, that captures the essence of her post:

For the superior performer the goal isn't just repeating the same thing again and again but achieving higher levels of control over every aspect of their performance. That's why they don't find practice boring. Each practice session they are working on doing something better than they did the last time.

i've blogged about becoming a pro too; you can find the post here.

self correlating ports (and parallel processing)

the image above belongs to an orchestration i wrote to update a table with a list of currencies. the process is rather simple: all we have to do, is get the list of currencies we support (from the database), and then update the exchange rate.

to do this, i chose to use and extremely parallel method, in which every currency spins off its own thread, calls the service that contains current exchange quotes, and updates its value in the db.

what you don't see in the image above is the code that gets the list from the database and then instantiates a new orchestration for each currency. what you do see, is the part where the orchestrations (when they have finished their work) call back to signal that they have finished. after this, we send out an email notifying that we've updated all currencies.

doing this in biztalk is rather simple, since the functionality to asynchronously start orchestrations comes out of the box, and call backs are easily implement with self correlating ports. i like how biztalk makes this parallel process extremely simple.

if you wish to learn more about how to setup self corrlating ports, visit Stephen W. Thomas BizTalk Blog here for detailed instructions.

who's on your bus?

i once read an interview with jim collins (author of from good to great and built to last) where he said the most important factor in a succesful company is whther or not "the right people are on the bus". if you have the right people, everything else will fall into place... at least that is the theory. and my gut feeling is that the theory is right; people are not everything, but they are important enough where if you have to just pick one thing, you pick the right people.

i mention this because of an experience i had at work today:
i found a problem trying to begin a transaction in one of our live servers. before calling the dbas, i made sure i had a pretty good idea of what was going on... the problem only surfaced when trying to start a transaction from a client that was on a different domain.

i called in one of the dbas and described what was happening and asked for his help to solve the problem. i was polite when i brought the issue up, but our dba immediately became aggresive and blamed the problem on the client. i was confident that the problem was on the server, so i insisted he should at least take a look at it.

we went back and forth, arguing where the problem was, for at least 10 minutes; i finally had to show our dba i wasn't kidding, and in a somewhat threatening (firm, more than anything) tone told him that (1)what he was saying made no sense and he knew, and (2) what he needed to do to get proof that it was the server that wasn't working.

after going through the steps i asked to go through, he finally admitted it was the server that wasn't working. he also told me he had no idea on how to fix it (i don't blame him on this... from the research i did before i called him, it was apparent to me that the problem wasn't trivial) and would get back to me whenever he had found a solution. i immedately thanked him for his time and for committing to solving the issue.

after i hung up, i started thinking: why did it have to come to this? why is it so hard to work with the dbas? (it's not the first time i've had a experience like this, and i'm not the only one that has problems working with them). i thought: "we need the right people on that job, it's a key job. we nee the right people on the bus".

my manager congratulated me on how i handled the situation (he actually sat there listening the whole time because this was a high priority issue for us). after a few comments i told him i thought the hardest part about being a "techie" is not the technology, but working rather working and communicating effectively with others; i also told him most problems in software arise because of problems in these areas... and thus the importance of getting the right people on the bus.

learning to drive

in his book extreme programming explained, kent beck compares software development to driving a car, and makes the following insight:

The driver of a software project is the customer. If the software doesn't do what they want it to do, you have failed. Of course, they don't know exactly what the software should do. That's why software development is like steering, not like getting the car pointed straight down the road. Our job as programmers is to give the customer a steering wheel and give them feedback about exactly where we are on the road.

talk about a useful mindset to have! on a similar note, eric (my current manager at work) always tells me about how in one of his most succesful projects, they presented the product to the customer on a daily basis to provide the feedback beck talks about.

wtf: sonic cineplay

i'm well aware of what wtf stands for; however, i choose to ignore that and have it mean Curious Pervasions in Information Technology.

here the is my first wtf:
on friday december 1st i rented a movie. for some unknown reason the dvd would not play correctly on our desktop... after about an hour of trying to make it work, we decided to to just watch the movie on our laptop.

a few days later, my wife tells me: "something is wrong with the desktop. it won't play the 'signing time' dvds either. i take a quick look, and sure enough, the dvds play but there is no sound.

after trying everything i could think of (checking codecs, checking settings on the player, checking audio drivers, etc.) for four days, i'm about to give up and roll back to a previous back up. before doing this however, i try one more google search... i find the following on one of roxio's customer forums:

All,

There is an issue with Cineplayer not working. That could be no video, no audio or corrupted video. We’re aware of the issue and are working with Sonic on getting this resolved shortly.

The problem only manifests when the clock is set to the month of December. So simply change the date to another month and it will work.

This is only a workaround. We are currently working with Sonic to resolve this issue. We’ve been told that a patch should be available by Wednesday, December 6th.

We apologize for the inconvenience and assure you we are working to get this resolved as quickly as possible.

this is a wtf on so many levels... i just can't even begin to comprehend why a codec does not work on the month of december. it would be great to look at the code that's causing this bug and then posting it for learning purposes.

i'm glad i didn't roll back to a previous back up; that would have failed and then i would have probably re-installed everything; after that i would have probably shipped my computer to dell... and pulled my hairs out :)

taking notes (and a little on UML)

i've written about this on my personal blog, but i'll mention it here again... taking notes helps you remember stuff, so i like to do it even if i never look at my notes again.

not too long ago i interviewed with bill, a manager in the company i work for. while we were talking he took copious notes. while he was doing this he told me not worry, that he always does this even though he normally just throws his notes away.

under the guidance of this principle, i'd then like to write down the following idea from Fowler's UML Distilled:
use cases help you determine requirements; and actors help you determine use cases.

i like the idea because it's simple, easy to remember, and it helps keep the end in mind.

asp 1.1 and 2.0 on the same box

the compnay i work for, and a lot of guys on my team have added the .net 2.0 framework to their boxes recently.

a lot of people have come to ask me either why their 1.1 web projects stopped working or why they can't get a 2.0 asp project to work. 9/10 times the problems arises because the box is trying to run asp 1.1 and 2.0 under the same app pool.

so, the answer to the problem is to make sure that at the very minimum all your 1.1 stuff is running under one app pool, and all of your 2.0 stuff is running under a different pool.

a lot of people have asked me why this is, and honestly, i don't know yet (but i'm looking)... i'll post a follow up as soon as i find out.

but there's an important lesson to learn here, not just some obscure fact about asp:
solving this issues is actually easier than what one would think; all one has to do is look in the eventviewer, where the following error would be logged:

It is not possible to run two different versions of ASP.NET in the same IIS process. Please use the IIS Administration Tool to reconfigure your server
to run the application in a separate process.

it's been my experience that most devs don't look in the eventviewer, iis logs, etc, to find clues to their problems. as i've worked with BizTalk (and thus have been forced to look there) i've learned what great tools the eventviwer, the performance counters, and others are.

a place holder

just an entry fot this new blog of mine.