a4host.net

i'm excited to announce that syscoders lc (the consulting company i started) has finally started a web hosting branch. please take a minute to visit our website at a4host.net!

some problems with biztalk's ftp adapter

there are two issues you need to be aware of when using the ftp as a receive location:

1. ftp does not have a way to lock files while you're reading them; under some conditions, you could have more than one machine pick up the file and you could end up with duplicate messages. obviously then, you cannot have the host running the ftp adapter live on more than one machine. the receive function of the ftp adapter MUST run under a singleton host.
   
2. the biztalk adapter closes the ftp connection after 3 minutes of inactivity. this means that you must process your files promptly. if you're processing a large file, and it takes more than 3 minutes for your process to commit the message(s) to the message box, you will not be able to delete the file from the ftp receive location once your process finishes. this means that next time your receive location runs again, you'll pick up the same file!

we've encountered both of these issues recently, and both are easy to solve:

1. like i've already mention, have your ftp receive location run under a singleton host.
2. just bring the file down (without processing it) through ftp, and put it on some file folder. have the process that would normally work on the file pick the file up from this new location.

the "guaranteed delivery" nature is a really nice feature, but you have to be aware of the problems it can create. hopefully this will help you in your development; i know i was scratching my head for a while trying to figure out why the ftp adapter wouldn't delete the files it had processed.

on throwing errors

not too long ago i spent quite a few days trying to solve what was a really easy bug.

after moving a class from one assembly to another, one of my biztalk orchestrations started throwing a runtime error saying that it couldn't serialize an instance of a XmlDocument.

because the runtime error did not return a stack trace, i was very confused: even though XmlDocument truly is not serialiazable, biztalk is able to dehydrate the contents of a XmlDocument instance by getting the XmlDocument's outerxml and storing it; upon rehydration, the XmlDocument is loaded with the saved string.

after a few days of thinking about this problem, I realized that the error was not coming from the orchestration's instance of XmlDocument, but instead from a class that held and instance of XmlDocument and which the orchestration was using.

i simply marked the XmlDocument variable declaration with [NonSerializable] and all the problems went away.

i mention all of this because if had just gotten the stack trace, all the hassle I went through could have been avoided.

i've noticed that the errors i throw are usually not very descriptive, because i understand the code and i know what's going on. however, i've also noticed that when co-workers get exception from my code, it's not clear to them what the error means.

thus the importance of throwing very specific and very clear error messages; even if they're a little verbose. and outsider needs to understand the context of what's going on in order to be able to understand and solve problems.

two important biztalk tips

without further ado, here are the two tips (in question form):

1. do you really need an orchestration?
2. do you really need an xsd/xml message?

most developers making the switch to biztalk are most comfortable with procedural code, and thus writing orchestrations feels natural and is a lot more intuitive. however, i've learned that messaging based solutions are extremely powerful, easier to maintain, and incredibly flexible. i'm not saying that you can't build a messaging based solution if you are using orhcestrations... all i'm saying is that most of the time, whatever you're doing in an orchestration, really doesn't need to be done in an orchestration; this is specially true if all you're doing is message manipulation.

also, most projects don't need and xsd/xml message. a .net class as a message is, for most solutions, the better option. there's one big con in using a .net class however. if you need to publish your message externally, the you have to go with xsd, because a .net class offers nothing as far as publication goes. nonetheless, if you don't need an externally published message, go with a .net class... it'll save you tons of times and headaches ;)

i'm on slashdot!!!

ok, so i'm really not on slashdot, but the project i'm working on was recently reviwed there.

since i don't particularly like touting my own horn (not to mention that no one reads this blog anyhow), i'll just say that i'm writing all of the back-end communication with sorenson genomics and internal systems.

i was excited about this project from the get-go, but the "national" attention it's gotten really makes this one something else.

communication!

i've always been acutely aware of the fact that a lot of problems in life arise out of the lack of proper communication - and IT is no exception to this rule.

the oracle guys at place i work, are all (with one exception) non-developers, and have decided to start writing some web-services in order to expose some features that haven't been available before.

one of the services they wrote gets the purchase order for any given sales order. as i tried to consume the service today, i noticed that the purchase order had a flag for whether the purchase order had been canceled - but here's the thing: the flag was not a boolean, it was a string and it returned either 'Yes' or null.

so i called them and explained that it might be better to use a boolean. they seemed a little reluctant, so i went over to their desks to try to persuade them. and when i got there i realized that i could not explain what i wanted to explain without technical terms, and in a way that would make sense to them. i guess that happens when you only communicate with other techies on a regular basis, but that is really no excuse.

everything ended up ok; i had to explain a few concepts and terms like coupling, cohesion, etc. but in a way that meant something to them and would show them the values of their code.

communication is hard, especially when we can't speak the same language. i'm looking forward to more opportunities to interact with the oracle guys, so i can get out of my comfort zone and expand my vocabulary.

how to be an expert

kathy sierra has an excellent article on how to become an expert. here's a quote, that captures the essence of her post:

For the superior performer the goal isn't just repeating the same thing again and again but achieving higher levels of control over every aspect of their performance. That's why they don't find practice boring. Each practice session they are working on doing something better than they did the last time.

i've blogged about becoming a pro too; you can find the post here.

self correlating ports (and parallel processing)

the image above belongs to an orchestration i wrote to update a table with a list of currencies. the process is rather simple: all we have to do, is get the list of currencies we support (from the database), and then update the exchange rate.

to do this, i chose to use and extremely parallel method, in which every currency spins off its own thread, calls the service that contains current exchange quotes, and updates its value in the db.

what you don't see in the image above is the code that gets the list from the database and then instantiates a new orchestration for each currency. what you do see, is the part where the orchestrations (when they have finished their work) call back to signal that they have finished. after this, we send out an email notifying that we've updated all currencies.

doing this in biztalk is rather simple, since the functionality to asynchronously start orchestrations comes out of the box, and call backs are easily implement with self correlating ports. i like how biztalk makes this parallel process extremely simple.

if you wish to learn more about how to setup self corrlating ports, visit Stephen W. Thomas BizTalk Blog here for detailed instructions.

who's on your bus?

i once read an interview with jim collins (author of from good to great and built to last) where he said the most important factor in a succesful company is whther or not "the right people are on the bus". if you have the right people, everything else will fall into place... at least that is the theory. and my gut feeling is that the theory is right; people are not everything, but they are important enough where if you have to just pick one thing, you pick the right people.

i mention this because of an experience i had at work today:
i found a problem trying to begin a transaction in one of our live servers. before calling the dbas, i made sure i had a pretty good idea of what was going on... the problem only surfaced when trying to start a transaction from a client that was on a different domain.

i called in one of the dbas and described what was happening and asked for his help to solve the problem. i was polite when i brought the issue up, but our dba immediately became aggresive and blamed the problem on the client. i was confident that the problem was on the server, so i insisted he should at least take a look at it.

we went back and forth, arguing where the problem was, for at least 10 minutes; i finally had to show our dba i wasn't kidding, and in a somewhat threatening (firm, more than anything) tone told him that (1)what he was saying made no sense and he knew, and (2) what he needed to do to get proof that it was the server that wasn't working.

after going through the steps i asked to go through, he finally admitted it was the server that wasn't working. he also told me he had no idea on how to fix it (i don't blame him on this... from the research i did before i called him, it was apparent to me that the problem wasn't trivial) and would get back to me whenever he had found a solution. i immedately thanked him for his time and for committing to solving the issue.

after i hung up, i started thinking: why did it have to come to this? why is it so hard to work with the dbas? (it's not the first time i've had a experience like this, and i'm not the only one that has problems working with them). i thought: "we need the right people on that job, it's a key job. we nee the right people on the bus".

my manager congratulated me on how i handled the situation (he actually sat there listening the whole time because this was a high priority issue for us). after a few comments i told him i thought the hardest part about being a "techie" is not the technology, but working rather working and communicating effectively with others; i also told him most problems in software arise because of problems in these areas... and thus the importance of getting the right people on the bus.