Blog Home  Home Feed your aggregator (RSS 2.0)  
Implements IVillage - Sunday, August 12, 2007
It takes a village to keep up with .Net
 
 Tuesday, July 10, 2007

We've had a problem with our BizTalk Servers for quite a while.  Receive ports to a particular file server would seem to shutdown almost at random.  After much searching of the internet, there was a consenus to build a port watcher application using BizTalk WMI.  We did this and it worked, ports would stop and the port watcher would kick them in the butt to get them going again.  But this was just a band-aid to a huge sore under the surface.  After almost a year of this, a re-researching of the problem finally gave way to a helpful post and then a solution

It turns out that when a BizTalk Server is checking LOTS of directories on the same server, the network BIOS command limit can be reached:

"This issue may occur if the client computer submits simultaneous, long-term requests against a file server that uses the Server Message Block (SMB) protocol. An example of a long-term request is when a client computer uses the FindFirstChangeNotification function to monitor a server share for changes."

"This issue may occur if the MaxCmds registry value setting on the client is less than 50, or the MaxMpxCt registry value setting on the server is less than 50."

So, per the kb article, an increase of the MaxCmds regsitry entry to 500 solves the problem (after a reboot)!  Not a BizTalk problem at all.  The limitation in the Windows 2003 OS designed to throttle its workload (for good reason) was causing the receive location to shutdown.  The error message wasn't very helpful from BizTalk as it really didn't provide a good error code or reason to point to the command limit that was eventually found to be the problem.  Now if I can only figure out why my File Recieve loactions to DFS shares do the same thing!

Tuesday, July 10, 2007 10:09:27 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]    |   | 
 Wednesday, June 13, 2007

There's a point in time where on one side, chaos reigns and the fundamental laws of order seem to be suspended.  Then on the other side is the light, the complete understanding of something that now seems simple.  This is a story of a boy, an HTTP Send Port and his trust in Microsoft's 'calculations.'  It all started with a MOM alert:

Severity: Error
Status: New
Source: BizTalk Server 2006http://localhost/xxxx/sendPoWS/sendPoWS.asmx/upDatePOData
Name: Error: An outbound message is being suspended by the adapter.
Description: A message sent to adapter "HTTP" on send port "scosPoOut" with URI "http://localhost/xxxx/sendPoWS/sendPoWS.asmx/upDatePOData" is suspended.
Error details: The HTTP send adapter cannot complete the transmission within the specified time.
Destination: http://localhost/xxxx/sendPoWS/sendPoWS.asmx/upDatePOData
MessageId: {DA614303-8AA6-444E-8312-4C41B4BBD29C}
InstanceID: {954FAA15-BAA0-453A-9AD5-C62E186AF05E}

Cannot Complete the Transmission?  It's a request-response HTTP port off of an orchestration... the webservice is on the local host... why can't it complete sending the XML message to the service?  Let's look at this IIS Logs...

#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
2007-06-10 10:59:18 W3SVC1 127.0.0.1 POST /xxxx/services_1/sendDemandWS.asmx/upDateData - 80 - 127.0.0.1 Microsoft+(R)+BizTalk+(R)+Server+2006+3.0.1.0 200 0 64

It has a sc-status of 200?  200 - OK?  Checked the web service logs and it successfully submitted the data.  But wait... what is this sc-win32-status of 64...type net helpmsg 64 at cmd prompt gets "The specified network name is no longer available".  This is localhost, why is it having problems talking to itself?  We've had server build issues in the past but how could the TCPIP stack be hosed?  That's impossible.  OK. Let's back up and go to BizTalk.

BizTalk is sending the message to the web service via the orchestrations HTTP send port.  It retries 5 times per the configuration and then fails.  Each time it tries, the message is received and processed by the web service.  BizTalk is just not getting back that final ACK.  But it's local Host...

After some Google'ing on sc-win32-status and 64, the collective wisdom said it's an indication that one side of the TCP connection is closing prematurley.  So am I throwing errors in this web service somebody else wrote that I have to maintain? No... there's a 200 returned by IIS - that's success.  So where's the other end of the connection... BizTalk!

The HTTP send port's Request Timeout is set to '0' which is default.  When set to zero, the time-out is calculated based on the request message size.  Should be fine.  But it isn't.  How do they calculate the size?  I found one posting with the answer:

Timeout = Min((180sec + ((MessageSize* 3)/1000)), 3600sec)

So, Ihave a message that is 6.4 Megs in size.  I do the calculation and it's going to top out at 3600 secs.  I turn on additional logging in IIS and see that the actual IIS run takes 5900 seconds (I know this is a poor design and is way too long for a TCP connection to hang open - v2 will change the architecture).  So back to the beginning.  BizTalk is giving up after an hour and retrying.  This why I see the retries going at intervals of 1 hour and 5 minutes.  1 Hour to timeout and 5 minutes for the retry interval.  Overriding the send port's default value of '0' to '8000' solves the problem.

It's really crazy that the solution was one of the first items that I dismissed.  And I hold no malice toward Microsoft's calculation as it is reasonable and this is not a reasonable circumstance.  I was really disapointed that the calculation method was not there in bold print.  Instead I meandered down many dead ends before I regrouped and went deep.  I am a wiser mand for having made the journey.

 

 

 

Thursday, June 14, 2007 6:04:42 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]    | 
 Monday, May 21, 2007

We took a pretty good hit at work a couple of weeks ago.  We lost one of the good guys to another companyPaul's leaving because he found himself one of those opportunities that is so well fit for him that it probably only comes along once in a lifetime.  Paul reads many of the same Blogs I do.  He reads one blog that I don't read and happened to read this interesting post.  After submitting his resume and some discrete omissions of the truth (Paul is ready for politics) to his co-workers, we get a resignation

My time with Paul has definitely increased my knowledge and understanding of some XP principles, repeatable build processes and source control.  Best of Luck to him and his family!

Monday, May 21, 2007 6:09:01 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]    | 
 Tuesday, May 15, 2007

David McNamee will be presenting on Windows Workflow on May 16th @ 6:30 PM.  The Sapce Coast Credit Union Building as usual.  See http://www.scdnug.org for details.  Hope to see everyone there.

Tuesday, May 15, 2007 10:55:41 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]    | 
 Monday, April 09, 2007

I was implementing some data access code for a personal project using the Enterprise Library's Data Access Application Block (DAAB).  I was running some code that checked for the existence of a row prior to issuing an insert.  Here is the original code:

SqlDatabase db = DatabaseFactory.CreateDatabase("mainDB") as SqlDatabase;

string sqlCommand = "ar_GetArrestReportItemByCaseNumber";
DbCommand dbCommand = db.GetStoredProcCommand(sqlCommand);

db.AddInParameter(dbCommand, "reportId", DbType.Int32, reportID);
db.AddInParameter(dbCommand, "caseNumber", DbType.String, caseNumber);

IDataReader dataReader = db.ExecuteReader(dbCommand);

exists = dataReader.Read();

After running through on some test data, I gave it a run against some real world data.  And it worked fine until the data's batches exceeded 100 rows.  Then I got the error message:

Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

I checked SQL Manager and saw 100 connections / processes accessing the table in question.  I am using the DAAB.  The Database class is supposed to take care of this sort of thing for me.  There was a bug in VS 2003, but this was fixed in VS 2005.  Where is the connection being left open... the DataReader.  I forgot to close the DataReader.  I promptly re-educated myself on some ADO and IDisposable.  Whenever an object implements IDisposable, there is probably a good reason for it... like say... releasing a connection object back to the connection pool.  Corrected code and much better performance:

SqlDatabase db = DatabaseFactory.CreateDatabase("mainDB") as SqlDatabase;

string sqlCommand = "ar_GetArrestReportItemByCaseNumber";
using (DbCommand dbCommand = db.GetStoredProcCommand(sqlCommand))
{
   db.AddInParameter(dbCommand, "reportId", DbType.Int32, reportID);
   db.AddInParameter(dbCommand, "caseNumber", DbType.String, caseNumber);

   using (IDataReader dataReader = db.ExecuteReader(dbCommand))
   {
      exists = dataReader.Read();
   }
}

Moral of the story... if IDisposable is implemented, chances are you should call it.

ADDENDUM
----------
For those who are not familiar, IDisposable is an interface for the implementation of the Dispose Pattern.  This is very useful in programming languages that have Garbage Collection (ie. .Net).  When you have Garbage Collection, you have what is called a non-deterministic destructor.  In other words, when you are done using an object, you have no idea when it's resources will be freed.  Garbage Collection occurs at an interval that is determined by the system.  By implementing IDisposable, you ad a Dispose method to your class where high contention resources can be freed (ie. File Handles, Database Connections, etc.).  The implementing of IDisposable signals the user that there is something valuable to be cleaned up ASAP that cannot wait for Garbage Collection.  So, even if Dispose is called, the Garbage Collector will still clean up the object's resources eventually.  But in the mean time, all of the high value resources have been released.

Monday, April 09, 2007 10:49:49 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]    | 
 Wednesday, April 04, 2007

I wanted to share this bit of SQL wisdom I picked up a few months ago on the ASP.Net forums (http://forums.asp.net). Let's say you have an orders table and you have an orderDate column that contains the Date & Time of the order.  If you wanted to create a query to select all orders made yesterday at any time, I cameup with this working (if not long winded) method:

 Where orderDate = CAST(MONTH(DATEADD(day, - 1, GETDATE())) AS varchar) + '/' + CAST(DAY(DATEADD(day, - 1, GETDATE())) AS varchar) + '/' + CAST(YEAR(DATEADD(day, - 1, GETDATE())) AS varchar))

I was treated to an extreme stream lining of this on the forums.  The resulting query was:

Where orderDate >= convert(Varchar, Getdate() -1, 101) And orderDate < convert(Varchar, Getdate() , 101)
 
The use of convert with the right format number negates a lot of work and is much easier to read.
Wednesday, April 04, 2007 9:32:33 PM (Eastern Daylight Time, UTC-04:00)  #    Comments [1]    | 
Copyright © 2008 Christian M Loris. All rights reserved.
DasBlog 'Portal' theme by Johnny Hughes.