Google analytics script

Latest jQuery CDN with code tiggling.

Sunday, 23 January 2011

How to correctly use IHttpModule to handle Application_OnStart event

In one of my previous blog posts (Writing a custom IHttpModule that handles Application_OnStart event) I've been talking about using IHttpModule to also handle application start event which is a non-documented feature. Sure it works, but you may see some strange behaviour of duplicated (or even multiplicated) functionality being executed. You probably won't see this with your applications in development, because your local IIS isn't really heavy duty workhorse, but in production environment you may see this strange behaviour. Investigating it is even a bit more complicated because of the running application serving live traffic. Let me help you. So I will point you in the right direction and more importantly show you a solution that makes things work reliably even on a heavy load IIS.

Symptoms

When you create a custom IHttpModule and put particular functionality in it that changes some static or shared resource you may see that its functionality has been run multiple times. That may seem strange since every module's initialization happens only once. At application start. Why does it seem to run multiple times then?

I've used my module to register Asp.net MVC framework to work with an existing Asp.net WebForms application. First thing that has to be done is registering routes. RouteTable has its routes stored in a static property Routes. That is already problematic. So registering routes multiple times will make route table X times larger, but you wouldn't see any errors, because the first set of routes will catch routes just fine so all subsequent route definitions won't get hit at all.

But routes were not the root of my invalid application execution. As mentioned I was registering Asp.net MVC in a non-MVC application. Namely Sharepoint 2010 (I know I know I'm arrogant enough to think I'm smart to do this blasphemy; I'd burn in medieval times for sure). The fact that I was also registering MVC's view engine was the problem I was hitting. View engines are also part of a static collection in the ViewEngines class. And they work more like a queue, not as a normal collection. Don't ask me why because I haven't investigated this any further than required but that's the way it is. Details aren't really important in this case. The problem was simply that view engine was registered multiple times in various locations in queue. There's no reasonable explanation to this other than that my module's Init() method was being called multiple times.

Diagnose

When I looked at HttpApplication class' code there was just one point where modules actually got initialized. At application start. Once only. So why was it that my module got initialized multiple times or at least it seemed as it was. The reason lies in the fact that Asp.net applications are run by IIS. And as you may know for a long time, IIS has something called application pools. You probably never gave too much thought to these. All you had to actually know was that application pools are separate from each other so when one application hangs others still run as expected without interruption. Thus you separated your applications into separate pools. But you already know this, don't you? You probably do.

Now lets think of the word itself. Application pool. Yes pool. It means that a certain web application is running multiple HttpApplication instances in one pool. Yes multiple. Otherwise it wouldn't be called a pool. »How many?« you may ask. That doesn't really matter as long as you know there could be more than one. We trust IIS to do its job. And it obviously does it so well that it made this fact completely transparent for us developers hence not many completely understand its inner workings. We rely on its robustness to provide the service. And it does. Each of these HttpApplication instances in the pool keeps its own list of HTTP modules that it uses with each request it processes.

Now we know why our modules get initialized multiple times. Once for every HttpApplication instance in the application pool. This is what happens on IIS when an application starts:

  1. when the first request comes in IIS/Asp.net creates as many HttpApplication instances as it needs to process requests simultaneously. For most applications, this number is limited to the number of threads and remains in the range of 1 through 100, depending on the hardware, server load, configuration, and so on (as per Application Instances, Application Events, and Application State in ASP.NET article);
  2. when IIS/Asp.net creates the very first HttpApplication instance it then calls its Init() method;
  3. because this is the first instance it calls Application_OnStart() method as well;
  4. it then creates as many HttpApplication additional instances as it needs and calls their Init() method;
  5. because these are subsequent instances it doesn't call their Application_OnStart() method;
  6. every call to HttpApplication.Init() also creates HTTP module instances and calls their Init() method right after creation;
  7. etc.
From this point on processing is irrelevant to understand and solve the problem at hand. Application ending works similarly but in the opposite order. You can read more about it here.

Cure (solution)

It is now easy to understand (and it also makes perfect sense) why certain module's Init() method gets called multiple times when application starts. For what's worth it could be called afterwards as well. I guess if IIS/Asp.net decides it needs some more HttpApplication instances I guess it can create few more during application life. I'm not saying it does but I suppose it could.

Knowing all this information we can envision our solution that will finally make it possible to create a custom IHttpModule that can handle Application_OnStart event. We will do this by using static variables and thread locking. But don't worry. Thread locking only happens once so it won't cripple your application. Lets look at some code.

   1:  public abstract class ApplicationStartModuleBase : IHttpModule
   2:  {
   3:      #region Static privates
   4:   
   5:      private static volatile bool applicationStarted = false;
   6:      private static object applicationStartLock = new object();
   7:   
   8:      #endregion
   9:   
  10:      #region IHttpModule implementation
  11:   
  12:      /// <summary>
  13:      /// Disposes of the resources (other than memory) used by the module that implements <see cref="T:System.Web.IHttpModule"/>.
  14:      /// </summary>
  15:      public void Dispose()
  16:      {
  17:          // dispose any resources if needed
  18:      }
  19:   
  20:      /// <summary>
  21:      /// Initializes the specified module.
  22:      /// </summary>
  23:      /// <param name="context">The application context that instantiated and will be running this module.</param>
  24:      public void Init(HttpApplication context)
  25:      {
  26:          if (!applicationStarted)
  27:          {
  28:              lock (applicationStartLock)
  29:              {
  30:                  if (!applicationStarted)
  31:                  {
  32:                      // this will run only once per application start
  33:                      this.OnStart(context);
  34:                      applicationStarted = true;
  35:                  }
  36:              }
  37:          }
  38:          // this will run on every HttpApplication initialization in the application pool
  39:          this.OnInit(context);
  40:      }
  41:   
  42:      #endregion
  43:   
  44:      /// <summary>Initializes any data/resources on application start.</summary>
  45:      /// <param name="context">The application context that instantiated and will be running this module.</param>
  46:      public virtual void OnStart(HttpApplication context)
  47:      {
  48:          // put your application start code here
  49:      }
  50:   
  51:      /// <summary>Initializes any data/resources on HTTP module start.</summary>
  52:      /// <param name="context">The application context that instantiated and will be running this module.</param>
  53:      public virtual void OnInit(HttpApplication context)
  54:      {
  55:          // put your module initialization code here
  56:      }
  57:  }
You can use this class as your base class for any modules that you need them to handle Application_OnStart event. Any other (usual) modules don't need this thread safety and checking. Unless of course required by your process.

Outcome

From the additional information we have about application pools we can deduct that handling Application_OnStart is easily possible. I've shown you a solution how to do this. But if we think of the other application event Application_OnEnd things get a bit more dodgy. Event is executed on the last HttpApplication just before its Discard() method gets called. So how in the world could we detect that? I guess the best way would be to change our code and instead of using a bool variable we could use an int and increment it with each initialization. Then we'd be checking its value in Discard() method and when it would get back to where it started this would mean that the last HttpApplication is being discarded hence executing Application_OnEnd functionality. I guess this could work but you can suggest a better solution if you know of any. The only problem could be thread locking that would happen on every module initialization to safely increment the counter. The same thing would be going on in the process of disposal. The good news is that this lock would be issued for a very very short period of time and just at application instantiation and disposal. Just long enough to increment/decrement the counter. Only the first and the last lock would take longer, because they would execute start and end event code.

If you feel like it you're more than welcome to leave a comment or at least vote on usefulness of this blog post. Click your preferred reaction below here.

35 comments:

  1. Very nice post that reflect some IIS web application handling internals, that can come handy in development. Worth to remember!

    ReplyDelete
  2. Good article! You are however, missing in your example code, the line that sets applicationStarted to true.

    ReplyDelete
  3. @GiddyUpHorsey You're right. I added the assignment statement.

    ReplyDelete
  4. Good info, thanks! Also, you could use a readwrite lock to add a little better performance. Although, as you pointed out, this locking should only be a trivial performance hit as it only starts once for every HttpApplication instance. However, on very high-traffic websites where an AppPool is recycled frequently this may be a more significant factor.

    -jtoast

    ReplyDelete
  5. I noticed this problem in my application when I did load tests.

    Thanks.
    Now everything is explained

    ReplyDelete
  6. @Caique Dourado: Glad I could be of some help clearing this up for you. Thanks for your comment.

    ReplyDelete
  7. One other thing to note which is often forgotten is that registration of events (e.g. Application.BeginRequest + Application.EndRequest) should be done in your OnInit method - so that the events is fired in all instances of HttpApplication.

    ReplyDelete
  8. @Rune Olsen: Thanks for pointing that out. My module template doesn't define any event subscription so I only stated using comments what should be defined and where. But don'0t define anything else apart from that.

    Hence comment before OnInit call:

    // this will run on every HttpApplication initialization in the application pool

    ReplyDelete
  9. this.OnInit(context) is an error because OnInit doesn't take any parameters.

    ReplyDelete
  10. I mean, it's just a typo, but still wanted to mention it.

    ReplyDelete
  11. @Anonymous: True true. Corrected it.

    And if you looked closely this parameter was added in methods XML documentation. It just wasn't added in method definition. :)

    Thanks for pointing it out anyway.

    ReplyDelete
  12. Many thanks for this article!
    Makes several things much clearer

    ReplyDelete
    Replies
    1. You're welcome. Always glad to help.

      Delete
  13. ... and because I've posted the comment above I cant see the article anymore 0.o
    I wasn't able to scroll back up after publish. After a reload all I get is a white page.
    I think you should check your javascripts ;)

    (using IE9)

    ReplyDelete
    Replies
    1. No no. This is highly likely related to Blogger's changed comment functionality. They allow comment replies that they didn't. Which BTW completely broke my comment styling because they're using different HTML element now. I will have to change my CSS. But apart from that I'm not using any special scripts on my blog. So they can't be the cause of it.

      Delete
  14. Thank you very much. I had similar problem and I was wondering why it is happening.
    There is a small problem in your example though. With double-check lock pattern you should use “volatile” keyword in the in the applicationStarted field declaration. There are some border cases in which the second thread may still read false value despite the thread lock.
    In the following post you can read pretty good explanation about the problem:
    http://geekswithblogs.net/akraus1/articles/90803.aspx

    ReplyDelete
    Replies
    1. Thank you very much for the volatile hint. You're right that without using volatile modifier on the boolean variable could pose a problem. Since this is much much harder to test I added the modifier to my code... Just to stay on the safe side. Thanks again.

      Delete
  15. Works great! I used this on a DotNetNuke module so I could enable/disable SQL Cache Dependency. I used your idea of changing applicationStarted to an int. In Dispose, just subtract one and if applicationStarted == 0, run the OnEnd() method. Worked perfect!

    ReplyDelete
    Replies
    1. Thanks for the info. I'm glad it works as expected.

      Delete
  16. Hi Robert,Thanks for sharing this. I am also in a similar quest to use just the URL routing feature in SharePoint 2010. I followed your post and created a httpmodule that register all my routes and the routehandlers for the same. However I end up with a nasty exception as below:

    System.ArgumentException: virtualPath
    at Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.IsExcludedPath(String virtualPath)
    at Microsoft.SharePoint.ApplicationRuntime.SPVirtualPathProvider.DirectoryExists(String virtualPath)
    at System.Web.Routing.RouteCollection.GetRouteData(HttpContextBase httpContext)
    at System.Web.Routing.UrlRoutingModule.PostResolveRequestCache(HttpContextBase context)
    at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
    at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

    Did you come across this while working on the above ?

    I have registered the modules in the order:
    <add name="BlogUrlRoutingModule" type="[Fully qualified name of my module]" />
    <add name="UrlRoutingModule" type="System.Web.Routing.UrlRoutingModule, System.Web.Routing, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />

    so that the routes are already defined before the UrlRoutingModule executes.

    I have read another post which talks about creating a custom Virtual path provider for sharepoint, which I havent tried yet.

    http://simone.codeplex.com/wikipage?title=Host%20Aps.Net%20Mvc

    Is this something you had to do to get it fully working.
    Your help/comments will be really appreciated.

    ReplyDelete
    Replies
    1. There's a bit more than just registering routing in Sharepoint 2010 to actually make it work. It also depends how deep you'd like it to integrate with Sharepoint. I wanted to use Sharepoint's masterpages as well not just run in it's context. That's why my virtual path provider is a bit more complex than the one you've linked.

      But to say the least. I have had this post long overdue where I'm about to give you all the code how my integration's working, but I never find enough time to do it. It will be a long one...

      My MVC is integrated inside an existing Sharepoint Site within a certain folder only. Everything else works as is.

      /Mysite - is normal Sharepoint
      /Mysite/MvcSubfolder - is an MVC app that uses all settings of it's parent Sharepoint app.

      MVC folder is configurable and is not necessarily the same as physical IIS virtual folder.

      My custom classes:
      1. MvcVirtualPathProvider
      2. MvcInitializerModule
      3. SubfolderUrlRoutingModule
      4. SubfolderWebFormViewEngine

      And likely some more... These are just from the top of my head.

      Module registration in web.config looks like this:


      <modules runAllManagedModulesForAllRequests="true">
      <remove name="AnonymousIdentification" />
      <remove name="FileAuthorization" />
      <remove name="Profile" />
      <remove name="WebDAVModule" />
      <remove name="Session" />

      <remove name="SubfolderUrlRoutingModule" />
      <remove name="MvcInitializer" />
      <add name="SPRequestModule" preCondition="integratedMode" type="Microsoft.SharePoint.ApplicationRuntime.SPRequestModule, Microsoft.SharePoint,..." />
      <!-- Make sure these two modules are listed right after SPRequestModule -->
      <add name="MvcInitializer" preCondition="" type="MvcWeb.Web.General.MvcInitializerModule, MvcWeb.Web,..." />
      <add name="SubfolderUrlRoutingModule" preCondition="" type="MvcWeb.Web.General.SubfolderUrlRoutingModule, MvcWeb.Web,..."/>

      <add name="ScriptModule" preCondition="integratedMode" type="System.Web.Handlers.ScriptModule, System.Web.Extensions,..." />
      <add name="SharePoint14Module" preCondition="integratedMode" />
      <add name="FederatedAuthentication" type="Microsoft.SharePoint.IdentityModel.SPFederationAuthenticationModule,..." />
      <add name="SessionAuthentication" type="Microsoft.SharePoint.IdentityModel.SPSessionAuthenticationModule,..." />
      <add name="SPWindowsClaimsAuthentication" type="Microsoft.SharePoint.IdentityModel.SPWindowsClaimsAuthenticationHttpModule,..." />
      </modules>


      Does this help? Probably not. And I must say that it took me quite some time to make it. But it works... Hopefully I will find the time to finally write that post...

      Delete
  17. Thanks for the update. Definitely looking forward to the above post and some sample code as to how everything ties together.

    ReplyDelete
  18. For handling app shutdown, would it not be safe to handle the HttpApplication.Disposed event?

    So:

    this.OnStart(context);
    context.Disposed += MyDisposalEventHandler;

    ReplyDelete
    Replies
    1. If you look at my class it doesn't have this event (it doesn't inherit from any class that has it, it merely implements IHttpModule interface) so it should be extended to support it.
      But I wonder why complicate the code if it works in its simplified version. The only reason I can think of is if you'd have more methods/handlers you'd want to execute. So you'd subscribe several event handlers to it. But in this simplified example (which is likely the production code as well) there's no need for such functionality.

      Delete
  19. You made some decent points there. I looked on the internet for the issue and found most individuals will go along with with your website.

    ReplyDelete
  20. Hi Robert,

    Got everything set up and I can browse my standard SharePoint pages normally, but would like to the do something like: http://www.mysharepoint.com/home instead of http://www.mysharepoint.com/pages/default.aspx. From my understanding of how routing works, and reading through your recommendations I presume that I have everything set up correctly, but still get a 404 when trying to initialize my route as stated above.

    Any ideas what I might be missing?

    - Wrote my own VirtualPathProvider for ~/
    - Wrote a custom HttpModule to handle the OnStart (as per your blog)
    - Added the necessary binding to the web.config
    - Ensured this is Web.Routing 3.5
    - Register routes correctly

    ReplyDelete
  21. in a webgarden environment use a mutex since lock doesn't work over processes

    ReplyDelete
    Replies
    1. I'd say depends. And I say that simply because this largely depends on what you do in your application start-up event. Every web server will be running its own set of IHttpApplication instances and each server will (although I can't prove I'm right) run application start when the first instance is created.

      But it depends whether you initialize server-wide or web garden-wide things in your application start-up. That's why it may not be necessary to use mutex...

      Delete
  22. It worked like a charm. Thanks you very much!

    ReplyDelete
  23. It works great, unfortunately only for one type inherited class. I created two modules classes inherited from that base class and i noticed overridden method OnStart(context) is called only for one from these modules.because of that applicationStarted is static.

    ReplyDelete
  24. The simplest approach for this issue is make base class as generic for example:
    ApplicationStartModuleBase
    HandleErrorHttpModule : ApplicationStartModuleBase

    ReplyDelete
    Replies
    1. Yes generics are best because static variables aren't shared between instances with different generic type.

      Delete
  25. I have an interesting scenario I have been tracing through trying to understand and debug which lead me to your blog posts.

    On certain servers we are seeing where Application_Start is being called multiple times, but no corresponding Application_End() - none in fact. This is causing our application to begin (many times) and never finish the data processing begun inside. Any help would be much appreciated.

    ReplyDelete
    Replies
    1. Please read this blog post again. Carefully. Because it talks about this scenario that you're having. Application pools on IIS. Ot is perfectly normal that Application_Start gets called several times without having intermediate "ends" in between. Each thread thst IIS starts executes "start" event. Hence the change in my upper code. I suggest you really re-read this blog post again. Carefully and diligently.

      Delete
  26. I have implemented a module which simulate "begin_application" and "end_application". My implementation doesn't lock the other module instances while the start code runs. This worked well in my case where i needed initialize and finalize a quartz scheduler.

    public class MyModule : IHttpModule
    {
    private static int applicationStarted = 0;

    public void Init(HttpApplication app)
    {
    if (Interlocked.Increment(ref applicationStarted) == 1)
    {
    /// Acao a ser executada quando a primeira aplicacao for criada.
    this.OnAppStart(app);
    }
    }

    public void Dispose()
    {
    if (Interlocked.Decrement(ref applicationStarted) == 0)
    {
    /// Finaliza o scheduler se for o Ășltimo.
    this.OnAppStop();
    }
    }

    private void OnAppStart(HttpApplication app)
    {
    new StdSchedulerFactory().GetScheduler().Start();
    }

    private void OnAppStop()
    {
    new StdSchedulerFactory().GetScheduler().Shutdown(false);
    }
    }

    ReplyDelete