Chris Taylor

Building Web Applications using a Windows Forms paradigm

taylorza — Sat, 07 Jul 2007 19:43:00 GMT

I don't typically post on a non-technical topic, but recently I started looking at Visual WebGUI and I was really impressed.

In essence it provides a Windows Forms design time interface for building web applications. The user experience and developer experience very very closely matches that of a traditional Windows Forms application, but the content is delivered as web code which seems from the little testing that I have done to at least support IE and FireFox.

The realy cool thing about the framework is that the client updates and events are all driven using ajax to lighten the network traffic. Once the server side code has executed only the required updates are transmitted back to the client and automagically integrated into the client presentation without any assitance or know how from the developer.

The framework seems to have a very friendly licencing model, essentially free unless you want guaranteed support (that's my simplified interpretation). The controls are all open-sourced so you can develop new controls and extend and modify the appeance and functionality of the intial control library.

I would do the framework an injustice to try an explain all the things it achieves, so I would strongly recommend you take a look at it. I would be very interested to hear your opinions on it.

Now I wonder if Microsoft would pick-up on something like this.

Using XBOX 360 Controller on Windows 2003

taylorza — Sun, 31 Dec 2006 16:23:00 GMT

Updated to include step by step instructions

Now that the Game Studio Express has released I intend to be playing around quite a bit with it. Since I am not a game programmer and the software I typically work on is server based my preferred development platform is Windows 2003, this poses a problem because the XBOX 360 Controller for windows driver refuses to install if the target platform is not Windows XP with the following message.

However, I found that all the required driver files have been extracted at this point and can be installed on Windows 2003 and so far (touch wood) with no ill-effect.

The driver files are extracted to a folder on the root of the drive you ran the installation from, the folder has a randomly generated name which looks something like the following

D:\f37260c804717b0288154486

The associated driver files can be located at

D:\f37260c804717b0288154486\xbox360\setup\files.

Now all you need to do is plug in the XBOX 360 Controller into a free USB port and point the hardware installation wizard to the folder that contains the driver files. Once the installation is complete you should be able to use the controller on Windows 2003.

Example Installation

1. Run Xbox360_32Eng.exe installation program. If you get the Open File - Security Warning dialog, click the Run button. Take note which drive you run the installation on.

2. On Windows 2003 you will be prompted with the message 'Setup cannot install this software because your computer does not meet the system requirements. To install this software, you must have a computer running Microsoft Windows XP SP1 or greater.'.
DO NOT DISMISS/CLOSE THIS MESSAGE BOX

3. While the message box is still on the screen, use explorer to browser the root directory of the drive you ran the installation on. In my case it was the 'D' drive. On the root of this drive you will find a folder with a randomly generated name, in my case the folder is named 'f7267e57e15594f2f5a62054b463fb0e', note your folder name will differ.

4. Once you have noted the folder name, you can proceed to plug in the XBOX 360 controller. Windows should detect the device and prompt you with the 'Found New Hardware Wizard'.

5. Select the 'Install from a list or specific location(Advanced)' radio button and click 'Next'.

6. Ensure that the 'Searc for the best driver in these locations.' radio button is selected and check the 'Include this location in the search' checkbox, nothing else needs to be checked.

7. Enter the path to the driver, this path is relative to the temporary folder created by the installation program, in my case the path is. D:\f7267e57e15594f2f5a62054b463fb0e\xbox360\setup\files. Click 'Next'. Windows should proceed to install the driver, do not be concerned, this can take a few moments.

8. Now all that is left to be done is to click on 'Finish' and you can use your controller for your XNA development. Oh and before I forget, you can dismiss that message box now :) You know the one that did not want you to install the driver on Windows 2003.

Enjoy!!

My foray into XNA : Basic Pixel Perfect Collision Detection

taylorza — Sat, 30 Sep 2006 09:55:00 GMT

So I decided to give XNA a quick spin. Now I am not a game developer, not even an aspiring game developer, but I generally take anything new for a drive just to familiarize myself with it.

I was pleasantly surprised, in about 10 minutes I have objects flying around the screen in all directions and that was written entirely from scratch using only the initially wizard. The 2D stuff is really easy to get started with, I have not ventured into the 3D side yet.

Anyway, once I had all these object flying around I decided to do some basic collision detection to test when they collide. I wanted to go further than just bounding box checks; I wanted pixel perfect collision detection, the kind that reports collisions on when the pixels of one image collide with the pixels of another.

To optimize the collision detection the detection is performed in two stages.

1. Check for potential collisions using the bounding box

2. If test 1 revealed a potential collision perform a pixel level comparison of the overlapping areas of the two textures.

Before I get started I first want to define some simplified terminology that I will be using.

Texture – Represents the graphical information required to render a Sprite.

Texture co-ordinate space – Rectangle defining the size of the texture. A texture that is 32x32 will have texture co-ordinate space (0, 0, 32, 32).

Sprite – A graphical object painted with a texture at a location defined by the sprite co-ordinates.

Sprite co-ordinates – Rectangle representing the space occupied by the sprite in the game.

Note that the routines defined here are very simplistic and do not cater to scaling and rotation transformations of the sprite. Therefore the width and height of the sprite must correspond to the width and height of the texture painted on the sprite!

See the notes below for more information.

Now back to the tests, test 1 is to check for the potential collision by checking for a overlap between the is easily accomplished using the following function which compares to rectangles and returns true if they overlap.

public static bool Intersect(Rectangle rectangle1, Rectangle rectangle2)
{
if (((rectangle1.X < (rectangle2.X + rectangle2.Width))
    && (rectangle2.X < (rectangle1.X + rectangle1.Width)))
    && (rectangle1.Y < (rectangle2.Y + rectangle2.Height)))
{
    return (rectangle2.Y < (rectangle1.Y + rectangle1.Height));
}
return false;
}

To determine if two sprites have overlapped the Intersect function is called with sprite co-ordinates of the two sprites you are interested in. If this function returns true we need to move on to Test 2.

Test 2 requires that we determine if any of the corresponding pixels in the overlap between the two sprites are visible i.e. not transparent. If both sprites contain corresponding visible pixels at the same location this represents a collision, if on the other hand none of the visible pixels overlap with visible pixels of the other texture then there is no collision. The following images depict the two situations graphically.

;

The first image shows two sprites which there bounding rectangles overlapping however none of the visible pixels overlap and therefore this is not a true collision. The second image shows that there is an overlap of visible pixels between the two sprites in the intersection and therefore represents a collision.

To do the pixel level tests we first need to determine the intersection rectangle, the rectangle highlighted in black in the above images. The following function can be used to achieve this.

public static Rectangle Intersection(Rectangle rectangle1, Rectangle rectangle2)
{
int x1 = Math.Max(rectangle1.Left, rectangle2.Left);
int y1 = Math.Max(rectangle1.Top, rectangle2.Top);
int x2 = Math.Min(rectangle1.Right, rectangle2.Right);
int y2 = Math.Min(rectangle1.Bottom, rectangle2.Bottom);

if ((x2 >= x1) && (y2 >= y1))
{
return new Rectangle(x1, y1, x2 - x1, y2 - y1);
}
return Rectangle.Empty;
}

The two rectangles passed to the function will be the sprite rectangles; the result is a new rectangle which represents the intersection of the two rectangles.

Armed with the intersection rectangle we need to convert the co-ordinate of the intersection rectangle to the texture co-ordinates of the two textures. In other words, we need to determine the rectangles within the textures that overlapped. Viewing each texture individually this would look like the following.

The following function will translate the co-ordinates one rectangle to the co-ordinate space of the reference rectangle.

public static Rectangle Normalize(Rectangle reference, Rectangle rectangle)
{
return new Rectangle(
    rectangle.X - reference.X,
    rectangle.Y - reference.Y,
    rectangle.Width,
    rectangle.Height);
}

The function is called passing the sprite co-ordinate rectangle as the reference and the intersection rectangle as the second rectangle argument. Call this function once for each of the sprites, resulting in two rectangles, representing the rectangle within each of the textures that we are interested in.

Now we need to extract the pixel data for the two rectangles and compare them pixel for pixel. The following function will perform this task in a reasonably efficient manner, but my focus was readability.

public static bool TestCollision(
Texture2D texture1, Rectangle texture1Intersection,
Texture2D texture2, Rectangle texture2Intersection)
{
int pixelCount = texture1Intersection.Width * texture1Intersection.Height;
uint[] texture1Pixels = new uint[pixelCount];
uint[] texture2Pixels = new uint[pixelCount];

texture1.GetData(0, texture1Intersection, texture1Pixels, 0, pixelCount);
texture2.GetData(0, texture2Intersection, texture2Pixels, 0, pixelCount);
for (int i = 0; i < pixelCount; ++i)
{
    if (((texture1Pixels[i] & 0xff000000) > 0)
      && ((texture2Pixels[i] & 0xff000000) > 0))
    {
      return true;
    }
}
return false;
}

The key to performance in this function is to extract only the pixel data we are interested in, because the area of intersection within both texture is the same size we can extract the data and run through it one element at a time. Each element index represents the corresponding pixel data in each of the textures. In the loop we mask out the Alpha channel bits by performing a bitwise and, if the alpha channel of both pixels (texture1 and texture2) is not 0 we have a visible pixel overlap which is considered a collision and we can break out of the loop.

Some notes:

1. As has already been mentioned the above does not address scaled or rotated sprites.

2. Additional performance can be achieved by compiling with the unsafe switch and using pointer arithmetic to iterate the pixel data. From my tests I get roughly 4 frames per second extra when using pointers, which is with 100 sprites bouncing around.

3. If the original textures are not powers of 2 in size XNA automatically scales the textures, this is for performance reasons. However the problem with that for this version of the algorithm is that the scaling introduces noise (anti-aliasing) into the image which improves the visual quality of the scaled image. While the artifacts from this process are not individually visible to the eye the algorithm does detect them as visible pixels and potentially results in a false positive, at least as far as the eye can see.

Basic Application Partitioning

taylorza — Fri, 21 Jul 2006 18:06:00 GMT

I often need to present a more practical view of the classical application partitioning
Presentation Layer -> Business Logic Layer -> Data Access Layer
I believe the diagram below depicts a clear if simplified representation of the interaction between layers.

The key tenant is that there is no SQL or ADO.NET data access code beyond the Data Access Layer (DAL). The DAL provides all data services to the higher level layers. In the cases where the where the business logic is hosted within the physical database the DAL provides a pass-through interface to expose the business logic to the business layer.

With in the database a similar logical partitioning is maintained restricting the DAL from directly accessing the underlying tables. Within the database it self this is relaxed and both the database access code and business code have direct access to the tables.

The two bars cutting across the layers Business Entities and Data Entities represent the data container classes that transport data and user service request arguments between the layers.

While the data entities represent basic entities within the system, the business entities are represent more complex aggregates of the data entities required to complete an autonomous business function. For example to save a customers application for an account could be broken down into the following. For simplicity this explanation forgoes the actual business logic involved in processing an application and focuses on storing the application.

The class definitions are simplified pseudo-code of entity classes

Data Entities

class ApplicationEntity
class CustomerEntity
class AccountEntity

Business Entity

class AccountApplicationEntity
{
ApplicationEntity ApplicationDetails;
CustomerEntity CustomerDetails;
AccountEntity AccountDetails;
}

Data Service Interface

SaveApplication(ApplicationEntity entity); // Saves the data in the Applications table
SaveCustomer (CustomerEntity entity); // Saves the data in the Customers table
SaveAccount(AccountEntity entity); // Saves the data in the Accounts table

Business Service Interface

SaveAccountApplication(AccountApplicationEntity entity)
{
DataService ds = new DataServiceInterface();
ds.SaveApplication(entity.ApplicationDetails);
ds.SaveCustomer(entity.CustomerDetails);
ds.SaveAccount(entity.AccountDetails);
}

You can assume that the data access is in a transaction.

From the consumer’s point of view creating an application is a matter of instantiating an instance of the AccountApplicationEntity and populating the ApplicationDetails, CustomerDetails and the AccountDetails, then invoking the business service for further processing.

Debugging WindowsIdentity and IsInRole 2.0

taylorza — Thu, 29 Jun 2006 18:12:00 GMT

Update: Accidentally posted an older version of the function, this is a slightly cleaner version.

In a previous post I showed how to get get a list of groups from a WindowsIdentity, this method used all kinds of under handed trick to get to the information and as warned the solution no longer works for the new version of the framework. Here is an updated version which takes advantage of officially supported techniques available in the 2.0 Framework, given a WindowsIdentity it returns a list of role names from the WindowsIdentity.

public string[] GetWindowsIdentityRoles(WindowsIdentity identity)
{
if (identity == null) throw new ArgumentNullException("identity");

IdentityReferenceCollection groups = identity.Groups.Translate(typeof(NTAccount));
string[] roles = new string[groups.Count];
for (int i = 0; i < groups.Count; ++i)
{
roles[i] = groups[i].Value;
}

return roles;
}

This function comes in handy when you are not certain of the string representation of a role you need to pass to the IsInRole function.

Gradient Shades

taylorza — Sun, 25 Jun 2006 17:53:00 GMT

Recently I was googling and came a routine to calculate the transision from one color to another color in 255 discreet steps.
Looking at the code it did not quite look like it was going to achieve what I understood goal of the routine to be. So I quickly
put some code together and then found that there was no way to post a comment on the blog. The original post can be found
here http://dalldorf.us/home/?m=200603

So rather than waste the code here it is.

public Color Gradient(Color from, Color to, byte step)
{
   double stepFactor = (double)step / 255;
   return Color.FromArgb(
      (int)(from.R + ((to.R - from.R) * stepFactor)),
      (int)(from.G + ((to.G - from.G) * stepFactor)),
      (int)(from.B + ((to.B - from.B) * stepFactor)));
}

Thread Local Storage the .NET way

taylorza — Thu, 18 Aug 2005 20:58:00 GMT

Those of you familiar with the intricacies of Win32 multi-threaded programming will be intimately familiar with thread local storage (TLS). And you might have wondered how .NET exposes this often useful construct.

Before we begin I would like to offer a brief description of what TLS is, for those of you that might not have had the fortune of learning Win32 multi-threaded programming. TLS is a means of storing data on a per-thread basis. Any methods accessing the data will access the data associated with the thread in which the method is running. And this is achieved without the method having any special knowledge of the thread in which it is executing.

Now the real question, how can we achieve the same results in the .NET managed environment? Well depending on your situation there are a number of options available. I will describe only 3 options here. First I will describe the more commonly known CallContext and then what in my experience is less widely known Thread Static variables and finally there is the thread data slots.

The CallContext
Most of you are probably familiar with the CallContext class and how it relates to .NET remoting. A CallContext is used to flow data across multiple calls with out having to pass the data along as some part of the argument list of each called method. And if the stored data implements the ILogicalThreadAffinative interface the data will flow across machine boundaries.

A call context is maintained by the .NET framework for each logical thread of execution. By storing data in the call context you are essentially storing data specific to the thread currently executing the method, and any method called in that thread of execution has access to the threads call context. As a simple example, if you wanted to maintain a count of the number of times a function was called per thread you could do some thing like the following.

class Class1
{
static void Main(string[] args)
{
    System.Threading.Thread t1 = new System.Threading.Thread(new System.Threading.ThreadStart(ThreadFunction));
    System.Threading.Thread t2 = new System.Threading.Thread(new System.Threading.ThreadStart(ThreadFunction));

    // Start the threads
    t1.Start();
    t2.Start();

    // Wait for the threads to complete
    t1.Join();
    t2.Join();

    System.Console.Read();
}

private const string PerThreadCallCounterKey = "CallCounter";
private static Random Rnd = new Random();

private static void ThreadFunction()
{
    // Initialize the per-thread CallCounter
    System.Runtime.Remoting.Messaging.CallContext.SetData(PerThreadCallCounterKey, 0);
    // Pick a randon number of times to call the target method
    int iterations = Rnd.Next(255);
    // Call the method iteration number of times
    for (int i = 0; i < iterations; ++i)
    {
      DoTheWork();
    }
    DisplayCounterValue();
  }

  private static void DoTheWork()
{
// Get the current threads CallCounter from the CallContext
    int counter = (int)System.Runtime.Remoting.Messaging.CallContext.GetData(PerThreadCallCounterKey);

    // Increment the counter and store the new value in the CallContext
    // Notice the pre-increment (++counter).
    System.Runtime.Remoting.Messaging.CallContext.SetData(PerThreadCallCounterKey, ++counter);

    //OK, so we updated the counter now do the real work of this method
    //...
}

  private static void DisplayCounterValue()
{
    // Get the current threads CallCounter from the CallContext
    int counter = (int)System.Runtime.Remoting.Messaging.CallContext.GetData(PerThreadCallCounterKey);
    System.Console.WriteLine("The method was called {0} times from this thread", counter);
}
}

Notice how neither the DoTheWork or DisplayCounterValue methods have any knowledge of the calling thread. They are only aware of context information which is thread specific.

Thread Static Members

As an alternative to using the CallContext class, .NET provides support for what I call Thread Static Members. Thread static members for the most part act like normal static members except that they have per-thread storage rather than per-AppDomain. Declaring a thread static member is very similar to declaring a normal static member, except that the ThreadStaticAttribute attribute is applied to the declaration as follows.

[ThreadStatic]
private static int PerThreadCallCounter;

From this point on any thing that is done to the PerThreadCallCounter member affects the instance of this member associate with the current thread of execution. The following achieves the same results as the earlier example, only this time rather than using the CallContext class a thread static member is used.

class Class1
{
static void Main(string[] args)
{
    System.Threading.Thread t1 = new System.Threading.Thread(new System.Threading.ThreadStart(ThreadFunction));
    System.Threading.Thread t2 = new System.Threading.Thread(new System.Threading.ThreadStart(ThreadFunction));

    // Start the threads
    t1.Start();
    t2.Start();

    // Wait for the threads to complete
    t1.Join();
    t2.Join();
    System.Console.Read();
}

[ThreadStatic]
private static int PerThreadCallCounter;

private static Random Rnd = new Random();

private static void ThreadFunction()
{
    // Pick a randon number of times to call the target method
    int iterations = Rnd.Next(255);

    // Call the method iteration number of times
    for (int i = 0; i < iterations; ++i)
    {
      DoTheWork();
    }
    DisplayCounterValue();
}

private static void DoTheWork()
{
    // Increment the static member associated with the current thread
    PerThreadCallCounter++;

    //OK, so we updated the counter now do the real work of this method
    //...
}

private static void DisplayCounterValue()
{
    // Display the value of the thread static member to the user
    System.Console.WriteLine("The method was called {0} times from this thread", PerThreadCallCounter);
  }
}

As you can see and I am sure you will agree this is a much simpler solution. However a word of caution, with this example the naming of the instance becomes critical, sooner or later you will need to come back to a piece of code and if you loose site of the fact that a particular static member is bound to a thread rather than the entire AppDomain you could be in for some interesting debugging in the early hours of the morning.

While thread static members appear to work just like there normal static member counter parts, there are a few caveats. Firstly you might be tempted to initialize your thread static members in a static constructor. Unfortunately the results might surprise you. At first inspection it might appear that your initialization was ignored, however this is not the case, it is just that the static constructor is not run per-thread, but per-AppDomain. That is it is only executed once which means that the Thread Static member is only initialized for one thread, the and the rest are left untouched. And no, you can not apply the ThreadStaticAttribute to the constructor, this attribute can only be applied to member variables (fields).

Of course as apposed to the CallContext solution the ThreadStatic member is limited to the AppDomain boundary, which for most requirements is sufficient.

Thread Data Slots
Thread data slots are allocated once for every thread by using the Thread.AllocateDataSlot(). Once a data slot has been allocated the slot can be used from any thread to access the data stored in the data slot for the executing thread. This most closely matches the workings of the TLS APIs in the Win32 api. Rather than bore you with another rendition of the demonstration application I will leave it to the reader to investigate this solution which is adequately documented in the MSDN documentation.

Conclusion
This brief overview of TLS the .NET way probably leaves more questions than what have been answered. One thing I would like to mention is why in the world you would require TLS in an Object Oriented environment like .NET.

In my case I often end up writing code to analyze/profile existing code or applications which I either do not want to tamper with or do not have access to tamper with. Using these techniques I have been able to attach tracking information to ASP.NET requests without impacting existing code. Prepare information in one place and access that information from a event handler that was not designed to carry this kind of ad-hoc tracking information etc.

While this is rife for abuse this does have its merits when used wisely.

See Also:
Along similar lines as the ThreadStaticAttribute, you might also want to take a look at the ContextStaticAttribute. Especially for those of you interested in the potential of AOP offered by context bound objects.

Writing a Windows Service using ATL 7.1

taylorza — Wed, 04 May 2005 03:36:00 GMT

After more than 12 years of C++ Windows development I have written my share of Windows Services. These days however I spend most my income generating time in the throws of .NET development and consulting. But every now and then I get to do some really cool task using C++, mostly real-time hardware interfaces.

Recently I had to develop a Windows Service and since the service is interacting with a hardware device I thought I would go with C++ rather than .NET. This would be the first non .NET Windows service I have developed in about 3 years. Of course I would have to dig out all my backups to find my tried and tested service framework. Then I thought since the service was really nothing special let me use the ATL framework to develop the service. Now ATL is a technology that I really had a handle on, but that was in the days of version 3 so what did version 7 have to offer?

I fired up Visual Studio and selected the Visual C++ ATL Project. From the ensuing wizard I selected the Application Settings tab and checked the Attributed checkbox and the Service (EXE) radio button. In the generated code I quickly came across the class that implemented the heart of the service, this is essentially a template class that derives from a specialization of CAtlServiceModuleT<>.

Looking through the code for the template class (this is in the file atlbase.h) I decided that the Run method was a good place to put the code that started the service, so to test my theory I made a call the the LogEvent utility function to log an event when the Run method was invoked. I compiled the service and registered it by executing the service with the /service command line argument. When I started the service the service started and immediately stop, some thing was up so I stepped through the code and found that the call to PreMessageLoop was not returning S_OK, in short the function was trying to register COM object, but I had not written and COM objects and nor was I intending to. Fortunately I noticed that Microsoft had place the code in a define _ATL_NO_COM_SUPPORT so I defined this in the stdafx.h file, recompiled and started the service, this time my services started and stayed that way. I could even stop my service without any problem. To indicated to the Service Control Manager (SCM) that the service can handle notifications other than Start/Stop I modified the m_status.dwControlsAccepted flag to include SERVICE_ACCEPT_PAUSE_CONTINUE and overrode the OnPause and OnContinue functions.

As it turns out building a standard Windows Service using ATL is very simple. In summary here are the steps I followed:

1) Add the following code to the stdafx.h, before any of the #include statements
#define _ATL_NO_COM_SUPPORT

2) Add a constructor to the Module class, this class will be called CxxxModule where xxx is the name used when creating the project. In the constructor modify the m_status.dwControlsAccepted as required, here is an example that enables support for Pause and Continue.
m_status.dwControlsAccepted |= SERVICE_ACCEPT_PAUSE_CONTINUE;

3) Override the Run method to start you service activities, ultimately this is called whenever the service is started. It is important that the base class implementation is called so that the message loop is started. A simple implementation would look like this.

          HRESULT Run(int nShowCmd = SW_HIDE)
          {
          // Start worker thread here
          return __super::Run(nShowCmd);
          }

4) Override OnStop to stop the service. Here you should stop your thread from running and call the base class OnStop which takes care of terminating the message loop and notifying the SCM that the service has stopped. If the process of stopping the service thread is time consuming then you need to take further steps to manage this, if there is enough interest I will cover this in a later post.

          void OnStop()
          {
            // Stop worker-thread
            __super::OnStop();
          }

5) Override whatever other handlers you have choosen to support eg. OnPause and OnContinue. In these overrides you should call the SetServiceStatus to update the SCM on the status of the service. For example a simple implementation of OnPause and OnContinue would be something like the following.

          void OnPause()
          {
          SetServiceStatus(SERVICE_PAUSE_PENDING);
          // Pause worker-thread activity
          SetServiceStatus(SERVICE_PAUSED);
          }

          void OnContinue()
          {
            SetServiceStatus(SERVICE_CONTINUE_PENDING);
            // Resume worker-thread activity
            SetServiceStatus(SERVICE_RUNNING);
          }

This is not intended as an introduction to implementing windows services, just a documentation of the steps required to implement a standard Windows service using ATL 7.1. I have not even mentioned message string resources etc. I only hope this is useful enough to get some started without having to dig around like I did.

Propagating Distributed Transactions – A .NET Remoting Solution

taylorza — Wed, 16 Feb 2005 04:26:00 GMT

I have published an early draft of an article that I am working on at http://dotnetjunkies.com/WebLog/chris.taylor/articles/54503.aspx. The article describes a simple strategy to flowing distributed transactions across remoting boundaries. It also includes information on enabling TIP (Transaction Internet Protocol) on Windows 2003 server as well as Windows XP SP2. The formating still needs work, especially the code.

Distributed Transactions without COM+ Components

taylorza — Sat, 12 Feb 2005 20:40:00 GMT

Update: I have posted a draft of an article that takes this topic further and addresses flowing distributed transactions. See the post at http://dotnetjunkies.com/WebLog/chris.taylor/articles/54503.aspx.

Not being in a position to wait for all the wonderful support for transactions in .NET Framework 2.0 namespace System.Transactions I have had to make use of existing services to enlist in Distributed transactions and further more to propagate those transactions across process and machine boundaries.

Initial solutions required the development of numerous COM+ components. Being a C++ developer with extensive COM experience this did not bother me but fortunately with COM+ 1.5’s Services Without Components I no longer needed jump these hoops to satisfy the transactional requirements of the software. Even better is .NET Framework 1.1 support for these services in the System.EnterpriseServices namespace. Through the next few posts I will be passing on some of the techniques I have used to propagate distributed transaction across WebServices and .NET Remoting. To kick start here is a watered down version of the TransactionContext class.

   1:    using System;

   2:    using System.EnterpriseServices;

   3:    using System.Runtime.InteropServices;

4:

   5:    public class TransactionContext : IDisposable

   6:    {

   7:      private bool _consistent = false;

8:

   9:      public TransactionContext() : this(TransactionOption.Required)

  10:      {

  11:      }

12:

  13:      public TransactionContext(TransactionOption option)

  14:      {

  15:        ServiceConfig config = new ServiceConfig();

  16:        config.Transaction = option;

  17:        config.TrackingEnabled = true;

  18:        ServiceDomain.Enter(config);

  19:      }

20:

  21:      public void Complete()

  22:      {

  23:        _consistent = true;

  24:      }

25:

  26:      public void Dispose()

  27:      {

  28:        try

  29:        {

  30:          GC.SuppressFinalize(this);

  31:          if (_consistent)

  32:            ContextUtil.SetComplete();

  33:          else

  34:            ContextUtil.SetAbort();

  35:        }

  36:        finally

  37:        {

  38:          ServiceDomain.Leave();

  39:        }

  40:      }

41:

  42:      ~TransactionContext()

  43:      {

  44:        System.Diagnostics.Debug.Fail("TransactionContext must de explicitly Disposed");

  45:      }

  46:    }

To use this class is relatively simple, all you do is create an instance of the TransactionContext and if everything completes without causing a problem you put the context into a consistent state by calling Complete() when the Dispose method is called the vote for the success of the transaction is cast based on the consistency flag, you must ensure that Dispose is called as soon as your work is complete. The following example demonstrates the use of the TransactionContext class.

   1:        SqlConnection oCon;

   2:        SqlCommand oCmd;

   3:        using (TransactionContext tx = new TransactionContext())

   4:        {

   5:          oCon = new SqlConnection("Your Connection String here");

   6:          oCmd = oCon.CreateCommand();

   7:          oCmd.CommandText = "insert into ...";

8:

   9:          oCon.Open();

  10:          oCmd.ExecuteNonQuery();

  11:          oCon.Close();

12:

  13:          tx.Complete();

  14:        }

In a future post I will address some of the more naïve solutions to propagate these transactions across process and machine boundaries.

A simple LineNumberReader

taylorza — Sun, 06 Feb 2005 16:46:00 GMT

For a small personal project I have been developing a parser, as part of the parser I developed the LineNumberReader modeled after the Java(TM) class of the same name. Since the class has been useful, I thought I would share it. For the purposes of the post I have removed comments and a number of the constructors.

One of the nice features is the ability to set a marker in the stream using the Mark() method and returning to that point in the stream with the Reset() method. Enjoy!

   1:  class LineNumberReader : StreamReader

   2:      {

   3:        public LineNumberReader(Stream stream) : base(stream){}

4:

   5:        public LineNumberReader(

   6:          Stream stream,

   7:          System.Text.Encoding encoding) : base(stream, encoding){}

   8:        // More constructors matching those from StreamReader ...

9:

  10:        public int LineNumber

  11:        {

  12:          get { return _lineNumber; }

  13:          set

  14:          {

  15:            if (value < 0 ) throw new ArgumentOutOfRangeException(

  16:              "LineNumber", "Must be greater or equal to 0.");

  17:            _lineNumber = value;

  18:          }

  19:        }

20:

  21:        public void Mark()

  22:        {

  23:          if (!BaseStream.CanSeek) throw new NotSupportedException(

  24:            "Mark is not supported, underlying stream is not seekable.");

  25:          _mark = new MarkData(_lineNumber, BaseStream.Position);

  26:          _markset = true;

  27:        }

28:

  29:        public void Reset()

  30:        {

  31:          if (!_markset) throw new IOException("Reader not marked");

  32:          _markset = false;

  33:          DiscardBufferedData();

  34:          BaseStream.Seek(_mark.Position, SeekOrigin.Begin);

  35:          _lineNumber = _mark.LineNumber;

  36:        }

37:

  38:        public override int Read()

  39:        {

  40:          int retval = base.Read();

  41:          if (retval == (int)'\r')

  42:          {

  43:            if (Peek() == (int)'\n')

  44:            {

  45:              base.Read();

  46:            }

  47:            retval = (int)'\n';

  48:          }

49:

  50:          if (retval == (int)'\n')

  51:            _lineNumber++;

52:

  53:          return retval;

  54:        }

55:

  56:        public override int Read(char[] buffer, int index, int count)

  57:        {

  58:          int bytesRead = base.Read(buffer, index, count);

  59:          _lineNumber += CountLinesInBuffer(buffer, index, bytesRead);

  60:          return bytesRead;

  61:        }

62:

  63:        public override string ReadLine()

  64:        {

  65:          string retval = base.ReadLine();

  66:          if (retval != null)

  67:            _lineNumber++;

  68:          return retval;

  69:        }

70:

  71:        private int CountLinesInBuffer(char[] buffer, int index, int count)

  72:        {

  73:          int lineCount = 0;

  74:          int i = index;

  75:          int lastIndex = index + count;

76:

  77:          do

  78:          {

  79:            char ch = buffer[i];

  80:            if (ch == '\r')

  81:            {

  82:              if ((i + 1 < lastIndex) && (buffer[i + 1] == '\n'))

  83:                i++;

84:

  85:              ch = '\n';

  86:            }

87:

  88:            if (ch == '\n')

  89:              lineCount++;

  90:          } while (++i < lastIndex);

91:

  92:          return lineCount;

  93:        }

94:

  95:        private struct MarkData

  96:        {

  97:          public MarkData(int lineNumber, long position)

  98:          {

  99:            LineNumber = lineNumber;

 100:            Position = position;

 101:          }

 102:          public int LineNumber;

 103:          public long Position;

 104:        }

 105:

 106:        private int _lineNumber = 0;

 107:        private MarkData _mark;

 108:        private bool _markset = false;

 109:      }

Nostalgia

taylorza — Wed, 19 Jan 2005 01:42:00 GMT

Sitting this evening talking to my wife, I realized that she had never seen any of the Microsoft Operating Systems prior to Windows 95. This got me all nostalgic and I am in the process of installing a virtual machine with DOS 6.22 and Window for Workgroups 3.11. I might even have an old Visual Studio 1.5 lying around some where. Ah the good old days……

Oracle Team Development – Locking Oracle Objects

taylorza — Tue, 23 Nov 2004 00:17:00 GMT

Current source control systems such as Visual Source Safe, CVS, and Subversion is that they all rely on having a working copy of the source code under development on the developers local machine. Unfortunately this does not map quite so easily to the database environment. This is particularly problematic when multiple developers are working on database objects on the same server.

The Problem

The in our environment we are using Oracle as the backend database. One of the problems the Oracle developers are facing is when more than one of them is working on the same Package at the same time. Most tools allow you to extract the source for the Package edit the package and then compile it. It is at the compile stage that the problem occurs. If two developers edit the same package at the same time, the one that compiles last will overwrite the changes made by the other developer. Obviously there is another option, each developer could have his/her own database instance to work against and then use version control software to maintain the sources, but for this team this is how they work and since I have already thrown a spoke in the wheels by introducing Subversion I did not want to push things.

Finding a solution

With my focus on Microsoft Technologies, I am not particularly knowledgeable about Oracle, but I decided that I would take up the challenge to find a solution to this problem and learn a thing or two about Oracle along the way. Obviously the problem with piecing together a solution for a technology that you are not particularly knowledgeable about is that you run the risk of re-inventing the wheel. But assuming that the database developers are familiar with there tools there did not seem to be a satisfactory solution to the problem.

On of the few things I do know about Oracle is that they allow you to create triggers on the schema. These triggers are fired when changes are made to the existing schema. With this I decided I would attempt to piece together an object locking scheme where the developer could lock a database object before working on it and then once he has compiled and tested the object he can unlock it. To compile an object the object must be locked by the developer attempting the compile, if not a message should be displayed indicating that the object is either not locked or is locked by another developer. The solution should also be independent of the tools that are being used by the developer.

The solution

As it turns out the solution to this problem was rather easily implemented even with my limited knowledge of Oracle PL/SQL. I created a table that maintains a list of locked objects.

create table DBSCC_LOCKS
(
OBJTYPE varchar2(19) not null,
OBJNAME varchar2(30) not null,
LOCKEDBY varchar2(64) not null,
constraint PK_DBSCC_LOCKS primary key (OBJTYPE, OBJNAME)
)

The table has the object type for instance PROCEDURE or PACKAGE etc. and the name of the object and who has the object locked. For my purposed I decided to use the machine name the complete solution actually uses the OS user for this purpose.

The next step was to provide a trigger that would check against the DBSCC_LOCKS table before allowing a object creation to proceed.

create or replace trigger TR_DBSCC_CHANGE
before create on schema
declare
vMachine v$session.MACHINE%TYPE;
vLockedBy DBSCC_LOCKS.LOCKEDBY%TYPE;
vObjExists integer;
begin
-- Only locking PROCEDURES, PACKAGE and PACKAGE BODY
if ora_dict_obj_type in ('PROCEDURE', 'FUNCTION', 'PACKAGE', 'PACKAGE BODY') then
    -- Get the machine name that is executing the current session
    select s.machine into vMachine from v$session s
     where s.audsid = userenv('sessionid');

    -- Check if the object already exists
    select count(*) into vObjExists from ALL_OBJECTS o
     where o.object_type = ora_dict_obj_type
       and o.object_name = ora_dict_obj_name;

    -- If the object does not exist then we allow the
    if vObjExists = 0 then
      return;
    end if;

    -- Query the DBSCC_LOCKS table to see if and who
    -- has the object locked
    begin
      select l.LOCKEDBY into vLockedBy from DBSCC_LOCKS l
       where l.OBJTYPE = ora_dict_obj_type
         and l.OBJNAME = ora_dict_obj_name;
    exception
      when NO_DATA_FOUND then
      vLockedBy := null;
    end;

    -- Test the lock state of the object
    if vLockedBy is null then -- Not locked
      raise_application_error(-20001,
        ora_dict_obj_type || '.' || ora_dict_obj_name || ' is not locked, please use DBSCC_LOCK before updating');
    elsif vLockedBy <> vMachine then -- Locked from another workstation
      raise_application_error(-20001,
        ora_dict_obj_type || '.' || ora_dict_obj_name || ' cannot be created/updated because it is currently locked by ' || vLockedBy);
    end if;
    -- If we get here then the workstation owning the session
    -- has the lock and the creation can proceed.
end if;
end;

Now this is where my lack of PL/SQL experience really shows itself, I feel that there are a number of things that can be improved but as it stands it works.

The trigger checks if the object being created is one of the types that we are expecting to be locked before creation. Then the machine name is extracted from the current session (the OS user could be used instead). The trigger checks that the object is an existing object; if it is further tests are done to check that the machine creating the object has a lock on the existing object. In the case that the object is locked by the machine issuing the create or this is the first time the object is being created the trigger allows the create to proceed. Otherwise an exception is raised indicating either that the object must be locked or that it is already locked from elsewhere. Now all that is left is to enable the developer to easily lock and unlock database objects.

The following procedure can be called by the developer to lock the intended database object.

create or replace procedure DBSCC_LOCK(
p_objtype in DBSCC_LOCKS.OBJTYPE%TYPE,
p_objname in DBSCC_LOCKS.OBJNAME%TYPE)
is
vLocks integer;
vCanLock integer;
vLockedBy DBSCC_LOCKS.LOCKEDBY%TYPE;
vMachine v$session.MACHINE%TYPE;
vObjType DBSCC_LOCKS.OBJTYPE%TYPE;
vObjName DBSCC_LOCKS.OBJNAME%TYPE;
begin
vObjType := upper(trim(p_objtype));
vObjName := upper(trim(p_objname));

-- Validate object type
if not vObjType in ('PROCEDURE', 'FUNCTION', 'PACKAGE') then
    raise_application_error(-20001, 'Only objects of type PROCEDURE, FUNCTION and PACKAGE can be locked');
end if;

-- Validate object that the object exists
  select count(*) into vCanLock
    from ALL_OBJECTS o
   where o.object_type = vObjType
     and o.object_name = vObjName;

if vCanLock = 0 then
    raise_application_error(-20001, vObjType || '.' || vObjName || ' does not exist');
end if;

-- Assume the object is already locked
vLocks := 1;

-- Query the locks table to determine if and who
-- has the object locked.
begin
    select scc.LOCKEDBY into vlockedBy
      from DBSCC_LOCKS scc
     where scc.OBJTYPE = vObjType
       and scc.OBJNAME = vObjName;
    exception
      -- NO_DATA_FOUND implies that the object is not locked
      when NO_DATA_FOUND then
      vLocks := 0;
  end;

-- If the object is locked inform the caller that the object is
-- already locked
if (vLocks > 0) then
    raise_application_error(-20001, vObjType || '.' || vObjName || ' is locked by ' || vLockedBy);
else
    -- Get the name of the workstation requesting the lock
    select s.MACHINE into vMachine
      from v$session s
     where audsid = userenv('sessionid');

    -- Special case PACKAGE objects so that both the
    -- PACKAGE and PACKAGE BODY are locked
    if vObjType = 'PACKAGE' then
      insert into DBSCC_LOCKS
        (OBJTYPE, OBJNAME, LOCKEDBY)
      values
        ('PACKAGE', vObjName, vMachine);

      insert into DBSCC_LOCKS
        (OBJTYPE, OBJNAME, LOCKEDBY)
      values
        ('PACKAGE BODY', vObjName, vMachine);
    else
      insert into DBSCC_LOCKS
        (OBJTYPE, OBJNAME, LOCKEDBY)
      values
        (vObjType, vObjName, vMachine);
    end if;
commit;
end if;
end DBSCC_LOCK;

The procedure mostly just checks that the object is not already locked and if not makes an entry into the DBSCC_LOCKS table. One special case is for packages, if a package is locked an entry is made for both the package and the package body. The reciprocal of this procedure is the DBSCC_UNLOCK, this procedure is used to unlock a database object.

create or replace procedure DBSCC_UNLOCK(
p_objtype in DBSCC_LOCKS.OBJTYPE%TYPE,
p_objname in DBSCC_LOCKS.OBJNAME%TYPE,
p_force in integer := 0)
is
vMachine v$session.MACHINE%TYPE;
vObjType DBSCC_LOCKS.OBJTYPE%TYPE;
vObjName DBSCC_LOCKS.OBJNAME%TYPE;
vLocks integer;
begin
vObjType := upper(trim(p_objtype));
vObjName := upper(trim(p_objname));

-- Validate object type
if not vObjType in ('PROCEDURE', 'FUNCTION', 'PACKAGE') then
    raise_application_error(-20001, 'Only objects of type PROCEDURE, FUNCTION and PACKAGE are supported');
end if;

-- Get the name of the workstation executing the request
select s.machine into vMachine
    from v$session s
   where audsid = userenv('sessionid');

-- Determine if the object is locked and if it is locked
-- by the caller, if p_force is non-zero then the machine
-- is ignored
select count(*) into vLocks
    from DBSCC_LOCKS l
   where l.OBJTYPE = vObjType
     and l.OBJNAME = vObjName
     and (l.LOCKEDBY = vMachine or p_force <> 0);

-- If locked then remove the entry from the DBSCC_LOCKS
-- table. PACKAGE and PACKAGE BODY are special cased
if vLocks > 0 then
    if vObjType = 'PACKAGE' then
      delete from DBSCC_LOCKS
       where OBJTYPE in ('PACKAGE', 'PACKAGE BODY')
         and OBJNAME = vObjName;
    else
      delete from DBSCC_LOCKS
       where OBJTYPE = vObjType
         and OBJNAME = vObjName;
    end if;
    commit;
else
    raise_application_error(-20001, 'You do not have ' || vObjType || '.' || vObjName ||' locked');
end if;
end DBSCC_UNLOCK;

DBSCC_UNLOCK does pretty much the reverse, it checks that the machine issuing the unlock in fact has the object locked and if so removes the entry from the DBSCC_LOCKS table. Again there is a special case for packages, where unlocking a package removes both the entry for the PACKAGE and PACKAGE BODY. Additionally the DBSCC_UNLOCK procedure includes a force parameter which when set will force an object to be unlocked even if it is not currently locked by the user.

So a few questions, if I keep saying the OS User can be used rather than the machine name why did I opt to use the machine name. Well, this might just be ignorance, but I recall reading somewhere that OS User might not be reliable from all environments especially when connecting via SQL*NET. Since this solution has been released to the team they have provided some nice enhancements. One of those was integrating this with PL/SQL Developer, the environment predominantly used by the team members, now they can just right click on an object in the object browser and lock/unlock, in the case of the object being locked the context menu even shows who has it locked, they have also changed the procedures to use the OS User rather than the machine name, maybe I should make that change some time soon.

Hope fully those of you out there that have more Oracle expertise than I do will point out the short comings in my implementation as well as maybe a better or alternative solution to the problem.