Passing on a link from The Daily WTF. This one was too good not to share.
Ever since I started reading about .net in earnest, I've been fascinated by the new programming paradigm created by reflection and metadata. But, it wasn't until today that I really got to put some of those high-level concepts to work
We're writing a central configuration management system for all the applications here at Group One. Ideally, the COMET (we love acronyms that spell stuff) should be able to store and retrieve any object we want to serialize and deserialize.
We looked at Microsoft's Application Block for configuration management but, after a week of banging my head on it, deemed it too complex to customize to our needs. If you want a centralized CM tool that writes to SQL Server or an XML file, I highly recommend it, though. It's pretty much drop in and go. We also tried using default serialization as provided by the SoapFormatter in the framework. But, the SoapFormatter has a horrible feature where it follows any active event handler back to its source and tries to serialize the source as well. In Windows applications, this usually means that, if you use events at all, you wind up trying to serialize one or more winforms along with your object and that is a mess.
So, we need to be able to serialize any object, but not use SoapFormatter's serialization. This includes a few hundred classes we already use common to all our applications, framework classes, UI classes, and any classes we may come up with in the future.
The obvious answer seemed to be an interface. So, I created one: IGroupOneSetting. To be able to be stored and retrieved in the COMET, a class just has to have a default constructor, implement IGroupOneSetting or be wrapped in a setting class that does.
Here's the interface:
public interface IGroupOneSetting : IComparable
{
XmlNode Serialize();
IGroupOneSetting Deserialize(XmlNode pattern);
string SettingName { get; }
object SettingValue { get; set; }
SettingScope Scope { get; set; }
}
So far, this is all pretty routine stuff that could have been done under COM. But, here's where the subtle beauty of .net comes into play. The COMET takes all of its settings and stores them together in a block of XML that resides in Active Directory. To regenerate all these serialized objects, all the comet has to do is loop through all the XmlNodes that are immediately under the root element and pass them to the following method:
public static IGroupOneSetting GetSetting(XmlNode settingNode)
{
string typeName = settingNode.Attributes["type"].Value;
string typeAssembly;
if(settingNode.Attributes["assembly"] == null)
{
typeAssembly = "GroupOne.Goat.Comet";
}
else
{
typeAssembly = settingNode.Attributes["assembly"].Value;
}
AppDomain thisDomain = AppDomain.CurrentDomain;
object theSettingObject = thisDomain.CreateInstanceAndUnwrap(typeAssembly, typeName);
IGroupOneSetting theSetting = (IGroupOneSetting)theSettingObject;
theSetting.Deserialize(settingNode);
return theSetting;
}
And that is something you never could have managed under COM.
In case you're farther out in the datasphere than I am, I'll mention that Somasegar announced in his blog today that there will be support for Edit & Continue in C# under Visual Studio 2005.
This should have been a no-brainer, but Microsoft nearly listened to the small, but vocal minority of curly-brace afficionados who cry foul whenever they see a feature being added that makes C# easier.
Does anyone know of any good .net-savvy source control system. Every tool I look at treats source code as text. How...primitive.
We just delivered the first iteration of the first application we tried to build using an agile-ish methodology. We delivered it functionally on time (read: one day late) and without the “I hate this project. We must be close to delivering it” phase.
We've only implemented about a third of the methods we put on the table as “things we plan to do” and it's already reaped huge benefits in the process. What we delivered wasn't 100% functional and was only somewhat pretty, but it ran and gave putative business value. It's what's referred to in Agile/Crystal Clear as a “walking skeleton.”
We've already got a metric boatload of ideas for what to add to it in the next cycle targeted at this particular application including serialization and configurability plus the drill-down tool that was at the bottom of our sequential list of features to add this cycle. But, more likely, we'll be focused on a different product.
There might be something to this whole “agile” business.
We're currently working in a multi-threaded environment and trying to come up with a good logging solution. Right now, we're running up against a problem in the following situation:
1. A message is added to the log which triggers ForceSave.
2. ForceSave launches a thread to save the logfile using FileSave.
3. The other threads are all higher priority than the thread trying to save the file.
4. The application raises an unhandled exception, forcing the developer to break in the IDE while the FileSave thread is still waiting for processor time.
5. The file doesn't get saved.
Does anyone know how to architect a solution to this? Preferrably, I'd rather not set the priority of the file saving thread higher than the main application because it will slow down performance when finally deployed (and I'm not even sure it will work.)
Here's the code:
Public Function ForceSave() As Boolean _
Implements ILogfile.ForceSave
'Don't let the slow disc save block incoming queue writes or slow down the
'application being logged.
Dim thWorker As New Thread(AddressOf FileSave)
thWorker.Priority = ThreadPriority.BelowNormal
thWorker.IsBackground = False
thWorker.Start()
'Don't wait for return value. FileSave will raise exception on failure.
Return True
End Function
Protected Sub FileSave()
'Lock all threads out of the queue while we copy it.
mWRLQueue.AcquireReaderLock(miQueueReadWriteTimeout)
'Copy the message queue to an array.
Dim amsg() As Object = mQueue.ToArray()
'Clear the queue
mQueue = New LogMessageQueue
mWRLQueue.ReleaseReaderLock()
'Saving the file is a critical section. Disc read/write must
'be done by exactly one thread at a time.
SyncLock FileSLMessage
'Open or create the logfile
Dim sr As StreamWriter = File.AppendText(msFileName)
'Write each message to the end of the logfile.
Dim thisObj As Object
Dim thisMsg As LogMessage
For Each thisObj In amsg
thisMsg = CType(thisObj, LogMessage)
'Only write messages that reach the required level
'of severity. This way, developers can change SeverityTolerance
'on later builds to get less verbose information.
If thisMsg.Severity >= miSeverityTolerance Then
sr.WriteLine(thisMsg.ToString())
End If
Next
'Save and close the file.
sr.Flush()
sr.Close()
End SyncLock
End Sub
TIA to anyone who might have a solution.
Every methodology I've ever researched for handling the development process has at least one concept that its detractors can hold up and say, “See? This is a crazy idea.” With Extreme Programming, it's pair coding. I'm starting to think that, with Crystal Clear, it's osmotic communication.
The concept behind osmotic communication is that you keep your development team in the same room and physically close. This way, questions can be addressed verbally and get answered with the fastest turnaround time possible. In addition, when a question gets asked, everyone hears it meaning, in theory, that people don't get asked the same questions over and over again.
Of course, there are periods during the actual code writing process where a developer needs to concentrate intently on what they're doing. I half-jokingly refer to this as “becoming one with the code.” The last thing you want to deal with at that point is someone asking a question, even if the information may be useful to you later.
Cockburn tries to address this with the idea of a “cone of silence”—a place developers can go to get away from osmotic communication. There are a few problems with this. The biggest one is that the need for intense concentration is not always predictable. Suddenly realizing that you need it, but needing to change environments to get it can very well kill exactly what you were trying to preserve. Besides that, developers on the same project generally get into deep concentration mode at the same time. If you don't have enough equipment in the designated cone of silence, it becomes a competition for limited resources. If you do, unless its partitioned, all you've done is moved the disruptive environment into the cone of silence. For most companies, individual offices for the developers to go into when they need to concentrate just isn't feasible.
In my current work environment, it's even trickier. We have four and two-half developers. The two “half developers” divide their time between writing code and being electronic traders with a significant weighting towards the latter. From 9:30 to 4:00, they're sitting in a room with eight or more computers to a person and two or more monitors to a computer. In order to manage the complexity, the computers all have audio cues tied to market events. Some of them are just a voice saying, “Buy Stock” or “Sell Puts.” But, there are also a few that say things like “What in blue blazes do you think you're doing,” “Can't we do something else for a change,” and “Yahoo!” Most of the time, these occur every few minutes, but during a volatile market, the room becomes quite cacophynous. The business knowledge being passed around is valuable to us as developers, but most of the sound is just business noise.
So, we've given up on osmotic communication, moving the full-time developers into a separate room where we can barely hear the electronic trading room. The critical word here is “barely.” On top of that, there are still three of us, which means that there are some times when two of us are working out a problem and the third really needs to concentrate.
As a result, we've resorted to the bane of osmotic communication—headphones. I suggested the company supply us all with iPods, but management buy-in is still low on that initiative. So, for the time being, when I need to concentrate, I slap on my Sony minidisc player and listen to the same disc over and over again until I just don't hear it anymore.
We made our first attempt to handle concurrent development under Perforce source control today. It coincided with my first day of writing a substantial amount of production code. Earlier this week was the one-month anniversary of my first day here. The day after that, I started writing code. The time before that has been taken up spending a lot of brainpower figuring out how to work as quickly as possible in the future.
This may sound like a paradox to some, but it really hasn't been. We've spent a month looking at development methodologies, tools, applications and policies meant to make the transformation of Group One from a 1.5 developer shop to one with four developers and an eye towards growing in the future. Adding new developers has coincided with the shop's support universe getting so large that it would soon be unmanageable.
Group One is an equity options market maker that trades both electronically and via floor traders on several different exchanges. Any trading applications we write have to work in a large number of different environments as a result. As a tiny shop, they did a phenomenal job at keeping everyone supported but it meant, among other things, that there were three different code bases for the main floor trading application and a dozen or so applications that were pulled out of the main market making application to be used by floor traders.
So, they decided to expand the team and, at the same time, implement a methodology for simplifying development, speeding up new feature releases, and minimizing duplicated effort. It's a big task and we're working towards a big answer centered around source control, agile methodology, and a unified architecture for existing and future applications.
A month into the process, I finally got around to writing some code under this system—a new data display module and bias generator for the electronic traders. At some point, we'll have the development architecture set up so that we can implement this sort of thing as a business library and Windows control. But, things are still too interdependent for that, so it has to go in the same project as the rest of the electronic trading application.
As I'm doing this, one of my coworkers is working on fixing a pre-existing feature under source control while another is (unbeknownst to me) working with a copy outside of source control to start unifying the business code into libraries and correct code drift. The second coworker can't work in source control because the first needs a compilable version for the changes he's making. There are ways to do this within source control, but our Perforce integration isn't complete yet, so we don't have the setup or expertise to configure it that way.
So, we got to spend half the day integrating the three versions of the electronic trading app into a single code base. It seems like any SCM system has a significant time cost at implementation that is, hopefully, recovered once everything is up and running. It looks like I owe Source Safe a small apology, but only a small one. It's still incredibly shitty software.
Now that I've mentioned incredibly shitty software and “paradox” in the same blog, I'll probably get a lot of false hits from people looking for Borland, but I need to exacerbate that a little by mentioning that our next big initiative once source control is tied down and mature will probably be installing a DBMS. Right now, I'm advocating we give Firebird a try Incidentally, when I give Borland a hard time, it's really Inprise I had a problem with—and it's really just one product: Paradox. And, strictly speaking, Paradox wasn't the problem, so much as PAL, the Paradox-specific “instead-of-SQL” language. It's amazing how one small aspect like that can make me dislike a company as much as I did Inprise. Fortunately, I've been promised that there's no PAL in Firebird. In fact, it's SQL-92 compliant.
With so many great DBMSs available for free now, I can't understand why anyone is still using MySQL.
I understand that a lot of people want GMail and, while I applaud people's efforts to spread the wealth, it's not doing much to improve the signal:noise ratio in the blogsphere, particularly with RSS starting to show signs of wear.
The people at isnoop.net have set up a service for GMail inviters looking for invitees and vice versa. If you have invitations to give, send them to gmail@isnoop.net. If you're looking to pick up a GMail address, you can pick one up here.
I know this is going to be interesting to, at a maximum, maybe one out of every one thousand developers out there. But, to those who are interested, I'm hoping to save you the twenty or so hours (cumulative) that I spent getting this to work.
There's a statistical function in Excel called NORMSDIST. It's used in calculating the values of options. It's obscure enough that I spent a lot of time figuring out how to duplicate it. Here's the function I wrote to do get an approximation accurate out to about eight decimal places:
Public Shared Function StandardNormalCumulativeDistribution(ByVal x As Double) As Double
Dim Z As Double = 1 / Sqrt(2.0 * System.Math.PI) * System.Math.Exp(-x ^ 2.0 / 2.0)
Dim p As Double = 0.2316419
Dim b1 As Double = 0.31938153
Dim b2 As Double = -0.356563782
Dim b3 As Double = 1.781477937
Dim b4 As Double = -1.821255978
Dim b5 As Double = 1.330274429
Dim t As Double = 1.0 / (1.0 + p * x)
Return 1.0 - Z * (b1 * t + b2 * t ^ 2.0 + b3 * t ^ 3.0 + b4 * t ^ 4.0 + b5 * t ^ 5.0)
End Function
Special thanks to the guys on the ADVANCED-DOTNET list and to DevelopMentor for hosting such an incredibly useful list.
A pretty clear indication that this job was not going to be like other jobs came on the very first day when I was given my first task. It was, "Download a copy of nUnit, install it, put it through its paces, and report back." Maybe this is typical for some people, but I was a consultant for ten years. In my experience, a typical first task is, "We have seventeen days before we go into the quiet period before our IPO. We need you to build a stock quote and index reporting system for our website using the Reuters feeds."
I mention that (very real) example because I'll probably get back to it when I talk about when it's the right choice to code the wrong way.
Anyway, one downside of this unusual request was that I was expected to vet a piece of unit testing software on day one--before I'd written any code that could be tested.
So, I went to the well of code I'd written for my own use. In this case, I pulled up the code from what I call the "fixed income project," a brainchild I came up with after writing CashFlow, FixedIncomeInstrument, and Coupon classes for the fourth time. It occured to me that a toolkit of pre-existing classes to represent these entities might have some market value.
I haven't finished the library. It turns out that there are a lot of different kinds of fixed-income instruments on the market. If there's a way to make money buying and selling debt, somebody's probably issued an instrument to do so.
Since fixed-income trading systems run on both Unix and Windows, I decided to write the functional classes in C++ and, if this .net thing took off, give them managed wrappers. Other paying projects had forced me to put the fixed-income project aside multiple times. Now, I dusted off one of its most basic classes: FinancialDate.
The FinancialDate class is pretty straightforward. Most fixed-income instruments (I would say "all," but I've learned to never use that word in finance) calculate their value, yield, and other critical information based on a day-by-day basis. If you buy a bond at 4 pm on Monday and want to determine its valuation relative to purchase price at 9 am Tuesday, you calculate the difference based on one day, not eleven hours. For most equations, the difference is tiny. But, it's significant enough, especially in situations where you're holding these instruments for short periods of time.
Datawise, the Fin