Development
Development
Here's a question that's bothered me for a long time. I remember hearing from marketing types and even Microsoft partners early in the SQL 2000 product cycle that the product is self tuning. I distinctly remember a situation thrown around with DBA types saying that they would delete all the indexes and run the server for 24 hours and the server's processor would be pegged at 100% utilization for this time-period. SQL Server would then miraculously self-tune itself and processor utilization would drop to 25%. Obviously these numbers are app specific, but I've never seen this 'self-tuning' aspect.
Now I guess if you count running a SQL Profiler Trace and then run the index tuning wizard against the result, then you could say the database is capable of tuning itself. But 'self-tuning' is a pretty loaded phrase. After all, when did the DBA become considered part of the database? I'd love to see an example of how to get SQL Server 2000 to be self-tuning, but I can't find this anywhere outside the marketing docs.
Here's a
link to the article on my new blog site.
I'm moving my blog site to: http://www.codebetter.com/blogs/steve.hebert.
I'll be dual-posting for a period of time here at DNJ. Thanks to Donny Mack and the group here at DNJ for the opportunity to blog here. I've enjoyed it and learned a great deal in the process.
The Intersoft WebGrid.NET product does not persist the AggregateFunction setting on the column between postbacks. Therefore, the first time you configure the grid it will work, but after subsequent postbacks it will no longer work.
Given a form where “complete_qty“ is being summed, the following code should be placed in the “InitializeLayout” event.
e.Layout.ColumnFooters = ColumnVisibility.Yes
Dim oCol As WebGridColumn
For Each oCol In sGrid.RootTable.Columns
If oCol.DataMember = "complete_qty" Then
oCol.AggregateFunction = AggregateFunctions.Sum
End If
Next
We've been using
elementool for bug tracking at work for a few years and we're pretty happy with it. Run entirely remotely, the product works well for our environment with implementers routinely on the road. We started by using their free service and upgraded given the number of users we have on it.
Is anyone else running bug tracking through an ASP?
I'm moving the .Math library from the GotDotNet servers to SourceForge. The server responsiveness and now repeated errors in getting to the old site has prompted the change.
I'm still happy with my decision to host the documentation on my own site and that will remain the same.
I just went digging for IIS compression details and found the following article. The article makes mention of the Port80 Software Blog... subscribed.
David Bank, the author of “The Revolt of the Corporate Consumer” that I blogged about here, sent me the following URL to his article made freely available by the Wall Street Journal.
David took the time to talk to the people controlling the content servers and they responded by making it available to non-subscribers as well. Thank you to David and my hat goes off to the people at the Wall Street Journal! Wow!
(Authors Note: I'd like to thank David Bank and the people at the Wall Street Journal for sending me the URL to a version of his article that's available to non-WSJ-subscribers. I've modified the link accordingly.)
“The Revolt of the Corporate Consumer” article by David Bank appears in today’s Wall Street Journal. Subtitled “How companies are squeezing tech suppliers to get a bigger bang for their software bucks”, the article does a nice job of reviewing the combining factors today that are changing the software market landscape. Mr. Bank hits on a number of factors affecting the software community, but I think it’s interesting to look at these in light of how the development community is responding.
Demanding Standards
I thought this entry was interesting in Bank’s article, but he blends this portion of the article with the fact that “customers are using their new clout to force vendors to deliver software that is more secure and reliable.” In the production scheduling area I’m involved in, we do not see strong push toward using standardized integration, however we are taking a lead role in providing these types of integrations – I believe this is only a matter of time as the market matures.
I think Bank’s comments on security and reliability really connect with the basic ideas behind recent approaches in the development community. For example, test-driven development is an approach aimed at providing a more stable and confident release pattern using a repeatable process with a full complement of tools. While TDD provides a number of other benefits on the development front, when viewed from the customers’ perspective it simply makes good business sense.
The security front is a different story right now, with this being a hot-topic across the industry. We are beginning to see security “sniffer” tools being released from companies like Compuware to introduce a regimented approach to the problem. Security is a different ballgame as developers are constantly playing catch-up to the “black-hat” hacker community. However the application space faces a slightly different set of intrusion priorities than an OS vendor faces. I think it’ll be interesting to see how tools will adapt to each concern.
Cutting Maintenance Costs
Banks discusses the fact that customers are dissatisfied with both cost and complexity of software. Both factors impact maintenance costs with annual maintenance contracts typically priced between 15% and 25% of the initial software license and the setup and maintenance burden imposed by installation. Customers are pushing back against large up-front license fees and software companies are developing models where customers pay for what they need.
On one front, development approaches such as “service oriented architecture” and standards surrounding XML Web Services are giving solution providers the ability to develop standards based integration and the opportunity to host data storage and configuration offsite even when relatively tight integrations with existing systems are needed. While reducing the clients maintenance burden, this separation also allows for usage monitoring and billing opportunities that are far more difficult in a remote-installed application.The “soa” mantra also forces (when used properly) a simpler integration pattern. By forcing the application provider to express integration points in terms of simple, knowable and describable transactions happening largely within one call forces development and corresponding documentation to focus on simplicity. As with any tool, the goal can be missed with a poor implementation, but SOA does not promote (in terms of operation, scalability, bandwidth, etc.) the types of integrations used with in-process/stateful objects typical of earlier systems.
Sharing the burden
In this section, Bank discusses the demand for “shared responsibility” that he characterizes for demands that include easier patch management, more secure-code and fewer bugs. While Banks already discussed secure code and fewer bugs, Bank infers the point that customers are demanding a more responsive development environment. I believe the biggest response to this point has been the daily build cycle and a correspondingly active QA process. Maintaining a constantly released piece of code forces the developers to focus not only on features, but on getting the code installable from day one. This idea has been around the industry for many years – I was first introduced to the concept in Jim McCarthy’s book nearly 10 years ago, but this approach is getting far more traction today than it has in recent years and build automation tools are making there way into development tool offerings (such as Microsoft Build).
Deep Discounts
Bank discusses the fact that in the Oracle/PeopleSoft battle, court documents revealed that both companies were routinely discounting their software prices 70-90%. While he discusses the significance of open-source software in this role, I didn't feel that he made a solid case for it as having a casual relationship to the “Deep Discount” issue. I think the more interesting angle is in the ERP market itself – when is someone going to pull a “Microsoft” on this market. What I mean by this is offer the product at a 10th the listed price and even make the product available to support people in non-operational situations virtually free. For example, give the IT people who need to tie into the systems access to the software before the purchase and the corresponding tools and you provide another door into companies that sale reps cannot create or reach. It sends a message to companies that you are confident in your product offering and willing to work hand-in-hand with your staff to be successful. I wonder if Microsoft will see the light, or simply try to play ball with everyone else in the ERP market.
Demanding Results
I find this section to be the most compelling and dead-on. Software companies must focus on getting benefits and results to the customer. This may go beyond the realm of the software itself and software companies need to realize that the software installation is merely a starting point in proving worth. To be completely fair, I believe there are many companies purchasing software that cast a weary eye and rightly so. The “maintenance contract” is usually viewed as something that benefits the vendor instead of the customer. Battling that perception while providing a hands-on approach to customers is not a simple battle, but one that will be necessary for software vendors to succeed.
I like Bank’s quote of Mike Lawrie of Siebel Systems who said “It’s not just about making the best software, it’s also about helping companies get to that business outcome and get the value from the investment they’ve made.”
Summary
I enjoyed Bank’s article and it’s very worthwhile reading. Living in the software-space it’s unsettling when you look at the number of forces conspiring to take down the software gravy-train that’s been in place for a number of years. That said, even a fool knows it had to end sometime. The important thing to remember is that changes and improvements to software methodology need to be viewed with a constant eye toward the customer. Each developer needs to make their own processes better not only to make the software and it’s corresponding processes better but for the sake of staying in business and growing market share.
I'm putting together a sql update script execution application and one concern is that someone could grab the script during execution by having a sql trace running. After checking out the sql server documentation, I came across the following select statement:
select * from :: fn_trace_getinfo(default)
If one or more rows are returned, at least one trace is being run on the system. Cool stuff!
This has been the one caveat to securing the process. I'm using the SharpZipLib to compress the script before using TripleDESServiceProvider class to provide encryption. As a test, I reversed the process to see how much repetition would appear in an uncompressed script. Here are my numbers:
Script File Size = 3.9MB
Encryped File Size = 3.8MB
Encrypted then Zipped File Size = ~<3.8MB
Zipped File Size = 370000 bytes
Zipped then Encrypted File Size = 369000 bytes
I found it interesting that the encryption creates a random enough pattern to the data that compression doesn't work well.
I've read that compressing a file before encrypting is supposed to be more secure because it makes cracking the encryption. I imagine this could be true since it introduces another step that must be discovered in the process of hacking the file, however I wonder if it would be easier to hack since you'd know what the zip header has to look like.
It's been over a week since I last blogged, I've been heads-down in a piece of code. Using the Intersoft WebGrid.NET, I needed the ability to make context-sensitive menus available based on the column that has been right-clicked and pass along configured values. The implementation supports default menu grouping that's integrated with the security system, but the solution also allows for overrides for custom right-clicks on a per-page basis. I'm pretty happy with the implementation, I did the same thing with the built-in datagrid for asp.net but this implementation is cleaner and I like the code a lot more. The solution is entirely client-side and performs very well. Writing this much javascript reminds me just how much I dislike that “programming language“, but that's a topic for another day. I could have built the client-side javascript using XSL, but those solutions tend to be unreadable/undebuggable and this situation would have been even worse.
I've blogged before about Intersoft's WebGrid.NET before, and I continue to be very happy with their product. The server- and client-side interfaces are well thought out and extremely functional. As is typical with a 3rd party tool, there is a learning curve to get the feel for the product's architecture and Intersoft's product is no different. I've built scaffolding around the product to handle the 95%+ usage scenarios we see in the company's application to (1) minimize coding and (2) virtually eliminate grid-specific code on each page.
Next I'm saving the grid's layout to the database per user - allowing the user to resize, reorder, hide and show columns and then save the configuration across sessions. But first, I'm taking the weekend off to enjoy Christmas with the family.
Have a Merry Christmas!
Our automated build process is complete (I've been blogging about the process over the past several weeks). I thought I'd take a minute to blog a few of the bits I learned along the way. For a short history, I started the build process with NAnt to create our database script build and came across the Visual Build Professional tool in time to complete the MSI creation. Now for the tidbits I learned:
- NAnt is a truely flexible and powerful piece of software.
- NAnt's learning curve is non-trivial, and the further you dig the bigger it gets.
- NAnt's documentation set is somewhat disjointed. The associated wiki is marginally helpful and most useful information comes from googling the user community.
- Visual Build allowed me to create the entire build of the MSI project in less than 8 hours. This included time just learning how to navigate their product to putting it into production.
- Visual Build does not allow running a C# or VB.NET script as can be done in NAnt. It only allows for classic VB script, Javascript or Perl Script.
- I ended up leaving my database build script running under NAnt due to item #5 and NAnt is free after all. I just call my NAnt script from Visual Build.
So there it is, there are some more distribution features I'd like to incorporate but these features are not specific to the build automation tool.
-Steve
I've been blogging about using NAnt in my spare time to get software builds fully automated at work. I ran into a product today that changed my approach.
Today I ran into Mike Gunderloy's posting that Visual Build Professional 5.6 is available and downloaded it. The package comes with 50-free uses as a trial period. I'm hooked. I have done more in 2 hours with Visual Build than I've been able to accomplish in 15 hours using NAnt. In the 2 hour time frame, I created a build that (1) downloads my source tree, (2) updates my project and installation build numbers (checkout/update/checkin), (3) builds the projects and (4) creates an installer package. Now for some minor cleanup to put this into production.
I'll pull my database script builder into Visual Build so I can have this entire process under one 'roof'.
The product is impressive so far and upon first glance it appears that the generated build file is very similar to NAnt's. I'll have to do more digging on this front.
How do I summarize the Visual Build tool so far? I think Visual Build is to NAnt as (pre-.net) InstallBuilder is to InstallShield. The more I dig into the product, the more flexible I find it to be and well thought out, too. I'll blog more as I go, but for now I've done a complete turn on my approach to the problem.
I listened to Spolsky's talk on "community" style interfaces at SDMagazine where he discussed products like IM, usenet, etc.. He had some interesting usenet history and how the interface has been carried forward (unfortunately) in other applications.
I enjoyed the talk and submitted one question that I wish he had addressed. It seemed to me that all of his recommendations revolved around the basic coding idea of “get out of the way” - or - essentially that programmatically trying to format communications to improve readability as a rule does just the opposite. I think that's a good message - provide the tools that allow the user to express their ideas/thoughts in their own manner (within reason) and publish that content in a sequential, ordered manner. I'm not sure if that's an oversimplification, but the ramfications of making process assumptions and rigidity with this type of software can quickly marginalize the content value. I've always believed that the NewsGroup/NNTP world is largely unusuable - it's valuable but I believe the publishing format leads to horribly disjointed conversations and a lot of poorly answered questions.
As I'm working on my NAnt implementation, I came across Eric Desch's article published on the 4 Guys From Rolla site titled “A Brief Introduction to NAnt”. He does a very nice job of providing an introduction to NAnt along with a rundown on how to get the software running. He discusses NAnt, NAntContrib and MSBuild and provides links to some applicable resources.
I wish I had this intro when I started with NAnt, it would have eliminated some initial wheel-spinning. If you're considering NAnt or looking for a starting point, this article is a nice resource.
I was reading Steve Eichert's blog entry titled “Way to go Sun” where sun is aggressively taking down websites that are supporting the java community.
It seems that some companies hit a phase where they have more lawyers than engineers and they aggressively try to protect old revenue streams or product uniqueness within the market. Companies need to protect their products, but certain companies seem to start going over the top. Take Apple in the late 80's in their scorched-earth “look-and-feel” compaign to protect the mac. The industry might look very different today had they put that money into developing their platform instead of lining the pockets of lawyers.
I worked with an embedded device communication protocol in '98 and '99 and the company that established the protocol had to have a 100:1 L/E ratio. They published a management protocol and charged everybody $50,000 for the toolkit and tried to charge additional licensing fees for each device you talked to. Nothing worked - their base tool, the protocol, everything. We wrote an open protocol based on COM and put it through an independent standards organization. Needless to say, they tried suing everybody and their uncle. I was contracting at the time and even the CEO of that company asked who I was. It definitely helps having an obscurely pronounced last name because they kept asking my name and never made contact when the lawsuits got handed out. All suits were summarily dismissed by a judge who told them “I don't want to see these kind of lawsuits out of you ever again”. The company was a “bubble” company in the market and their shares trade for far less than a 10th of what they traded for in '99. I don't mention the company name because I wouldn't be surprised if they would throw lawsuits out as a reflexive action.
If an L/E ratio existed, you could get an idea when a company is agressively innovating or when their sacred revenue streams are dying on the vine and they are consumed by fighting to protect them. I doubt the ratio would be possible to enforce (just outsource the lawyers). But that wouldn't necessarily be a bad thing - maybe it would drive the lawyer employment market out of the country!
I just got an
invite to Joel Spolsky's talk on usability and social interface design. I'm looking forward to this one.
I was reading Steve Eichert's blog entry asking “if you had it to do all over again would you go into computers?“ Then I started thinking, if you won the lottery what kind of programming would you do?
I'd probably make an offer to John Deere to do work like this for free out of my house. They take real-time yield information from a combine and drive the equipment accordingly. Then they can take this information the following year and feed it into the planting equipment to vary seed density and fertizilizer to maximize yield over the field according to soil quality (and other factors). That is so freakin' cool!
Added thought: Add a little GPS drive system and farmers could be using this interface for their field work.
Otherwise I'd do open-source software and miscellaneous projects for businesses that I find compelling and try to turn them into good business opportunities in my spare time.
I've picked up a couple books on Analysis Services including the 'MDX Solutions' book by George Spofford. While I'm happy with these books, I'm really looking for something different, but I'm not sure how to quantify it.
Back in '92, I wrote an “Analysis Services” style application that used dimension tables, a fact table, a cube abstraction and even query caching. Of course, back then the terms like “fact table“ and “dimension table“ didn't exist but the functionality was identical. The creation of these systems is methodical and interesting when you consider the types of data situations you can run into. Bottom line, it's a relational database problem and I've always felt that OLAP was a clever way of saying you're “denormalizing“ the database without digging up a religious war with the “n-th normal form zealots.“
The problem with todays set of books is that authors tend to get caught up in a methodical rundown of OLAP theory in presenting their how-to approach. OLAP theory is comprehensive and necessary - it's there for a reason. But I've got to think there is a book that cuts through the theory and develops a core understanding of using the tools to build a solution from a conventional dbms perspective and THEN role into the theory bits. For example don't ramble on talking about Star Schemas - show the reader an implementation, why it works, why it's necessary and then reveal that the user just learned the Star Schema bit. Expose some details to tie the lesson together and move on.
We used to refer to great books as being “content-dense”, where you can sit down and read a chapter and feel like the author has revealed great truths about a language/os/design/etc.. I just don't get that “I'm getting smarter by the minute feeling” with these books. The OLAP books I've seen tend to be content-sparse - one or two facts surrounded by a chapter of discussion. Yawn.
That said, I am happy with Spoffords book. His book focuses on MDX which is a very related but different problem set. I'll continue looking for a more general book and blog it if I find it...
Microsoft now has the “
Using CLR Integration in SQL Server 2005” whitepaper on their site. This is a good read and discusses the operational differences between T-SQL and managed code in the SQL Server environment