Galin Iliev's blog

Software Architecture & Development

Databases tips

I have been dealing with MS SQL Server 2005 and Microsoft SQL Server 2005 Analysis Services as well as Reporting services to display results to end users.

It was amazing how many issues are in the fields of databases. I developed projects on my dev box with snapshot of database (biggest table contains aprox 200,000 rows) and I though it was done. Unfortunately I was not able to deploy in production because of following reasons:

  • cube and dimensions processing was extremely slow - this was solved with playing with indecies
  • Memory error: The operation cannot be completed because the memory quota estimate (1678MB) exceeds the available system memory  - this was on 4xCPU and 4GB RAM on 32bit Windows 2003 std. This was solved with limiting SQL Server serive memory to 2GB and set /3GB startup switch in windows.
  • then came:  Arithmetic overflow error converting expression to data type int. This was tough also because I had to wait 2:30 hours before I get the error. then I make changes and try again... this was very, very long loop. For fully understand it check SQL Server Books and as resolution check Barnabas Kendall's blog entry.

there are  some good points in Barnabas Kendall's blog entry like:

I don’t enjoy reading documentation (lack of plot and character development),   :):)

  • Create the view with the SCHEMABINDING option.
  • Define the view to pre-aggregate the data (this is where the performance enhancement comes from). For example, I have a view that splits up the year, month, day, and hour of a click, groups by client, and also returns a total. I can also reuse this view to get totals by hour, day, month, year, or all time.
  • Don’t use COUNT, use COUNT_BIG.
  • Don’t use AVG; use SUM and COUNT_BIG. Beware of integer overflows in your final query; SUM always wants to make an INT. Why isn’t there a SUM_BIG? You can get around this by using CAST or CONVERT in your final query.
  • Make a clustered unique index on the view. This forces the database engine to persist the values to disk rather than calculating them on the fly, which is what leads to five-minute queries. I am surprised to learn that you don’t need to put all the values of the table in the index, just the ones necessary to make a clustered unique index.
  • When querying the view, use the NOEXPAND hint.

I would like to add: All aggregate functions return result depending on input parameter's type- if input column is of type int and result would be if type int:

select sum(numbers) from dbo.TestTable

And if you use SUM of multiple rows all that contains values like 2,000,000,000 you will end up with Arithmetic overflow error

In order to change result type this:

select sum(cast(numbers as bigint)) from dbo.TestTable

I employed some of these techniques and started the process again. I am keeping my fingers crossed...

VB 9 Relaxed Delegates

I am reading about new features in Visual Studio 9.0 (Orcas) (while I am playing with it :) ) and I found there is a new feature in Visual Basic called Relaxed Delegates.

What this means!? For instance you can convert methods to delegate even when methods doesn;t have all parameters (if not used)

This is the sample from Amada Silver:

Another great feature that you’ll notice in Beta1 is Relaxed Delegates. In short, relaxed delegates are a way to extend VB’s implicit conversions to delegate types. With relaxed delegates, you can write the following code:

Private Sub Button1_Click(ByVal sender As Object, ByVal e As EventArgs) _

  Handles Button1.Click, Button1.MouseClick

    MsgBox("Do Something")

End Sub

You can even omit *all* of the event arguments if your method body doesn’t need them. This improves readability without compromising type safety:

Option Strict On

Public Class Form1

  Private Sub Button1_Click() Handles Button1.Click, Button1.MouseClick

    MsgBox("Do Something")

  End Sub

End Class

A bit unfair to C# but still - this is Visual Basic :)

Use ASP.NET to authenticate users in Winform app

ASP.NET Authentication is very nice and could save a lot of time. Yes, although it covers many cases according Murphy laws you'll need that one that is not covered :). Fortunately there is a way to extend ASP.NET Authentication in order to include all you need. Or even build entirely custom autentication module that works fine with ASP.NET login controls.

But when you want to use single authentication for Windows form and Web applications and decide to use built in ASP.NET Authentication then you will have to write a lot code.

In VS 9.0 (codename Orcas) there is a easy way to use ASP.NET Authentication in Windows Forms application. See sceenshot below:

The screenshot is from Visual Studio codename Orcas beta 1
More will be covered soon.

C# 3.0 and Orcas chats

There are several upcoming chats regarding C# 3.0, LINQ and Orcas that could be very interesting

April 23, 2007
11:00 - 12:00 P.M. Pacific Time
Additional Time Zones

C# IDE Chat
C# IDE Chat: Code snippets, enhanced IntelliSense, type colorization, refactoring, improved code navigation, metadata as source, Edit and Continue! There are so many great new C# IDE features in Visual Studio 2005 - there's a lot to talk about! Or perhaps you have a question about Visual Studio 2003 or what we're planning for the next version? It's your choice! Join the C# IDE team for a chat completely directed by your questions.

Add to Calendar

May 7, 2007
11:00 - 12:00 P.M. Pacific Time
Additional Time Zones

C# Compiler and Language Chat
 C# Language Chat. Want to know more about anonymous delegates in C# 2.0, or our design rationale for them? Have some questions about our current thinking on C# 3.0 and LINQ? Perhaps you'd like to share your ideas with other C# users or the C# language team. Join the C# team for all that and more!
Add to Calendar

May 21, 2007
11:00 - 12:00 P.M. Pacific Time
Additional Time Zones

C# LINQ to SQL Chat
The LINQ Project is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. LINQ to SQL is a component of LINQ which allows access to data stored in relational databases. We’re actively designing and developing this technology - show up and join the LINQ to SQL team to participate in that process or just get a better understanding!

Add to Calendar

For full list of upcoming chats see MSDN Online Chats

LINQ Tutorial - Expression trees

Is has been a long time since my last update on LINQ tutorial. Last two months has been busier than I expected.

This week I found some time I could announce with pride that next part is done and published -LINQ Tutorial - Expression Trees. As there are many resources for English speaking devs this article target Bulgarian speaking audience.

for those who are interested in LINQ and C# 3.0 in English - watch the blog. I am posting interesting links when I find them :)

As always: your feedback is highly appreciated.

LINQ but in ... JavaScript

Probably you've heard about C# 3.0 and LINQ features but can you imagine this can be done in Javascript too?!

Well this is possible - see example here.
It seems this have very good potential in current Web 2.0 wave.

Here is some info from their home page:

» What is jsc?

jsc is a decompiler or a cross compiler if you will. It is not a source-code parser nor an IDE. It is a command line utility like any other compiler. jsc was originally an anagram for csharp to javascript.

» How does it work?

The compiler extracts CIL from a .net assembly (latest javascript screencast, old screencast). It filters out the classes which are marked with the ScriptAttribute. It selects the target language and emits the source.

Each assembly defines which namespaces will be filtered in. Each class in these namespaces mustScriptAttribute, to be opt-in for jsc compilation. Therefore, previously built assamblies will not define be processed.

» What is the current status?
Currently, the jsc project is non-commercial, unsupported and experimental , but free to use :) If any of you manage to create something others would like to see with this, be it java games or ajax apps, post a comment on the blog.
» Who would like to use it?

  • php, javascript developers
  • java developers
  • developers
  • you!

» What are the limitations? (Will be adressed when the Orcas reaches beta)
  • operator ?? is not supported.
  • lock is not supported.
  • yield is not supported.
  • switch is not supported.
  • goto is not supported.
  • return within try catch finally is not supported.
  • try handler filter is not supported.
  • ref out is not supported.

Interested?! See Build AJAX and JavaME apps in the c# 3.0 language @ Sourceforge

Visual Studio "Orcas" and .NET FX 3.5 Beta1 shipped!

Today MS  signed off on the Beta 1 release for Visual Studio “Orcas” and .NET FX 3.5. 

Go and check here.

Visual Studio Code Name “Orcas” Beta 1

Visual Studio code name "Orcas" is the next generation development tool for Windows Vista, the 2007 Office system, and the Web. Beta 1 consists of multiple releases including, Visual Studio Professional Edition, Visual Studio Team Suite and Visual Studio Team Foundation Server, which are available as installation media ISO images you can use to install the products. Alternatively, you can download VPC images with the software pre-installed. In addition, you can download prerelease versions of Visual Basic Express, Visual C++ Express, Visual C# Express, and Visual Web Developer.

For a better download experience, MSDN Subscribers should use MSDN Subscriber Downloads for both installation media ISO images and VPC images.


This news came from Soma

Architect or developer?

I recently came across good article The role of the Software Architect by Fabrice Marguerie and I have some comments I would like to share. There was an paragraph that says ( about comparing architects with developers ):
Michael Platt wrote: "Architects are a lot slower in getting a solution, especially if the problem is simple!". Probably we could add that "Architects are much likely to come up with complex and costly solutions, especially if the problem is simple!".

I am not quite sure this is true. Building software solution is not much different than building construction (or building a car) - the main principles are same:
  1. understand business needs
  2. create it durable (and usable)
  3. give time and price estimations (for implementation and use)
  4. try to understand future changes and integrations (or easy repairable)
I wont discuss how this is done but let's say it is quite difficult to let this out from the stakeholders (point 1 and 4).

Getting above things done is much different in software for number of reasons but mainly (according my opinion)
because software industry is much younger and technologies are changing ( and constantly lowering development cost and time).

But let's back to paragraph above: Why Architects should create much complex solutions!? Based on their experience they would try to guess future needs and cover them. This is the only explanation I have... But ( in opinion again ) I think customer/stakeholders should known main decisions taken regarding solution built for them. In this case they will be told all main assumptions and constraints. Knowing these ( as well as how important it is represented by time and money :) ) they will closely involved in defining goals and scope.

I agree that architects, based on their experience, would notice much more details and would have much more questions to stakeholders. But this doesn't make solution more complex or expensive. This makes goals much clearer and allows setting good solution architecture as well as avoids changes in later stages.

Indeed, an architect can have to write lines of code (at least to create prototypes or mock-ups), but also has to get his hands dirty with code to validate the quality of what is produced or resolve a particularly technically difficult situation.

About this paragraph I absolutely agree :) Definitely the architects should be able understand and influence what 's done by developers and even could be able to participate in first stages of development.  But  not too close because  pure developers won't have the chance to take design decisions on certain levels of solution and would be obstacle to for them in gaining valuable experience.

Thus make me think about my career :) Since very first days I was having to deal with clients, to get their needs and build appropriate solutions. According my knowledge in Bulgaria most of senior devs fulfill these activities. I am calling them senior devs because we are involved closely in development, team lead, solution design and, of course, matching clients' needs (and managing their expectations)

So what are we - Architects or Developers?! By being the person who gets the project, and make it happens, are we developers because we are highly involved in implementation/code phase or we are architects because we do much more things beside implementation?

What is your opinion on this?

I highly recommend reading the article - The role of the Software Architect by Fabrice Marguerie

Red bits and green bits in Orcas

Assemblies of next version of .NET Framework are divided in two groups - "green bits" and "red bits" as they are called inside Microsoft.

The red bits include all the libraries that shipped before as part of the .NET Framework 2.0 and 3.0 (such as mscorlib.dll and system.dll). To maintain a high assurance of backward compatibility for Visual Studio "Orcas," changes in the red bits have been greatly limited.

The green bits assemblies are the brand new libraries with additional classes that work on top of the red bits assemblies. Most of the classes listed in this column are in the green bits assemblies (such as system.core.dll), with a few involving limited changes in the red bits assemblies.

The CLR's contributions to the new libraries include:

  • A new add-in hosting model, which was discussed in the last two editions of CLR Inside Out
  • Support for the Suite B set of cryptographic algorithms, as specified by the National Security Agency (NSA)
  • Support for big integers
  • A high-performance set collection
  • Support for anonymous and named pipes
  • Improved time zone support
  • Lightweight reader/writer lock classes
  • Better integration with Event Tracing for Windows® (ETW), including ETW provider and ETW trace listener APIs

There are also new crypto classes that covers Suite B set of cryptographic algorithms.

for more info read New Library Classes in "Orcas"