Friday, December 26, 2008

Thinking WCF towards Enterprise SOA

You want know how to build your own UDDI based SOA?

Read the UDDI SOA Howto.

Update: Get the article including source code at http://www.codeproject.com/KB/WCF/uddiservicefactory.aspx


Is WCF out of the box a Service Oriented Architecture?


If you think that WCF is SOA then lets review part of the definition of "Service Oriented Architecture".

OASIS (the Organization for the Advancement of Structured Information Standards) defines SOA as the following:

"A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations.

Now lets translate "capability" with "service".

So we get in slightly other words: "A SOA consists of distributed services, some sort of mechanism to offer and find them, a mechanism to interact with the service and finally a mechanism to invoke the service.

If you try to break down this into web service technology terms you now translate this into:

  • Services (this is what WCF is all about)
  • Find and Offer (this could be a UDDI Registry (Enterprise), or WS-Discovery (UPnP like))
  • Interact (this could be WSDL)
  • Invoke (this could be SOAP)

So what part of SOA is covered by WCF?
The service, the interaction and the service invocation. The runtime discovery is not part of WCF.

Conclusion:
WCF offers no proper mechanism to offer and find services at runtime and hence is a classical 3-tier architecture technology, that we basically have for 15-20 years now. We just called it DCOM or Corba or whatever before and now the same technology gets relabeled with the current buzzword "Service". Other examples I love are:
"SAAS" or "Software as a Service" what was called "ASP" or "Application Service Provider" before
"ESB" or "Enterprise Service Bus" what was called "EAI" or "Enterprise Application Integration" before
The thing about buzzwords is that you can throw them around with having no clue what they are really supposed to mean. So the words service in "Service Oriented Architecture" and "Web Service" are commonly misunderstood to be same where in fact they mean two different things.

Just a web service is not a SOA.

WCF follows Web Services

The WCF basically combines all the different communication technologies of .Net 2.0 (Remoting, Queues etc) under one roof and a common programming model. The model itself follows the web service model introduced with .Net 2.0. One of the biggest disadvantages of web services was that the communication layer for web services pretty much fixed to WS-I standard (BasicHttpBinding in WCF terms). What Microsoft now basically did for WCF was to address this issue and introduced a completely flexible communication layer. So instead of a preassigned WS-I protocol we can now choose from a widespread of communication protocols that nearly fit every situation.

But from a programmers point of view, the implementation of WCF services/clients follows pretty much the way we programmed web services. On the client side instead of a "web service reference" you now just add a "service reference" to your project but then it feels just like before. On the service side you now have to define a service interface and you use some other attributes but then again it feels like being with web services.

But this flexibility has some implications.
  • With every service reference you now add a configuration file with a "ServiceModel" section that keeps all the protocol specific configuration options.
  • The path to the WSDL for web services was pretty much always http://ServiceAddress?WSDL . Now with non http protocols there must be another way to access the Service Metadata. So with WCF the "Mex" or "MetadataExchange" endpoint, that is supposed to be on the "ServiceURL/mex" address, was introduced.

Target Enterprise Architecture

Guess the following scenario: I have 10 different types of Services running, with 100 clients each. An hour of unscheduled service downtime during day hours can cost me up to a 100 million bucks.

So what are my requirements?
  • I must be able to relocate my services without any clients failing
  • Services must be redundant and clients must support failover
  • I must be able to dynamically add more services of same type for load balancing and fault safety
  • I want to have a quick overview where my services are running
  • I must be able to change communication parameters (e.g. enable transport encryption on the service, without any of my clients failing

Does standard WCF Implementations meet our requirements?
By "Standard" I refer to the way the the service and the client had been created. Means setting up the service, then creating a client, adding the service reference to the service and then calling the service function of interest.

What happens if we:
  • Move the service to another server?
    The client fails.
  • A service fails?
    The client fails.
  • Change communication parameters of the service such as binding or encryption?
    The client fails.
  • Want to introduce another service of the same kind for load balancing?
    Installed clients ignore it
  • I want to get a quick overview where my services are running?
    I can't.

As a result for every above change I would need to update 100 hundred clients config files, but I have no clue on what workstations the clients are located. As a result, any above change breaks my production process, which is not acceptable in a enterprise environment.

Conclusion:
The standard implementation of WCF is not suited for use in Enterprise environments.

Introducing a central Service registry to our architecture

As a mechanism to find and offer web services in an enterprise environment this rings the bell for UDDI which is included since Windows Server 2003, but which seems to be fairly unnoticed so far. If I say documentation and support is sparse, this is quite a extenuation.

Web Services in a UDDI environment.
Handling web services in a UDDI registry environment was pretty much straight-forward. You simply stripped everything but the Types, Message and Operations sections out of your WSDL, published it as a model, and then you added the service endpoints to the registry (where there was just one per Service).
As a client, you added the web service reference to the stripped WSDL, then at runtime you simply put in the address from the UDDI and everything was supposed to work just fine as the communication layer was fixed to WS-I and there was just one endpoint.

WCF Services in a UDDI enviroment
With WCF we have a whole bunch of possibilities on how our service can communicate. This includes the option that a service has multiple endpoints offering different communication protocols at the same time. If we have multiple instances of a service we no longer can assume that they will all use the same communication protocols.

But if we try handle WCF Services with UDDI just like a Web services then what happens?

At first this means:
First we strip the everything but the Types, Message and Operations sections from our WSDL.
This is where you first time stop. Looking at the WSDL you will find out that the "Types" section is not declared incline, but every namespace just has a import reference.

You have 2 options now.
  • You open all the imports URL's in a browser and then you replace the imports with the results (sucks like hell after the 10th time) or
  • You modify the WSDL behavior of your service with e.g. FlatWSDL. But also this way is not perfect as it sounds as there is a bug in Visual Studio that does not let you choose your array types any more under some circumstances. So if your Service returns an array you could configure it to be a List in your code, but under some circumstances Visual Studio now will not let you change the array type to be anything but array. This is quite nasty as the WSDL is perfectly valid and the type is perfectly recognized as MyType [] but you just will not get is as List anymore, no matter what you select as your ArrayType.

But after we made a decision on this first problem now, what happens if you simple exchange the clients ServiceEndPoint adress with the URL you get from the UDDI Registry.

What happens if we:
  • Move the service to another server?
    The client works if it does not use encryption, otherwise it fails.
  • A service fails?
    The client works if it does not use encryption, otherwise it fails.
  • Change communication parameters of the service such as binding or encryption?
    The client fails.
  • Want to introduce another service of the same kind for load balancing?
    The client works if it does not use encryption, otherwise it fails.
  • I want to get a quick overview where my services are running?
    I get anytime a nice overview from my registry.

So if we use any kind encryption the concept we used for web services fails all but one of our architecture requirements.

Can't WCF Services be integrated into a UDDI environment?

The problem is basically the "ServiceModel" section in our clients config file. It freezes the state of the service to the point where we added the service reference. If you use a binding with encryption which is by default based on Windows users you find for example a tag called "Identity" which is usually something like "Hostname\Username" and which is used for the clients encryption to the service. But once we move the service to another server this of course will fail due to the identity mismatch.

Integration of WCF Services into a UDDI environment

How to query the Service Metadata at runtime?
I remembered I had a little web service project where we dynamically queried a services WSDL, created a DOM object and the compiled the client at runtime. So my first approach to WCF was similar, and it works.
But then what I basically want to do is the more or less the same what the WCFTestClient.exe does. You simply put in a URL to a service, WCFTestClients discovers it and then creates a client. But doing a little little reverse engineering using Lutz Roeders excellent Reflector tool revealed that....

WCFTestClient.exe just invokes svcutil on the command line. :-))) . Who dared to ship a awkward solution like this?

But when I did the same with "svcutil" I stumbled across two classes in System.ServiceModel.Description namespace.

MetadataExchangeClient and MetaDataResolver

What both basically do is to read the services WSDL and then return a ServiceEndpointCollection. Now you just have to choose which endpoint you want to use for communication with your service at runtime.
But here the first questions start. What do I need MetaDataResolver for, that needs a MetadataExchangeClient and returns a ServiceEndpointCollection, when I can use the MetadataExchangeClients ImportAllEndpoints function? (Update: My Implementation is solely based on MetadataexchangeClient. I tried to save some lines of code by using MetaDataResolver, but it never worked out. So I just can discourage the use of MetaDataResolver.)

If you try get some light into the situation you will soon find out that in regards to WCF you now walk off the beaten track. Documentation and references will get very sparse from this point on.

But anyway. This is basically exactly what we need to do, if we want to integrate WCF Services into a UDDI environment. We get the main ServiceURL from the UDDI Registry, then we read the service metadata on ServiceURL/mex or ServiceURL?WSDL, then we pick an endpoint and then we call the desired service function.

So what we basically do is instead of freezing the services state to the point we add the service reference to our client project, we do the "Add Service reference" functionality every first time we want to talk to our service in our code! This basically means we do not need the <ServiceModel> in our config file anymore.

This works a while but then....
you will find out...

Why we needed the <ServiceModel> section
First thing going to crash is when you receive a service response that is larger than 64k. Your client will report that your "MaxReceivedMessageSize" quota is exceeded. And there quite frankly I ask myself the following question: Who the fuck decided that in the age of Tera- and Petabytes 64k would be a good default limit for receiving a service response? My electric toothbrush could handle a response larger than 64k. Why not 6k or even better nothing by default??? By the end of the project I am sure it will be more than a thousand times we had to adjust this pain in the ass default parameter. If you are moving on BasicHttpBinding you will need to modify the "MaxBufferSize" parameter as well. Interestingly this parameter has always to be the same as as "MaxReceivedMessageSize". A parameter that has always the same size like another parameter? Jeez...
The next superfluous default parameter modification that you will encounter is the "maxItemsInObjectGraph" that has the same arbitrary limit of 64k.

Ok, I could rant on for while but let's stop crying and look for a solution.

How to modify an unknown binding at runtime?

The core of the problem is that we do not know what bindings the service will offer.

MaxReceivedMessageSize and MaxBufferSize
Lets look at the bindings inheritance.
All Bindings are derived from class Binding but except from some timeout parameters there is nothing we can do here.
Lets look at the binding element stack.
In the stack of Binding Elements this parameter belongs to the Transport Binding Element, which is part of any Binding. But I tried 2 days to find a way to get access to the binding elements, but only for CustomBinding this will lead you somewhere. For all other bindings this is a dead end street. The CreateBindingElements() function will just create a copy of the actual bindings elements and the GetBindingProperty() function which comes along undocumented in official documentation is of no help either.

As we have no clue what binding the Service will offer we now have 3 options:
  1. The IT-Hall of Shame candidate
    Write a function that tries to cast the discovered binding into any known binding and then sets the parameter
  2. We use Reflection
    We use a function that first tries to cast the object into a custom binding. If the cast is successful it tries for every property of every binding element to set a property with "Name" to "Value" and if the cast fails just try to find a property with "Name" and then to set it to "Value".
  3. We return a CustomBinding
    We use the discovered bindings CreateBindingElement() function, modify the parameter on the TransportBindingElement and return a CustomBinding instead of the original binding.

I don't like any of the solutions but solutions, but finally we decided to for the reflection solution.

How to modify MaxItemsInObjectGraph at runtime?
Interestingly "MaxItemsInObjectGraph" is a property of the DatacontractSerializer which is also used in all bindings.
If you use the config file you now have to add an Endpoint Behavior of type DataContractSerializer, but if you try to find in the ServiceModel namespace you will find out that you can not instantiate a this class at runtime. But looking further there is a DataContractSerializerOperationBehavior that can be instantiated at runtime but what can only be applied to a operation.
So what I did, was to create a class that implements IEndPointBehavior and then simply sets this DataContractSerializerOperationBehavior with a useful value to all operations.

Now I could get rid of my <ServiceModel> config section and all my functions worked just as intended.

So we got it?

So if you think this is enough struggling with 2 64k default parameters (yes I mean it sarcastically) you are soon to be disappointed.
Everything worked now, I could switch my service binding forth and back and my clients just worked fine until one day..... They stopped working.
MetadataExchangeClient reported that "Some items can not be resolved". It took me 2 days to find out what happened. Our data model had gotten bigger and bigger until one day.... Guess it.
Yes, the WSDL got larger than 64k. And this is the point where are all reference implementations of MetadataExchangeClient and MetadataResolver will fail. In case of MetadataExchangeClient even with a bolloks error message. And guess again. To solve the issue you have to create a Binding on your own, set this pain in the ass MaxReceivedMessageSize parameter and then use some constructors which are unmentioned in any example.

Finally!

This was so far the last submarine that submerged in front of my target architectures course. I don't say this was the last issue until of the end of the project, but yet I am confident that we will reach all architecture requirements with WCF without any limitations.

So what happens now if
  • Move the service to another server?
    I change the address in UDDI and all my clients simply follow and continue working.
  • A service fails?
    The clients does a nice failover and reconnects no matter what.
  • Change communication parameters of the service such as binding or encryption?
    The client adopts to it.
  • Want to introduce another service of the same kind for load balancing?
    Just register it to the UDDI and enjoy
  • I want to get a quick overview where my services are running?
    I get anytime a nice overview from my registry.
All our target architecture goals are met.
Hooray! Awesome.
WCF implemention flaws
  • The 64k default quotas on MaxReceivedMessageSize, MaxBuffersize and MaxItemsInObjectGraph and maybe elsewhere
  • The default WSDL schema that uses "Import" statements in the "Types" section.
  • The bug in Visual Studio that does not let you choose your ArrayType for flat WSDL's (MSConnect ID:387245)
  • The inheritance of bindings that have highly redundant properties in the specific bindings and/or the lack of access to the binding elements for non "Custom Bindings"
  • Inconsistencies between between config file configuration and runtime configuration
  • Lack of documentation
Conclusion:
  • WCF standard implementations are due to their "all is static" approach not Enterprise ready
  • WCF Services can be integrated into a Enterprise SOA architecture but several implementation flaws prevent a seamless integration and increase project risk.
Suggestions to Microsoft for next WCF Release:
  • Default binding quotas
    Quotas should be disabled by default. Any quota you set by default is arbitrary and will never fit all project scopes. If I do not trust the service, its nice that I can set this quota, but please let me choose whats best here. And especially do not use quotas that later blow up your own framework in a timebomb manner.
  • "Imports" in WSDL by default
    This is only causing problems and has absolutely no use. Get rid of it.
  • The inheritance of Binding
    Instead of a 2 level hierarchy it should be much more fine grained. e.g. in class BasicHttpBinding there should be only configuration properties that are absolutely unique to BasicHttpBinding. All others should be inherited. Adding more inheritance could be easily be done without breaking anything. This would smooth the way to a sophisticated runtime configuration of arbitrary bindings. I prefer this much more over a access to the binding elements as in CustomBinding.Elements.
  • Runtime configuration and config configuration should be consistent
If there is some interest I may be able to publish some source code. Just leave a comment.

Wednesday, December 10, 2008

WCF: Binding.GetProperty(T) Method

Did you ever wondered what to put in for the Parameter?
Rather than finding the answer in the Microsoft Documentation you find the answer here.
Basically it seems to be a rather senseless method for the standard bindings.

Monday, November 24, 2008

Error 9009 in Visual Studio means....

that Visual Studio can not find the executable you specified probably in Pre- or Post-Build command line.

That can have 3 reasons:
1.) There is no PATH environment variable set. You need to include the full path to your executable.
2.) You need to quote the fully qualified path to your executable if the path includes any blanks.
3.) The file really does not exist.

Examples:
SvcUtil.exe parameter1 parameter2 will fail

C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\SvcUtil.exe
parameter1 parameter2 will fail

"C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\SvcUtil.exe"
parameter1 parameter2 will do.

Tuesday, November 18, 2008

Bug in Visual Studio 2008 WSDL Interpreter

When I started working with Flat-WSDL we soon came to a phenomenon where our list types suddenly become converted to

ArrayType[]

instead of

List<arraytype>

It took a while to drill down the cause to the following.

A common wsdl:types section looks like this:

<wsdl:types>
<xsd:schema>
</xsd:schema ....>
<xsd:schema>
</xsd:schema ....>
</wsdl:types>

but whenever there is the following section in the WSDL

<wsdl:types>
<xsd:schema ....>
</xsd:schema>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" ....>
</xs:schema>
</wsdl:types>

you can not switch the ListTypes anymore. They will all be ArrayType[] . This applies for the ServiceReference as well as for svcutil that simply ignores the /ct: Parameter .

Both WSDL descriptions are perfectly valid as both tokens xsd: and xs: reference the http://www.w3.org/2001/XMLSchema Namespace and hence are synonyms.

I reported the bug to Microsoft using Connect (MSConnect ID:387245). But I heavily disagree with their answer, which basically says "It's not a bug it's a feature". You should find some samples wsdls there.

IT Hall of Shame

Oracle

Oracle column alias length limitation

The following SQL:

select 1 as "Why shouldnt you have a very long column name" from dual

will result in

"ORA-00972 identifier is too long"

The maximum length for a column alias in Oracle is 30 characters. Guys, we live in age of Terabytes!

Trying to pass a list as parameter

Try the following SQL:

select 1 from dual where 1 in (:var)

with :var = 1 (works)
with :var = 1,2 (no error but no result. The list is interpreted as float)
with :var = 1,2,3 (error Invalid number)

There is no way with standard SQL instruments to pass a list as parameter in Oracle.

Saturday, November 8, 2008

The Variant strikes back

Anders Hejlsberg, lead-architect of C# announced at the PDC 2008 that C# 4.0 will be enhanced towards a dynamic language. With the keyword "dynamic" you can declare a variable as a datatype that will be bound at runtime rather than compile time.
The background is, that it will make the use of COM components much more convient. Today you link your code against the exact version. If the version of your component is not installed on the remote computer your code will fail, even if the interface the installed component has not changed.
Reflection is in its usage for emergencies only and not suited for complex interfaces. Your code will become a huge mess if you try to handle complex objects.

So this is a HUGE break with current C# dogmas..

But wait a minute! Don't all of you, who know what's behind the letters "VB" feel strangely remembered? Something we thought we had burried a long time ago?

The VARIANT strikes back!

And to be honest, coming down the C, C++ path I feared and hated it, whenever sub-average VB programmer colleagues started using it in their code. The Variant or Dynamic liberates developers from spending a minimal time on thinking before start coding. They simply declare some global "Variants" and use them to store whatever comes along. Absolutely impossible to tell whats the content of the variable when you read the code. The absolut horror of everyone who has to maintain or debug this kind of code.

I clearly see the advantages of runtime binding, but I see the immense risks the same time.

Maybe you should pass some sort of qualification exam to get this type enabled.

"Anders Hejlsberg, Chef-Architekt der .Net-Programmiersprache C#, hat im Rahmen der Microsoft Professional Developer Conference (PDC) in Los Angeles neue Sprachfunktionen für C# 4.0 und die darauf folgende Version erstmals vorgestellt.
Die Hauptneuerung in C# 4.0 ist die Erweiterung in Richtung dynamischer Sprachen. Mit dem Schlüsselwort dynamic können C#-Entwickler in Zukunft eine Variable so deklarieren, dass die Bindung erst zur Laufzeit stattfindet." (Heise.de)

Microsoft rewrites Workflow Foundation from the scratch

In a keynote at the PDC 2008 conference the speaker Kenny Wolf announced that Microsoft is rewriting Windows Workflow Foundation from the scratch. The basic concepts will be retained but the implementation and all components will change fundamentally. It will be shipped with .Net 4.0 and will not be compatible with current Workflow Foundation Code.

One of my next articles in this blog would have been titled "Why I do not believe in Windows Workflow Foundation". I think I can skip this article now, as Microsoft obviously has come to the same conclusion.
First of all. If you want to do workflow driven development, the current Microsoft strategy is a mess in itself. Do I use Workflow Foundation, Workflow Foundation for Sharepoint, Biztalk or what? Currently we have 3 products offering more or less the same. But even even if you get behind the point of simply chosing Workflow Foundation things do not lighten up.
If you look closer at the word "Workflow" you will soon encounter some philosophic problems.
What is a workflow? Are we talking about program level workflow, about technical workflows or do we even talk about interactive workflows, are we talking about business processes, do I store my business object in the workflow object, when do I use sequential workflows or state machines....
Workflow Foundation fails to answer any of these questions and hence gives you no clear strategy what WF is aimed at. For me it felt like getting a tool box tossed at my head with a sticker on it saying "Find out what it's good for".
To the point where I used Workflow Foundation in technical workflow, things where working out but when I got to the point where I tried to implement an interactive workflow with just 4 states things were getting messy.
The whole communication concept with workflows is a mess, the Workflow controller is unbearable slow, the workflow designer shipped with Visual Studio 2008 was ***unusable*** before service pack 1. If you want to user persistence or tracking you end up rebuilding the database frequently, because errors come up that can not be solved differently.
The WCF-WF integration example published by Microsoft definitely belongs into the Hall of Shame. A calculator that needs a Start() and Stop() Method before/after it does calculate something. Jeeeez. Who dared publishing something like that? It clearly shows that both techniques where designed independently and were never designed to interact.

But with the new approach am looking forward to see what the result will be. I definitely believe that the future of development will be workflow driven. Software definitely needs to get to the point where my business process reflects in software. There and only there I benefit from workflow driven development.


"In einem Vortrag auf der PDC 2008 offenbarte Sprecher Kenny Wolf, dass Microsoft die Windows Workflow Foundation (WF) noch einmal von Grund auf neu schreiben wird. Die Konzepte einer Laufzeitumgebung, die aus einzelnen verbundenen Aktivitäten bestehende Workflows ausführt, bleiben zwar bestehen. Auch die Workflow-Dienste wie Persistierung und Ablaufverfolgung wird es weiterhin geben.
Die Implementierung und alle Bausteine (Laufzeitumgebung, Aktivitäten und Dienste) werden sich jedoch in der kommenden Version fundamental ändern. Die neue Ausgabe wird nicht mehr kompatibel sein zu den derzeit am Markt verfügbaren Versionen." (Heise)

Friday, October 10, 2008

Download ASP.Net Files with IIS

If you try to download .cs or .vb files from an IIS Server you will soon see the following error:


Server Error in '/' Application.
This type of page is not served.

if your file is located in a simple directory or

The page cannot be displayed

You have attempted to execute a CGI, ISAPI, or other executable program from a directory that does not allow programs to be executed.

To solve this you first have to convert the directory into an application-directory.

To do so enter the "IIS Manager", right click the directory, select "Properties", go to the tab and click the "Create" Button. Then select the "Configuration" Button, browse to the appropriate extension and remove it (in this case the .cs extension). It will remove the extension registration for the given directory only.

If you now you try to download the file you probably get the following error:


The page cannot be found
The page you are looking for might have been removed, had its name changed, or is temporarily unavailable.


The reason behind this is, that there is no MIME type registered with .cs files and IIS does not know how to handle it.

So again open the "IIS Manager", but this time right click on the server itself. Now click the button "MIME Types", press the "New" button and enter ".cs" as extension and "text/plain" as MIME Type. Then restart the affected web site (Default Web Site in my case) by clicking its stop and start button.

Now you should be able to download .cs files.

Wednesday, October 1, 2008

Cognos Impromtu 6 and Oracle 10 Client

works!

find the file cogdmor.ini (usually in %ProgramFiles%\Cognos\cer1\bin ) .

Find the section:

[DLL Names]

and Add the line:

OR1002-Win32=ORACLIENT10.DLL,10

(I am using 10.2.0.3 client)

Thursday, September 4, 2008

.Net and Oracle 10. Welcome to DLL Hell

Until Oracle 8 or 9 you installed the client, chose whether you want to use Oracle Driver (Oracle.DataAccess.dll) or the Microsoft Driver (System.Data.OracleClient) and you had a happy connection to your database.

Installing a OracleClient and connecting to a Oracle Database with was until a day in August 2008 a Non-Event. 15 minutes and off you go.

Now we upgraded our database backend to Oracle 10 as anything else is running out of maintenance.

"Ok", I thought. "Business as usual". "Install the latest client, turn on, works".

What I didn't know was that I was standing on the doorstep to Oracle DLL Hell.

First everything looked just smooth. After upgrading the client from Oracle 8 to 10, the connection with System.Data.OracleClient to our Production environment on Oracle 9 was just fine. No problems here.

The problems started when I first tried to connect with an Oracle 10 the same database only on Oracle 10.

4 out of five queries worked fine but then

"ORA-01405: fetched column value is NULL" System.Data.OracleClient

The error message means basically that I tried to fetch a null value into a PL-SQL variable.

Only Problem here. I use a common SQL here. So no fetch anywhere.

We now invested much much time in solving this problem, upgraded the server to latest patchlevel and lots of different things. Nothing helped. So the only way to solve this problem was to switch to the native Oracle .NET driver.

After switching to the native Oracle Driver (Oracle.DataAccess.dll) the problem was solved. Only problem that remained now, was that my version of the client 9 was not installed on the server and hence I would not be able to install the software on the server with that dll.

Ok, I thought. Lets get rid of my Oracle 9 and install a Oracle 10.2.0.3 client. The same client as installed on the server.

Sooner said than done. I chose "Application Developer" option and off I went.

But when I tried to add the reference to the Oracle .Net driver (Oracle.DataAccess.dll) using the "Add reference" dialog in Visual Studio .Net 2008 I found.... nothing. Obviously .Net Application Development is no longer in focus when installing the Oracle Client, but I knew that Oracle has special packages for .Net.

Installing Oracle Data Provider for .Net (10g).

Why the fuck the installer recommends to create a new Oracle Home instead of installing it where it belongs (the exsting Oracle Home of my 10.2.0.3 client directory)?

Very important: At the point where you can choose the target directory, make sure you select the existing client directory, by selecting the proper Oracle Home Name.

I cost me 1.5 days to find out, that by default, the installer creates a new Home Directory, where it simply doesn't do anything but to throw exceptions.

After installing the ODAC correctly you find it in your "Add reference" Dialog unter .NET components under "Oracle.DataAccess".

5 Cents on .NET and Oracle 10

The story is longer but here are my 5 cents:

  • Make sure the Oracle client version on your development and production machine are the same
  • To get a working .NET driver, you need to additionally install the Oracle Data Access Components for .NET which can be found here .
  • Make sure, you select the proper version of the ODAC for your client.
  • When the bloody Oracle installer asks you for the directory make sure you select the existing directory of your client.
Dear guys at Oracle. In the last 2 days I really learned to hate your "Oracle Universal Installer". The setups are full of flaws, misleading informations, bugs, crc errors, bad recommendations or unnecessary components.
And why the hell, the Developer Oracle Client Package does not include a .Net Driver?
By today I can call myself Oracle Client Setup Expert (OCSE), for a thing that should be what it is: a non-event

Wednesday, July 30, 2008

First time Windows Vista or Disillusion in 30 minutes

I just got my brand new retail PC sold with Vista Home Premium from Atelco, decent seller for computer related products in Deutschland (Germany). So I expect that Atelco had put some though in their configuration, what is working best with Vista. It even came not just with Vista, it came with Vista SP1 preinstalled.

After hitting the market for over a year and with SP1 I was willing to take the challange.....

It took me 30 minutes until I wrecked my Vista for the first time.

Until the installation I got the installer error: "Error 1628: Failed to complete script based install". Too late I found a promising link so I decided to reinstall.

In total I reinstalled it 2 times this night. The first upgrade install did not fix the problem, so I had to do another complete reinstall.

But this was not the end of disappointments.

The next was named Bluetooth. I had a simple USB Dongle. After I personally found it disappointing that Microsoft had failed to ship XP with proper Bluetooth support, I was absolutely convinced that it would be no problem at all to connect my Microsoft Bluetooth keyboard and mouse as well as my Nokia N95 mobile with Vista.

Well think again. Although keyboard and mouse seemed to work at first, it soon turned out that after a certain time (between 1 hour and 1 minute) the keyboard would delay every key pressed for 10 seconds or more. Not really the ideal solution for playing an ego shooter game.
Even worst was my mobile. I got a lot of installation errors (sync profile) errors, that from now on popped up every time I started my Vista.
When I googled for the problem it soon turned out that the Bluetooth stack that comes with Windows Vista is complete crap. If you want to use Bluetooth you still need to install a 3rd party bluetooth stack.

Apart from that I already had a blue screen during boot (bad_pool header), I have a disc usage of almost 80 gigs without anything installed, the disc is running with intense activity for no obvious reason, the icons use up space like nothing, the program menu looks like crap, the permanent questioning to authorize this and that sucks like hell.

Just today I heard a story from a real good IT guy, who had the following problem. After starting his Vista Laptop in a Windows 2003 Server controlled domain, his Vista would start and connect fine the first time, but when you shutdown or reboot this was the last time your Vista will ever connect to ANY network.
The reason is that the server distributes a group policy (which is aimed at XP Workstations). Once Vista applies the group policy it disables everything that is not explicitily allowed. And this includes various network services. Even worse. Even the administrator won't be able to revive the services ("Access denied") and you won't ever be able to connect to any network!
This means under certain circumstances your Vista will commit suicide.

But apart from the fact that Vista is big, bloated, unstable, confusing and a potential suicide candidate, what are the advantages compared to XP?

...
.....
........
...........

(Still thinking)

Friday, July 25, 2008

Adding FlatWSDL to WCF WebService

WCF WebServices in a UDDI Environment

Download the SourceCode

If it comes to the point where you want to integrate your WCF WebServices into a UDDI environment you will stumble accross a feature introduced in WCF.

By default you will find in the WSDL types section:

<wsdl:types>
<xsd:schema targetNamespace="http://tempuri.org/Imports">
<xsd:import schemaLocation="http://localhost:3826/Service1.svc?xsd=xsd0" namespace="http://tempuri.org/" />
<xsd:import schemaLocation="http://localhost:3826/Service1.svc?xsd=xsd1" namespace="http://schemas.microsoft.com/2003/10/Serialization/" />
<xsd:import schemaLocation="http://localhost:3826/Service1.svc?xsd=xsd2" namespace="http://schemas.datacontract.org/2004/07/MyFirstWCFService" />
</xsd:schema>
</wsdl:types>


Looks great, but here it is causing a problem. Using a UDDI means you bind your Service at runtime. To do so you need a WSDL that contains only the abstract informations. But "http://localhost:3826/Service1.svc?xsd=xsd0" ist not very abstract, right?

You have 3 choices now.
a.) You have always your service at the given adress up and running.
This would just negate the whole UDDI concept.
b.) Load every data schema by hand, and replace the include statement with real schema. By hand. This is pain in the ass. Especially when you deal a number of datatypes from different namespaces.
c.) You find a way to somehow flatten your WSDL.

As I did not like a.) or b.) I looked for c.) and luckily stumbled across the articles of Tomas Restrepo and Cristian Weyers. Christian even provides the source for download and their FlatWSDL does a nice job. Thanks guys!

But their example was related to self-hosted services. As I mainly try to run IIS based WCF Services I came on day to the point where I had to integrate FlatWSDL with an IIS based WCF Service.

The problem. You do not directly control the ServiceHost object and thus

foreach(ServiceEndpoint endpoint in selfHost.Description.Endpoints)
endpoint.Behaviors.Add(new FlatWsdl());

won't work here.

But Ok then. Everything related to ServiceHost Configuration you can do either in Code or via the ConfigFiles. The WCF Config Editor is really a great help here.

So I just added FlatWSDL.cs to my project, recompiled it and fired up the WCF ConfigEditor.

The first step is to register FlatWSDL as a "behaviour element extension" what can be done under the "Advanced - Extensions" tab.

There is a button "New". Enter a Name, pick your Service.DLL and select.....

nothing.... which is about the whole choice you have.

When I googled for "wcf add custom endpoint behaviour config" I again stumbled accross an article from James Bender where he also implements an IEndPointBehaviour but here he derives his behaviour class from the abstract class BehaviorExtensionElement as well.

Related to BehaviorExtensionElement the help says:
"Represents a configuration element that contains sub-elements that specify behavior extensions, which enable the user to customize service or endpoint behaviors."
"Sounds great" I thought and derived the FlatWSDL class from BehaviorExtensionElement. So your class definition now looks like
using System.Collections;
using System.Collections.Generic;
using System.ServiceModel.Channels;
using System.ServiceModel.Configuration;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.Xml.Schema;
using ServiceDescription = System.Web.Services.Description.ServiceDescription;

namespace SCDServices.ServiceImplementation
{
/// <summary>
///
/// </summary>
public class FlatWsdl : BehaviorExtensionElement, IWsdlExportExtension, IEndpointBehavior
{
    public void ExportContract(WsdlExporter exporter, WsdlContractConversionContext context)
    {
    }

    public void ExportEndpoint(WsdlExporter exporter, WsdlEndpointConversionContext context)
    {
        XmlSchemaSet schemaSet = exporter.GeneratedXmlSchemas;

        foreach (ServiceDescription wsdl in exporter.GeneratedWsdlDocuments)
        {
            List<XmlSchema> importsList = new List<XmlSchema>();

            foreach (XmlSchema schema in wsdl.Types.Schemas)
            {
                AddImportedSchemas(schema, schemaSet, importsList);
            }

            wsdl.Types.Schemas.Clear();

            foreach (XmlSchema schema in importsList)
            {
                RemoveXsdImports(schema);
                wsdl.Types.Schemas.Add(schema);
            }
        }
    }

    private void AddImportedSchemas(XmlSchema schema, XmlSchemaSet schemaSet, List<XmlSchema> importsList)
    {
        foreach (XmlSchemaImport import in schema.Includes)
        {
            ICollection realSchemas =
            schemaSet.Schemas(import.Namespace);

            foreach (XmlSchema ixsd in realSchemas)
            {
                if (!importsList.Contains(ixsd))
                {
                    importsList.Add(ixsd);
                    AddImportedSchemas(ixsd, schemaSet, importsList);
                }
            }
        }
    }

    private static void RemoveXsdImports(XmlSchema schema)
    {
        for (int i = 0; i < schema.Includes.Count; i++)
        {
            if (schema.Includes[i] is XmlSchemaImport)
                schema.Includes.RemoveAt(i--);
        }
    }

    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    {
    }

    public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
    {
    }

    public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
    {
    }

    public void Validate(ServiceEndpoint endpoint)
    {
    }

    public override System.Type BehaviorType
    {
    get { return typeof(FlatWsdl); }
    }

    protected override object CreateBehavior()
    {
    return new FlatWsdl();
    }

    #region Ctor

    /// <summary>
    /// Default constructor.
    /// </summary>
    public FlatWsdl()
    {
    }
    #endregion
    }
} 


1.) We compile and fire up the config editor again.
We go to "Advanced - Extension - behaviour element Extension" and press new.
Now we browse to our Service.DLL in the bin directory and Bingo!



We select Thinktecture.ServiceModel.Extensions.Description.FlatWsdl from the List.

2.) We now browse to "Advanced - Endpoint Behaviours" und press "New Endpoint Configuration"
We enter a name (here MexFlatWSDLBehaviour), press "Add" and select FlatWSDL from the List.



In case there is no FlatWSDL in the List, save, close and reopen the ConfigEditor. It now complains that it can't find the DLL file. Press "Yes" browse to our Service.DLL in bin directory and click "Yes" when the editor warns you it is going to execute some code.
Now at latest there should be FlatWSDL in the list.

3.) Now add the newly created Endpointbehaviour to our ServiceEndPoint (not the mex endpoint).
You can recognize your ServiceEndpoint as the "Contract" attribute points to your ServiceInterface.
For "BehaviourConfiguration" select our "MexFlatWSDLBehaviour" from the list.


Done

When you now check the WSDL emitted from our WCF WebService you will find nice inline type schemas.

<wsdl:types>
<xsd:schema elementFormDefault="qualified" targetNamespace="http://tempuri.org/">
<xsd:element name="HelloWorld">
<xsd:complexType>
<xsd:sequence />
</xsd:complexType>
</xsd:element>
<xsd:element name="HelloWorldResponse">
<xsd:complexType>
 <xsd:sequence>
    <xsd:element minOccurs="0" name="HelloWorldResult" nillable="true" type="xsd:string" />
 </xsd:sequence>
</xsd:complexType>
</xsd:element>



Christian had found another approach through a custom ServiceHost and custom ServiceHostFactory. Look at the bottom of the article.

I recommend taking a look at WCFExtra. They just released Version 2. Among others it brings single WSDL support out of the box.

Download the SourceCode

Thursday, July 24, 2008

Must have Development Tools

ProcessExplorer

Gives you many details on running processes. Including IP Connections etc. Although these functions are really nice, the "Find Handle or DLL" functionality is a real life saver.

Every Developer gets the error message "can not access file because it is used by another process" once in a while.

But which programm/process is it? This is where the "Find Handle or DLL" functionality enters the stage. Simply enter the filename, and Process Explorer will tell you who is sitting on your file.





Web(Service)-Development

Fiddler

Fiddler is a HTTP Debugging Proxy which logs all HTTP traffic between your computer and the Internet.



.NET

Lutz Roeder's Reflector

Excellent Tool for .NET Assembly reverse-engineering. Many plug-Ins available

Subversion & TortoiseSVN & Ankhsvn

Subversion is a versioning system and TortoiseSVN is the fitting Subversion explorer client, implemented as a windows shell extension.
Ankhsvn makes these 2 a dream team as in brings a Visual Studio 2005 & 2008 plugin, that allows you to handle most tasks directly from within Visual Studio.

Once you understood the concept of versioning and Subversion you never want to live without it. Even for standalone developers it makes perfectly sense to use it. You need no server component, simply create your repository on your disc or filesystem, import in your stuff, checkout and never worry again about versions or unlucky modifications.

Thursday, July 17, 2008

Walktrough for mediawiki on Windows IIS - The gory details

I have the following setup:

Windows Server 2003 with Inernet Information Server 6 (IIS 6) running. So installing Apache is not an option.

First of all.

It works with no limitation. Even with complete Email-Support.


For running mediawiki you need:

mySQL (I used 5.0.51)
the MySQL GUI Tools
Php (I used 5.2.6)
mediawiki of course 1.12)
FastCGI for IIS 6
DiffUtils for Windows (Optional but useful)
ImageMagick (Optional)


A. Install ImageMagick

Just run the installer and remeber the path.

B. Install DiffUtils


Just run the installer.

1. Install FastCGI

Simply run the installer

2a. Install mySQL


Simply run the installer. Not much to do here. Remember the Password of the "root" user. You will need it later to autocreate the mediawiki Database.

2b. Install the MySQL GUI Tools
Simply run the installer.

3. Install PHP

Installation

This is more tricky.

When asked for the Web Server Setup choose :

IIS FastCGI

I chose the following Extension (do not select all):

MySQL

MySQLi
OpenSSL

Enable the following Extra

Pear Install

After setup completed

Configuration

Modify php.ini

Under Start - Programms - PHP 5 you find php.ini

Edit the following parameter:

fastcgi.impersonate = 1;

Modify fastcgi configuration fcgiext.ini file for php support

Click Start - Run and slap in

%WINDIR%\system32\inetsrv

Open fcgiext.ini in Notepad and add at the bottom

[Types]
php=PHP

[PHP]
ExePath=D:\Program Files\PHP\php-cgi.exe
InstanceMaxRequests=10000
EnvironmentVars=PHP_FCGI_MAX_REQUESTS:10000


Make sure the Exepath reflects the path to your PHP installation!

If you get the error:

FastCGI Error
The FastCGI Handler was unable to process the request.

Error Details:

* Could not find entry for "php" on site 1 in [Types] section.
* Error Number: 1413 (0x80070585).

* Error Description: Invalid index.


HTTP Error 500 - Server Error.
Internet Information Services (IIS)

you did not edit fcigext.ini properly

See also the full description here

Install MediaWiki to IIS

Installation

Create a directory for you wiki under the wwwRoot Dir of IIS ( I used "wiki" in this case)

Extract the mediawiki-1.12.0.tar.gz to the "wiki" IIS directory.

Modification of config/index.php installer script

If you installed the DiffUtils:

Go to the wiki/config/ directory and edit the file index.php

Find the line:

$diff3versioninfo = array( '$1 --version 2>&1', 'diff3 (GNU diffutils)' );

and replace with:

$diff3versioninfo = null ;

If you get the Message:

PHP Warning: shell_exec() [function.shell-exec]: Unable to execute 'D:\Program Files\GnuWin32\bin\diff3.exe --version 2>&1' in F:\secureinet\wwwroot\wiki\config\index.php on line 1824

you did not edit this line.

If you installed the ImageMagick:

Go to the wiki/config/ directory and edit the file index.php

find the line:

$imcheck = array( "/usr/bin", "/opt/csw/bin", "/usr/local/bin", "/sw/bin", "/opt/local/bin" );

and replace it with

$imcheck = array( "", "D:/MYPATH/ImageMagick-6.4.2-Q16", "/usr/local/bin", "/sw/bin", "/opt/local/bin" );

Remember to replace MYPATH with the correct path and to turn Backslash \ into Slash /

find the line

$im = "$dir/convert";

and replace with:

$im = "$dir/convert.exe"

Configure the wiki directory settings in IIS

Start - Settings - Administrative Tools - Internet Information Services

Configure the "wiki" directory appropriatly. To be honest I just enabled everything:

Script source access, Read, Write,Directory Browsing and Execute Permissions on Script and Executables.

For me it is fine as I use the wiki in the intranet. For Internet usage this could be potentially insecure. Post a comment if you know what the best settings are.

Edit the permissions on the config Directory.

In explorer go the wiki directory

Configure file permissions by rightclicking the "config" directory, go to the security tab and add "Everyone" with Permission "Full Control". The directory should be deleted anyway after the installation so it does not matter.

If you get the error message:

PHP Warning: fopen(LocalSettings.php) [function.fopen]: failed to open stream: Permission denied

you do not have permissions to write the file.

Create the mediaWiki

Now start the Browser and slap in the URL

http://MyServer/wiki/config/index.php

If you have installed diff and ImageMagick it should come up with the message that they have been found.

  • Choose a name for your wiki
  • Enter a wiki admin Password (and remember it)
  • Disable all Email Options including email authentication (they won't work out off the box. All options can be enabled later)

Choose
  • a Database Name
  • a Wiki Database Username and Password.

If the user and database do not exist, you must check the "Use superuser account" option. In this case the database and the user will be created for you. Put in the password of your MySQL root user.

All the rest leave unchanged.

Now hit the install Button and you should get a lovely success message.

Now go to wiki/config directory and copy the LocalSettings.php file into the wiki directoy. If you use mediaWiki in the internet you should delete the config directory now.

Now slap in http://MYSERVER/wiki/index.php and you should get a warm welcome on your newly created media wiki.

Nice!


Email Support for mediawiki on Windows IIS

Remember I recommended to install the PEAR Extra during PHP Setup? (If you forgot, you can simply rerun the installer and modify the installation)

Go to your PHP programm directory and run the pear.bat.

This will ask you a bunch of questions. I simply confirmed each recommendation by return.

After the installation you find a PEAR_ENV.reg . Double click it to add the configuration to the registry.

Create an Includes Directory in your PHP programm directory.

Modify php.ini

Under Start - Programms - PHP 5 you find php.ini

After installing PEAR your should have something like this. Edit the following parameter:

include_path=".;D:\Program Files\PHP\pear"

to

include_path = ".;D:/INSTALLATIONPATH/PHP/includes;D:\Program Files\PHP\pear"

Remember to adapt the path to reflect the correct path to your PHP directory.

On Windows make sure you do use ; and not : as seperator!

I had some strage issues, where the change of my include path was not reflected in the running php scripts. To avoid unnecessary stress it is advisable to reboot the server now.


Now Download Pear Mail Package (I used 1.2.0b1)

Extract the folders to your newly created include folder.

Now Create a folder "Net" in your "includes" folder

Download Pear Net SMTP Package (I used 1.3.1)
Download Pear Net Socket Package (I used 1.0.9)

Extract the folders to your newly created includes/Net folder.

your INSTALLATIONPATH/PHP/includes folders should now look like this:

[Mail]
[tests]
Mail.php
[Net]/SMTP.php
[Net]/Socket.php
[Net]/[Docs]

and some other subdirectories.


Now go to your wiki directory and edit the file LocalSettings.php

Configure the Mail-Server by adding the following to your LocalSettings.php

$wgSMTP = array (
"host" => 'mymailserver.somewhere.com',
"IDHost" => 'myDomain.com',
"port" => "25",
"auth" => false
);

if you need authentication use

$wgSMTP = array(
'host' => "mymailserver.somewhere.com",
'IDHost' => "myDomain.com",
'port' => 25,
'auth' => true,
'username' => "my_user_name",
'password' => "my_password"
);

To Enable User Emails

find the lines:

$wgEnableEmail = false;
$wgEnableUserEmail = false;

and set to

$wgEnableEmail = true;
$wgEnableUserEmail = true;

To Enable WatchList Notification

find

$wgEnotifWatchlist = false; # UPO

and set to

$wgEnotifWatchlist = true; # UPO

Remember you have to enable Email Notification for your Watchlist in your user profile as well.

Done.

Have fun with a fully operational wiki on Windows and IIS.

Btw.

If you are looking for WYSIWYG Editor have a look at fckeditor. A bit tricky to set up, but awesome. I am using version 2.6.2.

For source code syntax highlighting have a look at the Geshi extension