LearnVSXNow! #22 - Thinking about a new MPF

In the last few weeks I had busy days full with different kind of deep dives. As you could see on LearnVSXNow, my latest three articles were about deep dives. I was contracted to work with the VSX team on some kind of technical deep dive materials. At the end of May I took part in other kind of deep dives: I went to a diving safari near to border between Egypt and Sudan at the St. Johns reefs.

Six days of my life was about what I like the best after my wife, daughters and job: diving! Every day we had four dives interrupted with meals, short sleeps, sun tanning and briefings for the next dive. I was a dive master of a team of six divers and we had a great time together. We made many nice dives with turtles, big groupers, lots of rays; we saw a few shy sharks and many nudibranches. We dived together at spectacular reef walls, in maze-like caves and even in small wrecks.

I have a fear of sea sickness, so I always take my Egyptian pills with me—we call them “shock absorvers” with a funny name. This time “shock absorvers“ gave me an excellent help. For three days we had about 3-4 meters (9-12 feet) high waves in a boat with a length of 35 meters (105 feet) and we felt just like being on a rollercoaster. Without the pills fishes and other sea creatures could have been the real consumers of my breakfast and lunch.

I took my notebook with me to waste my time between dives and dealing with the other kind of deep dives—in VSX. High waves did not allow me working too much, so I “meditated” and collected thoughts about meaning of life and meaning of VSX. J

After one week of relaxation and returning to Hungary I found myself in the middle of a big buzz. In the last week we had a VSX day here in Budapest, it was the last stop of a roadshow with James Lau and Jean-Marc Prieur (read here how James saw the event). These great guys spent two days here in Budapest.

On the first day I spent I few hours with Jean-Marc. In the morning we had a talk about DSL and VS SDK. In the afternoon we had a four hours sightseeing walk in the castle district of Budapest and kept on talking. In a few hours we recognized we are thinking about Visual Studio Extensibility and on its future on the same way.

On the second day after the event we also had a short guided tour with James and her girlfriend; in the evening we could have a talk after having a few mugs of excellent Hungarian beer and a few bottles of great red wine.

My “in-the-middle-of-high-waves” meditation and the talks with Jean-Marc and James stimulated me to share a few thoughts about how I imagine a new version of VS SDK and the Managed Package Framework. I dedicate this post to this topic.

What I put down here is just a start. As you may know, I deal with VSX only since last November, so the picture I paint here is rather about the impressions of VSX novice than the real experiences of a senior.

Pains with VS SDK and MPF

VS SDK improved a lot since Visual Studio 2003 and today it is easier to start with and use than ever before. However, there are many points that could make developers’ perception much better by reducing the pain (I mean reducing the efforts to do something that should be much easier in our minds). Because I am a relatively new guy in VSX, I think if something is a pain for me it also will be for others starting with VSX.

Need for a better documentation

My first concern—especially in the first few weeks— was the quality of the documentation available for Visual Studio Extensibility. I spent hours in the first days to guess out where to start the documentation. The “Getting Started” part was a bit “in medias res” for me. Hands-on labs and tutorials were much more useful but I missed something that could give me a global overview of the whole VSX story and marking the points to start with the right scenario depending on what I expect from extensibility or what I am supposed to do with it.

That was one of the initiatives leading me to LearnVSXNow project. In the last five months I see great progress on this area. Now we have the Professional Visual Studio Extensibility book from Keyvan Nayyeri that is no doubt something we have been missing for a long time. More and more community content is coming out so the amount and quality of documentation is getting better.

However, there is one area where we need much more improvement. Starting with VSPackage development and moving to advanced topics is not covered with the right documentation. Just think about topics like custom editors, custom project systems, handling hierarchies, language services, using VS Shell, and many more.

What I mean when saying “right documentation”? The “Domain Specific Development with Visual Studio DSL Tools” book by Steve Cook, Gareth Jones, Stuart Kent and Alan Cameron Wills is a great example of what we need for VSPackage development.

Facing with COM everywhere

We know that the technology roots of Visual Studio go back to COM. I totally agree with others: changing those roots is a long and painful process—at least for the architects and developers. Personally I do not believe that it can be done earlier than the next third or fourth version of Visual Studio. I suppose the VSX team makes efforts to examine how to move toward a Visual Studio implemented with as much managed code as possible. However, I do not think all the COM and unmanaged stuff can be eliminated.

Even if moving toward managed code is slow and we will still have COM for a long time from now, it does not mean that we have to face with COM interop at every corner as we walk. There are many cases where we could wrap the COM interfaces and classes into a costume to imitate they are the same nice-looking and easy-to-use .NET classes as the ones in the .NET base class library.

Here is a list about the “top pains” I feel personally—I suppose you can extend the list with many more:

—  Using and finding GUIDs. COM objects and GUIDS cannot be pulled apart. This phenomenon itself would not be a big concern while it would be manifested only in using Guid attributes decorating our types or members. However, there are situations when we need to describe the same GUID value at several locations (think about VSCT files and command binding—as only one example). If we want to obtain predefined objects, commands, context IDs, etc. we need to find the right GUID value. Sometimes it can be found as a member of the VSConstants type; sometimes we can found them only in the header files coming with VS SDK or can ask for them in forums. In decorating attributes we can only use the Guid attributes initialized by GUID strings, because System.Guid constants cannot be used in attributes.

—  Poor naming conventions. Krzysztof Cwalina and Brad Abrams dedicated a very useful chapter about naming conventions in their “Framework Design Guidelines” book. Well, the COM interop assemblies and the COM objects behind them poorly use those great patterns. Actually they seem using divergent naming conventions. Instead of helping the usage of types and methods by applying a right name, sometimes those names increase the likelihood of misunderstanding.

—  Separate functional approach. There are many COM interfaces and types having a different name and approach from the types we use in .NET to provide the same functionality. For example all of us know the Console.Write and Console.WriteLine methods with their overloads. The IVsOutputWindowPane interface more or less has the same functions as the Console object has. However it has methods like OutputString and OutputStringThreadSafe but both methods lack the flexibility the Write and WriteLine methods have.

—  Differences in object lifecycle management. While .NET uses the garbage collection mechanism provided by the framework, COM uses the “good old” reference counting approach. Everywhere when we obtain references to COM objects we must know if we are the ones responsible to release the COM object by decrementing the reference counter or calling a method will decrement the counter. We must implement the IDisposable interface and must have a good knowledge about how COM objects are used and released in order our Dispose method can work as it is expected.

—  You must be a good friend with the System.Runtime.InteropServices.Marshal class. Converting COM types to .NET types or vice versa you cannot avoid using the services of the Marshal class. It is understandable, because you need to cross technology borders, but always having to face with them makes you code much error-prone and harder to read. Sometimes using the Marshal class totally removes the focus from the real things your method does.

—  HRESULT versus Exceptions. By design, using COM methods requires handling HRESULT values. Sometimes the two-third of code lines in a method is related to handling the HRESULT values and this syntax noise makes your code more difficult to read. Converting them to your custom Exception requires a lot of work and the same is true when an Exception must be converted back to an HRESULT.

This list represents only the top of the iceberg of COM issues in VSX. You can find many more points that can lead back to the COM interoperability as the root of issues.

Need for a better object model

What I really miss is a good object model above the COM interoperability types. Creating a layer over those COM interfaces and classes and establishing a useful, intuitive hierarchy of objects would decrease the startup costs of VSX programming to much lower level.

I feel a few objects—for example COM interfaces related to the text buffer and custom editors or hierarchy objects—have too many responsibilities.

Managed Package Framework is a kind of answer for the questions about a better object model. MPF has many nice features like registration attributes decorating the package class, the WindowPane, ToolWindowPane and DialogPage classes.  However, it covers only a small portion of the COM types behind VSX.

Improving VS SDK and MPF

After having a few talks with Jean-Marc, we picked up a few ideas about how to make the next version of VS SDK or MPF better. I thought them over several times and decided to create some demonstration patterns to show how a new version of MPF could make our development tasks easier. Here and the future posts I share these ideas with you.

Declarative approach

Using declarative programming—wherever possible—is a good practice by my opinion. The .NET framework intensively uses metadata about types and members and we can add custom metadata by decorating declarations with attributes. The reflection model makes it easy to access this information and we can use it for our own purposes.

The Managed Package Framework demonstrates this approach with registration attributes decorating the package class. During the build process the RegPkg.exe utility scans for these attributes and carries out all the registrations declared by them. Why does not use MPF this pattern for more elements where this approach could work as well? Let me share some imaginations about where it could be useful.

·       Declarations could be used to bind commands with their command handlers. I have also treated this idea in the following post: LVN! Sidebar #4 - Command handlers. Pablo Galiano used the same technique with a slightly different implementation in his PowerCommands utility, I treat it here: PowerCommands Deep Dive — Command Architecture.

·       Objects owned by a package (like tool windows, custom editors, services, etc.) can be decorated with attributes to set their initial parameters. For example we can set the initial caption or the associated bitmap of a tool window; we can flag a service as a global one; bind a tool window with a command that displays it, etc.

·       We can implement IOleComandTarget interface in a way that instead of using switch statements for dispatching commands, we can use simple methods decorated with the ID of the command they handle.

These are only a few examples of how declarative approach could be used. I like the declarative approach not because of its “trendiness” but for the fact declarative things can be easily supported by reflection or by such tools like a DSL designer.

Do we need VSCT file at all?

I often think about why and how we need the VSCT file. No doubt, VSCT is a much better format then the former CTC file, but in its state is yet not as easy to use as it could be. Only one of issues is the editing of the VSCT file. Difficulties related to searching for GUIDs and commands IDs declared by the Visual Studio Shell or difficult understandability can be removed by a good visual editor. However, the fact that command IDs have to be written both in the VSCT file and in the source code means redundancy and is a potential source of issues.

I can imagine a way where the command table information currently put in the VSCT file can be declared by .NET metadata. In this case the build process would simply extract the metadata of the assembly and register the command table in the format required by Visual Studio. This approach would allow avoiding redundancy of IDs and issues coming from mistyping them and also directly binding the command handlers with the appropriate command resources.

Of course, a visual editor also could add value to the development process in this case.

Using up-to-date .NET features

Examining the objects and the interfaces offered by the Managed Package Framework we can see that not only coolest .NET features but also some other useful .NET stuff is unused. One thing I really miss is generics. Using generic types would it make much easier to handle stuffs like the GetService call (Pablo Galiano used a nice pattern in PowerCommands), creating tool windows, custom editors, etc. You can get a few ideas about what I mean when reading LVN! Sidebar #3 - Simplifying tool window declaration or LearnVSXNow! #17 - Creating a simple custom editor — under pressure.

In those articles I show you a few examples how great it could be to use generics for MPF classes. In these cases we derive a few types from MFP classes to introduce generic behavior. If ancestor classes would be generic ones by themselves we could do even more.

Not only MPF types miss generics, but MPF itself does not leverage on generic types! For example, you can find intensive use of HashTable and ArrayList in the MPF source code.

The new C# 3.0 features and using LINQ also could add value to MPF. A few examples can be seen in PowerCommands, like here: LearnVSXNow! #20 - PowerCommands Deep Dive — Commands and UI and Pablo’s LinqToCodeModel.

Where we are?

I suppose, not I am the only one who can imagine a VS SDK that can be used better than the one we have today. In this post I enumerated only a few pains about it. I could carry on with this list—tools, issues with registration and cleanup, deployment issues, etc.

I treated only a few ideas about how to make the next VS SDK version better. My intention was not to create an exhaustive list of issues and suggested features but rather to start some kind of “brainstorming” about ideas and concepts to inspire the VSX community.

I started to work out some patterns and to refactor the current source of VS SDK. I do not aim to recreate all the stuff there but to demonstrate how those could make VS SDK better especially at the entry level. In the next few articles I try to show what results I have.

Please, share your imaginations, comments, additions and opinion with me!