Compiler extensions is just another framework

Over the years I’ve had several issues where I’ve longed for ways to better express my code that I felt that the language should have exposed grammar for me to make it more readable. Just like LINQ introduces an internal DSL, I’ve had quite a few cases were I also wanted to introduce my own internal DSL. Very often one ends up with working around it and do creative frameworks or libraries that helps doing the same thing without really solving the expressiveness.

I come to realize more and more that the programming language one chose is less important than the frameworks one use. As long as one can express the code the way one wants and perhaps even in a suitable manner for the domain you’re working in, the choice of language is really not important.

Attributes
Attributes in C# are great for expressiveness, they are great at providing additional meta information for classes, fields and properties. Often I find myself using attributes for expressing business rules and having evaluation code for classes that recognise the attributes and then validates according by calling the different validation attributes applied. In my opinion, this is a bit unnatural. A lot of the attributes found in the .net framework acts as more than just meta information at compiletime. The compiler recognizes a whole bunch of the framework attributes and alters the behavior of the execution accordingly.
Ofcourse, everything that could be solved at compiletime could certainly also be solved at runtime, but it affects runtime speed and scalability.

Modifying IL code
PostSharp is a project that solves this; one can alter the IL code generated after compilation and then commit it to the assembly. Attributes is a part of the language and with PostSharp we could solve this problem and we don’t need to extend the compiler.

Plumbing
Another good place were you’d might look at having compiler extensions are the places were you are basically just plumbing to make things work with how a framework wants it to work. A good candidate would be for instance the INotifyPropertyChanging and INotifyPropertyChanged interfaces were you basically are just doing the same code over and over again for all your properties. With a compiler extension you could plumb all this automatically.

Typically you would do the following:

[code:c#]
public class MyNotifyingObject : INotifyPropertyChanging
{
    public event PropertyChangedEventHandler PropertyChanging;

    protected virtual void OnPropertyChanging(string propertyName)
    {
       if( null != this.PropertyChanging )
       {
          this.PropertyChanging(this,new PropertyChangedEventArgs(propertyName));
       }
    }

    public string Name
    {
       get
       {
          return this._name;
       }

       set
       {
          if ((this._name != value))
          {
             this.OnPropertyChanging(“Name”);
             this._name = value;
             this.OnPropertyChanged(“Name”);
          }
       }
    }
}
[/code]

A more expressive way could be:

[code:c#]
public class MyNotifyingObject
{
    public notify string Name { get; set; }
}
[/code]

If you ever worked with WPF or Silverlight, you know the tedious parts where you want to have DependencyProperties on your controls. The syntax is like this:

[code:c#]
public static readonly DepedencyProperty MyStringProperty =
                  DependencyProperty.Register(“MyString”, typeof(string), typeof(ParentType), null);

public string MyString
{
     get
     {
           return this.GetValue(MyStringProperty);
     }
     set
     {
           this.SetValue(MyStringProperty,value);
     }
}
[/code]

Wouldn’t it be a lot more expressive if we could just do:

[code:c#]
public dependent string MyString { get; set; }
[/code]

The way a compiler extension would handle this, is to fill in all the code found above to make it all work like if you were to write it all. We would get a more expressive code and actually get rid of potential problems with the literal declaring the name of the property as we do today.

C# 4.0

During PDC there’s been a bit talk about C# 4.0 and further down the pipeline, Anders Hejlsberg talked about the new static type called dynamic, that will help us out getting more dynamic. I for one has been quite the skeptic over the years about dynamic languages, but I must admit that in many cases it is really convienient. My conclusion is that this is a really great step forward for the language.

Boo
The programming language Boo is a static language that can be dynamically altered by introducing new keywords into the language through compiler extensions. In addition to the compiler they’ve created a small framework that enables the developer to take part of the compilation and modify the AST. This feels very great and gives us developers the power we need to really express our code and give our programming model a domain specific feel. A colleague of mine; Tore Vestues blogs about Boo and all its glory. Have a look here.

Boo is great, but has a couple of obstacles before it will be embraced as a mainstream programming language by a lot of developers; syntax is one – even though I claim that the language shouldn’t really matter, it sort of does. If you are familiar with the C syntax of C++,Java,C#, JavaScript, then the chances are that you’d feel that swithching to a new language is too big and you fall back. So, what about C# – if this is so great, why aren’t we seeing this in C#? It is for sure, technically possible. At PDC there was a great panel on the future of programming languages where this among a ton of subjects was discussed. My impression was that Anders Hejlsberg was quite clear on this subject; they didn’t want to open it all up because of the danger of devolving the language. I think this is really sad, I really hope they turn around on this. After all, as the title of this post claims; it is basically just another framework. By extending the language, and doing it like Boo has done it, wouldn’t pose a threat to the language. The extensions one would build would in many cases be solving a domain specific problem and just act as just another framework one used to solve that problem.

Compiler Extensions – how
A compiler extension, would need access to the AST tree created during compile time and based upon the code and any extensions found extend the existing tree. Boo solves this in
a very graceful manner, the extensions are just part of your code as you compile or you can put them in a component. It is really not that intrusive as it might sound. We’re not talking about creating a whole new version of the language, rather than introducing domain specific problem-solvers or aiding in creating more expressiveness. In addition to compiler extensions, one need to hook into the IDE to get rid of all the errors one would get thrown in your face while writing code.

14 thoughts on “Compiler extensions is just another framework

  1. Gøran says:

    Hi Einar

    Interesting post. It’s great to see more people opening up for new techniques like compiler extensions and languages. I’m one of the suspects that have been playing around with Boo for a while, and I like it. And the one and only reason for that is the compiler extensions, because it enable meta-programming.

    This is a interesting topic that I care a great deal about, so I leave a few comments. My comments reflect my thoughts more than questioning the content of your post.
    I admit, it would be nice if C# and Anders would have opened up the compiler for us, but we don’t need him to. We as .NET developers will in the future have lots of languages to leverage upon, and if C# want to be a strict language – let it be. We can still move around using other framework and languages on the .NET platform, for instance like Boo. The problem with Boo is that it’s still a young project and there are lots missing, like documentation, good tools and as you said the syntax is not in the C-family. In .NET 4, DLR will be backed into the platform, and IronRuby will probably be shipped. Ruby is one of those languages that really enables us to create internal DSLs and do meta-programming. With IronRuby they are probably going to ship good tools, like Visual Studio which gives us IntelliSense, Refactoring etc. And when we are at this point, I don’t think it will be too hard to sell IronRuby to the customers.

    At the end of the day, these languages and platforms are built upon .NET, which enable us to use these languages together at some level. One of the things I’m working on with Boo at the moment is a DSL to help me create user interface applications faster using Silverlight/WPF and C#. I’m using Boo and the compiler extensibility to create the DSL which then will be used to output a .NET assembly. Then I reference this assembly and use it in the Silverlight/WPF project where I use C# as the language. At the moment the project is on the Proof of Concept stage. But I think it’s a good example of how we can combine different languages based on .NET and use the output from them together.

    I’m glad Anders and Microsoft are strict about what they put into the C# language and that they don’t have the one tool to rule them all mentality. C# is a great language in many scenarios, but it’s not always the right tool for the job.

    Great post by the way! I’m glad to see that it came from a fellow Norwegian .NET developer! I hope this will start a conversation in the Norwegian community about meta-programming and compiler extensions.

    Cheers

    Gøran

  2. Gøran says:

    Hi Einar

    Interesting post. It’s great to see more people opening up for new techniques like compiler extensions and languages. I’m one of the suspects that have been playing around with Boo for a while, and I like it. And the one and only reason for that is the compiler extensions, because it enable meta-programming.

    This is a interesting topic that I care a great deal about, so I leave a few comments. My comments reflect my thoughts more than questioning the content of your post.
    I admit, it would be nice if C# and Anders would have opened up the compiler for us, but we don’t need him to. We as .NET developers will in the future have lots of languages to leverage upon, and if C# want to be a strict language – let it be. We can still move around using other framework and languages on the .NET platform, for instance like Boo. The problem with Boo is that it’s still a young project and there are lots missing, like documentation, good tools and as you said the syntax is not in the C-family. In .NET 4, DLR will be backed into the platform, and IronRuby will probably be shipped. Ruby is one of those languages that really enables us to create internal DSLs and do meta-programming. With IronRuby they are probably going to ship good tools, like Visual Studio which gives us IntelliSense, Refactoring etc. And when we are at this point, I don’t think it will be too hard to sell IronRuby to the customers.

    At the end of the day, these languages and platforms are built upon .NET, which enable us to use these languages together at some level. One of the things I’m working on with Boo at the moment is a DSL to help me create user interface applications faster using Silverlight/WPF and C#. I’m using Boo and the compiler extensibility to create the DSL which then will be used to output a .NET assembly. Then I reference this assembly and use it in the Silverlight/WPF project where I use C# as the language. At the moment the project is on the Proof of Concept stage. But I think it’s a good example of how we can combine different languages based on .NET and use the output from them together.

    I’m glad Anders and Microsoft are strict about what they put into the C# language and that they don’t have the one tool to rule them all mentality. C# is a great language in many scenarios, but it’s not always the right tool for the job.

    Great post by the way! I’m glad to see that it came from a fellow Norwegian .NET developer! I hope this will start a conversation in the Norwegian community about meta-programming and compiler extensions.

    Cheers

    Gøran

  3. Compiler extensions in dynamic languages are definitively cool. However, I don’t think either of your examples really qualify for the need of them – they could easily be implemented today using f.ex Resharper Live Templates (and expressions to get around the issue with the literals). Plus just having

    I think there’s a danger here of getting too caught up in the trend of dynamic languages that its easy to forget why we’ve been so adamantly making things strongly typed these last few years.

    Besides, extending the compiler is not enough – you need to extend the IDE aswell. I’m not giving up my intellisense just yet… :p

    That said, when we’re talking about using compiler extensions for creating domain specific languages to help express the ubiquitous language of the business domain, THAT gets my cowbells a-ringing! 🙂

  4. Compiler extensions in dynamic languages are definitively cool. However, I don’t think either of your examples really qualify for the need of them – they could easily be implemented today using f.ex Resharper Live Templates (and expressions to get around the issue with the literals). Plus just having

    I think there’s a danger here of getting too caught up in the trend of dynamic languages that its easy to forget why we’ve been so adamantly making things strongly typed these last few years.

    Besides, extending the compiler is not enough – you need to extend the IDE aswell. I’m not giving up my intellisense just yet… :p

    That said, when we’re talking about using compiler extensions for creating domain specific languages to help express the ubiquitous language of the business domain, THAT gets my cowbells a-ringing! 🙂

  5. einar says:

    Thanks for the feedback guys..

    I see real business value in the power of having your DSL introduced into the language.
    My samples above was not necessarily the best samples to prove the concept. But I guess you already know what I mean.

    As for the extending the IDE, this is probably the thing that is the most complicated. Intellisense is one thing, another thing is refactoring.

    In a perfect world, where the compiler is a service as Anders was talking about at PDC for C# 5.0 or later. The service could expose functionality for retrieving the "current kewords" and such and all other services could then rely on this. It feels a bit strange that this hasn’t already been done. Every IDE feature has its own little compiler to parse whatever is going on in the code to be able to provide functionality like refactoring or intellisense.

    It is a very interesting subject in which I will devote more time to, to see what can be done with C# especially.

    Anders and the C# team has also got more reasons than technical to lock down the language. There is the motivation of having control. But I think one should really turn the question around a bit, because we’re not really talking about extending the language as a global extension, rather than providing domain specific alterations at compiletime for specific projects.

    This is probably not the last time I visit this subject.. 🙂

  6. einar says:

    Thanks for the feedback guys..

    I see real business value in the power of having your DSL introduced into the language.
    My samples above was not necessarily the best samples to prove the concept. But I guess you already know what I mean.

    As for the extending the IDE, this is probably the thing that is the most complicated. Intellisense is one thing, another thing is refactoring.

    In a perfect world, where the compiler is a service as Anders was talking about at PDC for C# 5.0 or later. The service could expose functionality for retrieving the "current kewords" and such and all other services could then rely on this. It feels a bit strange that this hasn’t already been done. Every IDE feature has its own little compiler to parse whatever is going on in the code to be able to provide functionality like refactoring or intellisense.

    It is a very interesting subject in which I will devote more time to, to see what can be done with C# especially.

    Anders and the C# team has also got more reasons than technical to lock down the language. There is the motivation of having control. But I think one should really turn the question around a bit, because we’re not really talking about extending the language as a global extension, rather than providing domain specific alterations at compiletime for specific projects.

    This is probably not the last time I visit this subject.. 🙂

  7. Tore Vestues says:

    Nice post, Einar.

    Personally I don’t see any good reason why C# should not include compiler extensibility. Actually, I see quite a few reasons why they should: It’s all about giving developers more power and that should always be a good thing. I’ve posted my thoughts on it at my blog.
    Some comments on the comments:

    I fear that Gøran’s dream that we could use the language we want (the right tool for the job) is just a dream. Remember that flexible code and the ability to change it is not only about code quality (although it is a big part of it). It’s also a matter of getting the people with the skills to work with the code. I wish every developer knew several languages well, but this is very seldom the case. Thus the safe bet is to go for C#, which "everybody" knows. Given less people master either Ruby or Boo, it will be harder to get competent people to work with the code after it has been created, and this sadly makes these languages less attractive for the customers.

    Fredrik, there is a major difference between compiler extensions and template/snippet-generated code. The point with compiler extensions is to hide the code completely. The real benefit here has to do with declarativeness, readability, simplicity and productivity. Using templates will at best have an impact on productivity; the code will look just as messy as before.

    And by the way, intellisense is nice, but note that in C# 4 you actually lose it when using the "dynamic"-type. So for some reason C# does actually not have intellisense for everything anymore.

  8. Tore Vestues says:

    Nice post, Einar.

    Personally I don’t see any good reason why C# should not include compiler extensibility. Actually, I see quite a few reasons why they should: It’s all about giving developers more power and that should always be a good thing. I’ve posted my thoughts on it at my blog.
    Some comments on the comments:

    I fear that Gøran’s dream that we could use the language we want (the right tool for the job) is just a dream. Remember that flexible code and the ability to change it is not only about code quality (although it is a big part of it). It’s also a matter of getting the people with the skills to work with the code. I wish every developer knew several languages well, but this is very seldom the case. Thus the safe bet is to go for C#, which "everybody" knows. Given less people master either Ruby or Boo, it will be harder to get competent people to work with the code after it has been created, and this sadly makes these languages less attractive for the customers.

    Fredrik, there is a major difference between compiler extensions and template/snippet-generated code. The point with compiler extensions is to hide the code completely. The real benefit here has to do with declarativeness, readability, simplicity and productivity. Using templates will at best have an impact on productivity; the code will look just as messy as before.

    And by the way, intellisense is nice, but note that in C# 4 you actually lose it when using the "dynamic"-type. So for some reason C# does actually not have intellisense for everything anymore.

  9. compile is a set of program.,..thanks for sharing your post and ideas i admire it !!i just want to say thank you very much!

  10. compile is a set of program.,..thanks for sharing your post and ideas i admire it !!i just want to say thank you very much!

  11. its was a nice frameworks , and actually works to me

  12. its was a nice frameworks , and actually works to me

Leave a Reply

Your email address will not be published. Required fields are marked *