Moved, again

March 3rd, 2014

Find me at http://www.hammettblog.com/

But yes, I’m keeping this one, frozen, for historical purposes since some content might be useful for some. But the theme and wordpress’ plugins are broken beyond repair.

Like Eric Cartman used to say ‘screw you guys, I’m going home’ :-)

Categories: General | Top Of Page | No Comments » |

MR3 – *very* high level overview

February 26th, 2012

Picture = thousands of words, right?

Castle.Extensibility

February 25th, 2012

One of the good things – of a few – of my tenure at the CLR team was being exposed to the metal. One of my interest areas was the loader, PM’ed by one of the most arrogant person I’ve ever met. Fortunately the senior dev in charge of it is an awesome person, and helped me with some explorations I was doing on my own, which I’ve blogged before. At that time, pre nuget, I pitched an idea to the asp.net leadership team of using runtime packages, similar to win8′s deployment system and OSGi. The idea gained some traction, but in order to have a composable framework many things would have to change, starting with asp.net mvc. A very charismatic and public known person whom should remain unnamed blocked the effort saying “no customers were asking for this”. Faulty logic since no customers were asking for cars, for iPads, for .net… Not to mention the “if-I-don’t-understand-it-must-be-a-hack” person, still in that team. So they settle for a simpler solution: Nuget.

Anyway, left MS but the lack of a framework that enforced a stronger modularity was an itchy to be scratched. In the end, the ultimate experience I wanted is to have an app shell that can be composed in runtime with additional functionality, in any level of granularity. To some extend, VS is like that. It’s a dumb shell, and the functionality we all know and enjoy is delivered through “packages”. Those themselves can be composed to augment in features and additional support plugins and vsix extensions (although it’s composed through COM aggregates, which is just sad)

However, Eclipse is far ahead by combining physical packages with isolated scoped: enters OSGi. In OSGi a bundle is a unit of deployment, versioning and execution. In contain all the artifacts required to its execution, including a declaration of its external world dependencies. In runtime, a class loader trick is used to load a bundle in an isolated context, so if you load a bundle that say, uses nhibernate 3.1, it won’t interfere with another loaded bundle that is using nhibernate 3.3. (no jar hell, and no remoting). This problem is also known to .net folks. Clashing of dependencies requires either recompilation of many dependencies or assembly binding redirects. Not fun!

Castle Extensibility aims to solve exactly this problem. Since it’s a complex space, I dont think I can fully explain how to use it or how it works in a single brief blog post. So I’ll save more details for later. It’s worth mentioning that MonoRail 3 is integrated with it, so it’s able to run several “copies” of itself in the same app domain, hosting then many isolated applications in a single web site. Also worth mentioned it’s being used on the stuff I’m working on.

Oh, and obviously it was built in F# :-)

Categories: Castle, OS | Top Of Page | 6 Comments » |

Help Emilie

January 13th, 2012

If you heard the story on radiolab, you know her. If you havent, then do it.

Categories: General | Top Of Page | No Comments » |

Using MEF scopes to support service overrides

December 28th, 2011

Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100

One of the coolest thing in the newest version of MEF is scoping support. Not because I’ve participated in designing and eventually patented some aspects of it – not at all – but because it’s actually quite helpful.

Scopes, as you can imagine, allow you to define isolated “bubbles” of composition.

On the components side, we use an ExportFactory (should have been ScopeFactory, but dont get me started on this one) with CreationPolicy.NewScope (ughh) to achieve this behavior. On the hosting side, you need to define an hierarchy of CompositionScopeDefinition and give it to the container.

One thing that is particularly handy in IoC containers is supporting default services, but allow for overrides. Windsor even use this taxonomy. The following is a trick that uses scopes to achieve the same behavior.

Before saying that it’s easier, simpler, faster to achieve that using Container X, Y or Z, remember that MEF is all about federated systems, and that comes with challenges and constraints.

Here’s the trick: I define three scopes: Default, Overrides and App. All default services are bound to the Default scope. The app level components live in the App scope. In the middle, the Override scopes will be used to replace default services.

 cs |  copy code |? 
01

02
    using System.ComponentModel.Composition;
03
    using System.ComponentModel.Composition.Hosting;
04
05
    class Program
06
    {
07
        static void Main(string[] args)
08
        {
09
            var ubber = new AssemblyCatalog(typeof (Program).Assembly);
10
11
            var defLevelCat = ubber.Filter(cpd => cpd.ContainsPartMetadata("Scope", "Default"));
12
            var overLevelCat = defLevelCat.Complement.Filter(cpd => cpd.ContainsPartMetadata("Scope", "Override"));
13
            var appLevelCat = overLevelCat.Complement.Filter(cpd => cpd.ContainsPartMetadata("Scope", "App"));
14
15
            var appDef = new CompositionScopeDefinition(appLevelCat, new CompositionScopeDefinition[0],
16
                                                        appLevelCat.Parts.SelectMany(p => p.ExportDefinitions));
17
            var overDef = new CompositionScopeDefinition(overLevelCat, new [] { appDef },
18
                                                         overLevelCat.Parts.SelectMany(p => p.ExportDefinitions));
19
            var defDef = new CompositionScopeDefinition(defLevelCat, new[] { overDef },
20
                                                        defLevelCat.Parts.SelectMany(p => p.ExportDefinitions));
21
22
            var cont = new CompositionContainer(defDef);
23
24
            var entryPoint = cont.GetExportedValue<EntryPoint>();
25
            var sndLevel = entryPoint.Creator.CreateExport().Value;
26
            var lastLevel = sndLevel.Creator.CreateExport().Value;
27
            var component = lastLevel.com;
28
29
        }
30
    }
31
32
    [Export]
33
    [PartMetadata("Scope", "Default")]
34
    public class EntryPoint
35
    {
36
        [Import(RequiredCreationPolicy = CreationPolicy.NewScope, AllowDefault = true)]
37
        public ExportFactory<OverrideLevelScopeCreator> Creator { get; set; }
38
    }
39
40
    [Export]
41
    [PartMetadata("Scope", "Override")]
42
    public class OverrideLevelScopeCreator
43
    {
44
        [Import(RequiredCreationPolicy = CreationPolicy.NewScope, AllowDefault = true)]
45
        public ExportFactory<AppLevelScopeCreator> Creator { get; set; }        
46
    }
47
48
    [Export]
49
    [PartMetadata("Scope", "App")]
50
    public class AppLevelScopeCreator
51
    {
52
        [Import]
53
        public Component com { get; set; }
54
    }
55
56
    public interface IService
57
    {
58
    }
59
60
    [Export(typeof(IService))]
61
    [PartMetadata("Scope", "Default")]
62
    public class DefaultService : IService
63
    {
64
    }
65
66
    [Export(typeof(IService))]
67
    [PartMetadata("Scope", "Override")]
68
    public class OverrideService : IService
69
    {
70
        [Import(Source = ImportSource.NonLocal)]
71
        public IService ServParent { get; set; }
72
    }
73
    
74
    [Export, PartMetadata("Scope", "App")]
75
    public class Component
76
    {
77
        [Import]
78
        public IService Serv { get; set; }
79
    }
80

Have fun!

Categories: .NET, MEF | Top Of Page | 2 Comments » |

A “decent” DirectoryCatalog implementation

December 27th, 2011

Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: file(http://svn.wp-plugins.org/devformatter/branches/langs/cs.php): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100 Warning: implode(): Invalid arguments passed in C:\websites\castle\blogs\hammett_castleblog\wp-content\plugins\devformatter\devgeshi.php on line 100

Yes, I’ve raised this with the MEF team many times. It’s easier to go and do it yourself.

The issue is that DirectoryCatalog does not guard against failures on loading types (which is quite common as soon as you start working on real projects as opposed to samples – hint hint!). So here’s a better version (in F#):

 cs |  copy code |? 
01
02
    type DirectoryCatalogGuarded(folder) = 
03
        inherit ComposablePartCatalog()
04
05
        let _catalogs = List()
06
        let mutable _parts : ComposablePartDefinition seq = null
07
08
        let load_assembly_guarded (file:string) = 
09
            try
10
                let name = AssemblyName.GetAssemblyName(file);
11
                Assembly.Load name
12
            with | ex -> null
13
14
        let guard_load_types (asm:Assembly) =
15
          try
16
              asm.GetTypes()
17
          with | :? ReflectionTypeLoadException as exn -> exn.Types
18
19
        do 
20
            let files = Directory.GetFiles(folder, "*.dll")
21
            for file in files do
22
                let asm = load_assembly_guarded file
23
                if asm <> null then
24
                    let types = guard_load_types(asm) |> Seq.filter (fun t -> t <> null) 
25
                    if not (Seq.isEmpty types) then
26
                        _catalogs.Add (new TypeCatalog(types))
27
            _parts <- _catalogs |> Seq.collect (fun c -> c.Parts) 
28
29
        override x.Parts = _parts.AsQueryable()
30
31
        override x.GetExports(definition) = 
32
            _catalogs |> Seq.collect (fun c -> c.GetExports(definition))
33

The “trick” is to catch ReflectionTypeLoadException – something I learned from David. This exception gives you a list of types it was able to load. Mind you, nulls may exists in the list hence the filtering.

The shortcoming here is that by bypassing the AssemblyCatalog, you wont be able to use CatalogReflectionContextAttribute – which I believe I’m one of the ten people on earth that knows what it’s for, so no biggy I guess?

Categories: MEF | Top Of Page | 2 Comments » |

FAKE impressions

August 8th, 2011

Weird post title, I know. But I refer to FAKE – F# Make.

It’s a nice and simple library to perform usual build related tasks. The fact that it’s pure F# makes it quite powerful – have you tried doing complex stuff with NAnt or MSBuild? If so, you know what I mean. Xml is *not* a programming language.

FAKE is simply organized as a collection of modules, each exposing related functions and|or operator overloads. So it’s not exposed in an OOP fashion, rather in a very flat public surface. That *may* make up for the lack of documentation. It’s somewhat easy to understand how things work by browsing the source – for those who are comfortable with that.

That said, in my first experience I couldn’t get the FileSet to generate any result. I also couldn’t find a way to turn on a more verbose logging. I went on a trial-and-error and realized the fileset was missing a BaseDir setting.

Since I’m a big fan of F# Project Extender, which adds metadata to define the order of the build items that are not recognized by MSBuild. So I managed to create a FscTask that takes that in consideration. In the process I noticed how FAKE lacks a few common NAnt features, such as resolving reference assemblies. I guess the author didn’t need them, which is fair in the OSS world. That said, a simple Fsc.exe invocation turned into a sizable task.

My fsc task is dirty and ugly, so beware. You can find it here. For now I’m sticking to Fake for MR builds and see how it goes.

Categories: .NET, F# | Top Of Page | 2 Comments » |

MR 3 update / Castle Blade

July 18th, 2011

Fun times! Since I’ve left MS I’ve been working on a real web site/backend/frontend – you know, doing actual work instead of long meaningless meetings, doc writing and one-on-ones with managers setting you up for failure (I wonder if I should make one of my ‘reviews’ public, so you’ll see the amount of BS I had to put up with).

WebSockets

Anyway, the first work was to create a websockets server. That was fun especially since F# offers its async workflow support and MailboxProcessor. It’s fun to think about it. While at MS, one of my friends there wouldn’t stop bitching about how F# is incredibly superior to C#. Gosh, he was right..

The company I’m doing the work for and I are discussing whether we will make the websocket server available commercially. It’s a PITA to implement the protocols, to test them and keep up, since the spec is in progress. So, sorry, won’t be doing all this work for free.

Ideas playground

On the MonoRail end, we’ve been using it extensively and coding up functionality as needed – by we I mean Henry and I. It’s been quite a journey to reassess all the design decisions in MR1/2, and also evaluate what’s out there. I’m sure we learn more from our mistakes than anything else. Mauricio is pushing some interesting ideas on web frameworks that are making me reevaluate our proposed API over and over again. Now that I’m quite familiar with FParsec, the idea of using combinators to compose forms (formlets) is enticing. It’s just strike me as something hard to expose in C#, we would need a different API and I’m sure it would be awfully verbose.

Castle.Blade = ++Razor

I love Razor’s simplicity and tooling support. Achieving simplicity is a major accomplishment, and the guys at the ASP.NET team did it. I remember when Scott Hunter gave us a preview of what at the time was code named Plan9. Awesome work!

I’ve then decided to make Razor the main view engine for MR3. However, when I started to experiment with a better API for our helpers library I bumped into some not-so-nice limitations from Razor. For example, I wanted to be able to express something like:

 Rails |  copy code |? 
1
2
@Form.For(..., {
3
	// this is a block
4
        @builder.EditorFor(m => m.Name)
5
})
6

Well, Razor does support translating a mix of content and code block into a delegate – which is neat. The limitation is that they cannot be nested, and the parameter name is restricted to “item”

 Rails |  copy code |? 
1
2
@Form.For(..., @{
3
        @item.EditorFor(m => m.Name)
4
})
5

The issue is that “item” is not very expression. I’ve spent many hours digging into Razor’s code trying for find a way to work-around this limitation (by using its extension points, not changing their code). At some point it was clear that coding up my own parser and translator would be easier. Castle Blade then came to fruition.

Blade intends to be 100% backward compatible with Razor, and introduces a few (one?) different transition marks to overcome Razor’s limitations. For the example above, we would use:

 Rails |  copy code |? 
1
2
@Form.For(..., @=> builder {
3
        @builder.EditorFor(m => m.Name)
4
})
5

The @=> transition signals that a delegate will be created for the block, and the parameter name is the one that follows. In theory more than a single parameter is supported.

We also support nested blocks, which allows for the something like the following:

 Rails |  copy code |? 
01
02
@Form.For(..., @=> builder {
03
        builder.FormTemplate(@=> t {
04
                <div>
05
                    @t.Label(): @t.Field()
06
                </div>                                                      
07
            });
08
 
09
        <fieldset id="contactForm">    
10
 
11
        @builder.TemplateFor( m => m.Name )
12
        @builder.TemplateFor( m => m.Email )
13
 
14
        </fieldset>
15
})
16

Feel free to give it a try.

MonoRail+++

MonoRail’s 3 goal is based on our my experience and perception of the “state of the union” and trends. If I could put them in three simple statements:

I’m mentally tired of crafting web sites despite huge functionality overlaps (combine/compose). I’m tired of REST being an afterthought to existing websites (rest support from the beginning). I’m tired of frameworks created by people without *actual* website building experience (frictionless).

The goals/roadmap/value-proposition were discussed in the past in our development list.

  • Since our underlying runtime (CLR) is keen on static typing then fully embrace it
  • Move forward: embrace HTML 5
  • Simplify special render for different form factors
  • Strive for simplicity, but no simpler

I’ll dive into what’s been done in practice to address each of the above bullets in upcoming blog posts.

Exposing extension methods in F#

April 13th, 2011

Thanks for Quora + Reflector + MSDN, I’ve figured out how to expose extension methods from F#

  1. Create a public module within a namespace
  2. Add the extension method as a function
  3. Decorate both the module and the function(s) with the ExtensionMethodAttribute

 C# |  copy code |? 
01
02
namespace Castle.MonoRail
03
 
04
[<System.Runtime.CompilerServices.ExtensionAttribute>]
05
module public ExtensionMethods = 
06
 
07
    open Castle.MonoRail.Routing
08
    open Castle.MonoRail.Hosting.Mvc
09
 
10
    [<System.Runtime.CompilerServices.ExtensionAttribute>]
11
    let Match(router:Router, path:string) = 
12
        router.Match(path, MonoRailHandlerMediator())
13

Categories: .NET, F# | Top Of Page | 2 Comments » |

Using Loading contexts effectively

March 24th, 2011

Long long time ago I promised a few people in my twitter that someday I’d post somewhere a sample on how to deal with “dependency hell”.

By that I mean something you’ve probably experienced yourself. Suppose you’re happily using log4net, and then you start using NHibernate which happens to use a different version of log4net. Hmmm. Easy. You can switch your own copy to the NHibernate uses. Then you add another dependency to your project, say SupperCoolWidget, and it happens to use yet another version of log4net.

You can rely on binding redirects, as long as you’re damn sure the API surfaced touched by these projects haven’t changed on the dependency (in this case log4net), otherwise you’ll get exceptions in runtime (cannot find member).

IMHO it’s especially problematic to rely on binding redirects because some code, somewhere, isn’t used very often, and in some special circumstance it may try to use an API that isn’t there.

log4net is an interesting example, but the problem applies to any scenario where you have common dependencies in different versions.

Looking at another camp – Java in particular – OSGi brought some interesting ideas to this very situation. There it’s even worse since jars are way more loose than assemblies. OSGi’s solution is to either have independent versioned bundles which your bundle may explicit say it depends upon, _or_ your bundle carries everything it needs to work. Multiple bundles can be loaded and executed in a single VM, and they are guarantee to not step on each others foot.

Java’s enabler to this magic is the ClassLoader.

In .NET there isn’t a concrete equivalent of the Class loader, but we have a loader. And it has different contexts. In fact, as many as you want. The Load and LoadFrom contexts are the typical ones you’re exposed to. More resources: Choosing a Binding Context and LoadFile vs LoadFrom

By using a combination of the right loading context and the AssemblyResolve event, you can achieve the behavior of isolated silos loading the same assemblies (with different versions) in the same AppDomain.

I created a sample to demonstrate the idea and you take it from there. I’ve tried to minimize the concepts, so no MEF, no Windsor, and it’s not a web app. The file structure is like the following

Capture

The build folder contains the app, which doesn’t do much:

private static readonly CustomBinder _binder = new CustomBinder();

static void Main()
{
    var curDir = AppDomain.CurrentDomain.BaseDirectory;

    // Each module is loaded in its own isolated context
    // so they can have conflict dependencies and work
    var modules = LoadModules(_binder, curDir);

    foreach (var module in modules)
    {
        Console.WriteLine(module);
    }
}

So it loads “modules” in a kind of late bound way, using a well-known contract: IModule.

Each module implementation depends on – guess what – log4net. But different versions of it. Each implementation of IModule looks pretty much the same, but the dependency version is quite different:

namespace FakeMod1
{
    using WellKnownContracts;

    public class Mod1Impl : IModule
    {
        private static log4net.ILog logger = log4net.LogManager.GetLogger(typeof(Mod1Impl));

        public Mod1Impl()
        {
            logger.Info("constructed");
        }
    }
}

namespace FakeMod2
{
    using WellKnownContracts;

    public class Mod2Impl : IModule
    {
        private static log4net.ILog logger = log4net.LogManager.GetLogger(typeof(Mod2Impl));

        public Mod2Impl()
        {
            logger.Info("Mod2Impl constructed");
        }
    }
}

When we run the app we expect the following to happen

  • module 1 is found
  • A logical context is created for it
  • Each dependency within the module 1 is satisfied within the boundary
  • module 2 is found
  • ditto ditto..

Running the app and watching the debugger confirms the expected behavior:

‘ParallelContexts.vshost.exe’: Loaded ‘ParallelContexts.exe’, Symbols loaded.

‘ParallelContexts.vshost.exe’: Loaded ‘WellKnownContracts.dll’, Symbols loaded.

‘ParallelContexts.vshost.exe’: Loaded ‘modules\mod1\FakeMod1.dll’, Symbols loaded.

‘ParallelContexts.vshost.exe’: Loaded ‘modules\mod1\log4net.dll’

log4net:ERROR No appenders could be found for logger (FakeMod1.Mod1Impl).

log4net:ERROR Please initialize the log4net system properly.

‘ParallelContexts.vshost.exe’: Loaded ‘modules\mod2\FakeMod2.dll’, Symbols loaded.

‘ParallelContexts.vshost.exe’: Loaded ‘modules\mod2\log4net.dll’

Notice that WellKnownContracts.dll isn’t loaded more than once, since it first loaded in the Load context, it’s always found – I’m a bit unsure if this one is even probed after it’s loaded for the first time.

 

How it works?

The code is simple. The class CustomBinder takes a “module folder”, and starts a new logical context for it.

public partial class CustomBinder : IDisposable
{
    ...


    public BindingContext Add(string modFolder)
    {
        var ctx = new BindingContext(this);
        var files = Directory.GetFiles(modFolder);

        string entryPointFromManifest = null;

        foreach (var file in files)
        {
            // manifest has an entry point which is the first type/assembly we load
            // this is just an optimization, so we dont have to load all assemblies found within a package/module
            if (Path.GetFileName(file).Equals("manifest.xml", StringComparison.InvariantCultureIgnoreCase))
            {
                entryPointFromManifest = GetEntryPointFromManifest(file);
                continue;
            }

            if (!file.EndsWith(".dll")) continue;

            var name = AssemblyName.GetAssemblyName(file);
            ctx.AddAssemblyName(name.Name, file);
        }

        if (entryPointFromManifest != null)
        {
            string[] split = entryPointFromManifest.Split(',');
            Debug.Assert(split.Length == 2);
            ctx.EntryPointTypeName = split[0];
            ctx.PreLoad(split[1].TrimStart());
        }

        return ctx;
    }

Then, whenever the loader probes for an assembly, we use the “requesting assembly” to bring the existing context back, and use it to load the right assembly.

private Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
{
    if (args.RequestingAssembly == null)
        return null;

    BindingContext ctx = GetBindingContext(args.RequestingAssembly);
    if (ctx == null) return null;

    Assembly assembly;
    if (ctx.TryGetAssembly(new AssemblyName(args.Name), out assembly))
    {
        return assembly;
    }

    return null;
}

As I mentioned before, this is just a proof of concept that shows what is possible. The sky is the limit for modular/composable frameworks out there. Enjoy!

Download sample