Have ORMs introduced extra complexity into our codebase?

Whilst at DDDSouthWest 2011, I was speaking to Chris Hay about his scalability session. One of the areas we touched upon was the issues that ORMs have introduced into code.

ORMs have become very widely used. In my most recent work project we started using Entity Framework 4, we then migrated to Entity Framework Code First, when that was not working for us we moved away from Code First and stuck with EF4. We have had serious issues with our ORMs. Firstly, we found that our repository layer was causing CPU usage to spike to over 80%. This was due to queries being compiled on the fly each time the query was called. Secondly, when we removed our web service calls (to be replaced by a service layer), we found that the object hierarchy was causing all sorts of issues. Many of these were down to lazy design. The entities have the following look to them:

Drawing1

This meant that we had some hideous queries being created due to some navigation properties of the objects we were using. This is, of course, not just something found in EF4 but can be found in any ORM that is misused. The trouble with this type of scenario is that in order to fine tune the SQL that we (via the ORM) are producing via the ORM requires a change to the entities or their navigational properties. This means a lot of testing, recompilation and a redeploy.

What would be the alternative to an ORM? A DAL layer that called stored procedures and passed parameters to get data back? That felt very old school to me. I think that’s what drove me to start using ORMs initially.

So, if we cannot change the sql being produced easily, why do we use ORMs? It was described to me, by Andreas Håkansson (better known as TheCodeJunkie) as follows:

“DAL was born as a reaction to SQL, ORMs is an attempt to provide a DSL for writing DALs”

Mark Rendle suggested it was because of the need for intellisense.

To my mind, both are correct. This is clearly a way that enables developers to not have to deal with writing SQL. Now, old person hat on, in the olden days, a developer would have to be proficient in SQL if there were no DBAs around to take care of those things. This meant that a developer knew what was being done “under the hood”. Now, would it be fair to say that newer developers, born into a world of ORMs, actually don’t understand the SQL that they are producing?

Let’s think again about dealing with stored procedures. What would we do if we had to fine tune a stored procedure? Well if we are lucky, then we may not have to change input and output parameters and just change the logic of the procedure itself. This would mean that we could work on it without a recompilation and deployment. But if we have to change the inputs and outputs then we would need to change one layer, the DAL. This seems a lot more desirable to me. I am changing my code in one place which means I am a lot more confident.

Returning to Chris, he said he needs his SQL to be so finely tuned that it runs in milliseconds rather than seconds and the only way he can achieve this is to use stored procedures. This would allow the stored procedure to be worked on without the application being changed. This begs the questions: has the introduction of ORMs made developers lazy in their DAL layers? And: have they been too widely adopted – were they ever really meant for enterprise applications?

I can have a simple application up and running with a simple data structure and an ORM in a very short amount of time. If I take the same amount of time when setting up a so-called enterprise application, then my entities will have such bad relationships between them that the SQL produced is very bad. Oh wait – that’s the issue that has happened to me already………

Comments (13) -

Dave
Dave
8/15/2011 3:37:41 PM #

Nothing wrong with old school; a DAL that executes stored procedures and returns strongly typed object. The DAL can be auto-generated; T4 is good for that sort of thing. I wrote a custom tool yers before T4 was around that took a table and created the object and the basic CRUD methods and stored procedures; it was a good starting point.

Mark Rendle
Mark Rendle
8/15/2011 4:02:41 PM #

The other reason EF causes performance problems is that it does Unit-of-Work change-tracking on all the objects that go through it. If only there were a really Simple solution that didn't do all this stuff... ;)

Matt Roberts
Matt Roberts
8/15/2011 4:04:43 PM #

Interesting post Smile

The #1 reason I want to use ORMs is not because I want to abstract the SQL away, it's because I want something to populate my domain objects for me, and I don't want to write code to saturate my objects. I like what I can do with EF4.1 code-first, it's such a time saver to be able to define classes and have them talking to the db in no time - but fair point, it's open to abuse. I know a lot of devs who never check the SQL they produce with their elaborate linq queries, and that makes me nervous.

I for one am much happier and productive in SQL than Linq (after all, SQL is made *FOR* databases), but I wouldn't want to go down a stored proc route - it was always far too time consuming in the days when I did that (admittedly we didn't do any fancy T4 templates), and also was a good place for lazy devs like me to hide business functionality.

Could micro-ORMs like Dapper ever take off in the enterprise?





stack72
stack72
8/15/2011 4:06:15 PM #

@Mark Rendle would that simple solution be Simple.Data? If you need to find more information on this "ORM" style framework then please visit http://en.wordpress.com/tag/simpledata/ Smile

stack72
stack72
8/15/2011 4:07:37 PM #

@Matt Roberts I think micro-ORMs could well be something we have to look at. Maybe there is a correlation between dev happiness with ORMs and the rise of the Micro-ORM framework

Kendall Miller
Kendall Miller
8/15/2011 4:18:02 PM #

For me with real-world applications the great benefits of ORMs are that they can bridge the compiler gap from the database to your code (so if you don't get a type right, you get a compile error early on instead of some bizzare error at runtime) and they can take over the vast amount of basic CRUD that goes on in a data model.  Then you can hand write queries & stored procedures for the really critical bits.  

However, using an ORM to try to avoid understanding RDBMS theory or SQL is a losing game. SQL is a fairly clean expression of set theory which is fundamental to scalable systems. In the end, you'll need to get how your ORM is going to map to SQL to know the difference between say committing changes each iteration in a loop and once at the end of the loop.

If you don't understand these fundamentals, an ORM can let you get much deeper into the swamp before your ignorance kills you.  

Ian Battersby
Ian Battersby
8/15/2011 5:58:36 PM #

Wise words from Kendall above and couldn't agree more; I would add that like any tool in a developers arsenal it needs to be used appropriately and with care, and you could argue that this can only be done with (at least) a basic understanding of what it is happening beneath the skin.

I have no particular lean to/away from an ORM, it for me very much lands in the category of "horses for courses" (and I would extend this beyond architecture/design into meeting commercial constraints); I am keen however that it's employed through transparent consideration and debate, and not purely through un-bounding enthusiasm of team member/s. Most importantly, as with any DAL, you want to make sure you can swap it out at a future date, perhaps thus acknowledging it's limitations/baggage is just another form of technical debt.

Tools like NHProf/EHProf are also your friends and provide the confidence those LEFT OUTER joins and nested queries don't sneak in; and for those like me who still like to write T-SQL it keeps us on our toes.

PS. I get ridiculed all the time for this, but I'd still use SPOIL if I thought the cap fits

Paul Stovell
Paul Stovell
8/16/2011 2:04:19 PM #

>> in the olden days, a developer would have to be proficient in SQL if there were no DBAs around to take care of those things

It's not about not knowing SQL - it's because SQL sucks.

SELECT *

DBA's will scream not to use that. So we get:

SELECT Column1, Column2, ... Column23

Then when we need the same entity, we use the amazing power of Clipboard Inheritance to do it all again:

SELECT Column1, Column2, ... Column23

Until we're asked to do pagination, where we need to earn a PhD on set theory to understand why T-SQL makes pagination so complicated.

Paul Stovell
Paul Stovell
8/16/2011 2:06:32 PM #

My point being 'enterprise developers', 9 times out of 10, have better things to do than copy and paste to do the simplest things with an outdated language (AKA SQL)

Guy Murphy
Guy Murphy
8/16/2011 2:18:41 PM #

If you absolutely must be throwing your objects into a backing store with abandon use an oodb. By this point in time they've been around for long enough to be regarded as safe.

If this gives you problems with reporting then decide whether you're building an application or a reporting engine. Reporting needs can be met without bending your application out of shape.

If you simply don't like relational systems, that okay, you have choices now. Use a document or XML database if you feel more comfortable with them... But if you're using a relational database then that's what you're using. Adding a whole layer to pretend that your relational store is really an object store is at best lazy and at worst dellusional.

Yes it adds complexity. Its a whole additional layer and in most cases an extra thrid party dependency. An additional layer between me and my data along with an extra dependency needs to meet needs other that laziness and dellusion to justify itself for me.

If you're knocking up a wee app for three local bakeries then yes, use an ORM. If you're developing for high load or high complexity then in my personal view you're going to regret an ORM bitterly.

Aaron
Aaron
8/17/2011 12:17:32 PM #

Based on your concerns, I would suggest giving netTiers a try.  It is a full ORM which generates SProcs as well.  Not just 'supports' stored procs but generates parameterized stored procedures.  I normally generate using the ServiceLayer option so I get a SOA.  I tried it because our DBA only permitted exec on sproc permissions, so the more 'dynamic' options were not possible.

IMHO: netTiers is a highly under recognized ORM.

Guy Murphy
Guy Murphy
8/17/2011 10:28:04 PM #

Thanks for the shout on netTiers, if nothing else the generated code was interesting to look over. Seems very nicely done. I liked what appeared to be a focus on code generation... Cheers.

DavidS
DavidS
10/17/2011 8:30:04 AM #

As a NHibernate newbie, I think the problem with ORM is the fact that it's yet another "framework" that we need to learn about, but are not given enough time to do so.

Hence developers will set up their entities and mappings in a less than efficient manner, resulting in less than optimal SQL being generated.

It's just like any other tool. You have to be given the appropriate training and time to learn how to best wield it.



Pingbacks and trackbacks (1)+

Add comment

  Country flag

biuquote
  • Comment
  • Preview
Loading

About Me

 

Web Developer. My most used framework is C# with MVC but use Webforms on occasion. Im an advocate of clean, maintainable code and am very passionate for what I do. Absolutely obsessed with Continuous Integration and how it should be used in every day development scenarios. Trying to move towards a system of Continuous Deployment
Follow Me on Twitter

Jetbrains Academy Member

 

MVB Blogger

 

Friends Of Redgate

Month List