Eager loading with cascaded associations

maiha : February 21st, 2006

I created a patch that removes limitations for depth of joining tables when we use eager loading. Now we can cascade associations as much as we want!

ticket #3913

Syntax (about :include option)

Now, the :include option recognizes the following objects.

  • a Symbol : the base table simply joins it.
  • a String : same as a symbol,
  • an Array : the base table joins those elements directly (same in traditional).
  • a Hash : the base table joins KEY part, and it joins VALUE part. (where VALUE is processed as same as :include options).

Example (result only hierarchy structure)

  • cascading association with one level (same in traditional)
1
2
3
4
5
6
7
8
9


>> Author.find(:all, :include=>:posts)
=> authors
     + posts

>> Author.find(:all, :include=>[:posts, :categorizations])
=> authors
     +- posts
     +- categorizations
  • cascading association with two levels
1
2
3
4
5
6
7
8
9
10
11
12


>> Author.find(:all, :include=>[{:posts=>:comments}, :categorizations])
=> authors
     +- posts
          +- comments
     +- categorizations

>> Author.find(:all, :include=>{:posts=>[:comments, :categorizations]})
=> authors
     +- posts
          +- comments
          +- categorizations
  • and more than two levels ...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15


# cascaded in three levels
>> Company.find(:all, :include=>{:groups=>{:members=>:favorites}})
=> companies
     +- groups
          +- members
               +- favorites

# although this is meanless, you can ping and pong :)
>> Author.find(:all, :include=>{:posts=>{:author=>{:posts=>...}}})
=> authors
     +- posts
          +- author
               +- posts
                    +- ...

It also has a feature that automatically aliases table name when conflicted. So we can use eager loading on same table names such as acts_as_tree.

1
2
3
4


>> TreeMixin.find(:all, :include=>"children")
=> mixins
     +- children

For developers

New three classes are introduced to ActiveRecord::Associations::ClassMethods in this patch.

  • JoinDependency : this manages dependency information for joining cascaded associations
  • JoinBase : this manages joining information about base table
  • JoinAssociation : this manages joining information about joined table

An instance of JoinDependency is created when 'findwithassociations' method is executed. And we treat it as joining information provider in the task. Let's see the mechanism concretely by using following code.

1
2


Author.find(:all, :include=>{:posts=>:comments})

Calls 'findwithassociations' method

with options set to '{:posts=>:comments}'

1
2
3
4
5
6


def find_with_associations(options = {})
  join_dependency = JoinDependency.new(self, options[:include])
  rows = select_all_rows(options, join_dependency)
  return join_dependency.instantiate(rows)
end

An instance of JoinDependency is created.

And then, dependency object is created.

1
2
3
4
5
6
7
8
9
10
11


>> join_dependency = JoinDependency.new(Author, {:posts=>:comments})
=> #<JoinDependency:
      @associations={:posts=>:comments},
      @base_records_hash={},
      @base_records_in_order=[],
      @joins=[
        #<JoinBase: ... >,         # this object knows that base table is presented by Author
        #<JoinAssociation: ... >,  # this object knows a reflection about ':posts'
        #<JoinAssociation: ... >,  # this object knows a reflection about ':comments'
      ]>

main instance methods are

  • associations : returns original include option
  • joins : returns an array composed of base table and joined tables
  • join_associations : returns same as joins except base table
  • instantiate : create instances by using result set from database
  • construct : control instantiating flow recursively by using join dependency
  • construct_association : actually instantiate object by using information of reflection

Construct finder sql

And then, We contruct finder query by using mainly following composed methods in 'selectallrows' method.

  • column_aliases : construct column fields with column name aliases
  • JoinAssociation#association_join : construct join table sentence with table name alias

Now we get following finder_sql.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15


SELECT authors.`id`       AS t0_r0,
       authors.`name`     AS t0_r1,
       posts.`id`         AS t1_r0,
       posts.`author_id`  AS t1_r1,
       posts.`title`      AS t1_r2,
       posts.`body`       AS t1_r3,
       posts.`type`       AS t1_r4,
       comments.`id`      AS t2_r0,
       comments.`post_id` AS t2_r1,
       comments.`body`    AS t2_r2,
       comments.`type`    AS t2_r3
FROM authors
LEFT OUTER JOIN posts    AS posts    ON posts.author_id = authors.id
LEFT OUTER JOIN comments AS comments ON comments.post_id = posts.id

Note that table aliased name is not same as prefix of column aliased name. This is because table aliased name would be referred by user with :order or :conditional sentences. I believe everybody prefers ':order=>"comments.id"' syntax than ':order=>"t3.id"'.

Result set

Now we can get result set from DB. It would be like this.

1
2
3
4
5
6


+-------+-------+-------+-------+--------------+-------+-------+-------+-------+--------------+----------------+
| t0_r0 | t0_r1 | t1_r0 | t1_r1 | t1_r2        | t1_r3 | t1_r4 | t2_r0 | t2_r1 | t2_r2        | t2_r3          |
+-------+-------+-------+-------+--------------+-------+-------+-------+-------+--------------+----------------+
|     1 | David |     4 |     1 | sti comments | hello | Post  |     6 |     4 | Special type | SpecialComment |
...

Instantiate from result set

'JoinDependency#instantiate(rows)' method create instances by using 'JoinDependency#construct' and 'JoinDependency#construct_association' methods. And we will get following objects.

1
2
3
4
5
6
7
8
9


[
  #<Post:0xb79b8d70
     @comments=[#<Comment:0xb79b84b0 @attributes={"body"=>"Thank you for the welcome", "type"=>"Comment", "post_id"=>"1", "id"=>"1"}>,
                #<Comment:0xb79b7f60 @attributes={"body"=>"Thank you again for the welcome", "type"=>"Comment", "post_id"=>"1", "id"=>"2"}>],
     @attributes={"body"=>"Such a lovely day", "title"=>"Welcome to the weblog", "type"=>"Post", "id"=>"1", "author_id"=>"1"}>,
  #<SpecialPost:0xb79b7754 @comments=[#<SpecialComment:0xb79b48d8
    ...
]

This part is most complex in my patch. To be honest, I've no absolute idea about which approach should I use, iterate? recursive? or some stuff? And I know this part is far from elegant code and would be a performance killer. So I hope someone rewrites it in near future.

TODO

I'd like to support options defined in included associations. In short, only macro and name information are used for cascading inclusion in current version. For example

1
2
3
4
5
6


class Post < ActiveRecord::Base
  has_one :special_comment, :include=>"posts", :conditions=>"..."
end

Author.find(:all, :include=>{:posts=>:special_comment})

:include and :conditions sentence of Post is ignored in this case.

Posted by maiha Filed in rails

8 Responses to “Eager loading with cascaded associations”

  1. Ryan Platte Says:
    February 21st, 2006 at 01:21 PM Yum!
  2. some1else Says:
    February 21st, 2006 at 03:32 PM I hope this gets approved asap, looks like great stuff!
  3. atmos Says:
    February 21st, 2006 at 04:54 PM double plus good
  4. procreate Says:
    February 21st, 2006 at 04:58 PM If you want this "approved asap," please install it, use it, abuse it, and comment your findings on the ticket (the url of which is at the beginning of this post). If it passes your test, a simple 'Works for me' is all that's needed!
  5. ssk Says:
    February 21st, 2006 at 05:57 PM good
  6. Eleo Says:
    March 10th, 2007 at 11:18 AM

    What if the tree is of indeterminate depth and I want to eager load a tree’s children and it’s children’s children, etc? You said something along the lines of :include=>{:posts=>{:author=>{:posts=>…}}}) was meaningless, is that to imply that such is handled automatically?

  7. Marston A, SugarStats Says:
    April 19th, 2007 at 02:14 AM

    This is great. Though posted a while ago, I was wondering if :conditions along side :include yet? Or when (if) that would be possible in the future?

  8. Eimantas Says:
    April 23rd, 2008 at 07:42 AM

    I think it forgets sorting for that association if parent model says something like has_many :foos, :order => 'bla', etc.

Leave a Reply

I am a human (check this)

Remember: escape your underscores \_ and indent code at least 4 spaces or incur the wrath of smartypants.