All Indexes being configured on every page load (seemingly)

[Max Williams]

Hi - i'm doing some DB optimisations on our site, mainly by watching the
log files (in dev mode) and seeing what db access is going on. I'm
seeing a lot of massive outputs like below - this is for one of my
ferret-indexed classes, but i have 5, and they all seem to output all
this stuff on every page that loads something from one of those tables
(just loading, not even updating).

I'm just wondering -
a) is this normal?
b) Is it really reconfiguring all my ferret indexes every time i load a
page?
c) is it necessary?
d) is it harming my site's performance?

thanks, max

SQL (0.001301) SHOW TABLES
configured index for class Lesson:
{:user_default_field=>nil, :enabled=>true,
:fields=>{:property_names=>{}, :asset_count=>{:index=>:untokenized},
:asset_paths=>{}, :resource_property_names=>{},
:description_for_sort=>{:index=>:untokenized}, :user_login=>{},
:name=>{}, :created_at_for_sort=>{:index=>:untokenized},
:officialness=>{}, :asset_names=>{}, :description=>{},
:name_for_sort=>{:index=>:untokenized}, :user_name=>{}},
:store_class_name=>true,
:index_dir=>"/home/max/work/e_learning_resource/trunk/index/development/lesson",
:mysql_fast_batches=>true, :name=>:lesson, :single_index=>false,
:index_base_dir=>"/home/max/work/e_learning_resource/trunk/index/development/lesson",
:reindex_batch_size=>1000, :registered_models=>[Lesson(id: integer,
name: string, description: text, user_id: integer, created_at: datetime,
privacy: integer, is_official: boolean, is_readonly: boolean,
comments_allowed: boolean, hours: integer, sessions: integer,
updated_at: datetime)],
:ferret=>{:path=>"/home/max/work/e_learning_resource/trunk/index/development/lesson",
:auto_flush=>true, :or_default=>false, :key=>[:id, :class_name],
:handle_parse_errors=>true, :create_if_missing=>true,
:default_field=>[:property_names, :asset_paths,
:resource_property_names, :name, :user_login, :officialness,
:description, :asset_names, :user_name]}, :raise_drb_errors=>false,
:ferret_fields=>{:property_names=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:property_names, :boost=>1.0}, :asset_paths=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:asset_paths, :boost=>1.0}, :asset_count=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets,
:index=>:untokenized, :via=>:asset_count, :boost=>1.0},
:resource_property_names=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:resource_property_names, :boost=>1.0},
:description_for_sort=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:untokenized,
:via=>:description_for_sort, :boost=>1.0}, :name=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:name, :boost=>1.0}, :user_login=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:yes, :via=>:user_login,
:boost=>1.0}, :created_at_for_sort=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:untokenized,
:via=>:created_at_for_sort, :boost=>1.0},
:officialness=>{:highlight=>:yes, :store=>:no,
:term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:officialness, :boost=>1.0}, :description=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:description, :boost=>1.0}, :asset_names=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:asset_names, :boost=>1.0}, :user_name=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets, :index=>:yes,
:via=>:user_name, :boost=>1.0}, :name_for_sort=>{:highlight=>:yes,
:store=>:no, :term_vector=>:with_positions_offsets,
:index=>:untokenized, :via=>:name_for_sort, :boost=>1.0}}}

[Jens Kraemer]

Hi!

Post by Max Williams
Hi - i'm doing some DB optimisations on our site, mainly by watching the
log files (in dev mode) and seeing what db access is going on. I'm
seeing a lot of massive outputs like below - this is for one of my
ferret-indexed classes, but i have 5, and they all seem to output all
this stuff on every page that loads something from one of those tables
(just loading, not even updating).
I'm just wondering -
a) is this normal?

yes, in development mode it is. You shouldn't see this in production.

Post by Max Williams
b) Is it really reconfiguring all my ferret indexes every time i load a
page?

yep. I think the biggest problem is the noisy output in this case,
which really makes log files hard to read. You can easily comment out
the debug statement responsible for this, it's around line 94 in
act_methods.rb.

Post by Max Williams
c) is it necessary?

It's a side effect of Rails reloading class definitions on each
request in dev mode, maybe there would be a way for aaf to work around
this. However I think this would lead to unexpected behaviour i.e. if
you modified some aaf option in dev mode and the change would only be
picked up by restarting the server.

Post by Max Williams
d) is it harming my site's performance?

If you're running in production mode this will happen only once at
application startup, so the answer is no.


Cheers,
Jens


- --
Jens Krämer
Finkenlust 14, 06449 Aschersleben, Germany
VAT Id DE251962952
http://www.jkraemer.net/ - Blog
http://www.omdb.org/ - The new free film database

GPG public key: http://www.jkraemer.net/static/keys/jk_jkraemer.net.key.asc

[Max Williams]

Ah, i see, that makes sense. I forgot about the class-reloading stuff
(even though i habitually make a change to a class and reload the page
to see it, while in dev mode).

Thanks Jens!
max