Class Ultrasphinx::Search
In: lib/ultrasphinx/search.rb
lib/ultrasphinx/search/internals.rb
lib/ultrasphinx/search/parser.rb
lib/ultrasphinx/search.rb
Parent: Object

Command-interface Search object.

Basic usage

To set up a search, instantiate an Ultrasphinx::Search object with a hash of parameters. Only the :query key is mandatory.

  @search = Ultrasphinx::Search.new(
    :query => @query,
    :sort_mode => 'descending',
    :sort_by => 'created_at'
  )

Now, to run the query, call its run method. Your results will be available as ActiveRecord instances via the results method. Example:

  @search.run
  @search.results

Options

Query format

The query string supports boolean operation, parentheses, phrases, and field-specific search. Query words are stemmed and joined by an implicit AND by default.

  • Valid boolean operators are AND, OR, and NOT.
  • Field-specific searches should be formatted as fieldname:contents. (This will only work for text fields. For numeric and date fields, see the :filters parameter, below.)
  • Phrases must be enclosed in double quotes.

A Sphinx::SphinxInternalError will be raised on invalid queries. In general, queries can only be nested to one level.

  @query = 'dog OR cat OR "white tigers" NOT (lions OR bears) AND title:animals'

Hash parameters

The hash lets you customize internal aspects of the search.

:per_page:An integer. How many results per page.
:page:An integer. Which page of the results to return.
:class_names:An array or string. The class name of the model you want to search, an array of model names to search, or nil for all available models.
:sort_mode:‘relevance‘ or ‘ascending‘ or ‘descending‘. How to order the result set. Note that time and ‘extended‘ modes are available, but not tested.
:sort_by:A field name. What field to order by for ‘ascending‘ or ‘descending‘ mode. Has no effect for ‘relevance‘.
:weights:A hash. Text-field names and associated query weighting. The default weight for every field is 1.0. Example: :weights => {‘title’ => 2.0}
:filters:A hash. Names of numeric or date fields and associated values. You can use a single value, an array of values, or a range. (See the bottom of the ActiveRecord::Base page for an example.)
:facets:An array of fields for grouping/faceting. You can access the returned facet values and their result counts with the facets method.
:location:A hash. Specify the names of your latititude and longitude attributes as declared in your is_indexed calls. To sort the results by distance, set :sort_mode => ‘extended‘ and :sort_by => ‘distance asc’.
:indexes:An array of indexes to search. Currently only Ultrasphinx::MAIN_INDEX and Ultrasphinx::DELTA_INDEX are available. Defaults to both; changing this is rarely needed.

Query Defaults

Note that you can set up your own query defaults in environment.rb:

  self.class.query_defaults = HashWithIndifferentAccess.new({
    :per_page => 10,
    :sort_mode => 'relevance',
    :weights => {'title' => 2.0}
  })

Advanced features

Geographic distance

If you pass a :location Hash, distance from the location in meters will be available in your result records via the distance accessor:

  @search = Ultrasphinx::Search.new(:class_names => 'Point',
            :query => 'pizza',
            :sort_mode => 'extended',
            :sort_by => 'distance',
            :location => {
              :lat => 40.3,
              :long => -73.6
            })

   @search.run.first.distance #=> 1402.4

Note that Sphinx expects lat/long to be indexed as radians. If you have degrees in your database, do the conversion in the is_indexed as so:

    is_indexed 'fields' => [
        'name',
        'description',
        {:field => 'lat', :function_sql => "RADIANS(?)"},
        {:field => 'lng', :function_sql => "RADIANS(?)"}
      ]

Then, set Ultrasphinx::Search.client_options[:location][:units] = ‘degrees‘.

The MySQL :double column type is recommended for storing location data. For Postgres, use <tt>:float</tt.

Interlock integration

Ultrasphinx uses the find_all_by_id method to instantiate records. If you set with_finders: true in Interlock's config/memcached.yml, Interlock overrides find_all_by_id with a caching version.

Will_paginate integration

The Search instance responds to the same methods as a WillPaginate::Collection object, so once you have called run or excerpt you can use it directly in your views:

  will_paginate(@search)

Excerpt mode

You can have Sphinx excerpt and highlight the matched sections in the associated fields. Instead of calling run, call excerpt.

  @search.excerpt

The returned models will be frozen and have their field contents temporarily changed to the excerpted and highlighted results.

You need to set the content_methods key on Ultrasphinx::Search.excerpting_options to whatever groups of methods you need the excerpter to try to excerpt. The first responding method in each group for each record will be excerpted. This way Ruby-only methods are supported (for example, a metadata method which combines various model fields, or an aliased field so that the original record contents are still available).

There are some other keys you can set, such as excerpt size, HTML tags to highlight with, and number of words on either side of each excerpt chunk. Example (in environment.rb):

  Ultrasphinx::Search.excerpting_options = HashWithIndifferentAccess.new({
    :before_match => '<strong>',
    :after_match => '</strong>',
    :chunk_separator => "...",
    :limit => 256,
    :around => 3,
    :content_methods => [['title'], ['body', 'description', 'content'], ['metadata']]
  })

Note that your database is never changed by anything Ultrasphinx does.

Methods

Included Modules

Internals Parser Internals Parser

Classes and Modules

Module Ultrasphinx::Search::Internals
Module Ultrasphinx::Search::Parser

Constants

SPHINX_CLIENT_PARAMS = { 'sort_mode' => { 'relevance' => :relevance, 'descending' => :attr_desc, 'ascending' => :attr_asc, 'time' => :time_segments, 'extended' => :extended, }   Friendly sort mode mappings
MODELS_TO_IDS = Ultrasphinx.get_models_to_class_ids || {}
IDS_TO_MODELS = MODELS_TO_IDS.invert #:nodoc:
MAX_MATCHES = DAEMON_SETTINGS["max_matches"].to_i
SPHINX_CLIENT_PARAMS = { 'sort_mode' => { 'relevance' => :relevance, 'descending' => :attr_desc, 'ascending' => :attr_asc, 'time' => :time_segments, 'extended' => :extended, }   Friendly sort mode mappings
MODELS_TO_IDS = Ultrasphinx.get_models_to_class_ids || {}
IDS_TO_MODELS = MODELS_TO_IDS.invert #:nodoc:
MAX_MATCHES = DAEMON_SETTINGS["max_matches"].to_i

Public Class methods

Builds a new command-interface Search object.

[Source]

     # File lib/ultrasphinx/search.rb, line 295
295:     def initialize opts = {} 
296: 
297:       # Change to normal hashes with String keys for speed
298:       opts = Hash[HashWithIndifferentAccess.new(opts._deep_dup._coerce_basic_types)]
299:       unless self.class.query_defaults.instance_of? Hash
300:         self.class.query_defaults = Hash[self.class.query_defaults]
301:         self.class.query_defaults['location'] = Hash[self.class.query_defaults['location']]
302:         
303:         self.class.client_options = Hash[self.class.client_options]
304:         self.class.excerpting_options = Hash[self.class.excerpting_options]
305:         self.class.excerpting_options['content_methods'].map! {|ary| ary.map {|m| m.to_s}}
306:       end    
307: 
308:       # We need an annoying deep merge on the :location parameter
309:       opts['location'].reverse_merge!(self.class.query_defaults['location']) if opts['location']
310: 
311:       # Merge the rest of the defaults      
312:       @options = self.class.query_defaults.merge(opts)
313:       
314:       @options['query'] = @options['query'].to_s
315:       @options['class_names'] = Array(@options['class_names'])
316:       @options['facets'] = Array(@options['facets'])
317:       @options['indexes'] = Array(@options['indexes']).join(" ")
318:             
319:       raise UsageError, "Weights must be a Hash" unless @options['weights'].is_a? Hash
320:       raise UsageError, "Filters must be a Hash" unless @options['filters'].is_a? Hash
321:       
322:       @options['parsed_query'] = parse(query)
323:   
324:       @results, @subtotals, @facets, @response = [], {}, {}, {}
325:         
326:       extra_keys = @options.keys - (self.class.query_defaults.keys + INTERNAL_KEYS)
327:       log "discarded invalid keys: #{extra_keys * ', '}" if extra_keys.any? and RAILS_ENV != "test" 
328:     end

Builds a new command-interface Search object.

[Source]

     # File lib/ultrasphinx/search.rb, line 295
295:     def initialize opts = {} 
296: 
297:       # Change to normal hashes with String keys for speed
298:       opts = Hash[HashWithIndifferentAccess.new(opts._deep_dup._coerce_basic_types)]
299:       unless self.class.query_defaults.instance_of? Hash
300:         self.class.query_defaults = Hash[self.class.query_defaults]
301:         self.class.query_defaults['location'] = Hash[self.class.query_defaults['location']]
302:         
303:         self.class.client_options = Hash[self.class.client_options]
304:         self.class.excerpting_options = Hash[self.class.excerpting_options]
305:         self.class.excerpting_options['content_methods'].map! {|ary| ary.map {|m| m.to_s}}
306:       end    
307: 
308:       # We need an annoying deep merge on the :location parameter
309:       opts['location'].reverse_merge!(self.class.query_defaults['location']) if opts['location']
310: 
311:       # Merge the rest of the defaults      
312:       @options = self.class.query_defaults.merge(opts)
313:       
314:       @options['query'] = @options['query'].to_s
315:       @options['class_names'] = Array(@options['class_names'])
316:       @options['facets'] = Array(@options['facets'])
317:       @options['indexes'] = Array(@options['indexes']).join(" ")
318:             
319:       raise UsageError, "Weights must be a Hash" unless @options['weights'].is_a? Hash
320:       raise UsageError, "Filters must be a Hash" unless @options['filters'].is_a? Hash
321:       
322:       @options['parsed_query'] = parse(query)
323:   
324:       @results, @subtotals, @facets, @response = [], {}, {}, {}
325:         
326:       extra_keys = @options.keys - (self.class.query_defaults.keys + INTERNAL_KEYS)
327:       log "discarded invalid keys: #{extra_keys * ', '}" if extra_keys.any? and RAILS_ENV != "test" 
328:     end

Public Instance methods

Returns the current page number of the result set. (Page indexes begin at 1.)

[Source]

     # File lib/ultrasphinx/search.rb, line 264
264:     def current_page
265:       @options['page']
266:     end

Returns the current page number of the result set. (Page indexes begin at 1.)

[Source]

     # File lib/ultrasphinx/search.rb, line 264
264:     def current_page
265:       @options['page']
266:     end

Overwrite the configured content attributes with excerpted and highlighted versions of themselves. Runs run if it hasn‘t already been done.

[Source]

     # File lib/ultrasphinx/search.rb, line 369
369:     def excerpt
370:     
371:       require_run         
372:       return if results.empty?
373:     
374:       # See what fields in each result might respond to our excerptable methods
375:       results_with_content_methods = results.map do |result|
376:         [result, 
377:           self.class.excerpting_options['content_methods'].map do |methods|
378:             methods.detect do |this| 
379:               result.respond_to? this
380:             end
381:           end
382:         ]
383:       end
384:   
385:       # Fetch the actual field contents
386:       docs = results_with_content_methods.map do |result, methods|
387:         methods.map do |method| 
388:           method and strip_bogus_characters(result.send(method)) or ""
389:         end
390:       end.flatten
391:       
392:       excerpting_options = {
393:         :docs => docs,         
394:         :index => MAIN_INDEX, # http://www.sphinxsearch.com/forum/view.html?id=100
395:         :words => strip_query_commands(parsed_query)
396:       }
397:       self.class.excerpting_options.except('content_methods').each do |key, value|
398:         # Riddle only wants symbols
399:         excerpting_options[key.to_sym] ||= value
400:       end
401:       
402:       responses = perform_action_with_retries do 
403:         # Ship to Sphinx to highlight and excerpt
404:         @request.excerpts(excerpting_options)
405:       end
406:       
407:       responses = responses.in_groups_of(self.class.excerpting_options['content_methods'].size)
408:       
409:       results_with_content_methods.each_with_index do |result_and_methods, i|
410:         # Override the individual model accessors with the excerpted data
411:         result, methods = result_and_methods
412:         methods.each_with_index do |method, j|
413:           data = responses[i][j]
414:           if method
415:             result._metaclass.send('define_method', method) { data }
416:             attributes = result.instance_variable_get('@attributes')
417:             attributes[method] = data if attributes[method]
418:           end
419:         end
420:       end
421:   
422:       @results = results_with_content_methods.map do |result_and_content_method| 
423:         result_and_content_method.first.freeze
424:       end
425:       
426:       self
427:     end

Overwrite the configured content attributes with excerpted and highlighted versions of themselves. Runs run if it hasn‘t already been done.

[Source]

     # File lib/ultrasphinx/search.rb, line 369
369:     def excerpt
370:     
371:       require_run         
372:       return if results.empty?
373:     
374:       # See what fields in each result might respond to our excerptable methods
375:       results_with_content_methods = results.map do |result|
376:         [result, 
377:           self.class.excerpting_options['content_methods'].map do |methods|
378:             methods.detect do |this| 
379:               result.respond_to? this
380:             end
381:           end
382:         ]
383:       end
384:   
385:       # Fetch the actual field contents
386:       docs = results_with_content_methods.map do |result, methods|
387:         methods.map do |method| 
388:           method and strip_bogus_characters(result.send(method)) or ""
389:         end
390:       end.flatten
391:       
392:       excerpting_options = {
393:         :docs => docs,         
394:         :index => MAIN_INDEX, # http://www.sphinxsearch.com/forum/view.html?id=100
395:         :words => strip_query_commands(parsed_query)
396:       }
397:       self.class.excerpting_options.except('content_methods').each do |key, value|
398:         # Riddle only wants symbols
399:         excerpting_options[key.to_sym] ||= value
400:       end
401:       
402:       responses = perform_action_with_retries do 
403:         # Ship to Sphinx to highlight and excerpt
404:         @request.excerpts(excerpting_options)
405:       end
406:       
407:       responses = responses.in_groups_of(self.class.excerpting_options['content_methods'].size)
408:       
409:       results_with_content_methods.each_with_index do |result_and_methods, i|
410:         # Override the individual model accessors with the excerpted data
411:         result, methods = result_and_methods
412:         methods.each_with_index do |method, j|
413:           data = responses[i][j]
414:           if method
415:             result._metaclass.send('define_method', method) { data }
416:             attributes = result.instance_variable_get('@attributes')
417:             attributes[method] = data if attributes[method]
418:           end
419:         end
420:       end
421:   
422:       @results = results_with_content_methods.map do |result_and_content_method| 
423:         result_and_content_method.first.freeze
424:       end
425:       
426:       self
427:     end

Returns the facet map for this query, if facets were used.

[Source]

     # File lib/ultrasphinx/search.rb, line 225
225:     def facets
226:       raise UsageError, "No facet field was configured" unless @options['facets']
227:       require_run
228:       @facets
229:     end

Returns the facet map for this query, if facets were used.

[Source]

     # File lib/ultrasphinx/search.rb, line 225
225:     def facets
226:       raise UsageError, "No facet field was configured" unless @options['facets']
227:       require_run
228:       @facets
229:     end

Delegates enumerable methods to @results, if possible. This allows us to behave directly like a WillPaginate::Collection. Failing that, we delegate to the options hash if a key is set. This lets us use self directly in view helpers.

[Source]

     # File lib/ultrasphinx/search.rb, line 431
431:     def method_missing(*args, &block)
432:       if @results.respond_to? args.first
433:         @results.send(*args, &block)
434:       elsif options.has_key? args.first.to_s
435:         @options[args.first.to_s]
436:       else
437:         super
438:       end
439:     end

Delegates enumerable methods to @results, if possible. This allows us to behave directly like a WillPaginate::Collection. Failing that, we delegate to the options hash if a key is set. This lets us use self directly in view helpers.

[Source]

     # File lib/ultrasphinx/search.rb, line 431
431:     def method_missing(*args, &block)
432:       if @results.respond_to? args.first
433:         @results.send(*args, &block)
434:       elsif options.has_key? args.first.to_s
435:         @options[args.first.to_s]
436:       else
437:         super
438:       end
439:     end

Returns the next page number.

[Source]

     # File lib/ultrasphinx/search.rb, line 285
285:     def next_page
286:       current_page < page_count ? (current_page + 1) : nil
287:     end

Returns the next page number.

[Source]

     # File lib/ultrasphinx/search.rb, line 285
285:     def next_page
286:       current_page < page_count ? (current_page + 1) : nil
287:     end

Returns the global index position of the first result on this page.

[Source]

     # File lib/ultrasphinx/search.rb, line 290
290:     def offset 
291:       (current_page - 1) * per_page
292:     end

Returns the global index position of the first result on this page.

[Source]

     # File lib/ultrasphinx/search.rb, line 290
290:     def offset 
291:       (current_page - 1) * per_page
292:     end

Returns the options hash.

[Source]

     # File lib/ultrasphinx/search.rb, line 203
203:     def options
204:       @options
205:     end

Returns the options hash.

[Source]

     # File lib/ultrasphinx/search.rb, line 203
203:     def options
204:       @options
205:     end

Returns the last available page number in the result set.

[Source]

     # File lib/ultrasphinx/search.rb, line 274
274:     def page_count
275:       require_run    
276:       (total_entries / per_page.to_f).ceil
277:     end

Returns the last available page number in the result set.

[Source]

     # File lib/ultrasphinx/search.rb, line 274
274:     def page_count
275:       require_run    
276:       (total_entries / per_page.to_f).ceil
277:     end

Returns the number of records per page.

[Source]

     # File lib/ultrasphinx/search.rb, line 269
269:     def per_page
270:       @options['per_page']
271:     end

Returns the number of records per page.

[Source]

     # File lib/ultrasphinx/search.rb, line 269
269:     def per_page
270:       @options['per_page']
271:     end

Returns the previous page number.

[Source]

     # File lib/ultrasphinx/search.rb, line 280
280:     def previous_page 
281:       current_page > 1 ? (current_page - 1) : nil
282:     end

Returns the previous page number.

[Source]

     # File lib/ultrasphinx/search.rb, line 280
280:     def previous_page 
281:       current_page > 1 ? (current_page - 1) : nil
282:     end

Returns the query string used.

[Source]

     # File lib/ultrasphinx/search.rb, line 208
208:     def query
209:       # Redundant with method_missing
210:       @options['query']
211:     end

Returns the query string used.

[Source]

     # File lib/ultrasphinx/search.rb, line 208
208:     def query
209:       # Redundant with method_missing
210:       @options['query']
211:     end

Returns the raw response from the Sphinx client.

[Source]

     # File lib/ultrasphinx/search.rb, line 232
232:     def response
233:       require_run
234:       @response
235:     end

Returns the raw response from the Sphinx client.

[Source]

     # File lib/ultrasphinx/search.rb, line 232
232:     def response
233:       require_run
234:       @response
235:     end

Returns an array of result objects.

[Source]

     # File lib/ultrasphinx/search.rb, line 219
219:     def results
220:       require_run
221:       @results
222:     end

Returns an array of result objects.

[Source]

     # File lib/ultrasphinx/search.rb, line 219
219:     def results
220:       require_run
221:       @results
222:     end

Run the search, filling results with an array of ActiveRecord objects. Set the parameter to false if you only want the ids returned.

[Source]

     # File lib/ultrasphinx/search.rb, line 332
332:     def run(reify = true)
333:       @request = build_request_with_options(@options)
334: 
335:       log "searching for #{@options.inspect}"
336: 
337:       perform_action_with_retries do
338:         @response = @request.query(parsed_query, @options['indexes'])
339:         log "search returned #{total_entries}/#{response[:total_found].to_i} in #{time.to_f} seconds."
340:           
341:         if self.class.client_options['with_subtotals']        
342:           @subtotals = get_subtotals(@request, parsed_query) 
343:           
344:           # If the original query has a filter on this class, we will use its more accurate total rather the facet's 
345:           # less accurate total.
346:           if @options['class_names'].size == 1
347:             @subtotals[@options['class_names'].first] = response[:total_found]
348:           end
349:           
350:         end
351:         
352:         Array(@options['facets']).each do |facet|
353:           @facets[facet] = get_facets(@request, parsed_query, facet)
354:         end        
355:         
356:         @results = convert_sphinx_ids(response[:matches])
357:         @results = reify_results(@results) if reify
358:         
359:         say "warning; #{response[:warning]}" if response[:warning]
360:         raise UsageError, response[:error] if response[:error]
361:         
362:       end      
363:       self
364:     end

Run the search, filling results with an array of ActiveRecord objects. Set the parameter to false if you only want the ids returned.

[Source]

     # File lib/ultrasphinx/search.rb, line 332
332:     def run(reify = true)
333:       @request = build_request_with_options(@options)
334: 
335:       log "searching for #{@options.inspect}"
336: 
337:       perform_action_with_retries do
338:         @response = @request.query(parsed_query, @options['indexes'])
339:         log "search returned #{total_entries}/#{response[:total_found].to_i} in #{time.to_f} seconds."
340:           
341:         if self.class.client_options['with_subtotals']        
342:           @subtotals = get_subtotals(@request, parsed_query) 
343:           
344:           # If the original query has a filter on this class, we will use its more accurate total rather the facet's 
345:           # less accurate total.
346:           if @options['class_names'].size == 1
347:             @subtotals[@options['class_names'].first] = response[:total_found]
348:           end
349:           
350:         end
351:         
352:         Array(@options['facets']).each do |facet|
353:           @facets[facet] = get_facets(@request, parsed_query, facet)
354:         end        
355:         
356:         @results = convert_sphinx_ids(response[:matches])
357:         @results = reify_results(@results) if reify
358:         
359:         say "warning; #{response[:warning]}" if response[:warning]
360:         raise UsageError, response[:error] if response[:error]
361:         
362:       end      
363:       self
364:     end

Returns whether the query has been run.

[Source]

     # File lib/ultrasphinx/search.rb, line 259
259:     def run?
260:       !@response.blank?
261:     end

Returns whether the query has been run.

[Source]

     # File lib/ultrasphinx/search.rb, line 259
259:     def run?
260:       !@response.blank?
261:     end

Returns a hash of total result counts, scoped to each available model. Set Ultrasphinx::Search.client_options[:with_subtotals] = true to enable.

The subtotals are implemented as a special type of facet.

[Source]

     # File lib/ultrasphinx/search.rb, line 240
240:     def subtotals
241:       raise UsageError, "Subtotals are not enabled" unless self.class.client_options['with_subtotals']
242:       require_run
243:       @subtotals
244:     end

Returns a hash of total result counts, scoped to each available model. Set Ultrasphinx::Search.client_options[:with_subtotals] = true to enable.

The subtotals are implemented as a special type of facet.

[Source]

     # File lib/ultrasphinx/search.rb, line 240
240:     def subtotals
241:       raise UsageError, "Subtotals are not enabled" unless self.class.client_options['with_subtotals']
242:       require_run
243:       @subtotals
244:     end

Returns the response time of the query, in milliseconds.

[Source]

     # File lib/ultrasphinx/search.rb, line 253
253:     def time
254:       require_run
255:       response[:time]
256:     end

Returns the response time of the query, in milliseconds.

[Source]

     # File lib/ultrasphinx/search.rb, line 253
253:     def time
254:       require_run
255:       response[:time]
256:     end

Returns the total result count.

[Source]

     # File lib/ultrasphinx/search.rb, line 247
247:     def total_entries
248:       require_run
249:       [response[:total_found] || 0, MAX_MATCHES].min
250:     end

Returns the total result count.

[Source]

     # File lib/ultrasphinx/search.rb, line 247
247:     def total_entries
248:       require_run
249:       [response[:total_found] || 0, MAX_MATCHES].min
250:     end

[Validate]