Package org.broadleafcommerce.core.search.service.solr.index

Class SolrIndexServiceImpl

java.lang.Object
org.broadleafcommerce.core.search.service.solr.index.SolrIndexServiceImpl
All Implemented Interfaces:
SolrIndexService, org.springframework.beans.factory.InitializingBean
@Service("blSolrIndexService") public class SolrIndexServiceImpl extends Object implements SolrIndexService, org.springframework.beans.factory.InitializingBean
Responsible for building and rebuilding the Solr index
Author:
Andre Azzolini (apazzolini), Jeff Fischer

  • Field Details

    • solrConfiguration

      @Qualifier("blCatalogSolrConfiguration") @Autowired(required=false) protected SolrConfiguration solrConfiguration

    • errorOnConcurrentReIndex

      @Value("${solr.index.errorOnConcurrentReIndex}") protected boolean errorOnConcurrentReIndex

    • pageSize

      @Value("${solr.index.product.pageSize}") protected int pageSize

    • commit

      @Value("${solr.index.commit}") protected boolean commit

    • softCommit

      @Value("${solr.index.softCommit}") protected boolean softCommit

    • waitSearcher

      @Value("${solr.index.waitSearcher}") protected boolean waitSearcher

    • waitFlush

      @Value("${solr.index.waitFlush}") protected boolean waitFlush

    • useLegacySolrIndexer

      @Value("${solr.catalog.useLegacySolrIndexer:true}") protected boolean useLegacySolrIndexer

    • productDao

      protected ProductDao productDao

    • catalogService

      protected CatalogService catalogService

    • fieldDao

      protected FieldDao fieldDao

    • localeService

      protected org.broadleafcommerce.common.locale.service.LocaleService localeService

    • shs

      protected SolrHelperService shs

    • extensionManager

      protected SolrIndexServiceExtensionManager extensionManager

    • transactionManager

      protected org.springframework.transaction.PlatformTransactionManager transactionManager

    • solrIndexDao

      protected SolrIndexDao solrIndexDao

    • sandBoxHelper

      protected org.broadleafcommerce.common.sandbox.SandBoxHelper sandBoxHelper

    • searchFacetDao

      protected SearchFacetDao searchFacetDao

    • indexFieldDao

      protected IndexFieldDao indexFieldDao

  • Constructor Details

    • SolrIndexServiceImpl

      public SolrIndexServiceImpl()

  • Method Details

    • afterPropertiesSet

      public void afterPropertiesSet() throws Exception
      Specified by:
      afterPropertiesSet in interface org.springframework.beans.factory.InitializingBean
      Throws:
      Exception

    • performCachedOperation

      public void performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation) throws org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService
      SolrIndexService exposes #buildIncrementalIndex(int, int, boolean). By wrapping the call to this method inside of a SolrIndexCachedOperation.CacheOperation, a single cache will be used for all the contained calls to buildIncrementalIndex. Here's an example: performCachedOperation(new SolrIndexCachedOperation.CacheOperation() { @param cacheOperation the block of code to perform using a single cache for best performance @throws ServiceException @Override public void execute() throws ServiceException { int page = 0; while ((page * pageSize) < numProducts) { buildIncrementalIndex(page, pageSize); page++; } } }); Note SolrIndexService.rebuildIndex() already internally wraps its call using CacheOperation, so it is not necessary to call performCacheOperation for calls to rebuildIndex().
      Specified by:
      performCachedOperation in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • rebuildIndex

      public void rebuildIndex() throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService

      Executes a full rebuild of the Solr index. This will rebuild the index on a separate core/collection and then swap out the active core/collection with the new version of the index (essentially replacing all documents that are currently in the index).

      The order of methods that are apart of rebuilding the entire index: 

      1. SolrIndexService.preBuildIndex()
      2. SolrIndexService.buildIndex()
      3. SolrIndexService.postBuildIndex()
      Specified by:
      rebuildIndex in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • preBuildIndex

      public void preBuildIndex() throws org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService
      Executed before we do any indexing when rebuilding the index. Usually this handles deleting the current index. This is called at the beginning of SolrIndexService.rebuildIndex()
      Specified by:
      preBuildIndex in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • buildIndex

      public void buildIndex() throws IOException, org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService

      Handles all the document building for the current index rebuild. This is where all of the SolrIndexOperation's need to be created, executed and the documents built and added to the Solr index

      This is the method that should be overridden to specify which operations should be run to build the correct index.

      Specified by:
      buildIndex in interface SolrIndexService
      Throws:
      IOException
      org.broadleafcommerce.common.exception.ServiceException

    • postBuildIndex

      public void postBuildIndex() throws IOException, org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService
      Executed after we do any indexing when rebuilding the current index. Usually this handles optimizing the index and swapping the cores.
      Specified by:
      postBuildIndex in interface SolrIndexService
      Throws:
      IOException
      org.broadleafcommerce.common.exception.ServiceException

    • getReindexOperation

      public SolrIndexOperation getReindexOperation()
      Description copied from interface: SolrIndexService
      Creates the SolrIndexOperation for rebuilding the current index, used by SolrIndexService.buildIndex(). This is the primary index operation used to rebuild the index.
      Specified by:
      getReindexOperation in interface SolrIndexService
      Returns:
      a SolrIndexOperation capable of rebuilding the current index

    • executeSolrIndexOperation

      public void executeSolrIndexOperation(SolrIndexOperation operation) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      Executes the given operation in the correct method order
      Specified by:
      executeSolrIndexOperation in interface SolrIndexService
      Parameters:
      operation - the SolrIndexOperation that is to be executed
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • getTotalPageCount

      protected long getTotalPageCount(Long numItemsToIndex)

    • buildPageNumberMessage

      protected String buildPageNumberMessage(int page, Long totalPages)

    • countIndexableItems

      protected Long countIndexableItems()
      Returns:

    • deleteAllDocuments

      @Deprecated protected void deleteAllDocuments() throws org.broadleafcommerce.common.exception.ServiceException
      Deprecated.
      use deleteAllReindexCoreDocuments() instead

      This method deletes all of the documents from SolrContext#getReindexServer()

      Throws:
      org.broadleafcommerce.common.exception.ServiceException - if there was a problem removing the documents

    • deleteAllReindexCoreDocuments

      protected void deleteAllReindexCoreDocuments() throws org.broadleafcommerce.common.exception.ServiceException

      This method deletes all of the documents from SolrContext#getReindexServer()

      Throws:
      org.broadleafcommerce.common.exception.ServiceException - if there was a problem removing the documents

    • deleteAllNamespaceDocuments

      public void deleteAllNamespaceDocuments(String collection, org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException
      Specified by:
      deleteAllNamespaceDocuments in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • deleteAllDocuments

      public void deleteAllDocuments(String collection, org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException
      Specified by:
      deleteAllDocuments in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • buildIncrementalIndex

      protected Long buildIncrementalIndex(int pageSize, Long lastId, SolrIndexOperation operation) throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • buildIncrementalIndex

      public Collection<org.apache.solr.common.SolrInputDocument> buildIncrementalIndex(String collection, List<? extends Indexable> indexables, org.apache.solr.client.solrj.SolrClient solrServer) throws org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService
      Builds a set of Indexables against the given SolrServer
      Specified by:
      buildIncrementalIndex in interface SolrIndexService
      Parameters:
      collection - The collection to be used for incremental indexing
      indexables - the list of items to index
      solrServer - if non-null, adds and commits the indexed documents to the server. If this is null, this will simply return the documents that were built from indexables
      Returns:
      the SolrInputDocuments that were built from the given indexables
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • readAllActiveIndexables

      protected List<? extends Indexable> readAllActiveIndexables(int pageSize, Long lastId)

    • getAllLocales

      public List<org.broadleafcommerce.common.locale.domain.Locale> getAllLocales()
      Specified by:
      getAllLocales in interface SolrIndexService
      Returns:
      a list of all possible locale prefixes to consider

    • buildDocument

      public org.apache.solr.common.SolrInputDocument buildDocument(Indexable indexable, List<IndexField> fields, List<org.broadleafcommerce.common.locale.domain.Locale> locales)
      Description copied from interface: SolrIndexService
      Given a product, fields that relate to that product, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.
      Specified by:
      buildDocument in interface SolrIndexService
      Returns:
      the document

    • attachIndexableDocumentFields

      public void attachIndexableDocumentFields(org.apache.solr.common.SolrInputDocument document, Indexable indexable, List<IndexField> fields, List<org.broadleafcommerce.common.locale.domain.Locale> locales)
      Description copied from interface: SolrIndexService
      Iterates through the fields for this Indexable and indexes any IndexFields
      Specified by:
      attachIndexableDocumentFields in interface SolrIndexService

    • attachAdditionalDocumentFields

      protected void attachAdditionalDocumentFields(Indexable indexable, org.apache.solr.common.SolrInputDocument document)
      Implementors can extend this and override this method to add additional fields to the Solr document.
      Parameters:
      indexable -
      document -

    • attachBasicDocumentFields

      protected void attachBasicDocumentFields(Indexable indexable, org.apache.solr.common.SolrInputDocument document)

    • buildFullCategoryHierarchy

      protected void buildFullCategoryHierarchy(org.apache.solr.common.SolrInputDocument document, CatalogStructure cache, Long categoryId, Set<Long> indexedParents)
      Walk the category hierarchy upwards, adding a field for each level to the solr document
      Parameters:
      document - the solr document for the product
      cache - the catalog structure cache
      categoryId - the current category id

    • getPropertyValues

      protected Map<String,Object> getPropertyValues(Indexable indexedItem, Field field, FieldType fieldType, List<org.broadleafcommerce.common.locale.domain.Locale> locales) throws IllegalAccessException, InvocationTargetException, NoSuchMethodException
      Returns a map of prefix to value for the requested attributes. For example, if the requested field corresponds to a Sku's description and the locales list has the en_US locale and the es_ES locale, the resulting map could be

      { "en_US" : "A description", "es_ES" : "Una descripcion" }

      Parameters:
      indexedItem -
      field -
      fieldType -
      locales -
      Returns:
      the value of the property
      Throws:
      IllegalAccessException
      InvocationTargetException
      NoSuchMethodException

    • convertToMappedProperty

      @Deprecated protected String convertToMappedProperty(String propertyName, String listPropertyName, String mapPropertyName)
      Deprecated.
      see SolrHelperService.getPropertyValue()
      Converts a propertyName to one that is able to reference inside a map. For example, consider the property in Product that references a List, "productAttributes". Also consider the utility method in Product called "mappedProductAttributes", which returns a map of the ProductAttributes keyed by the name property in the ProductAttribute. Given the parameters "productAttributes.heatRange", "productAttributes", "mappedProductAttributes" (which would represent a property called "productAttributes.heatRange" that references a specific ProductAttribute inside of a product whose "name" property is equal to "heatRange", this method will convert this property to mappedProductAttributes(heatRange).value, which is then usable by the standard beanutils PropertyUtils class to get the value.
      Parameters:
      propertyName -
      listPropertyName -
      mapPropertyName -
      Returns:
      the converted property name

    • saveState

      public Object[] saveState()
      Description copied from interface: SolrIndexService
      Saves some global context that might be altered during indexing.
      Specified by:
      saveState in interface SolrIndexService
      Returns:

    • restoreState

      public void restoreState(Object[] pack)
      Description copied from interface: SolrIndexService
      Restores state that was saved prior to indexing that might have been altered.
      Specified by:
      restoreState in interface SolrIndexService
      See Also:

    • optimizeIndex

      public void optimizeIndex(String collection, org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      Triggers the Solr optimize index function on the given server. This is typically called at the end of a SolrIndexService.rebuildIndex()
      Specified by:
      optimizeIndex in interface SolrIndexService
      Parameters:
      collection - The collection to optimize
      server - The server to use to optimze the given collection
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • commit

      public void commit(String collection, org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      Allows a commit to be called. By default, the details of the commit will depend on system properties, including:

      solr.index.commit - if false, then no commit will be performed. autoCommit (and autoSoftCommit) should be configured in Solr. solr.index.softCommit - indicates if a soft commit should be performed solr.index.waitSearcher - indicates if the process should wait for a searcher to be configured solr.index.waitFlush - indicates if the process should wait for a flush to disk

      Specified by:
      commit in interface SolrIndexService
      Parameters:
      collection - The collection to commit
      server - The server used to commit the specified collection
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • commit

      public void commit(String collection, org.apache.solr.client.solrj.SolrClient server, boolean softCommit, boolean waitSearcher, boolean waitFlush) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      This allows an external caller to force a commit to the SolrClient. See Solr Documentation for additional details. If using softCommit, you should ensure that a hardCommit is performed, either using autoCommit, or at the end of the commit process to flush the changes to the disk.

      Note that this method will force a commit even if solr.index.commit=false

      Specified by:
      commit in interface SolrIndexService
      Parameters:
      collection - The collection to commit
      server - - the SolrClient to use to commit the provided collection
      softCommit - - soft commit is an efficient commit that does not write the data to the file system
      waitSearcher - - whether or not to wait for a new searcher to be created
      waitFlush - - whether or not to wait for a flush to disk.
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • logDocuments

      public void logDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents)
      Description copied from interface: SolrIndexService
      Prints out the docs to the trace logger
      Specified by:
      logDocuments in interface SolrIndexService

    • convertDisplayOrderToLong

      protected Long convertDisplayOrderToLong(CatalogStructure cache, String displayOrderKey)
      We multiply the BigDecimal by 1,000,000 to maintain any possible decimals in use the displayOrder value.
      Parameters:
      cache -
      displayOrderKey -
      Returns:

    • deleteByQuery

      public void deleteByQuery(String deleteQuery) throws org.apache.solr.client.solrj.SolrServerException, IOException
      Specified by:
      deleteByQuery in interface SolrIndexService
      Throws:
      org.apache.solr.client.solrj.SolrServerException
      IOException

    • addDocuments

      public void addDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents) throws IOException, org.apache.solr.client.solrj.SolrServerException
      Specified by:
      addDocuments in interface SolrIndexService
      Throws:
      IOException
      org.apache.solr.client.solrj.SolrServerException

    • logDeleteQuery

      public void logDeleteQuery(String deleteQuery)
      Specified by:
      logDeleteQuery in interface SolrIndexService

    • buildIncrementalIndex

      public Collection<org.apache.solr.common.SolrInputDocument> buildIncrementalIndex(List<? extends Indexable> indexables, org.apache.solr.client.solrj.SolrClient solrServer) throws org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexService
      Builds a set of Indexables against the given SolrServer
      Specified by:
      buildIncrementalIndex in interface SolrIndexService
      Parameters:
      indexables - the list of items to index
      solrServer - if non-null, adds and commits the indexed documents to the server. If this is null, this will simply return the documents that were built from indexables
      Returns:
      the SolrInputDocuments that were built from the given indexables
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • optimizeIndex

      public void optimizeIndex(org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      Triggers the Solr optimize index function on the given server. This is typically called at the end of a SolrIndexService.rebuildIndex()
      Specified by:
      optimizeIndex in interface SolrIndexService
      Parameters:
      server - The server to use to optimze the default collection
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • commit

      public void commit(org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      Allows a commit to be called. By default, the details of the commit will depend on system properties, including:

      solr.index.commit - if false, then no commit will be performed. autoCommit (and autoSoftCommit) should be configured in Solr. solr.index.softCommit - indicates if a soft commit should be performed solr.index.waitSearcher - indicates if the process should wait for a searcher to be configured solr.index.waitFlush - indicates if the process should wait for a flush to disk

      Specified by:
      commit in interface SolrIndexService
      Parameters:
      server - The server used to commit the default collection
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • commit

      public void commit(org.apache.solr.client.solrj.SolrClient server, boolean softCommit, boolean waitSearcher, boolean waitFlush) throws org.broadleafcommerce.common.exception.ServiceException, IOException
      Description copied from interface: SolrIndexService
      This allows an external caller to force a commit to the SolrClient. See Solr Documentation for additional details. If using softCommit, you should ensure that a hardCommit is performed, either using autoCommit, or at the end of the commit process to flush the changes to the disk.

      Note that this method will force a commit even if solr.index.commit=false

      Specified by:
      commit in interface SolrIndexService
      Parameters:
      server - - the SolrClient to commit with
      softCommit - - soft commit is an efficient commit that does not write the data to the file system
      waitSearcher - - whether or not to wait for a new searcher to be created
      waitFlush - - whether or not to wait for a flush to disk.
      Throws:
      org.broadleafcommerce.common.exception.ServiceException
      IOException

    • deleteAllNamespaceDocuments

      public void deleteAllNamespaceDocuments(org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException
      Specified by:
      deleteAllNamespaceDocuments in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • deleteAllDocuments

      public void deleteAllDocuments(org.apache.solr.client.solrj.SolrClient server) throws org.broadleafcommerce.common.exception.ServiceException
      Specified by:
      deleteAllDocuments in interface SolrIndexService
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • useLegacyIndexer

      public boolean useLegacyIndexer()
      Description copied from interface: SolrIndexService
      Indicates if this should be used. The alternative is to use CatalogSolrIndexUpdateService. By default, this is driven by the property, 'solr.catalog.useLegacySolrIndexer', which defaults to true.
      Specified by:
      useLegacyIndexer in interface SolrIndexService
      Returns: