Package org.broadleafcommerce.core.search.service.solr.indexer

Class CatalogSolrIndexUpdateCommandHandlerImpl

java.lang.Object
org.broadleafcommerce.core.search.service.solr.indexer.AbstractSolrIndexUpdateCommandHandlerImpl
org.broadleafcommerce.core.search.service.solr.indexer.CatalogSolrIndexUpdateCommandHandlerImpl
All Implemented Interfaces:
CatalogSolrIndexCommandHandler, SolrIndexUpdateCommandHandler, org.springframework.beans.factory.DisposableBean
@Service("blCatalogSolrUpdateCommandHandler") public class CatalogSolrIndexUpdateCommandHandlerImpl extends AbstractSolrIndexUpdateCommandHandlerImpl implements CatalogSolrIndexCommandHandler, org.springframework.beans.factory.DisposableBean
Default command handler to handle Catalog Solr index commands. This is multi-threaded. SKU browsing is not supported with this implementation.
Author:
Kelly Tisdell

  • Field Details

    • solrConfiguration

      @Qualifier("blCatalogSolrConfiguration") @Autowired(required=false) protected SolrConfiguration solrConfiguration

    • localeService

      protected org.broadleafcommerce.common.locale.service.LocaleService localeService

    • shs

      protected SolrHelperService shs

    • siteService

      protected org.broadleafcommerce.common.site.service.SiteService siteService

    • extensionManager

      protected SolrIndexServiceExtensionManager extensionManager

    • transactionManager

      protected org.springframework.transaction.PlatformTransactionManager transactionManager

    • solrIndexDao

      protected SolrIndexDao solrIndexDao

    • sandBoxHelper

      protected org.broadleafcommerce.common.sandbox.SandBoxHelper sandBoxHelper

    • indexFieldDao

      protected IndexFieldDao indexFieldDao

    • productDao

      protected ProductDao productDao

    • queueProvider

      protected SolrIndexQueueProvider queueProvider

    • documentBuilder

      protected CatalogDocumentBuilder documentBuilder

    • pageSize

      @Value("${solr.index.product.pageSize:100}") protected int pageSize

    • workerThreads

      @Value("${solr.index.product.workerThreads:10}") protected int workerThreads

    • reindexCommitInterval

      @Value("${solr.index.product.reindexCommitInterval:30000}") protected long reindexCommitInterval

  • Constructor Details

    • CatalogSolrIndexUpdateCommandHandlerImpl

      public CatalogSolrIndexUpdateCommandHandlerImpl()

  • Method Details

    • executeCommand

      public <C extends SolrUpdateCommand> void executeCommand(C command) throws org.broadleafcommerce.common.exception.ServiceException
      Description copied from interface: SolrIndexUpdateCommandHandler
      Entry point from which this component can delegate action based on the type of SolrUpdateCommand.

      This should never be called directly. Rather it will be called from inside the AbstractSolrIndexUpdateServiceImpl.

      Specified by:
      executeCommand in interface SolrIndexUpdateCommandHandler
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • buildDocument

      public org.apache.solr.common.SolrInputDocument buildDocument(Indexable indexable, List<IndexField> fields, List<org.broadleafcommerce.common.locale.domain.Locale> locales)
      Description copied from interface: SolrIndexUpdateCommandHandler
      Provides an interface for a caller to convert an Indexable into a SolrInputDocument. This may return null if the implementor does not want the specified Indexable indexed.
      Specified by:
      buildDocument in interface SolrIndexUpdateCommandHandler
      Returns:

    • buildDocument

      public org.apache.solr.common.SolrInputDocument buildDocument(Indexable indexable)
      Description copied from interface: SolrIndexUpdateCommandHandler
      Provides an interface for a caller to convert an Indexable into a SolrInputDocument. This may return null if the implementor does not want the specified Indexable indexed.
      Specified by:
      buildDocument in interface SolrIndexUpdateCommandHandler
      Returns:

    • executeFullReindexCommand

      protected void executeFullReindexCommand(FullReindexCommand command) throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • executeCatalogReindexCommand

      protected void executeCatalogReindexCommand(CatalogReindexCommand command) throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • executeSiteReindexCommand

      protected void executeSiteReindexCommand(SiteReindexCommand command) throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • getSolrConfiguration

      protected SolrConfiguration getSolrConfiguration()
      Specified by:
      getSolrConfiguration in class AbstractSolrIndexUpdateCommandHandlerImpl

    • getAllLocales

      protected List<org.broadleafcommerce.common.locale.domain.Locale> getAllLocales()

    • performCachedOperation

      protected void performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation) throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • populateIndex

      protected void populateIndex(ReindexStateHolder holder, Long catalogId, Long siteId, org.broadleafcommerce.common.sandbox.domain.SandBox sandbox) throws org.broadleafcommerce.common.exception.ServiceException
      This method populates the index.

      It is not recommended that you override this method. Rather, consider overriding one of the other methods as this one generally coordinates and delegates to others.

      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • getReadIdsOperation

      protected org.broadleafcommerce.common.util.tenant.IdentityOperation<List<Long>,Exception> getReadIdsOperation(ReindexStateHolder holder, Long catalogId, Long siteId, Integer batchSize, Long lastId)
      Provides an IdentityOperation (function that runs in the context of a Site and/or Catalog) to read batches of IDs.
      Parameters:
      holder -
      catalogId -
      siteId -
      batchSize -
      lastId -
      Returns:

    • deleteAllDocuments

      protected void deleteAllDocuments(String collection, boolean commit) throws org.broadleafcommerce.common.exception.ServiceException
      By default this deletes all items in the collection, since there is a good chance that the collection will be re-aliased or swapped into the foreground after reindexing is complete.
      Parameters:
      collection -
      commit -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • finalizeChanges

      protected void finalizeChanges(String collection, boolean error, boolean swap) throws org.broadleafcommerce.common.exception.ServiceException
      Composite method that issues a commit or rollback (if there is an error), and optionally swaps (if there is no error).
      Parameters:
      collection -
      error -
      swap -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • swapCollections

      protected void swapCollections() throws org.broadleafcommerce.common.exception.ServiceException
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • destroy

      public void destroy() throws Exception
      Specified by:
      destroy in interface org.springframework.beans.factory.DisposableBean
      Throws:
      Exception

    • getBackgroundOperationExecutor

      protected org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor getBackgroundOperationExecutor()

    • getThreadsForBackgroundExecution

      protected int getThreadsForBackgroundExecution()
      Number of worker threads (page processing threads) in the thread pool.

      Default is 10.

      Returns:

    • createBackgroundOperationExecutor

      protected org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor createBackgroundOperationExecutor()

    • createBackgroundRunnable

      protected org.broadleafcommerce.common.util.EntityManagerAwareRunnable createBackgroundRunnable(ReindexStateHolder holder, List<Long> ids, Semaphore sem, Long catalogId, Long siteId, org.broadleafcommerce.common.sandbox.domain.SandBox sandBox)
      This is where most of the heavy lifting happens.
      Parameters:
      holder -
      ids -
      sem -
      catalogId -
      siteId -
      sandBox -
      Returns:

    • getIncrementalIndexOperation

      protected org.broadleafcommerce.common.util.tenant.IdentityOperation<Void,Exception> getIncrementalIndexOperation(ReindexStateHolder holder, org.broadleafcommerce.common.site.domain.Catalog catalog, org.broadleafcommerce.common.site.domain.Site site, List<Long> ids)
      Returns an IdentityOperation who may be executed in the context of a Catalog and/or Site to read products by IDs, and then build the index incrementally.
      Parameters:
      holder -
      catalog -
      site -
      ids -
      Returns:

    • buildPage

      protected List<org.apache.solr.common.SolrInputDocument> buildPage(List<Long> productIds, List<Product> products, List<org.broadleafcommerce.common.locale.domain.Locale> locales, List<IndexField> fields, ReindexStateHolder holder) throws Exception
      Given the arguments, this builds a list of SolrInputDocuments. This does not write them to Solr.
      Parameters:
      productIds -
      products -
      locales -
      fields -
      holder -
      Returns:
      Throws:
      Exception

    • readProductsByIds

      protected List<Product> readProductsByIds(ReindexStateHolder holder, List<Long> productIds) throws Exception
      Reads a batch of products by IDs.
      Parameters:
      holder -
      productIds -
      Returns:
      Throws:
      Exception

    • getIndexFields

      protected List<IndexField> getIndexFields()

    • buildIncrementalIndex

      protected void buildIncrementalIndex(List<Long> productIds, List<Product> products, ReindexStateHolder holder, org.broadleafcommerce.common.site.domain.Catalog catalog, org.broadleafcommerce.common.site.domain.Site site) throws Exception
      Builds an incremental or batch portion of the index. Activities include reading Locales and IndexFields, building a page, adding the documents to the index, and incrementally committing. It is not recommended that you override this method. Rather, consider overriding one of the other methods as this is largely responsible for orchestrating and delegating.
      Parameters:
      productIds -
      products -
      holder -
      catalog -
      site -
      Throws:
      Exception

    • readIdBatch

      protected List<Long> readIdBatch(ReindexStateHolder holder, Long catalogId, int batchSize, Long lastId) throws Exception
      Reads a batch of Product IDs from the DB.
      Parameters:
      holder -
      catalogId -
      batchSize -
      lastId -
      Returns:
      Throws:
      Exception

    • beforeProcess

      protected void beforeProcess(ReindexStateHolder holder) throws org.broadleafcommerce.common.exception.ServiceException
      Hook point that is executed at the beginning of the reindex process. If you extend this class and implement this method, consider calling the super method.
      Parameters:
      holder -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • afterProcess

      protected void afterProcess(ReindexStateHolder holder) throws org.broadleafcommerce.common.exception.ServiceException
      Hook point at that is executed at the end of the reindex process. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occured during the process.
      Parameters:
      holder -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • beforeBackgroundThread

      protected void beforeBackgroundThread(ReindexStateHolder holder, org.broadleafcommerce.common.site.domain.Catalog catalog, org.broadleafcommerce.common.site.domain.Site site, org.broadleafcommerce.common.sandbox.domain.SandBox sandBox) throws org.broadleafcommerce.common.exception.ServiceException
      The reindex process, specifically page processing, is executed in background threads. This executes prior to the main execution of the thread. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      catalog -
      site -
      sandBox -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • afterBackgroundThread

      protected void afterBackgroundThread(ReindexStateHolder holder, org.broadleafcommerce.common.site.domain.Catalog catalog, org.broadleafcommerce.common.site.domain.Site site, org.broadleafcommerce.common.sandbox.domain.SandBox sandBox) throws org.broadleafcommerce.common.exception.ServiceException
      This executes after to the main execution of the background (page) thread(s). If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      catalog -
      site -
      sandBox -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • beforePage

      protected void beforePage(List<Long> productIds, List<Product> products, List<org.broadleafcommerce.common.locale.domain.Locale> locales, List<IndexField> fields, ReindexStateHolder holder) throws org.broadleafcommerce.common.exception.ServiceException
      This is executed immediately before a page is processed. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      productIds -
      products -
      locales -
      fields -
      holder -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • afterPage

      protected void afterPage(List<Long> productIds, List<Product> products, List<org.broadleafcommerce.common.locale.domain.Locale> locales, List<IndexField> fields, ReindexStateHolder holder) throws org.broadleafcommerce.common.exception.ServiceException
      This executes immediately after a page is processed. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      productIds -
      products -
      locales -
      fields -
      holder -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • beforeReadProducts

      protected void beforeReadProducts(ReindexStateHolder holder, List<Long> productIds) throws org.broadleafcommerce.common.exception.ServiceException
      This executes immediately prior to reading a batch of products. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      productIds -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • afterReadProducts

      protected void afterReadProducts(ReindexStateHolder holder, List<Long> productIds) throws org.broadleafcommerce.common.exception.ServiceException
      This executes immediately after reading a batch of products. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      productIds -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • beforeReadIdBatch

      protected void beforeReadIdBatch(ReindexStateHolder holder, Long catalogId, int batchSize, Long lastId) throws org.broadleafcommerce.common.exception.ServiceException
      This executes immediately prior to reading a batch of product IDs for processing in a background thread. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      catalogId -
      batchSize -
      lastId -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • afterReadIdBatch

      protected void afterReadIdBatch(ReindexStateHolder holder, Long catalogId, int batchSize, Long lastId) throws org.broadleafcommerce.common.exception.ServiceException
      This executes immediately after reading a batch of product IDs for processing in a background thread. If you extend this class and implement this method, consider calling the super method. Note that you can query the ReindexStateHolder to determine if an error has occurred during the process.
      Parameters:
      holder -
      catalogId -
      batchSize -
      lastId -
      Throws:
      org.broadleafcommerce.common.exception.ServiceException

    • findCatalog

      protected org.broadleafcommerce.common.site.domain.Catalog findCatalog(Long catalogId)
      This reads a Catalog by ID. If the ID is null (default, unless in Multi-Tenant mode), this returns null.
      Parameters:
      catalogId -
      Returns:

    • findSite

      protected org.broadleafcommerce.common.site.domain.Site findSite(Long siteId)
      This reads a Site by ID. If the ID is null (default, unless in Multi-Tenant mode), this returns null.
      Parameters:
      siteId -
      Returns:

    • getIncrementalCommitInterval

      protected long getIncrementalCommitInterval()
      The amount of time between commits during a full reindex. Commits are global, so there's no need to have every thread committing every page.

      Default is 30 seconds.

      Returns:

    • incrementalCommit

      protected void incrementalCommit(ReindexStateHolder holder) throws Exception
      Issues incremental commits. Commits in Sorl are global, so we don't need multiple threads issuing commits at the same time, or even back-to-back. This uses a commit interval to determine if a commit should be issued.
      Parameters:
      holder -
      Throws:
      Exception

    • isReindexSuccessful

      protected boolean isReindexSuccessful(ReindexStateHolder holder)
      Method to determine whether a full reindex was successful. By default, this simply returns whether the ReindexStateHolder.isFailed() returns false and that at least 1 document was indexed. Implementors may wish to override this to determine whether the ReindexStateHolder.getIndexableCount() is a certain number, or if ReindexStateHolder.getUnindexedItemCount() is at least a percentage of total items or of the items that were indexed, given that the failure to build a single document / product does not stop the reindex process.
      Parameters:
      holder -
      Returns: