org.broadleafcommerce.core.search.service.solr

Class SolrIndexServiceImpl

java.lang.Object

org.broadleafcommerce.core.search.service.solr.SolrIndexServiceImpl

All Implemented Interfaces:

SolrIndexService

@Service(value="blSolrIndexService")
public class SolrIndexServiceImpl
extends Object
implements SolrIndexService

Responsible for building and rebuilding the Solr index

Author:

Andre Azzolini (apazzolini), Jeff Fischer

Field Summary

Fields

Modifier and Type	Field and Description
protected CategoryDao	categoryDao
protected boolean	commit
protected boolean	errorOnConcurrentReIndex
protected SolrSearchServiceExtensionManager	extensionManager
protected FieldDao	fieldDao
protected boolean	IS_LOCKED
protected LocaleService	localeService
protected Object	LOCK_OBJECT
protected int	pageSize
protected ProductDao	productDao
protected SandBoxHelper	sandBoxHelper
protected SolrHelperService	shs
protected SkuDao	skuDao
protected boolean	softCommit
protected SolrIndexDao	solrIndexDao
protected org.springframework.transaction.PlatformTransactionManager	transactionManager
protected boolean	useSku
protected boolean	waitFlush
protected boolean	waitSearcher

Constructor Summary

Constructors

Constructor and Description
SolrIndexServiceImpl()

Method Summary

Modifier and Type	Method and Description
protected void	attachAdditionalDocumentFields(Product product, org.apache.solr.common.SolrInputDocument document) Implementors can extend this and override this method to add additional fields to the Solr document.
protected void	attachAdditionalDocumentFields(Sku sku, org.apache.solr.common.SolrInputDocument document) Implementors can extend this and override this method to add additional fields to the Solr document.
protected void	attachBasicDocumentFields(Product product, org.apache.solr.common.SolrInputDocument document)
protected void	attachBasicDocumentFields(Sku sku, org.apache.solr.common.SolrInputDocument document) Adds the ID, category, and explicitCategory fields for the product or sku to the document
protected void	buildCategoryDocument(Long categoryId, Product product, org.apache.solr.common.SolrInputDocument document, CatalogStructure cache)
org.apache.solr.common.SolrInputDocument	buildDocument(Product product, List<Field> fields, List<Locale> locales) Given a product, fields that relate to that product, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.
org.apache.solr.common.SolrInputDocument	buildDocument(Sku sku, List<Field> fields, List<Locale> locales) Given a sku, fields that relate to that sku, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.
protected void	buildFullCategoryHierarchy(org.apache.solr.common.SolrInputDocument document, CatalogStructure cache, Long categoryId, Set<Long> indexedParents) Walk the category hierarchy upwards, adding a field for each level to the solr document
protected void	buildIncrementalIndex(int page, int pageSize)
void	buildIncrementalIndex(int page, int pageSize, boolean useReindexServer) The internal method for building indexes.
void	buildIncrementalProductIndex(List<Product> products, boolean useReindexServer) This can be used in lieu of passing in page sizes, The reason is that one might want to apply filters or only index certain products.
void	buildIncrementalSkuIndex(List<Sku> skus, boolean useReindexServer) This can be used in lieu of passing in page sizes, The reason is that one might want to apply filters or only index certain skus.
void	commit(org.apache.solr.client.solrj.SolrServer server) Allows a commit to be called.
void	commit(org.apache.solr.client.solrj.SolrServer server, boolean softCommit, boolean waitSearcher, boolean waitFlush) This allows an external caller to force a commit to the SolrServer.
protected Long	convertDisplayOrderToLong(BigDecimal displayOrder) We multiply the BigDecimal by 1,000,000 to maintain any possible decimals in use the displayOrder value.
protected String	convertToMappedProperty(String propertyName, String listPropertyName, String mapPropertyName) Deprecated. see SolrHelperService.getPropertyValue()
protected void	deleteAllDocuments() Deprecated. use deleteAllReindexCoreDocuments() instead
protected void	deleteAllReindexCoreDocuments() This method deletes all of the documents from SolrContext.getReindexServer()
protected Long	determineDisplayOrderValue(CatalogStructure cache, Long categoryId, Product product) Utility method to first try to read display order from cache before making Database call.
List<Locale>	getAllLocales()
protected Map<String,Object>	getPropertyValues(Object indexedItem, Field field, FieldType fieldType, List<Locale> locales) Returns a map of prefix to value for the requested attributes.
protected Tuple<CatalogStructure,Boolean>	getStructuredCache(Product product)
boolean	isReindexInProcess() Allows a query to determine if a full reindex is currently being performed.
void	logDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents) Prints out the docs to the trace logger
void	optimizeIndex(org.apache.solr.client.solrj.SolrServer server) Triggers the Solr optimize index function on the given server.
void	performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation) SolrIndexService exposes SolrIndexService.buildIncrementalIndex(int, int, boolean).
protected List<Product>	readAllActiveProducts() This method to read all active products will be slow if you have a large catalog.
protected List<Product>	readAllActiveProducts(int page, int pageSize) This method to read active products utilizes paging to improve performance over readAllActiveProducts().
protected List<Sku>	readAllActiveSkus(int page, int pageSize) This method to read active skus utilizes paging to improve performance.
void	rebuildIndex() Rebuilds the current index.
void	restoreState(Object[] pack) Restores state that was saved prior to indexing that might have been altered.
Object[]	saveState() Saves some global context that might be altered during indexing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOCK_OBJECT

protected final Object LOCK_OBJECT

IS_LOCKED

protected boolean IS_LOCKED

errorOnConcurrentReIndex

@Value(value="${solr.index.errorOnConcurrentReIndex}")
protected boolean errorOnConcurrentReIndex

pageSize

@Value(value="${solr.index.product.pageSize}")
protected int pageSize

useSku

@Value(value="${solr.index.use.sku}")
protected boolean useSku

commit

@Value(value="${solr.index.commit}")
protected boolean commit

softCommit

@Value(value="${solr.index.softCommit}")
protected boolean softCommit

waitSearcher

@Value(value="${solr.index.waitSearcher}")
protected boolean waitSearcher

waitFlush

@Value(value="${solr.index.waitFlush}")
protected boolean waitFlush

productDao

protected ProductDao productDao

skuDao

protected SkuDao skuDao

categoryDao

protected CategoryDao categoryDao

fieldDao

protected FieldDao fieldDao

localeService

protected LocaleService localeService

shs

protected SolrHelperService shs

extensionManager

protected SolrSearchServiceExtensionManager extensionManager

transactionManager

protected org.springframework.transaction.PlatformTransactionManager transactionManager

solrIndexDao

protected SolrIndexDao solrIndexDao

sandBoxHelper

protected SandBoxHelper sandBoxHelper

Constructor Detail

SolrIndexServiceImpl

public SolrIndexServiceImpl()

Method Detail

performCachedOperation

public void performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation)
                            throws ServiceException

Description copied from interface: SolrIndexService

SolrIndexService exposes SolrIndexService.buildIncrementalIndex(int, int, boolean). By wrapping the call to this method inside of a SolrIndexCachedOperation.CacheOperation, a single cache will be used for all the contained calls to buildIncrementalIndex. Here's an example: {@code performCachedOperation(new SolrIndexCachedOperation.CacheOperation() {

Specified by:

performCachedOperation in interface SolrIndexService

Parameters:

cacheOperation - the block of code to perform using a single cache for best performance

Throws:

ServiceException

isReindexInProcess

public boolean isReindexInProcess()

Description copied from interface: SolrIndexService

Allows a query to determine if a full reindex is currently being performed. NOTE: There is no guarantee that reindexing is not happening in another process or another server. This method simply indicates whether this instance, within a single JVM is currently performing a reindex.

Specified by:

isReindexInProcess in interface SolrIndexService

Returns:

rebuildIndex

public void rebuildIndex()
                  throws ServiceException,
                         IOException

Description copied from interface: SolrIndexService

Rebuilds the current index.

Specified by:

rebuildIndex in interface SolrIndexService

Throws:

ServiceException

IOException

deleteAllDocuments

@Deprecated
protected void deleteAllDocuments()
                                       throws ServiceException

Deprecated. use deleteAllReindexCoreDocuments() instead

This method deletes all of the documents from SolrContext.getReindexServer()

Throws:

ServiceException - if there was a problem removing the documents

deleteAllReindexCoreDocuments

protected void deleteAllReindexCoreDocuments()
                                      throws ServiceException

This method deletes all of the documents from SolrContext.getReindexServer()

Throws:

ServiceException - if there was a problem removing the documents

buildIncrementalIndex

protected void buildIncrementalIndex(int page,
                                     int pageSize)
                              throws ServiceException

Throws:

ServiceException

buildIncrementalProductIndex

public void buildIncrementalProductIndex(List<Product> products,
                                         boolean useReindexServer)
                                  throws ServiceException

Description copied from interface: SolrIndexService

This can be used in lieu of passing in page sizes, The reason is that one might want to apply filters or only index certain products.

Specified by:

buildIncrementalProductIndex in interface SolrIndexService

Throws:

ServiceException

buildIncrementalSkuIndex

public void buildIncrementalSkuIndex(List<Sku> skus,
                                     boolean useReindexServer)
                              throws ServiceException

Description copied from interface: SolrIndexService

This can be used in lieu of passing in page sizes, The reason is that one might want to apply filters or only index certain skus.

Specified by:

buildIncrementalSkuIndex in interface SolrIndexService

Throws:

ServiceException

buildIncrementalIndex

public void buildIncrementalIndex(int page,
                                  int pageSize,
                                  boolean useReindexServer)
                           throws ServiceException

Description copied from interface: SolrIndexService

The internal method for building indexes. This is exposed via this interface in case someone would like to more granularly control the indexing strategy.

Specified by:

buildIncrementalIndex in interface SolrIndexService

useReindexServer - - if set to false will index directly on the primary server

Throws:

ServiceException

See Also:

SolrIndexService.restoreState(Object[])

readAllActiveProducts

protected List<Product> readAllActiveProducts()

This method to read all active products will be slow if you have a large catalog. In this case, you will want to read the products in a different manner. For example, if you know the fields that will be indexed, you can configure a DAO object to only load those fields. You could also use a JDBC based DAO for even faster access. This default implementation is only suitable for small catalogs.

Returns:

the list of all active products to be used by the index building task

readAllActiveProducts

protected List<Product> readAllActiveProducts(int page,
                                              int pageSize)

This method to read active products utilizes paging to improve performance over readAllActiveProducts(). While not optimal, this will reduce the memory required to load large catalogs. It could still be improved for specific implementations by only loading fields that will be indexed or by accessing the database via direct JDBC (instead of Hibernate).

Returns:

the list of all active products to be used by the index building task

Since:

2.2.0

readAllActiveSkus

protected List<Sku> readAllActiveSkus(int page,
                                      int pageSize)

This method to read active skus utilizes paging to improve performance. While not optimal, this will reduce the memory required to load large catalogs. It could still be improved for specific implementations by only loading fields that will be indexed or by accessing the database via direct JDBC (instead of Hibernate).

Returns:

the list of all active SKUs to be used by the index building task

Since:

2.2.0

getAllLocales

public List<Locale> getAllLocales()

Specified by:

getAllLocales in interface SolrIndexService

Returns:

a list of all possible locale prefixes to consider

buildDocument

public org.apache.solr.common.SolrInputDocument buildDocument(Sku sku,
                                                              List<Field> fields,
                                                              List<Locale> locales)

Description copied from interface: SolrIndexService

Given a sku, fields that relate to that sku, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.

Specified by:

buildDocument in interface SolrIndexService

Returns:

the document

buildDocument

public org.apache.solr.common.SolrInputDocument buildDocument(Product product,
                                                              List<Field> fields,
                                                              List<Locale> locales)

Description copied from interface: SolrIndexService

Given a product, fields that relate to that product, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.

Specified by:

buildDocument in interface SolrIndexService

Returns:

the document

attachAdditionalDocumentFields

protected void attachAdditionalDocumentFields(Sku sku,
                                              org.apache.solr.common.SolrInputDocument document)

Implementors can extend this and override this method to add additional fields to the Solr document.

Parameters:

sku -

document -

attachAdditionalDocumentFields

protected void attachAdditionalDocumentFields(Product product,
                                              org.apache.solr.common.SolrInputDocument document)

Implementors can extend this and override this method to add additional fields to the Solr document.

Parameters:

product -

document -

attachBasicDocumentFields

protected void attachBasicDocumentFields(Sku sku,
                                         org.apache.solr.common.SolrInputDocument document)

Adds the ID, category, and explicitCategory fields for the product or sku to the document

Parameters:

sku -

document -

attachBasicDocumentFields

protected void attachBasicDocumentFields(Product product,
                                         org.apache.solr.common.SolrInputDocument document)

buildCategoryDocument

protected void buildCategoryDocument(Long categoryId,
                                     Product product,
                                     org.apache.solr.common.SolrInputDocument document,
                                     CatalogStructure cache)

Parameters:

categoryId -

product -

document -

cache -

getStructuredCache

protected Tuple<CatalogStructure,Boolean> getStructuredCache(Product product)

Parameters:

product -

Returns:

determineDisplayOrderValue

protected Long determineDisplayOrderValue(CatalogStructure cache,
                                          Long categoryId,
                                          Product product)

Utility method to first try to read display order from cache before making Database call.

Parameters:

cache -

categoryId -

product -

Returns:

buildFullCategoryHierarchy

protected void buildFullCategoryHierarchy(org.apache.solr.common.SolrInputDocument document,
                                          CatalogStructure cache,
                                          Long categoryId,
                                          Set<Long> indexedParents)

Walk the category hierarchy upwards, adding a field for each level to the solr document

Parameters:

document - the solr document for the product

cache - the catalog structure cache

categoryId - the current category id

getPropertyValues

protected Map<String,Object> getPropertyValues(Object indexedItem,
                                               Field field,
                                               FieldType fieldType,
                                               List<Locale> locales)
                                        throws IllegalAccessException,
                                               InvocationTargetException,
                                               NoSuchMethodException

Returns a map of prefix to value for the requested attributes. For example, if the requested field corresponds to a Sku's description and the locales list has the en_US locale and the es_ES locale, the resulting map could be { "en_US" : "A description", "es_ES" : "Una descripcion" }

Parameters:

field -

fieldType -

locales -

Returns:

the value of the property

Throws:

IllegalAccessException

InvocationTargetException

NoSuchMethodException

convertToMappedProperty

@Deprecated
protected String convertToMappedProperty(String propertyName,
                                                     String listPropertyName,
                                                     String mapPropertyName)

Deprecated. see SolrHelperService.getPropertyValue()

Converts a propertyName to one that is able to reference inside a map. For example, consider the property in Product that references a List, "productAttributes". Also consider the utility method in Product called "mappedProductAttributes", which returns a map of the ProductAttributes keyed by the name property in the ProductAttribute. Given the parameters "productAttributes.heatRange", "productAttributes", "mappedProductAttributes" (which would represent a property called "productAttributes.heatRange" that references a specific ProductAttribute inside of a product whose "name" property is equal to "heatRange", this method will convert this property to mappedProductAttributes(heatRange).value, which is then usable by the standard beanutils PropertyUtils class to get the value.

Parameters:

propertyName -

listPropertyName -

mapPropertyName -

Returns:

the converted property name

saveState

public Object[] saveState()

Description copied from interface: SolrIndexService

Saves some global context that might be altered during indexing.

Specified by:

saveState in interface SolrIndexService

Returns:

restoreState

public void restoreState(Object[] pack)

Description copied from interface: SolrIndexService

Restores state that was saved prior to indexing that might have been altered.

Specified by:

restoreState in interface SolrIndexService

See Also:

SolrIndexService.saveState()

optimizeIndex

public void optimizeIndex(org.apache.solr.client.solrj.SolrServer server)
                   throws ServiceException,
                          IOException

Description copied from interface: SolrIndexService

Triggers the Solr optimize index function on the given server. NOTE: This should rarely be called.

Specified by:

optimizeIndex in interface SolrIndexService

Throws:

ServiceException

IOException

commit

public void commit(org.apache.solr.client.solrj.SolrServer server)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

Allows a commit to be called. By default, the details of the commit will depend on system properties, including: solr.index.commit - if false, then no commit will be performed. autoCommit (and autoSoftCommit) should be configured in Solr. solr.index.softCommit - indicates if a soft commit should be performed solr.index.waitSearcher - indicates if the process should wait for a searcher to be configured solr.index.waitFlush - indicates if the process should wait for a flush to disk

Specified by:

commit in interface SolrIndexService

Throws:

ServiceException

IOException

commit

public void commit(org.apache.solr.client.solrj.SolrServer server,
                   boolean softCommit,
                   boolean waitSearcher,
                   boolean waitFlush)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

This allows an external caller to force a commit to the SolrServer. See Solr Documentation for additional details. If using softCommit, you should ensure that a hardCommit is performed, either using autoCommit, or at the end of the commit process to flush the changes to the disk. Note that this method will force a commit even if solr.index.commit=false

Specified by:

commit in interface SolrIndexService

Parameters:

server - - the SolrServer to update

softCommit - - soft commit is an efficient commit that does not write the data to the file system

waitSearcher - - whether or not to wait for a new searcher to be created

waitFlush - - whether or not to wait for a flush to disk.

Throws:

ServiceException

IOException

logDocuments

public void logDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents)

Description copied from interface: SolrIndexService

Prints out the docs to the trace logger

Specified by:

logDocuments in interface SolrIndexService

convertDisplayOrderToLong

protected Long convertDisplayOrderToLong(BigDecimal displayOrder)

We multiply the BigDecimal by 1,000,000 to maintain any possible decimals in use the displayOrder value.

Returns: