org.broadleafcommerce.core.search.service.solr.index

Class SolrIndexServiceImpl

java.lang.Object

org.broadleafcommerce.core.search.service.solr.index.SolrIndexServiceImpl

All Implemented Interfaces:

SolrIndexService

@Service(value="blSolrIndexService")
public class SolrIndexServiceImpl
extends Object
implements SolrIndexService

Responsible for building and rebuilding the Solr index

Author:

Andre Azzolini (apazzolini), Jeff Fischer

Field Summary

Fields

Modifier and Type	Field and Description
protected CatalogService	catalogService
protected boolean	commit
protected boolean	errorOnConcurrentReIndex
protected SolrIndexServiceExtensionManager	extensionManager
protected FieldDao	fieldDao
protected IndexFieldDao	indexFieldDao
protected LocaleService	localeService
protected int	pageSize
protected ProductDao	productDao
protected SandBoxHelper	sandBoxHelper
protected SearchFacetDao	searchFacetDao
protected SolrHelperService	shs
protected boolean	softCommit
protected SolrConfiguration	solrConfiguration
protected SolrIndexDao	solrIndexDao
protected org.springframework.transaction.PlatformTransactionManager	transactionManager
protected boolean	waitFlush
protected boolean	waitSearcher

Constructor Summary

Constructors

Constructor and Description
SolrIndexServiceImpl()

Method Summary

Modifier and Type	Method and Description
void	addDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents)
protected void	attachAdditionalDocumentFields(Indexable indexable, org.apache.solr.common.SolrInputDocument document) Implementors can extend this and override this method to add additional fields to the Solr document.
protected void	attachBasicDocumentFields(Indexable indexable, org.apache.solr.common.SolrInputDocument document)
void	attachIndexableDocumentFields(org.apache.solr.common.SolrInputDocument document, Indexable indexable, List<IndexField> fields, List<Locale> locales) Iterates through the fields for this indexable and indexes any SearchField's or SearchFacet's.
org.apache.solr.common.SolrInputDocument	buildDocument(Indexable indexable, List<IndexField> fields, List<Locale> locales) Given a product, fields that relate to that product, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.
protected void	buildFullCategoryHierarchy(org.apache.solr.common.SolrInputDocument document, CatalogStructure cache, Long categoryId, Set<Long> indexedParents) Walk the category hierarchy upwards, adding a field for each level to the solr document
protected Long	buildIncrementalIndex(int pageSize, Long lastId, SolrIndexOperation operation)
Collection<org.apache.solr.common.SolrInputDocument>	buildIncrementalIndex(List<? extends Indexable> indexables, org.apache.solr.client.solrj.SolrClient solrServer) Builds a set of Indexables against the given SolrServer
Collection<org.apache.solr.common.SolrInputDocument>	buildIncrementalIndex(String collection, List<? extends Indexable> indexables, org.apache.solr.client.solrj.SolrClient solrServer) Builds a set of Indexables against the given SolrServer
void	buildIndex() Handles all the document building for the current index rebuild.
protected String	buildPageNumberMessage(int page, Long totalPages)
void	commit(org.apache.solr.client.solrj.SolrClient server) Allows a commit to be called.
void	commit(org.apache.solr.client.solrj.SolrClient server, boolean softCommit, boolean waitSearcher, boolean waitFlush) This allows an external caller to force a commit to the SolrClient.
void	commit(String collection, org.apache.solr.client.solrj.SolrClient server) Allows a commit to be called.
void	commit(String collection, org.apache.solr.client.solrj.SolrClient server, boolean softCommit, boolean waitSearcher, boolean waitFlush) This allows an external caller to force a commit to the SolrClient.
protected Long	convertDisplayOrderToLong(CatalogStructure cache, String displayOrderKey) We multiply the BigDecimal by 1,000,000 to maintain any possible decimals in use the displayOrder value.
protected String	convertToMappedProperty(String propertyName, String listPropertyName, String mapPropertyName) Deprecated. see SolrHelperService.getPropertyValue()
protected Long	countIndexableItems()
protected void	deleteAllDocuments() Deprecated. use deleteAllReindexCoreDocuments() instead
void	deleteAllDocuments(org.apache.solr.client.solrj.SolrClient server)
void	deleteAllDocuments(String collection, org.apache.solr.client.solrj.SolrClient server)
void	deleteAllNamespaceDocuments(org.apache.solr.client.solrj.SolrClient server)
void	deleteAllNamespaceDocuments(String collection, org.apache.solr.client.solrj.SolrClient server)
protected void	deleteAllReindexCoreDocuments() This method deletes all of the documents from SolrContext#getReindexServer()
void	deleteByQuery(String deleteQuery)
void	executeSolrIndexOperation(SolrIndexOperation operation) Executes the given operation in the correct method order
List<Locale>	getAllLocales()
protected Map<String,Object>	getPropertyValues(Indexable indexedItem, Field field, FieldType fieldType, List<Locale> locales) Returns a map of prefix to value for the requested attributes.
SolrIndexOperation	getReindexOperation() Creates the SolrIndexOperation for rebuilding the current index, used by SolrIndexService.buildIndex().
protected long	getTotalPageCount(Long numItemsToIndex)
void	logDeleteQuery(String deleteQuery)
void	logDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents) Prints out the docs to the trace logger
void	optimizeIndex(org.apache.solr.client.solrj.SolrClient server) Triggers the Solr optimize index function on the given server.
void	optimizeIndex(String collection, org.apache.solr.client.solrj.SolrClient server) Triggers the Solr optimize index function on the given server.
void	performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation) SolrIndexService exposes #buildIncrementalIndex(int, int, boolean).
void	postBuildIndex() Executed after we do any indexing when rebuilding the current index.
void	preBuildIndex() Executed before we do any indexing when rebuilding the index.
protected List<? extends Indexable>	readAllActiveIndexables(int pageSize, Long lastId)
void	rebuildIndex() Executes a full rebuild of the Solr index.
void	restoreState(Object[] pack) Restores state that was saved prior to indexing that might have been altered.
Object[]	saveState() Saves some global context that might be altered during indexing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

solrConfiguration

@Qualifier(value="blCatalogSolrConfiguration")
 @Autowired(required=false)
protected SolrConfiguration solrConfiguration

errorOnConcurrentReIndex

@Value(value="${solr.index.errorOnConcurrentReIndex}")
protected boolean errorOnConcurrentReIndex

pageSize

@Value(value="${solr.index.product.pageSize}")
protected int pageSize

commit

@Value(value="${solr.index.commit}")
protected boolean commit

softCommit

@Value(value="${solr.index.softCommit}")
protected boolean softCommit

waitSearcher

@Value(value="${solr.index.waitSearcher}")
protected boolean waitSearcher

waitFlush

@Value(value="${solr.index.waitFlush}")
protected boolean waitFlush

productDao

protected ProductDao productDao

catalogService

protected CatalogService catalogService

fieldDao

protected FieldDao fieldDao

localeService

protected LocaleService localeService

shs

protected SolrHelperService shs

extensionManager

protected SolrIndexServiceExtensionManager extensionManager

transactionManager

protected org.springframework.transaction.PlatformTransactionManager transactionManager

solrIndexDao

protected SolrIndexDao solrIndexDao

sandBoxHelper

protected SandBoxHelper sandBoxHelper

searchFacetDao

protected SearchFacetDao searchFacetDao

indexFieldDao

protected IndexFieldDao indexFieldDao

Constructor Detail

SolrIndexServiceImpl

public SolrIndexServiceImpl()

Method Detail

performCachedOperation

public void performCachedOperation(SolrIndexCachedOperation.CacheOperation cacheOperation)
                            throws ServiceException

Description copied from interface: SolrIndexService

SolrIndexService exposes #buildIncrementalIndex(int, int, boolean). By wrapping the call to this method inside of a SolrIndexCachedOperation.CacheOperation, a single cache will be used for all the contained calls to buildIncrementalIndex. Here's an example: {@code performCachedOperation(new SolrIndexCachedOperation.CacheOperation() {

Specified by:

performCachedOperation in interface SolrIndexService

Parameters:

cacheOperation - the block of code to perform using a single cache for best performance

Throws:

ServiceException

rebuildIndex

public void rebuildIndex()
                  throws ServiceException,
                         IOException

Description copied from interface: SolrIndexService

Executes a full rebuild of the Solr index. This will rebuild the index on a separate core/collection and then swap out the active core/collection with the new version of the index (essentially replacing all documents that are currently in the index).

The order of methods that are apart of rebuilding the entire index:

1. SolrIndexService.preBuildIndex()

2. SolrIndexService.buildIndex()

3. SolrIndexService.postBuildIndex()

Specified by:

rebuildIndex in interface SolrIndexService

Throws:

ServiceException

IOException

preBuildIndex

public void preBuildIndex()
                   throws ServiceException

Description copied from interface: SolrIndexService

Executed before we do any indexing when rebuilding the index. Usually this handles deleting the current index. This is called at the beginning of SolrIndexService.rebuildIndex()

Specified by:

preBuildIndex in interface SolrIndexService

Throws:

ServiceException

buildIndex

public void buildIndex()
                throws IOException,
                       ServiceException

Description copied from interface: SolrIndexService

Handles all the document building for the current index rebuild. This is where all of the SolrIndexOperation's need to be created, executed and the documents built and added to the Solr index

This is the method that should be overridden to specify which operations should be run to build the correct index.

Specified by:

buildIndex in interface SolrIndexService

Throws:

IOException

ServiceException

See Also:

#rebuildIndex()}, #preBuildIndex()}

postBuildIndex

public void postBuildIndex()
                    throws IOException,
                           ServiceException

Description copied from interface: SolrIndexService

Executed after we do any indexing when rebuilding the current index. Usually this handles optimizing the index and swapping the cores.

Specified by:

postBuildIndex in interface SolrIndexService

Throws:

IOException

ServiceException

getReindexOperation

public SolrIndexOperation getReindexOperation()

Description copied from interface: SolrIndexService

Creates the SolrIndexOperation for rebuilding the current index, used by SolrIndexService.buildIndex(). This is the primary index operation used to rebuild the index.

Specified by:

getReindexOperation in interface SolrIndexService

Returns:

a SolrIndexOperation capable of rebuilding the current index

executeSolrIndexOperation

public void executeSolrIndexOperation(SolrIndexOperation operation)
                               throws ServiceException,
                                      IOException

Description copied from interface: SolrIndexService

Executes the given operation in the correct method order

Specified by:

executeSolrIndexOperation in interface SolrIndexService

Parameters:

operation - the SolrIndexOperation that is to be executed

Throws:

ServiceException

IOException

getTotalPageCount

protected long getTotalPageCount(Long numItemsToIndex)

buildPageNumberMessage

protected String buildPageNumberMessage(int page,
                                        Long totalPages)

countIndexableItems

protected Long countIndexableItems()

Returns:

deleteAllDocuments

@Deprecated
protected void deleteAllDocuments()
                                       throws ServiceException

Deprecated. use deleteAllReindexCoreDocuments() instead

This method deletes all of the documents from SolrContext#getReindexServer()

Throws:

ServiceException - if there was a problem removing the documents

deleteAllReindexCoreDocuments

protected void deleteAllReindexCoreDocuments()
                                      throws ServiceException

This method deletes all of the documents from SolrContext#getReindexServer()

Throws:

ServiceException - if there was a problem removing the documents

deleteAllNamespaceDocuments

public void deleteAllNamespaceDocuments(String collection,
                                        org.apache.solr.client.solrj.SolrClient server)
                                 throws ServiceException

Specified by:

deleteAllNamespaceDocuments in interface SolrIndexService

Throws:

ServiceException

deleteAllDocuments

public void deleteAllDocuments(String collection,
                               org.apache.solr.client.solrj.SolrClient server)
                        throws ServiceException

Specified by:

deleteAllDocuments in interface SolrIndexService

Throws:

ServiceException

buildIncrementalIndex

protected Long buildIncrementalIndex(int pageSize,
                                     Long lastId,
                                     SolrIndexOperation operation)
                              throws ServiceException

Throws:

ServiceException

buildIncrementalIndex

public Collection<org.apache.solr.common.SolrInputDocument> buildIncrementalIndex(String collection,
                                                                                  List<? extends Indexable> indexables,
                                                                                  org.apache.solr.client.solrj.SolrClient solrServer)
                                                                           throws ServiceException

Description copied from interface: SolrIndexService

Builds a set of Indexables against the given SolrServer

Specified by:

buildIncrementalIndex in interface SolrIndexService

Parameters:

collection - The collection to be used for incremental indexing

indexables - the list of items to index

solrServer - if non-null, adds and commits the indexed documents to the server. If this is null, this will simply return the documents that were built from indexables

Returns:

the SolrInputDocuments that were built from the given indexables

Throws:

ServiceException

readAllActiveIndexables

protected List<? extends Indexable> readAllActiveIndexables(int pageSize,
                                                            Long lastId)

getAllLocales

public List<Locale> getAllLocales()

Specified by:

getAllLocales in interface SolrIndexService

Returns:

a list of all possible locale prefixes to consider

buildDocument

public org.apache.solr.common.SolrInputDocument buildDocument(Indexable indexable,
                                                              List<IndexField> fields,
                                                              List<Locale> locales)

Description copied from interface: SolrIndexService

Given a product, fields that relate to that product, and a list of locales and pricelists, builds a SolrInputDocument to be added to the Solr index.

Specified by:

buildDocument in interface SolrIndexService

Returns:

the document

attachIndexableDocumentFields

public void attachIndexableDocumentFields(org.apache.solr.common.SolrInputDocument document,
                                          Indexable indexable,
                                          List<IndexField> fields,
                                          List<Locale> locales)

Description copied from interface: SolrIndexService

Iterates through the fields for this indexable and indexes any SearchField's or SearchFacet's.

Specified by:

attachIndexableDocumentFields in interface SolrIndexService

attachAdditionalDocumentFields

protected void attachAdditionalDocumentFields(Indexable indexable,
                                              org.apache.solr.common.SolrInputDocument document)

Implementors can extend this and override this method to add additional fields to the Solr document.

Parameters:

sku -

document -

attachBasicDocumentFields

protected void attachBasicDocumentFields(Indexable indexable,
                                         org.apache.solr.common.SolrInputDocument document)

buildFullCategoryHierarchy

protected void buildFullCategoryHierarchy(org.apache.solr.common.SolrInputDocument document,
                                          CatalogStructure cache,
                                          Long categoryId,
                                          Set<Long> indexedParents)

Walk the category hierarchy upwards, adding a field for each level to the solr document

Parameters:

document - the solr document for the product

cache - the catalog structure cache

categoryId - the current category id

getPropertyValues

protected Map<String,Object> getPropertyValues(Indexable indexedItem,
                                               Field field,
                                               FieldType fieldType,
                                               List<Locale> locales)
                                        throws IllegalAccessException,
                                               InvocationTargetException,
                                               NoSuchMethodException

Returns a map of prefix to value for the requested attributes. For example, if the requested field corresponds to a Sku's description and the locales list has the en_US locale and the es_ES locale, the resulting map could be { "en_US" : "A description", "es_ES" : "Una descripcion" }

Parameters:

product -

sku -

field -

fieldType -

locales -

Returns:

the value of the property

Throws:

IllegalAccessException

InvocationTargetException

NoSuchMethodException

convertToMappedProperty

@Deprecated
protected String convertToMappedProperty(String propertyName,
                                                     String listPropertyName,
                                                     String mapPropertyName)

Deprecated. see SolrHelperService.getPropertyValue()

Converts a propertyName to one that is able to reference inside a map. For example, consider the property in Product that references a List, "productAttributes". Also consider the utility method in Product called "mappedProductAttributes", which returns a map of the ProductAttributes keyed by the name property in the ProductAttribute. Given the parameters "productAttributes.heatRange", "productAttributes", "mappedProductAttributes" (which would represent a property called "productAttributes.heatRange" that references a specific ProductAttribute inside of a product whose "name" property is equal to "heatRange", this method will convert this property to mappedProductAttributes(heatRange).value, which is then usable by the standard beanutils PropertyUtils class to get the value.

Parameters:

propertyName -

listPropertyName -

mapPropertyName -

Returns:

the converted property name

saveState

public Object[] saveState()

Description copied from interface: SolrIndexService

Saves some global context that might be altered during indexing.

Specified by:

saveState in interface SolrIndexService

Returns:

restoreState

public void restoreState(Object[] pack)

Description copied from interface: SolrIndexService

Restores state that was saved prior to indexing that might have been altered.

Specified by:

restoreState in interface SolrIndexService

See Also:

SolrIndexService.saveState()

optimizeIndex

public void optimizeIndex(String collection,
                          org.apache.solr.client.solrj.SolrClient server)
                   throws ServiceException,
                          IOException

Description copied from interface: SolrIndexService

Triggers the Solr optimize index function on the given server. This is typically called at the end of a SolrIndexService.rebuildIndex()

Specified by:

optimizeIndex in interface SolrIndexService

Parameters:

collection - The collection to optimize

server - The server to use to optimze the given collection

Throws:

ServiceException

IOException

commit

public void commit(String collection,
                   org.apache.solr.client.solrj.SolrClient server)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

Allows a commit to be called. By default, the details of the commit will depend on system properties, including: solr.index.commit - if false, then no commit will be performed. autoCommit (and autoSoftCommit) should be configured in Solr. solr.index.softCommit - indicates if a soft commit should be performed solr.index.waitSearcher - indicates if the process should wait for a searcher to be configured solr.index.waitFlush - indicates if the process should wait for a flush to disk

Specified by:

commit in interface SolrIndexService

Parameters:

collection - The collection to commit

server - The server used to commit the specified collection

Throws:

ServiceException

IOException

commit

public void commit(String collection,
                   org.apache.solr.client.solrj.SolrClient server,
                   boolean softCommit,
                   boolean waitSearcher,
                   boolean waitFlush)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

This allows an external caller to force a commit to the SolrClient. See Solr Documentation for additional details. If using softCommit, you should ensure that a hardCommit is performed, either using autoCommit, or at the end of the commit process to flush the changes to the disk. Note that this method will force a commit even if solr.index.commit=false

Specified by:

commit in interface SolrIndexService

Parameters:

collection - The collection to commit

server - - the SolrClient to use to commit the provided collection

softCommit - - soft commit is an efficient commit that does not write the data to the file system

waitSearcher - - whether or not to wait for a new searcher to be created

waitFlush - - whether or not to wait for a flush to disk.

Throws:

ServiceException

IOException

logDocuments

public void logDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents)

Description copied from interface: SolrIndexService

Prints out the docs to the trace logger

Specified by:

logDocuments in interface SolrIndexService

convertDisplayOrderToLong

protected Long convertDisplayOrderToLong(CatalogStructure cache,
                                         String displayOrderKey)

We multiply the BigDecimal by 1,000,000 to maintain any possible decimals in use the displayOrder value.

Parameters:

cache -

displayOrderKey -

Returns:

deleteByQuery

public void deleteByQuery(String deleteQuery)
                   throws org.apache.solr.client.solrj.SolrServerException,
                          IOException

Specified by:

deleteByQuery in interface SolrIndexService

Throws:

org.apache.solr.client.solrj.SolrServerException

IOException

addDocuments

public void addDocuments(Collection<org.apache.solr.common.SolrInputDocument> documents)
                  throws IOException,
                         org.apache.solr.client.solrj.SolrServerException

Specified by:

addDocuments in interface SolrIndexService

Throws:

IOException

org.apache.solr.client.solrj.SolrServerException

logDeleteQuery

public void logDeleteQuery(String deleteQuery)

Specified by:

logDeleteQuery in interface SolrIndexService

buildIncrementalIndex

public Collection<org.apache.solr.common.SolrInputDocument> buildIncrementalIndex(List<? extends Indexable> indexables,
                                                                                  org.apache.solr.client.solrj.SolrClient solrServer)
                                                                           throws ServiceException

Description copied from interface: SolrIndexService

Builds a set of Indexables against the given SolrServer

Specified by:

buildIncrementalIndex in interface SolrIndexService

Parameters:

indexables - the list of items to index

solrServer - if non-null, adds and commits the indexed documents to the server. If this is null, this will simply return the documents that were built from indexables

Returns:

the SolrInputDocuments that were built from the given indexables

Throws:

ServiceException

optimizeIndex

public void optimizeIndex(org.apache.solr.client.solrj.SolrClient server)
                   throws ServiceException,
                          IOException

Description copied from interface: SolrIndexService

Triggers the Solr optimize index function on the given server. This is typically called at the end of a SolrIndexService.rebuildIndex()

Specified by:

optimizeIndex in interface SolrIndexService

Parameters:

server - The server to use to optimze the default collection

Throws:

ServiceException

IOException

commit

public void commit(org.apache.solr.client.solrj.SolrClient server)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

Allows a commit to be called. By default, the details of the commit will depend on system properties, including: solr.index.commit - if false, then no commit will be performed. autoCommit (and autoSoftCommit) should be configured in Solr. solr.index.softCommit - indicates if a soft commit should be performed solr.index.waitSearcher - indicates if the process should wait for a searcher to be configured solr.index.waitFlush - indicates if the process should wait for a flush to disk

Specified by:

commit in interface SolrIndexService

Parameters:

server - The server used to commit the default collection

Throws:

ServiceException

IOException

commit

public void commit(org.apache.solr.client.solrj.SolrClient server,
                   boolean softCommit,
                   boolean waitSearcher,
                   boolean waitFlush)
            throws ServiceException,
                   IOException

Description copied from interface: SolrIndexService

This allows an external caller to force a commit to the SolrClient. See Solr Documentation for additional details. If using softCommit, you should ensure that a hardCommit is performed, either using autoCommit, or at the end of the commit process to flush the changes to the disk. Note that this method will force a commit even if solr.index.commit=false

Specified by:

commit in interface SolrIndexService

Parameters:

server - - the SolrClient to commit with

softCommit - - soft commit is an efficient commit that does not write the data to the file system

waitSearcher - - whether or not to wait for a new searcher to be created

waitFlush - - whether or not to wait for a flush to disk.

Throws:

ServiceException

IOException

deleteAllNamespaceDocuments

public void deleteAllNamespaceDocuments(org.apache.solr.client.solrj.SolrClient server)
                                 throws ServiceException

Specified by:

deleteAllNamespaceDocuments in interface SolrIndexService

Throws:

ServiceException

deleteAllDocuments

public void deleteAllDocuments(org.apache.solr.client.solrj.SolrClient server)
                        throws ServiceException

Specified by:

deleteAllDocuments in interface SolrIndexService

Throws:

ServiceException