Property Index in AEM - Index properties and steps to create custom index

Property Index is for queries involving property constraints - equality, exist/not null check on property.

Oak property Index: 
  • Node of type "oak:QueryIndexDefinition" with 

  • Mandatory Properties
    NameTypeValue
    typeStringproperty
    propertyNamesName[]list of properties for which the index needs to be created.
    Example:
    jcr:title, cq:lastModified, hideInNav property for a page content
    cq:ParentPath property for an AEM Asset
    Optional/Supporting Properties
    declaringNodeTypesName[]list of node types for which the index is applicable for
    Example:
    cq:PageContent
    rep:Authorizable
    uniqueBooleantrue 
    • Applicable for queries with unique constraint
    • This property should be used along with declaringNodeTypes
    • Example: /oak:index/authorizableId
    includedPathsString[]paths that are included in this index. If not set, it is "/"
    For queries with path restrictions that is not excluded and part of included paths
    Example:

    Scenario 1: If two properties are used together. (as highlighted in red)
    Index will be picked if
    • query has "path" predicate to be "/content/dam/we-retail/en/people" 
    will not be picked if
    • query has "path" predicate to be "/content/dam/we-retail/en/products"
    • query has "path" predicate to be "/content/dam/we-retail/en" (this path is available as part of excluded paths)
    Scenario 2: If includedPaths property alone is mentioned. Then
    Index will be picked if
    • query has "path" predicate to be "/content/dam/we-retail/en" or 
    • anything under  "/content/dam/we-retail/en" (/en/.*) heirarchy.
    will not be picked if
    • query has "path" predicate to be "/content/dam/we-retail"
    excludedPathsString[]paths that are excluded in this index. If not set, it is "none"
    For queries with path restrictions that is not excluded and part of included paths
    Example:
    Scenario 1:
    Same as above
    Scenario 2: In the same example snap aboveIf excludedPaths property alone is mentioned
    Index will not be picked if
    • query has "path" predicate to be "/content/dam/we-retail/en/products"
    will be picked if
    • query has "path" predicate to be "/content/dam/we-retail/en/.*" 
    valuePatternStringRegex of all indexed values.
    The index will be used when the property value matches the pattern
    Example:
    If a property value is all lowercase alphabets, then valuePattern -> [a-z]+
    Index will be picked if 
    • query has "property.value" predicate matching this pattern.
    valueExcludedPrefixesString[]Prefix of property values to be excluded
    Index will be picked if the property value does not start with the given prefix 
    Example:
    Index will be picked if 
    • query has "property.value" predicate that doesn't start with prefix
    Use case:
    When cq:tag is tagged to page/asset, value will be namespace:tagid
    Index definition can then restricted for a specific namespace value using this valueExcluded/valueIncludedPrefixes
    valueIncludedPrefixesString[]Prefix of property values to be included
    Index will be picked if the property value starts with the given prefix 
    Used for equality, like conditions.
    Example:
    Index will be picked if 
    • query has "property.value" predicate that start with the prefix
    entryCountLongestimated number of path entries 
    This is related to cost estimation. high entry count -> high cost
    keyCountLongestimated number of key entries
    Again related to cost estimation. It is inversely proportional here.
    Low key count -> high cost; high key count -> low cost.
    useIfExistsStringUseful in blue-green deployments, when using Composite Node Store
    (Since Oak version 1.10.0)
    In AEM, it is 6.5 version which has Oak version to be 1.10.2
    Property to support async reindexing
    reindex-asyncBooleantrue
    (Asynchronous Reindexing is explained in detail below with video demo)
    Properties that gets created automatically
    reindexBooleanfalse
    reindexCountLong1, very first time when the property index is created.
    number gets incremented by 1 everytime reindex is triggered.
    async Stringasync-reindex 
    (Related to asynchronous reindexing, explained below)
Steps to create custom property index:
  • Sample query involving property constraint - Get all live copy pages which has overwritten the properties/which has cq:propertyInheritanceCancelled

    • path=/content/we-retail
    • type=cq:Page
    • 1_property=@jcr:content/cq:propertyInheritanceCancelled
    • 1_property.operation=exists
    • p.limit=-1

  • "Explain Query" response for the above query :

  • explainquery-console

  • Now is the time to create Property Index for the property - cq:propertyInheritanceCancelled

    • Navigate to CRXDE -> /oak:index node
    • Being on oak:index node, Create new node of 
      • name  -> cqPropertyInheritanceCancelled (in par with naming convention of OOB indexes) 
      • type  -> oak:QueryIndexDefinition
    • Add below mandatory properties
      • NameTypeValue
        typeStringproperty
        propertyNamesName[]cq:propertyInheritanceCancelled
    • Behind the scene: Property Index works in Synchronous mode of Indexing, the moment index definition is persisted, it will start indexing the related content. 
    • On "Refresh" of newly created index node  -> below two properties are reflected automatically
      • NameTypeValue
        reindex Booleanfalse
        reindexCount Long1
    • Note: 
      • Refresh to reflect the properties that are automatically created. 
      • New property index node along with two mandatory properties to be persisted in one shot. We will get exception otherwise - highlighted in demo video below

  • Execute sample query again and observe the index used: 

  • explain-query-console

  • Observation/Conclusion:
    • For a query involving property constraint, if both property and lucene index is available -> In general, Property Index will be picked. (Cost value is low for property index compared to lucene index as evident from the above screenshot for this use case)
    • Initially when we executed the query without property index, lucene index related to the type(cq:Page) we used in the query is picked which is "cqPageLucene"
With this, we are done with creating Property Index for a specific property with mandatory properties.
Usage of other supporting properties in a Property Index will be drafted in next follow up post with video demo for better clarity.

Reindexing Property Index:
  • One of the possible cause for reindexing is, change in the existing index definition. Reindexing can be done in two ways for a Property Index.
    • Synchronous way
      • By updating reindex property to true
      • Once it is finished indexing, it will automatically set it back to false with an update to reindexCount value(value will be incremented by 1 from the existing value)
    • Asynchronous reindexing
      • Property index updates is assigned to a dedicated background job and when the indexing is done, the property definition will be switched back to synchronous updates mode
      • To enable this, 
        • ActionNameTypeValue
          Addreindex-asyncBooleantrue
          Update
          (value from false->true)
          reindexBooleantrue
      • Note: Both these updates to be persisted in one shot (do not trigger save with one property update)
      • As a result of this, below property would be automatically created. 
        • NameTypeValue
          asyncStringasync-reindex
      • Navigate to JMX console - http://localhost:4502/system/console/jmx and invoke the MBean - PropertyIndexAsyncReindex#startPropertyIndexAsyncReindex.
      • Once when the job is completed, 
        • automatically created property async will be removed
        • reindex property would be set to false
        • reindexCount would be incremented to 1 from its existing value.
Video Demo:
  • Creating Property Index for cq:propertyInheritanceCancelled:

  • Property Index reindexing - Asynchronous

Comments

Popular posts from this blog

Embedding Third party dependency/OSGi bundle in AEM application hosted in AEMasCS

OSGI Factory Configuration implementation

Creation of Template Types for Editable templates