Lucene Index in AEM - Part 2

This post illustrates about full text search scenario and steps to create custom Lucene Full Text Index.

Lucene Index in AEM

On a high level, for full text search, we need to index all nodes and properties (these are the two major means in which our content is held in the repository) 

For property:
  • In order to make a specific property to be indexed for both full text and property constraint scenario, then in the property definition of respective indexRule, we need to add below property for
  • Full text:
    • NameTypeValue
      nodeScopeIndexBooleantrue
  • For property constraint: (name and isRegexExp are interrelated as already explained in previous post)
    • NameTypeValue
      propertyIndexBooleantrue
      nameStringpropertyname or regex pattern
      isRegexExpBooleanfalse or true
  • Example : jcr:title property of a page might be used in queries with property constraints or we might need to get the results of full text/contains queries based on the jcr:title or both. 
For node:
  • indexRules specific to a nodeType together with aggregates definition for its related child nodes to be indexed implies the nodes to be indexed as part of Lucene Full text Index.
  • Example : 
    • For a full text functionality to display all pages and assets as part of specific project say, xyx. Then we can create custom Lucene Full Text Index targeting project specific paths, related node types - cq:Page and dam:Asset (in indexRules and aggregates defintion) and hence its properties.
    • If we are not creating custom + if we use "type" predicate as "cq:Page" and "dam:Asset" for our fulltext query(group query to bring in both "type") -> OOB cqPageLucene and damAssetLucene will be considered based on its definition.
OOB Lucene Full Text Index:(/oak:index/lucene)
  • We are aware that nt:base is the super type of all available node types like nt:unstructured, cq:Page, dam:Asset, dam:AssetContent and so on (for any node type for that matter)
  • Given this understanding, in our local instance (without any custom index definition), execute a query with "fulltext" predicate -> "/oak:index/lucene" (OOB Lucene Full Text Index) will be used.
  • If we observe this index, we have indexRules configured for the nodeType - nt:base with whole list of properties defined under that. 
  • Except for last three properties, rest all would have property named "index" -> false (screenshot below for reference)

  • lucene full text index

  • Key to note here is the last property (highlighted in below screenshot) which will index all the properties of node nt:base (which in turn implies all nodes as nt:base is the super type of all available node types)
  • For this reason, for a full text query without "type" predicate, this index(/oak:index/lucene) will be used.
  • Also, on the nt:base node, we have a property named "includePropertyTypes" with value being String and Boolean. It indicates that it can index properties which are of type String and Boolean data type.

  • Lucene Full Text Index
With this understanding on how nodes and properties can be indexed as part of Lucene Full text Index, we will create a custom Lucene Full Text index with sample use case. 

Use case: 
Get all the assets which are related to "biking" in we-retail assets. 
Will divide into two scenarios - > 
  • Full text without "type" predicate (to show case the explanation above related to OOB Lucene full text index - /oak:index/lucene)
  • Full text with desired "type" predicate. 
Without "type" predicate:
  • path=/content/dam/we-retail
  • fulltext=biking
  • p.limit=-1
Video demo:

With "type" predicate:
  • path=/content/dam/we-retail
  • type=dam:Asset
  • fulltext=biking
  • p.limit=-1
Video demo: 

For the sake of creating custom Lucene Full text Index for a new node type, have amended the query as below with type as "dam:AssetContent" (In previous post, we have created damAssetContentLucene for illustrating Lucene Property Index, will be amending on top of it for fulltext query)
  • path=/content/dam/we-retail
  • type=dam:AssetContent
  • fulltext=biking
  • p.limit=-1
Video demo:

Comments

Post a Comment

Popular posts from this blog

Embedding Third party dependency/OSGi bundle in AEM application hosted in AEMasCS

Image
This post illustrate the ways of embedding third party dependency in AEM maven project. Steps are mentioned considering the project structure to be followed for AEMasCS while the same can be followed for classic versions too.  Before we begin, I would like to explain the scenario of including a dependency in our AEM maven project.  When we would like to make use of an API that is not already available in our project, we look for that in maven-central repository and specify the dependency in main pom.xml and desired module's pom.xml file. (Lets say, AWS S3 Java SDK V2 dependency) Upon updating the project in the IDE, we would see our class files (making use of those APIs) are compiled/build successfully without any errors.  Now, in the context of AEM/OSGi environment, when the same package and hence the bundle is deployed to AEM instance, dependencies that we added new might not be resolved/available in the OSGi as a bundle. Something like the below This indicates that the added dep
Read more

OSGI Factory Configuration implementation

OSGI Factory configuration is used to have many instances of an OSGI config (represented/identified via an identifier) for the single OSGI service/PID. Most often used OOB Factory configurations for example: Logger Factory/writer configuration (for log statements) Service user Mapper service Amendment (for using service resource resolver in projects) Few sample real time use case for the need of OSGI factory config: For multi tenant projects, if we have generic service used across tenants with properties/attributes specific to tenant. For single tenant, generic service targeting all locales with properties/attributes specific to locale. (where generic service -> holds logic in the lines of calling third party service for retrieving/updating information.) Implementation: ( In terms of OSGI R6 annotations ): Create Object Class Definition(OCD) defining desired properties for the OSGI service. - OSGIFactoryConfigOCD.java OSGI Service/Factory service to retrieve
Read more

Embed Third party dependency using bnd-maven-plugin

Image
This post explains the use of bnd-maven-plugin to embed third party dependency  Two main instructions of bnd-maven-plugin that serves the purpose are detailed below.  -conditionalpackage  pick packages from respective library and copy them in the bundle. Any dependencies of those copied packages will also be copied if they match the selectors. This means we need to explicitly list all the Java package names of desired dependency as part of this instruction (including its transitive dependencies) More about conditionalpackage instruction from official docs -  https://bnd.bndtools.org/instructions/conditionalpackage.html -includeresource  For including resource inline -includeresource: @artifactId-version.jar!/* For embedding as a jar with an entry in bundle class path -includeresource: artifactId-version.jar;lib:=true For several other possible combinations for including the resource, please check - https://bnd.bndtools.org/instructions/includeresource.html With complete understandin
Read more