Problem
What does the term, "Configuring JCR collation support", mean as mentioned during the setup of DB2 for Lotus Web Content Management (WCM)?
Resolving the problem
In IBM WebSphere Portal, you can use (store or retrieve) contents using Unicode or a locale-specific character set. If you are storing the content using Unicode, collation has minimum importance. Unicode is generic so each character is a predefined size that is the same across characters. If you are storing content using a locale-specific character set, collation becomes a priority.
If you use a filter on the content, the database must use the same character set as that used to encode the content. For example, if you have a column that stores names using a Chinese character set, the comparison will not be the same as if ASCII characters were used. Each character in the Chinese set has a predefined index associated with it so the database must follow these indexes when sorting the names in the result.
This is why you must configure your database and enable collation up front so that in response to your query, the database can pick up the desired character set and use it when ordering the data.
Showing posts with label jcr. Show all posts
Showing posts with label jcr. Show all posts
Monday, April 27, 2009
Saturday, April 25, 2009
JCR troubleshooting topic: troubleshooting common exceptions
Problem
You need to troubleshoot JCR (Java Content Repository) error codes that appear in the logs of your IBM Web Content Management (WCM) system.
Cause
JCR exceptions in logs
Resolving the problem
PathNotFound Exception and ItemNotFound exceptions
Within a workspace, each node is identified by its path and its uuid. In order to retrieve a node you must specify either its path or its uuid. The retrieval methods within the repository throw the exceptions "PathNotFoundException" and "ItemNotFoundException" to indicate that the node or property that you attempted to retrieve (either by path or uuid) does not exist in the current workspace. This exception is the repository's method of telling the WCM application that is does not have the item being requested.
NOTE: From the perspective of the API, both a node and a property possess a "path" which can be requested. Both Node and Property are sub-interfaces of the common parent interface Item, and hence the "ItemNotFoundException" notification.
javax.jcr.nodetype.NoSuchNodeTypeException
Example: javax.jcr.nodetype.NoSuchNodeTypeException: Node Type clb:clbLibrary not found
This problem indicates that the application is expecting a JCR Node Type which has not been defined.
Two scenarios are known to lead to this problem.
1. The node types have not been defined during the Portal upgrade. This is the most common scenario, and occurs when either Portal update has failed, or was never correctly invoked. To resolve this problem, we need to find out what happened during Portal update, and why the node types were not created correctly at this time.
2. The application is looking for a Quickr Node Type on 6.0.1.1. Since Quickr Node Types are not guaranteed to be installed on a base WebSphere Portal installation, this error calls for an update for the application. In this customer-reported case, the issue was found with PZN and the issue is resolved with PK51123.
Error deleting a library: SQL0101N
The SQL0101N error indicates that the SQL is too large or complex. One needs to ensure that the database statistics are current, especially if running DB2. Depending on the nature of your environment's data set, specific database tuning may be required to resolve this error. In the case of using DB2, the presence of the large number of versions is creating a SQL statement that exceeds the settings for DB2 cache size.
DB2:
* Increase the DB2 parameter: pckcachesz. This parameter is allocated out of the database shared memory, and is used for caching of sections for static and dynamic SQL and XQuery statements on a database. Try using 16382 for value.
NOTE: The database manager must be restarted for these updates to take effect.
* Verify the new value is set before retrying. If it fails again, then double the value specified in the previous step.
NOTE: the value can be reduced once the library is removed.
java.lang.OutOfMemoryError
For complete details about OutOfMemoryError during export, read JCR Troubleshooting topic: Export and Import, technote 1322146.
JCR Troubleshooting topics: Export and Import
There are often cases where the OutOfMemoryError is caused by the application processing too much data when using a StrongWorkspaceState. You can recognize this situation when you see the StrongWorkspaceState holding a lot of memory when you analyze the heap dumps. The StrongWorkspaceState is an internal cache of all JCR nodes referenced in the current workspace, and the OutOfMemoryError here indicates that more nodes are referenced than the current memory can handle.
The application needs to change to do one of the following:
1. Reduce the number of nodes processed per workspace
2. Manually clear the workspace state at an interval that can be handled by the system's memory
3. Use Weak workspace state
Applying PK70951 will not resolve the problem alone, but it will help identify what part of the application that is improperly using the StrongWorkspaceState.
After applying PK70951:
1. Enable trace to com.ibm.icm.jcr.StrongWorkspaceState=all, and
2. Recreate the failure.
After the next OutOfMemoryError, the trace will contain debug stack traces which can be used by IBM Support to identify the failure. Otherwise, this error needs more diagnostic information to determine the cause, including Java core output and the heap dump of the failure.
StaleValueException
The StaleValueException is a normal exception by JCR and does not indicate a JCR failure. JCR throws StaleValueException when the current node being saved is different from the latest version of that node in the database.
There is a known issue with JCR nodes which have been created or updated within the first hour after the time switch to Daylight Savings Time (DST). The cause of this problem is that Java compensates for DST differently than the database. During this hour of overlap, Java advances internal timestamps one hour ahead. The database does not advance the timestamp during this time period, so the result is an inconsistency between timestamps from Java and database timestamps. The end result is that the user encounters a StaleValueException every time they try to update nodes from this hour of overlap.
IBM support has a utility to fixup the invalid nodes during this hour of overlap. Contact IBM Support to review the issue and provide the utility if indicated.
javax.jcr.query.InvalidQueryException
Example:
javax.jcr.query.InvalidQueryException: Query exception is thrown for query statement: /contentRoot/element(*,icm:documentLibrary)/Taxonomies/mig/My Taxonomy/My Organization/North Carolina: QRY0552E: Syntax of query string is incorrect. Parsing error at position 71. Detail: Error: could not match input.
The application needs to correctly encode the query before sending it to JCR.
javax.jcr.PathNotFoundException
This message is not necessarily an error. It indicates that the caller is looking for a JCR object that does not exist. You need to look up the call stack to the calling application to determine where to debug further.
javax.jcr.ItemExistsException
This message is not necessarily an error. javax.jcr.ItemExistsException indicates that the caller is trying to save a JCR object that already exists. The object may already exist by either the same path or the same UUID.
We have also seen ItemExistsException occur during import. For more information, read JCR Troubleshooting topic: Export and Import, technote 1322146.
Data field is too long
Example: javax.jcr.nodetype.ConstraintViolationException: The value for the string property ibmcontentwcm:writeAccess on node /contentRoot/icm:libraries[5]\/Workflow/Stages/test/ibmcontentwcm:effective exceeds its max length.
The maximum length is 252 while the value is of length 263
This message indicates that the application is sending a data property value which is greater than its definition size. This is most commonly seen when migrating WCM data from 5.1 to V6. If this problem is occurring during WCM migration, this error indicates that a 5.1 property needs to be shortened in order to fit into the V6 size restrictions.
If this problem is not occurring during WCM migration, implement application debugging to proceed further.
SQLSTATE=54001 - The statement is too long or too complex.
This error can occur if you have not regularly executed database statistics. Ensure that the database statistics are current.
For more information, read the IBM WebSphere Portal Information Center topic: Database performance.
If using DB2 as the database:
1. Increase the values of the DB2 statement heap (stmtheap), the application heap (applheapsz) and the cache size (pckcachesz).
2. Restart the database manager for these updates to take effect.
NOTE: If this failure occurs during a one-time operation such as library delete, you can reduce these values after delete has completed.
java.io.IOException: Bad file number
Example:
java.io.IOException: Bad file number
at java.io.FileInputStream.available(Native Method)
This exception is reported because the data stream being stored inside JCR is invalid.
getMimeTypeId errors
Example: java.lang.NullPointerException at com.ibm.icm.ci.data.impl.dautils.PCreateNodeImpl.getMimeTypeId()
This error is encountered when the MimeType extensions had been manually edited. Make sure that WCMConfigServices.properties has the correct extension mappings.
Problems in LibraryDeleteModule
Older versions of LibraryDeleteModule would cause database exceptions when deleting drafts (internal locks). This has been resolved by PK64844. Ensure that your system has the latest version of the LibraryDeleteModule.
DB2 zOS
Fail to create database
The prefix must be unique enough to contain only the databases used for the content repository. Use a prefix that provides unique database name conventions, such as DPTJCRXXX, or something similar.
jcr.ZosDbPrefix=
Oracle
Error with ordered results
There is a known defect with ordering on Oracle 9.2.0.6. The result is that an ordered list from JCR may not be returned in the correct order. IBM recommends upgrading to Oracle 10.0.2.3 or higher to resolve this problem. For more information, read "Menu ordering is broken in Web Content Management (WCM) due to bug in Oracle" technote 1266428.
Sort issues may also indicate a query problem. To troubleshoot SQL queries:
1. Collect the database response
2. Look to the output from the repository class that identifies the SQL statement that will be issued against the database.
3. Once this SQL statement is identified, manually execute the query against the database to see if the results are ordered correctly and as expected. If the results do not appear in the expected order, open a support ticket with the database vendor support team.
Portal 6.0
DB2 engine SQL error, SQLCODE = -104, SQLSTATE = 42601, error tokens = ,1;+ - AS
There is a problem on a German locale where the comma character is allowed as a period. Ensure that you have the latest JCR cumulative fix to resolve the problem.
You need to troubleshoot JCR (Java Content Repository) error codes that appear in the logs of your IBM Web Content Management (WCM) system.
Cause
JCR exceptions in logs
Resolving the problem
PathNotFound Exception and ItemNotFound exceptions
Within a workspace, each node is identified by its path and its uuid. In order to retrieve a node you must specify either its path or its uuid. The retrieval methods within the repository throw the exceptions "PathNotFoundException" and "ItemNotFoundException" to indicate that the node or property that you attempted to retrieve (either by path or uuid) does not exist in the current workspace. This exception is the repository's method of telling the WCM application that is does not have the item being requested.
NOTE: From the perspective of the API, both a node and a property possess a "path" which can be requested. Both Node and Property are sub-interfaces of the common parent interface Item, and hence the "ItemNotFoundException" notification.
javax.jcr.nodetype.NoSuchNodeTypeException
Example: javax.jcr.nodetype.NoSuchNodeTypeException: Node Type clb:clbLibrary not found
This problem indicates that the application is expecting a JCR Node Type which has not been defined.
Two scenarios are known to lead to this problem.
1. The node types have not been defined during the Portal upgrade. This is the most common scenario, and occurs when either Portal update has failed, or was never correctly invoked. To resolve this problem, we need to find out what happened during Portal update, and why the node types were not created correctly at this time.
2. The application is looking for a Quickr Node Type on 6.0.1.1. Since Quickr Node Types are not guaranteed to be installed on a base WebSphere Portal installation, this error calls for an update for the application. In this customer-reported case, the issue was found with PZN and the issue is resolved with PK51123.
Error deleting a library: SQL0101N
The SQL0101N error indicates that the SQL is too large or complex. One needs to ensure that the database statistics are current, especially if running DB2. Depending on the nature of your environment's data set, specific database tuning may be required to resolve this error. In the case of using DB2, the presence of the large number of versions is creating a SQL statement that exceeds the settings for DB2 cache size.
DB2:
* Increase the DB2 parameter: pckcachesz. This parameter is allocated out of the database shared memory, and is used for caching of sections for static and dynamic SQL and XQuery statements on a database. Try using 16382 for value.
NOTE: The database manager must be restarted for these updates to take effect.
* Verify the new value is set before retrying. If it fails again, then double the value specified in the previous step.
NOTE: the value can be reduced once the library is removed.
java.lang.OutOfMemoryError
For complete details about OutOfMemoryError during export, read JCR Troubleshooting topic: Export and Import, technote 1322146.
JCR Troubleshooting topics: Export and Import
There are often cases where the OutOfMemoryError is caused by the application processing too much data when using a StrongWorkspaceState. You can recognize this situation when you see the StrongWorkspaceState holding a lot of memory when you analyze the heap dumps. The StrongWorkspaceState is an internal cache of all JCR nodes referenced in the current workspace, and the OutOfMemoryError here indicates that more nodes are referenced than the current memory can handle.
The application needs to change to do one of the following:
1. Reduce the number of nodes processed per workspace
2. Manually clear the workspace state at an interval that can be handled by the system's memory
3. Use Weak workspace state
Applying PK70951 will not resolve the problem alone, but it will help identify what part of the application that is improperly using the StrongWorkspaceState.
After applying PK70951:
1. Enable trace to com.ibm.icm.jcr.StrongWorkspaceState=all, and
2. Recreate the failure.
After the next OutOfMemoryError, the trace will contain debug stack traces which can be used by IBM Support to identify the failure. Otherwise, this error needs more diagnostic information to determine the cause, including Java core output and the heap dump of the failure.
StaleValueException
The StaleValueException is a normal exception by JCR and does not indicate a JCR failure. JCR throws StaleValueException when the current node being saved is different from the latest version of that node in the database.
There is a known issue with JCR nodes which have been created or updated within the first hour after the time switch to Daylight Savings Time (DST). The cause of this problem is that Java compensates for DST differently than the database. During this hour of overlap, Java advances internal timestamps one hour ahead. The database does not advance the timestamp during this time period, so the result is an inconsistency between timestamps from Java and database timestamps. The end result is that the user encounters a StaleValueException every time they try to update nodes from this hour of overlap.
IBM support has a utility to fixup the invalid nodes during this hour of overlap. Contact IBM Support to review the issue and provide the utility if indicated.
javax.jcr.query.InvalidQueryException
Example:
javax.jcr.query.InvalidQueryException: Query exception is thrown for query statement: /contentRoot/element(*,icm:documentLibrary)/Taxonomies/mig/My Taxonomy/My Organization/North Carolina: QRY0552E: Syntax of query string is incorrect. Parsing error at position 71. Detail: Error: could not match input.
The application needs to correctly encode the query before sending it to JCR.
javax.jcr.PathNotFoundException
This message is not necessarily an error. It indicates that the caller is looking for a JCR object that does not exist. You need to look up the call stack to the calling application to determine where to debug further.
javax.jcr.ItemExistsException
This message is not necessarily an error. javax.jcr.ItemExistsException indicates that the caller is trying to save a JCR object that already exists. The object may already exist by either the same path or the same UUID.
We have also seen ItemExistsException occur during import. For more information, read JCR Troubleshooting topic: Export and Import, technote 1322146.
Data field is too long
Example: javax.jcr.nodetype.ConstraintViolationException: The value for the string property ibmcontentwcm:writeAccess on node /contentRoot/icm:libraries[5]\/Workflow/Stages/test/ibmcontentwcm:effective exceeds its max length.
The maximum length is 252 while the value is of length 263
This message indicates that the application is sending a data property value which is greater than its definition size. This is most commonly seen when migrating WCM data from 5.1 to V6. If this problem is occurring during WCM migration, this error indicates that a 5.1 property needs to be shortened in order to fit into the V6 size restrictions.
If this problem is not occurring during WCM migration, implement application debugging to proceed further.
SQLSTATE=54001 - The statement is too long or too complex.
This error can occur if you have not regularly executed database statistics. Ensure that the database statistics are current.
For more information, read the IBM WebSphere Portal Information Center topic: Database performance.
If using DB2 as the database:
1. Increase the values of the DB2 statement heap (stmtheap), the application heap (applheapsz) and the cache size (pckcachesz).
2. Restart the database manager for these updates to take effect.
NOTE: If this failure occurs during a one-time operation such as library delete, you can reduce these values after delete has completed.
java.io.IOException: Bad file number
Example:
java.io.IOException: Bad file number
at java.io.FileInputStream.available(Native Method)
This exception is reported because the data stream being stored inside JCR is invalid.
getMimeTypeId errors
Example: java.lang.NullPointerException at com.ibm.icm.ci.data.impl.dautils.PCreateNodeImpl.getMimeTypeId()
This error is encountered when the MimeType extensions had been manually edited. Make sure that WCMConfigServices.properties has the correct extension mappings.
Problems in LibraryDeleteModule
Older versions of LibraryDeleteModule would cause database exceptions when deleting drafts (internal locks). This has been resolved by PK64844. Ensure that your system has the latest version of the LibraryDeleteModule.
DB2 zOS
Fail to create database
The prefix must be unique enough to contain only the databases used for the content repository. Use a prefix that provides unique database name conventions, such as DPTJCRXXX, or something similar.
jcr.ZosDbPrefix=
Oracle
Error with ordered results
There is a known defect with ordering on Oracle 9.2.0.6. The result is that an ordered list from JCR may not be returned in the correct order. IBM recommends upgrading to Oracle 10.0.2.3 or higher to resolve this problem. For more information, read "Menu ordering is broken in Web Content Management (WCM) due to bug in Oracle" technote 1266428.
Sort issues may also indicate a query problem. To troubleshoot SQL queries:
1. Collect the database response
2. Look to the output from the repository class that identifies the SQL statement that will be issued against the database.
3. Once this SQL statement is identified, manually execute the query against the database to see if the results are ordered correctly and as expected. If the results do not appear in the expected order, open a support ticket with the database vendor support team.
Portal 6.0
DB2 engine SQL error, SQLCODE = -104, SQLSTATE = 42601, error tokens = ,1;+ - AS
There is a problem on a German locale where the comma character is allowed as a period. Ensure that you have the latest JCR cumulative fix to resolve the problem.
JCR Troubleshooting topic: query performance and XPath
Problem
As system administrator of you IBM Web Content Management system, you need to analyze the JCR (Java Content Repository) query performance.
Resolving the problem
The following steps show how to search through a JCR trace for data showing the query performance. Note that this assumes a trace of the query using com.ibm.icm.*=all
The following lines in the trace will show the XPath query being sent to JCR, the generated SQL being sent to the database, and the performance for each:
1. The entry to JCR query:
Search string: QueryImpl execute includeLocks
Example:
[8/8/08 4:51:28:490 GMT] 0000006c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute includeLocks=false includeReferences=false includePaths=false statement=//element(*, icm:documentLibrary)[@jcr:uuid = '5375be0046c9f315bf53bf996f9fe841']//(element(*, ibmcontentwcm:webContent) | element(*, ibmcontentwcm:draftSummary))[@ibmcontentwcm:workflowStage = '3d115a0046c9fef2bf88bf996f9fe841' and (not(@ibmcontentwcm:isPrototype) or @ibmcontentwcm:isPrototype = fn:false()) and (@ibmcontentwcm:classification = 'Content' or @ibmcontentwcm:draftClassification = 'Content')] propertiesToRetrieve=null 2. The SQL sent to the database:
Search string: Generated SQL with param markers
[8/8/08 4:51:28:740 GMT] 0000006c ResultSetProc 3 Generated SQL with param markers included:
WITH NONLEAFS AS (SELECT Links_18.SIID , Links_18.SVID , Links_19.TIID , Links_19.TVID , Links_19.TIX , Links_19.TCTID ,
<200 more lines....>
NodesTab_17.IID)
3. The actual query:
Search string: executeQuery
[8/8/08 4:51:28:744 GMT] 0000006c PPreparedStat 3 com.ibm.icm.da.portable.common.sql.PPreparedStatement executeQuery() ==> [...] 4. The return from the query:
Search string: openQueryCursor
[8/8/08 4:51:43:472 GMT] 0000006c Query 2 com.ibm.icm.da.portable.query.Query openQueryCursor() Note: The difference between the executeQuery() and the openQueryCursor() is the time spent by the database server executing the SQL (about 15 seconds here). 5. The result size of the query:
Search string: query result size
[8/8/08 4:51:43:785 GMT] 0000006c QueryResultIt 2 com.ibm.icm.jcr.query.QueryResultIteratorImpl QueryResultIteratorImpl query result size=34 6. The total query execute time:
Search string: total query execute time
[8/8/08 4:51:43:785 GMT] 0000006c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute total query execute time=15294
Xpath query issues.
the JCR spec declares that nodes within the repository must be retrievable by way of a query language. The query language supported by our repository implementation is XPath. Technically, a subset of the full XPath specification is supported by the repository for retrieving nodes from the repository. To use this function, an application will construct a "query" object and execute that "query" against the repository returning an "iterator" of results. The results are the nodes that met the XPath query criteria.
The execution of an XPath query against the repository includes two primary steps: validation and execution. The validation step ensures that the XPath query string specified by the application is of a syntactically valid form and can be processed by the repository. If the XPath query is of an invalid form, then the query will throw an InvalidQueryException which indicates to the application that the XPath was not syntactically valid. If the XPath is syntactically valid, then the query is accepted and processed by the repository and a "result set" is returned.
In order to debug any issues with XPath queries that are valid but not returning results as expected there are the following steps:
1. Recreate the failing query with both the application trace (e.g. WCM trace) and JCR trace (com.ibm.icm.*=finest). The application trace will identify the XPath query being issued, as well as the context of that query. The JCR trace will identify the generated SQL that has been generated from the XPath.
2. Once you have gathered the traces, obtain the entry and exit points from the query, as well as the generated SQL.
See also the appropriate MustGather documents at the bottom of the technote for help collecting the JCR trace output.
--------------------------------
JCR MustGather documentation for collecting JCR trace output:
Version 6.0.x: http://www-01.ibm.com/support/docview.wss?uid=swg21303048
Version 6.1.x: http://www-01.ibm.com/support/docview.wss?uid=swg21316273
As system administrator of you IBM Web Content Management system, you need to analyze the JCR (Java Content Repository) query performance.
Resolving the problem
The following steps show how to search through a JCR trace for data showing the query performance. Note that this assumes a trace of the query using com.ibm.icm.*=all
The following lines in the trace will show the XPath query being sent to JCR, the generated SQL being sent to the database, and the performance for each:
1. The entry to JCR query:
Search string: QueryImpl execute includeLocks
Example:
[8/8/08 4:51:28:490 GMT] 0000006c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute includeLocks=false includeReferences=false includePaths=false statement=//element(*, icm:documentLibrary)[@jcr:uuid = '5375be0046c9f315bf53bf996f9fe841']//(element(*, ibmcontentwcm:webContent) | element(*, ibmcontentwcm:draftSummary))[@ibmcontentwcm:workflowStage = '3d115a0046c9fef2bf88bf996f9fe841' and (not(@ibmcontentwcm:isPrototype) or @ibmcontentwcm:isPrototype = fn:false()) and (@ibmcontentwcm:classification = 'Content' or @ibmcontentwcm:draftClassification = 'Content')] propertiesToRetrieve=null 2. The SQL sent to the database:
Search string: Generated SQL with param markers
[8/8/08 4:51:28:740 GMT] 0000006c ResultSetProc 3 Generated SQL with param markers included:
WITH NONLEAFS AS (SELECT Links_18.SIID , Links_18.SVID , Links_19.TIID , Links_19.TVID , Links_19.TIX , Links_19.TCTID ,
<200 more lines....>
NodesTab_17.IID)
3. The actual query:
Search string: executeQuery
[8/8/08 4:51:28:744 GMT] 0000006c PPreparedStat 3 com.ibm.icm.da.portable.common.sql.PPreparedStatement executeQuery() ==> [...] 4. The return from the query:
Search string: openQueryCursor
[8/8/08 4:51:43:472 GMT] 0000006c Query 2 com.ibm.icm.da.portable.query.Query openQueryCursor() Note: The difference between the executeQuery() and the openQueryCursor() is the time spent by the database server executing the SQL (about 15 seconds here). 5. The result size of the query:
Search string: query result size
[8/8/08 4:51:43:785 GMT] 0000006c QueryResultIt 2 com.ibm.icm.jcr.query.QueryResultIteratorImpl QueryResultIteratorImpl query result size=34 6. The total query execute time:
Search string: total query execute time
[8/8/08 4:51:43:785 GMT] 0000006c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute total query execute time=15294
Xpath query issues.
the JCR spec declares that nodes within the repository must be retrievable by way of a query language. The query language supported by our repository implementation is XPath. Technically, a subset of the full XPath specification is supported by the repository for retrieving nodes from the repository. To use this function, an application will construct a "query" object and execute that "query" against the repository returning an "iterator" of results. The results are the nodes that met the XPath query criteria.
The execution of an XPath query against the repository includes two primary steps: validation and execution. The validation step ensures that the XPath query string specified by the application is of a syntactically valid form and can be processed by the repository. If the XPath query is of an invalid form, then the query will throw an InvalidQueryException which indicates to the application that the XPath was not syntactically valid. If the XPath is syntactically valid, then the query is accepted and processed by the repository and a "result set" is returned.
In order to debug any issues with XPath queries that are valid but not returning results as expected there are the following steps:
1. Recreate the failing query with both the application trace (e.g. WCM trace) and JCR trace (com.ibm.icm.*=finest). The application trace will identify the XPath query being issued, as well as the context of that query. The JCR trace will identify the generated SQL that has been generated from the XPath.
2. Once you have gathered the traces, obtain the entry and exit points from the query, as well as the generated SQL.
See also the appropriate MustGather documents at the bottom of the technote for help collecting the JCR trace output.
--------------------------------
JCR MustGather documentation for collecting JCR trace output:
Version 6.0.x: http://www-01.ibm.com/support/docview.wss?uid=swg21303048
Version 6.1.x: http://www-01.ibm.com/support/docview.wss?uid=swg21316273
JCR troubleshooting topic: search index
Problem
As the system administrator of an implementation of IBM Web Content Management, you must troubleshoot the Java Content Repository (JCR) search index component.
Cause
Search not working properly
Resolving the problem
Rebuild Search Index
Title: How to rebuild the WebSphere Portal Document Manager search index
Doc #: 1296931
URL: http://www.ibm.com/support/docview.wss?rs=899&uid=swg21296931
NOTE: The SystemOut.log displays the following lines when rebuilding the search index:
* Start: START rebuilding juru index:
* End: DONE rebuilding juru index:
Stellent Conversion Errors
There is a database fix for Stellent Conversion Errors with htmlElement.elementData. contact IBM Support for more information
If you encounter other Stellant Conversion Errors while rebuilding the search index, contact IBM Support to involve the ODC/DCS team to help resolve the issues with running the conversion.
ParentIDNotFoundException
Example: com.ibm.icm.ts.path.ParentIDNotFoundException: Original parent not found for id: 120337072591476.
This error can occur if search indexing attempts to index an item which has been deleted. This is normal processing, and is not an error. The failed event is logged in the ICMJCRSTERRORS table.
Note that PK56104 will remove a lot of these messages. The former APAR, PK56033, has been replaced by PK56104.
After PK56104 has been applied, if search indexing encounters a deleted item, it will process that item only once and log the exception. The next time it will not attempt to process the deleted item again.
NOTE: PK56104 has been part of the JCR Cumulative fix since PK60132 (JCR Cumulative Fix #3).
This exception can cause the ICMJCRSTERRORS to grow very large. It is safe to remove the contents of this table after the JCR Cumulative Fix has been applied. Contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
Exception during Search
Previous Exceptions
If there is an exception starting a WebSphere service, this may lead to search problems later on. These are the exceptions that have been seen so far:
* java.lang.IllegalStateException: I18N0012I: The Internationalization service is not started on WebSphere_Portal
Duplicate exceptions during search
Example: COM.ibm.db2.jdbc.DB2Exception: SQL0601N The name of the object to be created is identical to the existing name "JCR.TSSTBL_2" of type "TABLE".
You should clean up the temporary search tables to resolve this error. To clean up the temporary search tables, contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
unique constraint violated
Example: java.sql.BatchUpdateException: ORA-00001: unique constraint (WCMICMADMIN.SYS_C0036649) violated
This error has been fixed in 6.0.1.1, but can occur if the search index has not been rebuilt since upgrading to 6.0.1.1.
Solution: Rebuild the search index.
Temp search tables
PK58346 is now available for 6.0.1.1 and later, which will greatly reduce the use use temporary search tables.
Note: PK58346 is part of the JCR Cumulative Fix since PK60132 (JCR Cumulative Fix #3).
To clean up the temporary search tables, contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
Incorrect Results from Search
Search index failures
If the search is not yielding correct results, make sure that the search index was created with no errors.
You can verify if a single document was reindexed successfully by the following steps:
1. Change the index maintenance interval to 2 minutes
2. Enable JCR trace at com.ibm.icm.*=finest
3. Edit the document
4. Allow 2 minutes (the index maintenance interval) for the document to be reindexed
Trace search results
Collect the following information:
* Search criteria
* Portal page from where search was invoked
* Any other search options (if advanced search)
* The user's locale
* Expected results
* Actual results
* JCR trace of the failure: com.ibm.icm.*=finest
(For instructions, read the MustGather documentation at the end of this technote)
Trace information:
1. Look for the entry to JCR query:
This will show the actual query that is being executed (including the text search):
Search string: QueryImpl execute includeLocks
Example:
[3/16/08 7:28:05:161 PDT] 0000009c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute includeLocks=false includeReferences=false includePaths=true statement=//element(, icm:documentLibrary)[@jcr:uuid = 'e8b5dc8046f1eb03a15db108d7e720a9']//(element(, ibmcontentwcm:authoringTemplate)|element(, ibmcontentwcm:webCategory))[@ibmcontentwcm:workflowStatus and @icm:authors = 'cn=userid,o=all users'][text-contains(.,'board')] order by text-score(.,'board*') descending propertiesToRetrieve=null
2. Look for the entry to text search (Juru):
This will show what is being sent to text search, and if any truncation has occurred:
Search string: executing search:
Example:
[3/16/08 7:28:05:416 PDT] 000000b6 JCRCFLLoggerI 3 com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]] com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: executing search: 'board*' with language: en wildcard expansion size: 20
Wildcard term expansion truncated for search: 'board*
3. Look for exit from text search:
This will identify how many results were found from text search (Juru).
Search string: num results:
Example:
[3/16/08 7:28:05:416 PDT] 000000b6 JCRCFLLoggerI 3 com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]] com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: num results: 4
4. Look for exit from query:
This will identify how many of the search results are returned after JCR has performed a query based on the results from text search.
Search string: query result size
Example:
[3/16/08 7:28:16:216 PDT] 0000009c QueryResultIt 2 com.ibm.icm.jcr.query.QueryResultIteratorImpl QueryResultIteratorImpl query result size=0
If the number of results returned from Juru is different than what is expected, then we must pursue the incorrect search with Juru.
If the number of results returned from Juru is what is expected, then we must pursue the incorrect search with IBM JCR Support, to find out if/where JCR has changed the result list.
If trace indicates "expansion truncated" as above, it indicates that Juru search is working as designed, but the search terms yield more results than are allowed, and so they are truncated by Juru. Note that you can increase the number of search terms with the jcr.textsearch.wildcardTermExpansionSize property in icm.properties. However, note that a larger wildcard expansion size will impact search performance.
Number of search results returned
At present, JCR cannot retrieve more than 100 results from a Juru search. Note that this number may be further reduced by JCR based on either access control, or additional query criteria. The best way to identify if the incorrect number of search results is being limited by the maximum number of results returned from Juru is to look at the search trace (see above). If the number of results returned from Juru is 100, it is very likely that the current search exceeds this maximum of 100.
Miscellaneous topics on search
Search performance issue in 6.0.1.3
We have encountered a performance issue with search on 6.0.1.3 and JCR Cumulative Fix #3. This problem causes the search performance to degrade with a large number of search results. This problem has been fixed with PK64038.
Search Across Locales
Nodes which are indexed in one language are not guaranteed to be searchable from another language. For example, a Turkish language node "fulya" is not searchable from English. This is working as designed.
To verify the search index language, compare the language from the search trace with the workspace language in icm.properties: jcr.workspace.defaultLanguage
Reorganize search index
If a lot of PDM or WCM content has been removed but the search index continues to grow, you may need to perform the administrator function to reorganize the search index. This capability is provided with PK61534. See the readme for PK61534 for instructions.
Reference information
Web Content Management Authoring Inline Advanced Search known limitations:
http://www-1.ibm.com/support/docview.wss?rs=688&uid=swg21259650
Known limitations and issues for Juru search utilized by WCM Authoring UI search:
http://www-1.ibm.com/support/docview.wss?rs=1041&uid=swg21259649
Web Content Management advanced authoring search does not return new or changed content:
http://www-1.ibm.com/support/docview.wss?rs=688&uid=swg21259884
--------------------------------
As the system administrator of an implementation of IBM Web Content Management, you must troubleshoot the Java Content Repository (JCR) search index component.
Cause
Search not working properly
Resolving the problem
Rebuild Search Index
Title: How to rebuild the WebSphere Portal Document Manager search index
Doc #: 1296931
URL: http://www.ibm.com/support/docview.wss?rs=899&uid=swg21296931
NOTE: The SystemOut.log displays the following lines when rebuilding the search index:
* Start: START rebuilding juru index:
* End: DONE rebuilding juru index:
Stellent Conversion Errors
There is a database fix for Stellent Conversion Errors with htmlElement.elementData. contact IBM Support for more information
If you encounter other Stellant Conversion Errors while rebuilding the search index, contact IBM Support to involve the ODC/DCS team to help resolve the issues with running the conversion.
ParentIDNotFoundException
Example: com.ibm.icm.ts.path.ParentIDNotFoundException: Original parent not found for id: 120337072591476.
This error can occur if search indexing attempts to index an item which has been deleted. This is normal processing, and is not an error. The failed event is logged in the ICMJCRSTERRORS table.
Note that PK56104 will remove a lot of these messages. The former APAR, PK56033, has been replaced by PK56104.
After PK56104 has been applied, if search indexing encounters a deleted item, it will process that item only once and log the exception. The next time it will not attempt to process the deleted item again.
NOTE: PK56104 has been part of the JCR Cumulative fix since PK60132 (JCR Cumulative Fix #3).
This exception can cause the ICMJCRSTERRORS to grow very large. It is safe to remove the contents of this table after the JCR Cumulative Fix has been applied. Contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
Exception during Search
Previous Exceptions
If there is an exception starting a WebSphere service, this may lead to search problems later on. These are the exceptions that have been seen so far:
* java.lang.IllegalStateException: I18N0012I: The Internationalization service is not started on WebSphere_Portal
Duplicate exceptions during search
Example: COM.ibm.db2.jdbc.DB2Exception: SQL0601N The name of the object to be created is identical to the existing name "JCR.TSSTBL_2" of type "TABLE".
You should clean up the temporary search tables to resolve this error. To clean up the temporary search tables, contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
unique constraint violated
Example: java.sql.BatchUpdateException: ORA-00001: unique constraint (WCMICMADMIN.SYS_C0036649) violated
This error has been fixed in 6.0.1.1, but can occur if the search index has not been rebuilt since upgrading to 6.0.1.1.
Solution: Rebuild the search index.
Temp search tables
PK58346 is now available for 6.0.1.1 and later, which will greatly reduce the use use temporary search tables.
Note: PK58346 is part of the JCR Cumulative Fix since PK60132 (JCR Cumulative Fix #3).
To clean up the temporary search tables, contact IBM Support to review the issue and provide the required SQL if indicated. It is recommended to make a full database backup before directly modifying the database.
Incorrect Results from Search
Search index failures
If the search is not yielding correct results, make sure that the search index was created with no errors.
You can verify if a single document was reindexed successfully by the following steps:
1. Change the index maintenance interval to 2 minutes
2. Enable JCR trace at com.ibm.icm.*=finest
3. Edit the document
4. Allow 2 minutes (the index maintenance interval) for the document to be reindexed
Trace search results
Collect the following information:
* Search criteria
* Portal page from where search was invoked
* Any other search options (if advanced search)
* The user's locale
* Expected results
* Actual results
* JCR trace of the failure: com.ibm.icm.*=finest
(For instructions, read the MustGather documentation at the end of this technote)
Trace information:
1. Look for the entry to JCR query:
This will show the actual query that is being executed (including the text search):
Search string: QueryImpl execute includeLocks
Example:
[3/16/08 7:28:05:161 PDT] 0000009c QueryImpl 2 com.ibm.icm.jcr.query.QueryImpl execute includeLocks=false includeReferences=false includePaths=true statement=//element(, icm:documentLibrary)[@jcr:uuid = 'e8b5dc8046f1eb03a15db108d7e720a9']//(element(, ibmcontentwcm:authoringTemplate)|element(, ibmcontentwcm:webCategory))[@ibmcontentwcm:workflowStatus and @icm:authors = 'cn=userid,o=all users'][text-contains(.,'board')] order by text-score(.,'board*') descending propertiesToRetrieve=null
2. Look for the entry to text search (Juru):
This will show what is being sent to text search, and if any truncation has occurred:
Search string: executing search:
Example:
[3/16/08 7:28:05:416 PDT] 000000b6 JCRCFLLoggerI 3 com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]] com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: executing search: 'board*' with language: en wildcard expansion size: 20
Wildcard term expansion truncated for search: 'board*
3. Look for exit from text search:
This will identify how many results were found from text search (Juru).
Search string: num results:
Example:
[3/16/08 7:28:05:416 PDT] 000000b6 JCRCFLLoggerI 3 com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]] com.ibm.icm.ts.tss.JuruIndexImpl.result [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: num results: 4
4. Look for exit from query:
This will identify how many of the search results are returned after JCR has performed a query based on the results from text search.
Search string: query result size
Example:
[3/16/08 7:28:16:216 PDT] 0000009c QueryResultIt 2 com.ibm.icm.jcr.query.QueryResultIteratorImpl QueryResultIteratorImpl query result size=0
If the number of results returned from Juru is different than what is expected, then we must pursue the incorrect search with Juru.
If the number of results returned from Juru is what is expected, then we must pursue the incorrect search with IBM JCR Support, to find out if/where JCR has changed the result list.
If trace indicates "expansion truncated" as above, it indicates that Juru search is working as designed, but the search terms yield more results than are allowed, and so they are truncated by Juru. Note that you can increase the number of search terms with the jcr.textsearch.wildcardTermExpansionSize property in icm.properties. However, note that a larger wildcard expansion size will impact search performance.
Number of search results returned
At present, JCR cannot retrieve more than 100 results from a Juru search. Note that this number may be further reduced by JCR based on either access control, or additional query criteria. The best way to identify if the incorrect number of search results is being limited by the maximum number of results returned from Juru is to look at the search trace (see above). If the number of results returned from Juru is 100, it is very likely that the current search exceeds this maximum of 100.
Miscellaneous topics on search
Search performance issue in 6.0.1.3
We have encountered a performance issue with search on 6.0.1.3 and JCR Cumulative Fix #3. This problem causes the search performance to degrade with a large number of search results. This problem has been fixed with PK64038.
Search Across Locales
Nodes which are indexed in one language are not guaranteed to be searchable from another language. For example, a Turkish language node "fulya" is not searchable from English. This is working as designed.
To verify the search index language, compare the language from the search trace with the workspace language in icm.properties: jcr.workspace.defaultLanguage
Reorganize search index
If a lot of PDM or WCM content has been removed but the search index continues to grow, you may need to perform the administrator function to reorganize the search index. This capability is provided with PK61534. See the readme for PK61534 for instructions.
Reference information
Web Content Management Authoring Inline Advanced Search known limitations:
http://www-1.ibm.com/support/docview.wss?rs=688&uid=swg21259650
Known limitations and issues for Juru search utilized by WCM Authoring UI search:
http://www-1.ibm.com/support/docview.wss?rs=1041&uid=swg21259649
Web Content Management advanced authoring search does not return new or changed content:
http://www-1.ibm.com/support/docview.wss?rs=688&uid=swg21259884
--------------------------------
JCR Troubleshooting topic: locks and deadlocks
Problem
How does one perform troubleshooting in IBM® Web Content Management regarding locking issues with JCR?
Resolving the problem
Locks
What is the best way to approach locks (AccessDenied, object has violated one or more lock constraints)?
There are two primary types of locks leveraged by the repository--external and internal. External locks are defined by the JCR specification and allow users to place locks on items to prohibit certain actions from other users. WCM supports only a "write" lock on a per-node basis within the product's implementation which means that user with the proper permission (PAC Action EDIT (EDITOR role)) can "lock" a node within the system. This "external" lock prohibits any other user from being able to persist changes to that node. In particular to be able to call save on that node alone. Interestingly enough it doesn't prohibit another user from indirectly modifying the node by operating on its parent (ie: another user could still delete the locked node by deleting its parent node).
Internal locks are known as "consistency" locks. These locks are used by the internal implementation to attempt to prohibit situations where merge conflicts or consistency conflicts could be encountered. For instance, if one creates a "dynamic workspace" from a stable workspace and then adds a node to a node that exists in the stable workspace, the system MUST place an "internal" lock on the node in the stable workspace to prohibit any other user from deleting it. If another user deleted the node, then the merge of the dynamic workspace would later fail. The consistency locks prevent situations like this from occurring.
In an effort to allow applications to know when such locks may prohibit actions, all internal locks are EXPOSED as "external" locks on nodes owned by the workspace itself. This allows applications to investigate locks and to take correct actions when "internal" locks exist.
When faced with an operation that is being reported as causing an AccessDeniedException (object has violated one or more lock constraints) this indicates that there are one or more locks that exist that prohibit this user from executing the operation just requested. Again, please note this isn't a bug it is the repository's way of alerting an application that they are prohibited from the operation at this moment due to a lock constraint. To help identify what the source of the constraint is, the following steps should again be utilized:
1. Identify the WCM class that is making the request that is failing and enable FINEST trace point for it.
2. Enable com.ibm.icm.*=finest trace point
3. Recreate and capture the traces. Look for the entry into the WCM class. From that point, look for the AccessDeniedException. From that exception walk upward on the thread id until you find a trace point for com.ibm.icm.jcr.NodeImpl save. This will output any locks that prohibited the save from completing. With this knowledge you can engage IBM Support. Please note within the output for the Lock object, the owner of the lock will be shown. If the lock is an "internal" lock the owner will be shown in some form as "Workspace XXXXX". This is how you can identify if an internal lock is prohibiting the operation vs and external lock owned by another true user of the system.
In addition, you can use selectableDisplayLocks.jsp to display all locks on a given node (and its children). Contact IBM Support for a copy of this jsp.
The following common exceptions are related to JCR Access and Locking exceptions:
User Name contains a comma (javax.jcr.LoginException)
Example: javax.jcr.LoginException: Login failed for UserId: cn=Smith, John, cn=users,dc=ibm,dc=com. Retrieved authenticated subject with unmatching UserId: CN=Smith\, John,CN=Users,dc=ibm,dc=com
The user name cannot contain a comma. If so, its name cannot be correctly processed by JCR internals.
AccessDeniedException
When the user sees an AccessDeniedException from WCM, it can mean one of the following possibilities:
The logged in user does not have available Portal Access Control for this object
This is a valid exception if the user does not have the correct access rights for the requested action. The administrator must grant the necessary rights through Portal for that object and action.
The JCR node is locked by another workspace
Example:
NodeImpl 3 com.ibm.icm.jcr.NodeImpl save(false, false) Found lock on path: /contentRoot/icm:libraries[8]/Content/epfsite/welcome owned by: Workspace 7c2ba800465031b597d5f719fed3c258
SystemErr R com.ibm.icm.jcr.access.AccessDeniedException: The requested operation violates one or more lock constraints.: [ErrorCode:7591]
This is the common occurrence if the node is being held by another draft. In this case, you must first delete the node and its draft workspace before proceeding.
Older version of Portal have seen problems after a failed library delete where drafts are still left in the library. If this is the case, contact IBM Support to review the issue and provide the cleanup tools as needed. It is recommended to make a full database backup before using any tools which directly modify the database.
The JCR node is locked by another user
This is the common occurrence where another user is working on the same node, and is considered to be normal behavior. The best course of action for this failure is to log in as the other user and unlock the node.
If that user has been removed, IBM Support has the tools to remove the locks for that user. Contact IBM Support to review the issue and provide the needed tools if indicated. It is recommended to make a full database backup before directly modifying the database.
Delete all JCR locks for a node.
WARNING: Incorrectly updating the database tables can lead to database inconsistencies and deadlocks. You should not remove all of the locks for a node unless you are in the process of deleting that node, and have exhausted all other possibilities.
To delete all of the JCR locks for a node, you need to know the UUID for that node. IBM Support has a utility to internally remove the locks for that node. Contact IBM Support to review the issue and provide the required utility if indicated. It is recommended to make a full database backup before using any tools which directly modify the database.
Tracing all SQL statements (including host variables)
Enabling the pls.debug.trackStatementCursorLeakage setting in icm.properties, combined with JCR trace (com.ibm.icm.*=all), will trace all of the SQL from JCR, combined with the host variables. Note that this setting will significantly slow down performance, so you you should reset pls.debug.trackStatementCursorLeakage to false after collecting the necessary trace data.
To enable this value, do the following:
1. Stop Portal Server
2. Edit/jcr/lib/com/ibm/icm/icm.properties, and set the following property:
pls.debug.trackStatementCursorLeakage=+
3. Set trace to com.ibm.icm.*=all and restart Portal
Deadlocks
Derby (Portal 6.1)
SQL Exception: A lock could not be obtained within the time requested
Check the derby.log file to verify that the customer is running at at least build 639536 of Derby 10.1.3.2.
DB2
Database hang during Portal upgrade
We have seen a problem where the customer will see a database hang while upgrading Portal version, for example upgrading to 6.0.1.4. This issue can occur when the database user does not have DBADM authority. Note that it is not enough to only grant SYSADM authority to the user, but the user must have explicit DBADM authority.
How does one perform troubleshooting in IBM® Web Content Management regarding locking issues with JCR?
Resolving the problem
Locks
What is the best way to approach locks (AccessDenied, object has violated one or more lock constraints)?
There are two primary types of locks leveraged by the repository--external and internal. External locks are defined by the JCR specification and allow users to place locks on items to prohibit certain actions from other users. WCM supports only a "write" lock on a per-node basis within the product's implementation which means that user with the proper permission (PAC Action EDIT (EDITOR role)) can "lock" a node within the system. This "external" lock prohibits any other user from being able to persist changes to that node. In particular to be able to call save on that node alone. Interestingly enough it doesn't prohibit another user from indirectly modifying the node by operating on its parent (ie: another user could still delete the locked node by deleting its parent node).
Internal locks are known as "consistency" locks. These locks are used by the internal implementation to attempt to prohibit situations where merge conflicts or consistency conflicts could be encountered. For instance, if one creates a "dynamic workspace" from a stable workspace and then adds a node to a node that exists in the stable workspace, the system MUST place an "internal" lock on the node in the stable workspace to prohibit any other user from deleting it. If another user deleted the node, then the merge of the dynamic workspace would later fail. The consistency locks prevent situations like this from occurring.
In an effort to allow applications to know when such locks may prohibit actions, all internal locks are EXPOSED as "external" locks on nodes owned by the workspace itself. This allows applications to investigate locks and to take correct actions when "internal" locks exist.
When faced with an operation that is being reported as causing an AccessDeniedException (object has violated one or more lock constraints) this indicates that there are one or more locks that exist that prohibit this user from executing the operation just requested. Again, please note this isn't a bug it is the repository's way of alerting an application that they are prohibited from the operation at this moment due to a lock constraint. To help identify what the source of the constraint is, the following steps should again be utilized:
1. Identify the WCM class that is making the request that is failing and enable FINEST trace point for it.
2. Enable com.ibm.icm.*=finest trace point
3. Recreate and capture the traces. Look for the entry into the WCM class. From that point, look for the AccessDeniedException. From that exception walk upward on the thread id until you find a trace point for com.ibm.icm.jcr.NodeImpl save. This will output any locks that prohibited the save from completing. With this knowledge you can engage IBM Support. Please note within the output for the Lock object, the owner of the lock will be shown. If the lock is an "internal" lock the owner will be shown in some form as "Workspace XXXXX". This is how you can identify if an internal lock is prohibiting the operation vs and external lock owned by another true user of the system.
In addition, you can use selectableDisplayLocks.jsp to display all locks on a given node (and its children). Contact IBM Support for a copy of this jsp.
The following common exceptions are related to JCR Access and Locking exceptions:
User Name contains a comma (javax.jcr.LoginException)
Example: javax.jcr.LoginException: Login failed for UserId: cn=Smith, John, cn=users,dc=ibm,dc=com. Retrieved authenticated subject with unmatching UserId: CN=Smith\, John,CN=Users,dc=ibm,dc=com
The user name cannot contain a comma. If so, its name cannot be correctly processed by JCR internals.
AccessDeniedException
When the user sees an AccessDeniedException from WCM, it can mean one of the following possibilities:
The logged in user does not have available Portal Access Control for this object
This is a valid exception if the user does not have the correct access rights for the requested action. The administrator must grant the necessary rights through Portal for that object and action.
The JCR node is locked by another workspace
Example:
NodeImpl 3 com.ibm.icm.jcr.NodeImpl save(false, false) Found lock on path: /contentRoot/icm:libraries[8]/Content/epfsite/welcome owned by: Workspace 7c2ba800465031b597d5f719fed3c258
SystemErr R com.ibm.icm.jcr.access.AccessDeniedException: The requested operation violates one or more lock constraints.: [ErrorCode:7591]
This is the common occurrence if the node is being held by another draft. In this case, you must first delete the node and its draft workspace before proceeding.
Older version of Portal have seen problems after a failed library delete where drafts are still left in the library. If this is the case, contact IBM Support to review the issue and provide the cleanup tools as needed. It is recommended to make a full database backup before using any tools which directly modify the database.
The JCR node is locked by another user
This is the common occurrence where another user is working on the same node, and is considered to be normal behavior. The best course of action for this failure is to log in as the other user and unlock the node.
If that user has been removed, IBM Support has the tools to remove the locks for that user. Contact IBM Support to review the issue and provide the needed tools if indicated. It is recommended to make a full database backup before directly modifying the database.
Delete all JCR locks for a node.
WARNING: Incorrectly updating the database tables can lead to database inconsistencies and deadlocks. You should not remove all of the locks for a node unless you are in the process of deleting that node, and have exhausted all other possibilities.
To delete all of the JCR locks for a node, you need to know the UUID for that node. IBM Support has a utility to internally remove the locks for that node. Contact IBM Support to review the issue and provide the required utility if indicated. It is recommended to make a full database backup before using any tools which directly modify the database.
Tracing all SQL statements (including host variables)
Enabling the pls.debug.trackStatementCursorLeakage setting in icm.properties, combined with JCR trace (com.ibm.icm.*=all), will trace all of the SQL from JCR, combined with the host variables. Note that this setting will significantly slow down performance, so you you should reset pls.debug.trackStatementCursorLeakage to false after collecting the necessary trace data.
To enable this value, do the following:
1. Stop Portal Server
2. Edit
pls.debug.trackStatementCursorLeakage=+
3. Set trace to com.ibm.icm.*=all and restart Portal
Deadlocks
Derby (Portal 6.1)
SQL Exception: A lock could not be obtained within the time requested
Check the derby.log file to verify that the customer is running at at least build 639536 of Derby 10.1.3.2.
DB2
Database hang during Portal upgrade
We have seen a problem where the customer will see a database hang while upgrading Portal version, for example upgrading to 6.0.1.4. This issue can occur when the database user does not have DBADM authority. Note that it is not enough to only grant SYSADM authority to the user, but the user must have explicit DBADM authority.
Subscribe to:
Posts (Atom)
