Super Slowness on Content Presenting Rendering due to Large Number of Profiles, Region Definitions or Static Lists

Update:

The Oracle Patch on the bug has been released: Patch 17328920: CONTENT PRESENTER THROWS STUCK THREAD AND OOM WHEN UCM RETURNS MANY PROFILES.

 In this fix, a system parameter "contentTypeLazyCaching=true" is requested to be added on the JVM startup.

Custom code has been added into the ContentTypeFactory.class in the two places:

• %ORACLE_HOME%/webcenter/modules/oracle.webcenter.content.integration_11.1.1/ucm_spi.jar/oracle/webcenter/content/integration/spi/ucm/factory/ContentTypeFactory.class
• %ORACLE_HOME%/webcenter/modules/oracle.webcenter.content.integration_11.1.1/content-app-lib.ear/APP-INF/lib/ucm_spi.jar/oracle/webcenter/content/integration/spi/ucm/factory/ContentTypeFactory.class

The directory could be either on your local Jdev folder or on the remote WebCenter portal server.

The added custom code is a new method getMinimalContentTypes():

  public synchronized List<ContentType> getMinimalContentTypes()
    throws RepositoryException
  {
    List typeNames = getTypeNames();
    List minContentTypes = new ArrayList();

    for (String typeName : typeNames)
    {
      if (!typeName.startsWith("IDC:Profile:"))
      {
        ContentType type = UCMCacheHelper.getContentType(this.repoName, typeName);
        if (type != null)
        {
          minContentTypes.add(type);
          continue;
        }

        try
        {
          type = buildContentType(typeName);

          minContentTypes.add(type);
          if (type.isSearchable())
          {
            this.searchableContentTypes.add(typeName);
          }

        }
        catch (NoSuchContentTypeException re)
        {
          logger.log_NO_SUCH_CONTENT_TYPE_BY_NAME_ERROR(typeName, re);
        }
      }
    }
    return minContentTypes;
  }

  private void ensureAllContentTypesLoaded() {
    try {
      if (this.contentTypeLazyCaching)
        getMinimalContentTypes();
      else
        getAllContentTypes();
    }
    catch (RepositoryException re) {
      throw new RepositoryRuntimeException(re);
    }
  }

So instead of get all content types, the custom code bypasses all the profile content types as all profile content type starts with "IDC:Profile:".

This fix actually resolves the content presenter issue on the customer's environment. But it's not the final fix yet. As VCR_GET_CONTENT_TYPES returns all profiles, region definitions and static lists on the UCM server, the large number of region definitions and static lists could also cause additional performance problem.

More to come on the final resolution to this problem.

====================================================================
Original Post:

Content presenting is a great feature introduced in WebCenter 11g. As Portal can create its own content display template, it makes the feature very powerful to present content data file in ADF-native way. It also allows the region template (created in site studio) to be reused in WebCenter Portal and make it even greater. During my recent practice, I have come across a bug on the content presenting rendering in a very long time if the content server (UCM) contains a large number of profiles, region definitions or static lists.

In my target environment (11.1.1.7.1), the content server has over 1800 profiles, over 100 region definitions and 10-20 static lists. Using the OOTB content presenter task flow, to render a HTML file (less than 1 Kb) would take around 35-40 minutes!!!

This problem is only happening to content presenter task flow not other document task flows. The issue wasn't tested on 11.1.1.8 but it should be replicable on 11.1.1.8 as well.

After troubleshooting, we found when content presenter task flow tries to consume a content file, it sends request to the UCM server. UCM will initiate the following service call in sequence:

VCR_GET_DOCUMENT_BY_NAME
VCR_GET_CONTENT_TYPES
VCR_GET_CONTENT_TYPE
VCR_GET_CONTENT_TYPE
......
GET_ADVANCED_SEARCH_OPTIONS
GET_USER_PERMISSIONS
GET_FILE

The problem happens at the "ellipsis". The service call VCR_GET_CONTENT_TYPES (in plural) will return a collection of all the profiles, regions definitions and static lists in the content server. Then it's followed by looping through of the collection of the result set with one VCR_GET_CONTENT_TYPE call. Therefore if the VCR_GET_CONTENT_TYPES returns 2000 rows, it will result in 2000 calls on VCR_GET_CONTENT_TYPE. That's why it's causing the super slowness in the content presenting rendering.

To find out how many rows that the result set from VCR_GET_CONTENT_TYPES return, simply do a service call:  http://ucmhost:port/cs/idcplg?IdcService=VCR_GET_CONTENT_TYPES&IsSoap=1

Such call only happens once which is the first time a content is consumed by the content presenter task flow. The result set of the all content types will be saved in cache and serves for subsequent requests. Therefore such implementation is intended caching, but it certainly didn't consider the scenario of large number of the profiles, etc.

Currently the issue is filed as an Oracle bug (17328920) and the fix is soon to be released.

There are two temporary workarounds before the official fix is released.

1. Reduce the # of the profiles, region definitions and static lists. If you can reduce the total # of such assets down to 40-50, you should be fine. As you increase the #, you would expect to see increased degradation of the performance on the first time rendering. This is not an option if total # of the assets cannot be reduced.

2. Customize the VCR_GET_CONTENT_TYPES service call and make it only return a subset of the profiles, region definitions and static lists. This requires an evaluation of the WebCenter Portal application to determine what profiles, region definitions and static lists are required. Those required elements have to be part of the result set from the service call. Otherwise, errors like "resource not found" would be thrown.

I will update the post again once the official bug fix is released.