Tuning Tutorial

Scipio ERP is initially setup to run well for a long range of systems. By default, the system launches with a minimum of allocated memory. The database configuration templates aren’t optimized for your own configuration. All caching mechanisms (browser and internal caching) are largely turned off. Javascript and CSS files are to a large extend left unminified. Even the rendered html is uncompressed. In short: the initial configuration is meant to give a satisfying experience to most users, but it isn’t optimized for performance.

For live environments or simply faster running systems, here’s a list of things you can do to get the most out of your Scipio ERP system:

 

Memory Allocation

Scipio ERP requires a minimum of 4GB of allocated memory, which also happen to be the default start parameters. That doesn’t mean, however, that more isn’t desired. The recommended specs are ~12GB of allocated memory per server (more doesn’t necessarily return better results). Be aware, however, that the allocated memory should never be set higher than 3/4 of your overall available memory to leave some room for other system applications.

The maximum amount of memory Scipio ERP can use is set as -xmx parameters in the various start scripts (start.bat, start.sh or the various unix background scripts located under /tools/scripts/rc.*). Per default Scipio ERP sets the following:

MARKDOWN_HASHac328ab1af1ece462fe240d83f538c34MARKDOWN_HASH

 

-Mxx3512M sets the maximum allocated heaps size to 3.5GB. Increase the value to match your own system specs and restart the system.

CPU Threads

By default the service engine is set to use up to 4 CPU threads, which means that the service engine will use up to 4 cpu threads to process services. Most modern systems have a much greater number of threads available and it is recommendable to leverage them, to manage a larger number of services in parallel. This is in particular useful, if a service is spawning a large number of sub-services, or if bulk-operations are performed.

To set the maximum number of cpu threads, edit framework/service/config/serviceengine.xml:

<thread-pool send-to-pool="pool"
                 purge-job-days="4"
                 failed-retry-min="3"
                 ttl="120000"
                 jobs="100"
                 min-threads="2"
                 max-threads="5"
                 poll-enabled="true"
                 poll-db-millis="30000">
        <run-from-pool name="pool"/>
    </thread-pool>

 

Note that the number of cpu-threads is independent from the amount of cores available. Many modern CPUs feature a greater number of cpu threads than cores available.

Database Configuration

By default Scipio ERP uses a built-in DERBY database. This works fairly well for developer systems. For production/live environments a switch to a more sophisticated RBD is advised. PostgreSQL is a great, modern relational database, which is well performing and easy to setup. Follow our database configuration guide to setup your connector.

During configuration, we recommend to take a second look at the max-poolsize. There can be significant performance improvements with the right poolsize. Scipio ERP will over-time keep a pool of database connections open. This reduces the time spent opening up connections and therefore can slightly improve the overall database query time. However, the downside to this is that if the amount of open connection is greater than the amount of available connections within your database, the system will take a significant longer (sometimes endlessly longer) time to run individual queries. A deadlock is possible. This scenario can usually be triggered through a load-test, in which a lot of requests will spawn a mass of db queries.

To circumvent this scenario and improve overall query performance, it is therefore recommended to keep the max-poolsize parameter on Scipio ERP side. Ideally, the number of connections in the connection pool should be equal the number of the exec threads configured. This article gives a good idea on pool sizes. We recommend a poolsize of 50-100 connections, but results can differ depending on the overall load, your own components and the system environment. Set the pool-size inside of your database configuration.

On the database side, make sure that the max connection size matches the following formula:

RDB max connection size = Configured pool-size * Number of Scipio ERP systems + Other application connections

 

Server Side Caching

Scipio ERP offers system-wide caching on all three layers (entity, service & view). Since caching can negatively impact the work of developers, most caches are disabled by default. For a live-system we recommend to optimize this by turning on cache on all system levels (edit framework/base/config/cache.properties):

 


# Default Settings
#cache.file.store=runtime/data/utilcache
#default.maxSize=0
#default.expireTime=0
#default.useSoftReference=false

# SCIPIO: allow file listeners to trigger clear cache
cache.fileupdate.enable=false

# No maxSize for properties.UtilPropertiesResourceCache
#properties.UtilPropertiesResourceCache.maxSize=0
#properties.UtilPropertiesResourceCache.expireTime=0

# No maxSize for properties.UtilPropertiesUrlCache
#properties.UtilPropertiesUrlCache.maxSize=0
#properties.UtilPropertiesUrlCache.expireTime=0

# This should be increased if more users will be simultaneously on the system.
security.UserLoginSecurityGroupByUserLoginId.maxSize=1000
# 30 min
security.UserLoginSecurityGroupByUserLoginId.expireTime=18000000

security.SecurityGroupPermissionCache.maxSize=0
# 30 min
security.SecurityGroupPermissionCache.expireTime=18000000

# Examples for per-delegator Entity Engine cache settings
#entitycache.entity.default.ServerHitType.expireTime=0
#entitycache.entity.default.ServerHitType.useSoftReference=true
#entitycache.entity-list.default.ProductPriceRule.expireTime=0
#entitycache.entity-list.default.ProductPriceRule.useSoftReference=true

# product.content.rendered cache settings, set to expire in 1 minutes by default to avoid too much administrative confusion, can comment this out or increase for better performance
product.content.rendered.expireTime=60000000
product.content.rendered.useSoftReference=true

# product.config cache settings, used by product (pc like) configurations, set to expire in 1 minutes by default to avoid too much administrative confusion, can comment this out or increase for better performance
product.config.expireTime=60000000
product.config.useSoftReference=true

# SCIPIO: category content cache settings; should mirror those of product.content.rendered above (added 2017-11-17)
category.content.expireTime=60000
category.content.useSoftReference=true

# SCIPIO: Additional ContentWrapper caches, for reference/tweaking if needed
# WARN: Some of these could cause unwanted browsing behavior if cached inappropriately, notably order.content.
# The .useSoftReference=true is probably necessary (is default on all of these, but should be uncommented if other settings set here)
#order.content.expireTime=
#order.content.useSoftReference=true
#party.content.rendered.expireTime=
#party.content.rendered.useSoftReference=true
#configItem.content.expireTime=
#configItem.content.useSoftReference=true
#product.promo.content.rendered.expireTime=
#product.promo.content.rendered.useSoftReference=true
#workeffort.content.rendered.expireTime=
#workeffort.content.rendered.useSoftReference=true

# Development Mode - comment these out to better cache groovy scripts, etc
#These are all cleared by default every 10000 milliseconds, or ten seconds
minilang.SimpleMethodsDirect.expireTime=100000000
minilang.SimpleMethodsResource.expireTime=10000000
minilang.SimpleMethodsURL.expireTime=100000000

script.BshLocationParsedCache.expireTime=100000000
script.BshBsfParsedCache.expireTime=100000000
script.ParsedScripts.expireTime=100000000
script.GroovyLocationParsedCache.expireTime=100000000

# Uncomment this for more flexibility with service changes.
# Clearing this cache will reload service defs, for all service and so it takes time and results in lots of log messages, so it is commented by default.
#service.ModelServiceMapByDispatcher.expireTime=10000

webapp.BsfEvents.expireTime=100000000
webapp.JasperReportsCompiled.expireTime=10000000
webapp.ControllerConfig.expireTime=100000000

widget.form.locationResource.expireTime=10000000
widget.form.webappResource.expireTime=100000000
widget.menu.classResource.expireTime=100000000
widget.menu.locationResource.expireTime=100000000
widget.menu.webappResource.expireTime=100000000
widget.screen.locationResource.expireTime=100000000
widget.screen.webappResource.expireTime=100000000
widget.tree.locationResource.expireTime=100000000
widget.tree.webappResource.expireTime=100000000

template.ftl.location.expireTime=100000000
template.ftl.general.expireTime=100000000
widget.screen.template.ftl.general.expireTime=100000000
# SCIPIO: 2016-10: new cache for widget-implementing ftl macros (added for 1.14.2)
widget.screen.template.ftl.macro.expireTime=100000000
# SCIPIO: 2016-11-10: new caches for screen inline templates (added for 1.14.3)
widget.screen.template.ftl.inline.expireTime=100000000
widget.screen.template.ftl.inline.basic.expireTime=100000000

ModelDataFile.expireTime=100000000

# SCIPIO: 2017-05-08: targeted rendering expression cache. They are public-facing so they must have entry limit.
widget.renderer.targeted.expr.maxSize=1000
widget.renderer.targeted.expr.expireTime=0
widget.renderer.targeted.multiexpr.maxSize=1000
widget.renderer.targeted.multiexpr.expireTime=0

# SCIPIO: contains-expression cache - usually internal, but bad coding could cause problems so put a limit
widget.screen.containsexpr.maxSize=10000
widget.screen.containsexpr.expireTime=0

# SCIPIO: cache for some SEO alt url lookups - this should be treated similar to the product content cache (product.content.rendered)
seo.filter.product.alturl.part.expireTime=10000
seo.filter.product.alturl.part.useSoftReference=true
seo.filter.category.alturl.part.expireTime=10000
seo.filter.category.alturl.part.useSoftReference=true

Further optimize for your own setup.

Browser Side Caching

The provided code example will ask the browser to cache the entire content of a page. There are negative sideeffects for dynamic pages, so apply with caution!

To enable browser side caching on all requests, open up urlrewirte.xml in your own component (located under webapp/component-name/WEB-INF). Uncomment the following:


   
 <rule>
    <from>.*</from>
    <set type="response-header" name="Cache-Control">max-age=3600, must-revalidate</set>
 </rule>

HTML Compression

It is recommended to remove verbose logging and enable auto-compression on the HTML renderer. This will effectively remove the auto-generated <!– Comments –>  from the HTML output and further compress the whitespace, resulting in less overhead per request.

To do so, open framework/widget/widget.properties in an editor and set the following:


 widget.verbose=false;


 compress.HTML=true

Open the web.xml file located under webapp/component-name/WEB-INF/. Uncomment the following:


 <context-param>
   <description>Remove unnecessary whitespace from HTML output.</description>
   <param-name>compressHTML</param-name>
   <param-value>true</param-value>
 </context-param>

Javascript & CSS Compression

By default, most themes will rely on unminified versions of Javascript and CSS files. In order to reduce the filesize, we recommend to recompile the theme sources (located under theme/your-theme/webapp/your-theme/) for distribution.

For gulp (bootstrap, ignite*) use:

MARKDOWN_HASHdadc01d6f6ad8f99a5196ddffdf5bb25MARKDOWN_HASH

Update your themes seed data afterwards by editing the Seed data file located under /themes/your-theme/data/*Data.xml. Update all references to the newly created /dist versions.

Image Compression

High-resolution images are having a huge impact on the overall load-time of a website. To decrease image file-sizes, we recommend to make use of the Scipio ERP CMS component. The CMS component will automatically generate variations of the original in different sizes and apply a standard set of image compression. A list of all variants can be found inside the image viewer:

CMS – Image viewer showing resized variants

Alternatively, we can recommend the use of third-party software to optimize images prior to uploading them to the system. Tiny PNG offers great results for most images.

Webserver Tweaks

We recommend the use of a web server in combination with Scipio ERP. Once setup, web-servers can apply a wide range of options to improve the overall system performance (HTTP2, media file caching and more come to mind).

Google has released a page speed module for the Apache and nginx webserver, which already delivers a wide range of optimizations: Google Pagespeed Module. The module is easy to install and the various improvements can further trim the felt performance.