Service Development

This topic provides guidelines for developing production quality Groovy Services.

While developing Groovy services is easy, you need to be take care to follow secure practices and protect against failure modes which could adversely impact the Journey Manager (JM) server.

The first step in developing a Groovy service is to determine what type of service you need to create. The Transaction Processing Sequence illustrates the Service type extension points.

Creating Services

To create a new Service in Journey Manager, navigate to System > Service Definitions and click New. The Service Definitions page is where you manage your Groovy services.

On the New Service page, select a Service Type and the Service Template you wish to base your new Groovy service on.

Now, you have a new Groovy service which you can start developing. By using a service template, we have a starting point for our new Groovy script service.

The Groovy Script template includes comments detailing the script parameters or variables. You can also click the Help Doc tab to get more information about the script parameters and the expected return values.

Developing Service Scripts

Unit Tests

To develop your services business logic, write your logic in the Groovy Script tab then write unit test code in the Unit Test tab to exercise your Groovy script. Developing your services in this way improves your productivity and helps to develop much more robust code. Unit tests enable you to exercise Groovy services in ways which are really difficult to produce manually.

Most Groovy service templates provide you with a working Groovy script and unit test to get started with.

On the Unit Test tab, select the Enable Unit Testing option then click Run Test. If the unit test completes successfully and does not throw any exceptions, a green bar is displayed.

If the unit test throws an exception, a red bar is displayed indicating failure. When developing unit tests, use the assert keyword to ensure your code is producing the correct results. In the example below, the expected value returned for the address first line is '12 Wall Street', however the Groovy Script returns the value '123 Wall Street' instead.

Debugging Scripts

While it is very quick to write Groovy code online, there is no IDE debugger support. So how should you debug problems? We use the GroovyLogger class which enables you to examine variables or determine the code paths being executed.

We also use GroovyLogger to monitor the execution of production code and provide error diagnostics, so in the real world its actually a lot more useful than just a debugger which is only useful after the fact. When developing your script, use the logger object to log useful debugging info messages.

In the code example below, a debug message with the script result data is logged.

import com.avoka.core.groovy.GroovyLogger as logger
import com.avoka.tm.vo.*
import javax.servlet.http.*

class FluentDynamicData {

    /*
     * Perform Form Dynamic Data service call.
     *
     * returns: REST response text data
     */
    String invoke(SvcDef svcDef, Txn txn, HttpServletRequest request, User user) {

        // TODO: replace with data lookup call

        String data = '''{
          "address": {
            "firstLine": "123 Wall Street"
          }
        }'''

        logger.debug 'data=' + data

        return data
    }
}

In the example unit test output below, we have a logger DEBUG message from the Groovy script, and a logger INFO message from the unit test script.

For more details, see Service Logging.

Runtime Security

The Groovy Script runtime environment has some restrictions to prevent the execution of code which could cause system integrity issues or result in access to unauthorized data. If the runtime determines the script is performing an illegal operation, the script is not executed and an error message is displayed:

Groovy Script contains an illegal reference

If the script attempts to access data that it is not authorized to access, a null value is returned.

Most Groovy Services are executed in a client Organization context with restricted access to transaction data. Only system administrators with explicitly configured global access policies can access multiple organizations transaction data.

Any Groovy Script executed in the Groovy Console is logged in the Security Audit Log along with details of the user and when this was performed.

note

The execution of Groovy Services is logged in the database Groovy Service Log. Changes to Service Parameters are logged to the Security Audit Log.

Service Timeouts

Sometimes Groovy Script services may not return because an external system responds extremely slowly, or because of some coding error causing an infinite loop.

To guard against these types of issues, Groovy Services are configured with an execution timeout. If a Groovy script doesn't complete execution in the given time, it is interrupted and a GroovyScriptException is thrown. By default, most of our Groovy services have a default execution timeout of 1 minute. This value is configured as a service parameter called executionTimeout.

If a Groovy Service does not define an executionTimeout then the system Deployment Property Groovy Script Timeout is used instead. The default value for Groovy Script Timeout is 10 minutes.

Best Practices

The following are recommended development best practices and guidelines to follow when developing Groovy services.

Unit Testing

Develop your Groovy services using unit tests, and ensure you test the error case scenarios in addition to the standard flows.

tip

You can use the unit test framework provided to develop Groovy services faster.

Use Service Test Suites to enable Continuous Integration (CI) testing of Groovy Services.

Error Handling

When developing Groovy scripts, you have a lot of power at your disposal, but you also have a lot of responsibility. The Groovy scripts you write will be executing in the same Java process as the application server. This provides very fast access to data and services, but you need to take care. Badly written scripts or scripts calling badly behaving remote services can impact everyone.

Be aware of the context in which your service script is running and use the appropriate error handling guidelines as specified in the Groovy Service API.

If you're performing business logic in the user transaction thread when using a Form Saved Processor or Submission Completed Processor, you need to ensure you dont throw unexpected exceptions or the entire transaction may be rolled back and the user will lose their submitted data.

Call Frequency

The Journey Manager application server uses a pool of threads shared between multiple requests. When developing service scripts, you need to ensure you don't exhaust this thread pool with long running scripts. While the application server grows this thread pool dynamically, expect it to be limited to around 200 threads.

If your script will be executed extremely frequently, such as in a Dynamic Data service performing a field auto-suggest lookup, it needs to run very quickly as the server may receive hundreds of requests per second.

If you are calling a remote service for form data, consider caching the results in memory for use in future calls.

If your script is running in the end user thread, and it's calling an external service which may fail, then you should attempt to fail fast. The speed at which you fail will depend upon the frequency of the call and its business value. Dynamic Data auto-suggest lookups should fail very fast (less than 1 sec), while a Submission Completed Processor calling an internal business system can be more tolerant (less than 60 sec).

When calling remote services, ensure you set the appropriate connection timeouts. For more details, see Remote Service Calls.

Memory Management

The Journey Manager application server provides a sufficient amount of memory for you to do your work in which should be enough for most purposes. With Groovy scripts you don't have to manage memory: this is done automatically for you.

That said, be aware of the following.

• If you read data from an external system into memory, take care not to read too much data into memory. For example, don't load 1 million data database rows into memory, or download all the content of Wikipedia.
• If you need to cache very large amounts of data in memory (more than 1 GB), you may need to request the Journey Manager servers RAM be increased.

Managing Complexity

How big should your Groovy scripts get? There is no hard rule. Performance is not an issue. The main concern is ensuring your scripts are maintainable.

In Groovy, 500 lines of well written and properly commented script can be readily maintained, while another person while 50 lines of cryptic and compact script may be completely unintelligible. Remember compact code does not run faster, favour readability to make your code maintainable.

Create Separate Groovy Services

For larger scripts (more than 500 lines), consider decomposing them into separate Groovy Services which can then be called by your scripts. You can use this technique to re-use the same script block in multiple services, saving on development and testing effort.

import com.avoka.core.groovy.GroovyLogger as logger
import com.avoka.tm.svc.*
import com.avoka.tm.vo.*
import javax.servlet.http.*

class ServiceInvokerService {

    String invoke(SvcDef svcDef, Txn txn, HttpServletRequest request, User user) {

    // Create a map of parameters
    def parameters = [:]
    parameters["txn"] = txn
    parameters["user"] = user
    
    // Invoke the Groovy service passing in the parameters
    def result = new ServiceInvoker()
                .setServiceName("My Service")
                .setClientCode("Maguire")
                .withHighestVersion()
                .invoke(parameters)

    return result;
    }
}

Use External Systems

In some cases, you can maintain complex business logic in another system and expose it to your Groovy Service via a remote service call. This provides similar benefits to script decomposition.

Secure Coding Practices

Journey Manager solutions are typically public facing, and involve the capture and handling of sensitive data. While developing a solution, you need to consider how to protect end users data and prevent criminals from hacking your solution. The Open Web Application Security Project Top 10 provides a great resource for developing secure web applications.

Logging Sensitive Data

By default Personally Identifiable Information (PII) transaction data is encrypted by Journey Manager, however, you need to be careful you don't accidentally log sensitive personal information to server log files or insecure data stores.

For example, don't do the following as this will write PII to the server log file:

println "Form Data: " + submissionXml

logger.info "Form Data: " + submissionXml

JavaScript Cross Site Scripting Attacks

If your service is handling GET or POST request parameters, and using these in service calls or injecting them into XML form prefill data, you need to prevent criminals from attacking your solution with JavaScript based attacks.

Journey Manager provides a Security class to identify cross-site scripting (XSS) values based on the OWASP XSS Filter Evasion Cheat Sheet rules. The script example below is redirecting the user to a Not Authorized page if a URL request parameter contains a dangerous XSS value.

import com.avoka.core.groovy.GroovyLogger as logger
import com.avoka.tm.util.Security
import com.avoka.tm.vo.*
import javax.servlet.http.*

class FluentFormSecurityFilter {

    String invoke(SvcDef svcDef, Txn txn, HttpServletRequest request, User user) {

        def accountId = request.getParameter('accountId');

        if (!Security.isXssSafeText(accountId)) {
            throw new RedirectException('../not-authorized.htm')
        }

        return "";
    }
}

SQL Injection Attacks

If you're using form data in database code, take care to prevent SQL injection attacks. Use prepared statements that automatically escape text values for you.

// Get the contact details out of the form submission XML
def xmlDoc = new XmlDoc(submissionXml)
def firstName = xmlDoc.getText("//Contact/FirstName")
def lastName = xmlDoc.getText("//Contact/LastName")
def email = xmlDoc.getText("//Contact/Email")

// Create a database SQL object
def sql = Sql.newInstance("jdbc:mysql://localhost:3306/mydb", "root", "password", "com.mysql.cj.jdbc.Driver")

// Use a prepare statement to insert values
sql.execute("insert into contact_record (first_name, last_name, email) values (?, ?, ?)", [firstName, lastName, email])