Modularization, Part 2

From our previous discussion in part 1, let us return to variable scope. Programming languages which enforce variable scope do so by logical constraints built into the language framework. There is nothing physical that enforces variable scope in a computer’s memory. The language’s compiler handles the grunt-work details to create code which achieves variable scoping in memory that is otherwise globally-accessible. We can do the same in ACL scripts, albeit manually, by following a few rules:

  • Scripts declare a contract which defines the necessary inputs and the promised outputs
  • Scripts cannot change input data
  • Scripts cannot use any data element other than what is defined in the contract or created by themselves
  • Scripts do not output anything other than what is defined in the contact

By following these rules, each script becomes a black box and knows nothing about the execution of other scripts. The only things scripts know are what values they provide other scripts and the expected data to be returned. How the other scripts arrive at the output data becomes completely irrelevant. Because scripts become black boxes, they can be plugged and unplugged at will without requiring additional coding maintenance. This causes your code base to take a giant leap toward becoming modularized. Let’s take a brief look at how each rule might be accomplished.

Scripts declare a contract

At the beginning of each script is a comment block that identifies 3 things: What the script does, what inputs are required, and what output is provided. A sample block might look like this:

COMMENT
** This script imports invoice data from the ERP system
**
** The required inputs are:
** v_InvoiceStartDate – A date variable defining the start of the period to import
** v_InvoiceEndDate – A date variable defining the end of the period to import
**
** The output is a table called ‘Invoices’ containing invoice data from the EPR invoice tables.
END

Below the comment block may be code which tests for the existence of the required inputs. This is not strictly required, but does help to quickly identify when the script is improperly called.

PAUSE “You are missing an input variable” IF NOT MATCH(‘d’ FTYPE(“v_InvoiceStartDate”) FTYPE(“v_InvoiceEndDate”))
ESCAPE ALL IF NOT MATCH(‘d’ FTYPE(“v_InvoiceStartDate”) FTYPE(“v_InvoiceEndDate”))

If you use AuditExchange, you may want something different than ESCAPE ALL. In situations where an input is missing, I want the script to exit with an error, not complete successfully. The solution I have come up with is to issue a DIALOG command with a customized error message. When used in AN developers will see the error and when used in AX the scripts will exit with an error.

When defining the inputs, consider forcing input tables to be defined in variables.  For example, if a script requires a table containing invoice data, the required input should be a variable that contains the table name, not the table itself.  This allows the script to be used with any input table and frees it from dependencies to specific import scripts.

Note: In those situations where a second script file is used to perform looping (DO…WHILE), I consider both files to be a single script. I do not create a contract in the second file because that file is not meant to stand alone.

Scripts cannot change input data

This one is easy – every input must have the same value at the end of the script. For input tables, this means extracting to temporary tables before adding/removing fields or changing table layouts. For variables, this means never assigning values to any of the input variables.

Scripts cannot use any data element other than what is defined in the contract or created by themselves

At first glance, this seems easy. If the input table or variable is not defined in the comment header or created by the script itself, then we simply do not use it. But this rule is also very easy to disregard for the sake of convenience. How many times do we see a table or variable created in another script and decide to use it in our current script because it is already there? Unfortunately, doing this creates a dependency between the scripts and results in another string of spaghetti that tangles our code.

This rule is also easy to inadvertently violate if variables are not given descriptive, unique names in each script. Multiple scripts may declare a counter variable to use in a DO loop. If the counter variable has the same name in each script, it is possible that they could overwrite each other and cause unintended consequences. It thus becomes important to give variables unique names.

Scripts do not output anything other than what is defined in the contract

Put another way, scripts must clean up after themselves. Any non-output variable that is created by the script must be deleted at the end of the script. All temporary tables must be deleted at the end of the script. I even go so far as to delete the script itself (I store my scripts in .aclscript files on a shared folder, so the in-project copy is extraneous).

Note that it would be inappropriate to use the DELETE ALL command because the script could delete variables used by other scripts. This means the end of each script could have a long list of ‘DELETE variable OK’ commands. Deal with it – it is a one-time effort that cements the modularity of your scripts.

Putting these rules together would give us an import script that looks something like the script below. The commands in red are the framework commands to achieve the 4 rules. It may seem like a lot of extra coding, but there are 2 things to note: 1) The pain of coding the extra commands can be alleviated by using a small handful of templates and 2) these extra commands do not noticeably affect the performance of the scripts.

COMMENT
** This script imports invoice data from the ERP system
**
** The required inputs are:
** v_InvoiceStartDate – A date variable defining the start of the period to import
** v_InvoiceEndDate – A date variable defining the end of the period to import
**
** The output is a table called ‘Invoices’ containing invoice data from the EPR invoice tables.
END

COMMENT Make sure the input part of the contract is satisfied
PAUSE “You are missing an input variable” IF NOT MATCH(‘c’ FTYPE(“v_InvoiceStartDate”) FTYPE(“v_InvoiceEndDate”))
ESCAPE ALL IF NOT MATCH(‘c’ FTYPE(“v_InvoiceStartDate”) FTYPE(“v_InvoiceEndDate”))

v_InvoiceStartDateStr = ALLTRIM(DATE(v_InvoiceStartDate ‘YYYY-MM-DD’))
v_InvoiceEndDateStr = ALLTRIM(DATE(v_InvoiceEndDate ‘YYYY-MM-DD’))
IMPORT ODBC SOURCE “ErpDsn” TABLE “InvoiceTable” TO “Invoices_T01” WIDTH 100 MAXIMUM 200 FIELDS “InvoiceNum”,”InvoiceDate”,”IsReversed”,”Amount”,”Customer” WHERE “InvoiceDate BETWEEN ‘%v_InvoiceStartDateStr%’ AND ‘%v_InvoiceEndDateStr%'”

EXTRACT FIELDS ALL TO Invoices IF IsReversed = ‘False’ OPEN

COMMENT Clean up the non-output data created by this script
DELETE v_InvoiceStartDateStr OK
DELETE v_InvoiceEndDateStr OK

DELETE FORMAT Invoices_T01 OK
DELETE “Invoices_T01.FIL” OK

With a little bit of extra coding to force a modular architecture, you will find that you will be able to maximize the flexibility of your scripts so that they can be re-used when and where they are needed.  You will also find the scripts to be easier to maintain and should see improvements in the stability of your code.  I am constantly working in both our legacy CCM code and our new code written in this modular format, and find the new code to be much easier to work with.  I have less fear of making changes and perform less testing when I do have to make changes.  Most importantly, the new code is more stable and runs with greater reliability, allowing me to focus on new development rather than fixing old code.  During your next development or project, try to ruthlessly implement a modular architecture and compare it to your old way of scripting.  You will not look back – I certainly have not.

Advertisements

2 thoughts on “Modularization, Part 2

  1. Great blog, Tom. Very easy to read. And the detailed examples in your GIT post and your GROUP posts are really helpful. In your GROUP2 examples, a few screen shots of the output files have a column for “TaxAmt” but that column in the script is “RunTotal”.

    We also use modular scripts. I wish AX had the ability to call utility scripts without using .aclscript files. Here’s the template we use at the top of every analytic:

    com ************************************************************************************
    com Script Name:
    com Script Purpose:
    com Original script author: <>, <>
    com Date script originally moved to production:
    com Version: 1.1, <> – See modification log below
    com
    com Required Inputs:
    com 1) N/A – all inputs optional
    com
    com Optional Inputs:
    com 1) optional variable “v_example”: description of variable use
    com
    com Outputs:
    com 1)
    com
    com User Prompts:
    com 1) Optional start date when running in AX
    com 2) Optional end date when running in AX
    com
    com Default Values:
    com 1) Example: Start date defaults to 90 days ago (today – 90) if not set at runtime
    com 2) Example: End date defaults to yesterday (today – 1) if not set at runtime
    com
    com Limitations:
    com 1) Describe known limitations or built-in assumptions
    com 2) example: Keywords must be less than 30 characters each
    com
    com Notes:
    com 1)
    com
    com Modifications:
    com Modified by:
    com Date modified:
    com Reason for modification:
    com Description of modifications to script code:
    com
    com Modifications:
    com Modified by:
    com Date modified:
    com Reason for modification:
    com Description of modifications to script code:
    com ************************************************************************************

    SET SAFETY OFF
    SET DATE “YYYYMMDD”
    SET SUPPRESSXML ON
    SET EXACT ON
    SET FILTER
    SET LOOP 100

    Like

  2. Thanks for your comment, Chris. I am glad you find the detailed examples in the more technical posts to be useful; that was my intention when I was creating this blog.

    Interesting how even after double-checking my work many times, something like the column names still slips through. Thanks for letting me know – I will work to update those pictures.

    Thanks also for sharing your template. There are many ways data analysts can go about defining their scripts that suits their style, and yours is another good option. I like how you break out optional vs required inputs. The limitations and default values are also good things to note – I may incorporate those into my own headers. The section on modifications is also good to have for those who do not have something like Git that can track that for them.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s