Deploying Reflexis Flow systems
by Ashwin Hirschi, Reflexis, March 2012 (updated: July 2014).
This note describes how to set up a Reflexis Flow system. We will detail the required components and explain the tools we provide to help you deploy a system quickly.
More specifically, we've created tools to help you initialise a Flow instance and package the various bits and pieces that actually make up the Flow solution you've created. Using them will not only speed up initial installation, but also makes updating an existing instance fast and easy.
We will not describe how to configure your target machine or set up things like a web server. Reflexis Flow works fine with any web server that supports CGI applications. But since web servers come with their own manuals, we feel there is little need to repeat them here.
Finally, as this is a draft document we very much welcome your feedback. If you feel anything is missing, incorrect or unclear, please let us know. We thank you kindly in advance.
Initialising an instance
Deploying web applications involves many different parts, moving or otherwise:
In this note we will only briefly touch upon the web server itself. Different web servers have different ways of doing things. You will have to figure out the specific needs of your server yourself. Here, we'll focus on how you can initialise your Flow instance.
A Flow instance is basically an installation of the Reflexis Flow system. Reflexis Flow allows you to set up many instances on the same server machine. Separate instances are located in separate folder trees. Deploying different projects in different instances, enables you to evolve them independently.
To set up and initialise an instance, we've created a tool: deploy. After you've copied the i runtime (i.e. the trio i.exe, i.lua & i.rfx) to your target machine, you can run it like this:
where you replace folder above with the actual path of the new folder you'd like to create and install the Flow instance in. Be sure not to deploy into an already existing instance, though!
When you start the deploy tool, it will create the folders that Flow needs and place the required executables, system files and databases in them. Note: deploy does not make any other changes to your machine. So, uninstalling is simply a matter of removing the instance folder.
You can browse the newly created folder structure to get an idea of where is what. In the root of the instance you'll see indigo.exe and indigo.lua. Together they act as the CGI-interpreter for the Flow system. Support files are distributed over the main subfolders: bin and dat.
The most important locations are the instance root folder itself and the dat folder tree. You'll need to remember the root folder, since you must let your web server know that URLs using the .rfx extension must be associated with the indigo executable in this folder.
Also, as the deploy tool will mention, you'll need to place your actual Flow package in the root folder as well. We'll discuss packages in a minute. For now we'll just say that a package contains the actual Flow system software plus any extensions you may have created (and need) yourself.
The dat folder tree is noteworthy because it contains important folders, like:
- flw, for the flow definition repository (task.db) and application databases
- cfg, for configuration files, like smtp.dat and web services configurations
- ses, where session files (*.ses) are stored
- log, where several system databases (alerts, block and catch) reside
- que, for subfolders representing queues (like outbox, for email, and ws, for web services)
Undoubtedly, dat/flw is the most important folder of the bunch. As your web solution will contain one or more flows, you'll need to make sure the repository (i.e. task.db) of this instance contains your flows. Depending on how you've modelled your flows, there are several ways to accomplish this.
If you have a flow definition available in text format, adding a flow to a repository is easy:
where you replace filename and task-db above with the filenames of your flow definition and your flow repository, respectively. If the task.db you specify doesn't exist, syncflow will create one for you.
Note that you can run syncflow repeatedly to add several flows to the same repository. If a flow is already present in the repository, syncflow will update it using the definition you provide.
If you do not yet have your flow definition available in text format, you can retrieve it from the repository that you modelled your flow in with the readflow tool:
will export your flow definition from the task repository you've indicated and write it to filename. You can now use syncflow to import your flow definition into your target repository.
Note that readflow requires you to provide the name of a file that does not yet exist. This is a simple safeguard to prevent you from accidentally overwriting important files.
Lastly, it may be easier to create the repository for your new instance on your development machine. If the tools and definitions are present, it's often more convenient to simply prepare the repository here instead of transferring all the ingredients to the target server separately and assemble things there.
Producing a package
In the previous section we discussed how to set up a fresh Flow instance and transfer your project flows. In this section we'll discuss Flow packages and how you can create and manage them.
Like its name implies, a Flow package is essentially a mechanism to collect various software components. For Reflexis Flow the most important components are its parts and scripts.
While parts are simply building blocks for your flows, scripts denote a wider-ranging group of software. They include library and application scripts, worker scripts and starters (to name but a few).
As different projects may require different sets of parts and scripts, we provide a tool that enables you to quickly define, build and update your packages. The pack tool uses manifests to figure out what to put into the package for your project:
will read the manifest you've provided and assemble the indicated scripts and parts into a package. If the package does not yet exists, it will simply be created from scratch. If the package does exist, it will be updated: missing components will be added and out-of-date items will be refreshed.
Like syncflow, you can run pack multiple times on the same package using different manifests. This enables you to keep your manifests to-the-point and run selective updates quickly.
So, in order to create your own, project-specific packages, you need to know how to create your own manifests. A manifest is simply a text file where each item (i.e. script or part) occupies one line and items can be grouped into sections.
Several sections are pre-defined, so you can quickly create manifests from scratch. These sections are called: scripts, parts and apps. But before we explain the remaining details, here's an example to give you an idea of what manifests look like:
Note that both scripts and parts do not have extensions in their filenames. The pack tool will first look for a file with the .lua extension, and if that fails try the same file but with an .lc extension. If a script is available in source form, it will be compiled and checked, before being stored in the package.
At this point you should realise that to create a project-specific package you'll need to start with a "project-neutral" base package, containing the basic Reflexis Flow system. For this purpose we've created a special, minimal distribution package: runtime.rfx.
To wrap up, the package build-and-transfer procedure thus becomes:
- create a project manifest to specify (all) your own scripts & parts
- copy the (default) runtime.rfx to your-project.rfx
- run pack to add your own material to your-project.rfx
- copy your-project.rfx to the root of its Flow instance on the server machine
- rename the package to indigo.rfx
The final step is necessary for indigo.exe to actually detect and use your package.
Taking the tour
Before closing we'll provide a point-by-point, high-level take on what we've described above:
- to start a Reflexis Flow system: initialise an instance & produce a suitable package
- use deploy to create & initialise a Flow instance
- use pack to create (or update) a Flow package
- create manifests to tell pack about your own scripts & parts
- copy the runtime.rfx base package to create a starting package
- transfer your project flows to the repository (dat/flw/task.db) of the Flow instance
- use readflow to export flow definitions from a repository
- use syncflow to import flow definitions into a repository
- to run the tools above, you'll need the i trio (i.e. i.exe/lua/rfx)
About your data and configuration:
- don't forget to put your application database(s) in dat/flw of your instance
- if your system uses configuration files, place them in dat/cfg
And a few points about configuring the web server:
- associate the .rfx extension with the indigo CGI-interpreter in the instance root
- place the flow folder (with web resources) and the flow.rfx file in the web server doc root
- place your own web resources (if any) in the web server doc tree
- always check your web server manual about managing rights and making adjustments
We recommend you always keep project-specific web resources in their own folder, separate from the Flow-specific "flow" folder that we distribute. This makes it much easier to perform this particular step of deploying your web application.
One final point remains: Reflexis does not allow its source code to be distributed!
In other words, please do not distribute source code we've provided you with, either in original or in modified form. You're free to modify our material to suit your needs. But you may only distribute material in non-compiled form if it does not contain any of our code.
We've nearly reached the end of this note. By now you should have a clear understanding of what's involved in setting up a Flow system. If the process seems overwhelming at first: Don't worry! Once the basic principles sink in, the process is actually very straight-forward.
Even while preparing complex software for deployment, there's very little "magic" involved. That is, nothing happens behind the scenes. During every step of the way you're in full control. After you've walked this way once, travelling the same road anew should prove quite relaxing.
In fact, we've designed the system specifically to speed up (and ease) the process of applying updates. In practice, many changes to your web application will mean you only need to "sync" your newly updated flow into the repository of your instance.
If you also had to change some of your Flow parts, library or worker scripts, type handlers, the starter, or perhaps some web service definition wrappers, the only extra update step required is to "pack" everything together. Simply place your updated package in the instance root and you're done!
In short, we hope we've created a system that makes both the initial deployment as well as the subsequent evolution of (potentially complex) web applications the proverbial breeze. We'll let our Reflexis Flow users decide if we succeeded in our aim and welcome your feedback.