Jeremy Davis
Sitecore, C# and web development
One of the interesting announcements from last year's Symposium was Sitecore's approach to how we might join up our composable applications. They announced that they were working on "Sitecore Connect" at this event, but didn't really get into detail of what it would be outside of "we're re-badging Workato". So what is that, and what does it suggest we might get out of Connect in the future? Read on, for an attempt at working that out...
Sadly it's too early for me to get my hands on an actual Connect instance, but I have had a chance to watch a demo at a user group, and spend some time with the core Workato product and its training materials. So my guess is this is fairly close to what we'll end up with, but branding (and maybe some features) are likely to change.
When we move away from "The Monolith" and towards a collection of composable SaaS systems we have much more need for integrations. Aas well as the internal systems we used to have to wire our websites into, we also need to join together the composable parts of the website itself. If our ecommerce system receives an order, we might want to record that in our customer data platform in order to drive personalisation. Or maybe we need to make sure form submissions get written into our CRM. The content-change WebHook events I discussed a while back could get wired into another system... There are loads of potential examples here, but the broad concept is that we need a convenient way to manage moving data between our individual systems when some event occurs that we care about.
Enter IPaaS - Integration Platform as a Service - SaaS services which provide tooling to help you set up these integrations without needing custom code or development projects. They provide mechanisms to detect and read the events that you want to trigger an integration, an engine which can run your integration workflow, and then logging and management tools to help you monitor your processes.
Having realised there's a product-lineup gap for a tool like this in their SaaS platform, Sitecore had a choice. Broadly they could do one of three things:
Despite their previous approach with Data Exchange Framework, Sitecore have chosen to go down road #2 here. Though this time they've not purchased a company. They're licensing a product that they can re-badge with their logo and customisations. They've gone for Workato, which is a well regarded platform in this space according to the big analysts.
(Though despite this choice, you can absolutely still bring your own tool - Sitecore's open approach and the broad concepts of "composable" means if you want to make use of something else, you absolutely can. But you might find yourself having to do a but more of the connectivity work)
When you sign in to Workato (and likely to Connect in the future) you get a UI that lets you manage the set of projects you're working on. A project here is a container for your workflows (or recipies as Workato calls them) and the connections to external systems that they make use of. It's a folder-like structure that lets you organise your resources to match your work, and you can apply role-based security to suit your organisation's needs to control access. Their training docs show an example with a collection of projects, and assorted assets in each one:
When you create or select a recipe you get an editor you're presented with an editing view. You get a fairly intuitive view of the flow that will run, and the steps that involves:
Each flow has to start with some sort of trigger at the top. It can be time-based (with a cron-like scheduler), event based (with a Webhook call, or a more specific event raised by one of their specialist connectors) and it also supports chat-bot style triggers. There are around 200 custom connector that I could see in the core product, ranging from Active Directory to Zoom, and there are further community-provided ones. Sitecore have said they intent to provide custom connectors for their APIs as part of Connect - but obviously these weren't available in the base Workato product I was able to look at. But what you can currently see is a big list:
Once you have picked a source for your trigger, you can configure it. Depending on what you choose that might involve authenticating with that service (creating a connection, in Workato's terminology) or configuring its options. These options vary dramatically based on your choice of trigger, but the UI is clean and simple. Configuring their scheduler gives you this:
Where there are choices you get dropdowns or on/off selections, and where there's more freeform data you have edit boxes which specify their data type (the "123" at the left of "Trigger Every" here specifies its a numeric field) and their units where relevant (the "minutes" at the right of that box - taken from the "time unit" dropdown above). There are also good help documents availabe which explain the options in more detail.
After you set the trigger, you get to specify the actions. Clicking the "add step" button in the design view gives you a set of core choices for the next step:
The "if", "if/else" and "repeat" actions are the basic logic constructs for testing state and repeating actions. The "stop job" and "handle error" options are fairly obvious. But the core of the work happens under "action in an app". That's their way of describing the set of behaviors they support with all their connected apps. Again, the list of behaviours here is big - all the systems with connectors are represented here, as well as more generic operations like "make a web request" or "parse a json or xml document". They roughly split up between actions that fetch data ("ask SalesForce for a particular record"), transform data ("parse some json and load it into the workflow's context") or write data ("use GMail to send a message"). And they all have the same interface for configuring settings.
You can add new actions anywhere in your flow - the "add new" button is available at the end of your workflow and in between any existing steps - always giving you the choices shown above. And you have the option to copy/paste actions, delete them or add comment text to explain their purpose in your flow:
Loops and if/then constructs are shown clearly in the flow, making the flow visible:
Wherever you have a configurable property for an action you're free to supply a constant value, or you can add what the UI describes as a "data pill". That's basically a value in the current state of your execution. It might be a setting from a trigger or an action, or some data that was returned by a previous action. The editor is aware of the schema of the data - either because you provide examples when configuring the system, or because the schema is set up by the connector or action itself. That makes picking data pretty friendly.
When you're providing these values you can add "simple" data by default: the constants and data pills described. When you're filling in one of the property fields, a chooser pops up to the left of these fields, listing out values you can use:
For more complex operations you can add a formula. There's a toggle on the top right of the relevant fields where you can switch from the light-coloured "text" mode to the dark coloured "formula" mode. That allows you to use code to affect the value. The formula entry uses a Ruby-like syntax where there are no parentheses after method names, but otherwise it's fairly obvious. And it comes with good context sensitive help, which appears below your chosen field and shows you options for the data types you're working on:
So between the data pill chooser and the context sensitive help for functions it's fairly easy to get to grips with mapping data from one system to another, and it's nice and visual so it doesn't scream "this needs a developer".
Once you're happy with your job, you can click from the editing view to the testing view. This will try to run your job - though that means if your trigger relies on external data you need to make sure its set up and accessible. If it fails, it will report where errors have occurred, and if it succeeds you'll see which path the execution took:
Again, it's a nice visual approach to seeing what's going on. And you can toggle back and forth between editing and testing as you need.
But there're are still a collection of things we don't know much about which might affect the real success of this tool in projects:
For a start, it's not clear to me right now how we would handle the process of migrating work from a development environment through testing and on to production. You get versions of your recipes by default, but that's for reverting bad changes and doesn't fit the requirement here. Workato appears to offer environments as an add-on to their basic product, which seems like a much better fit to this deployment process issue. Now I couldn't find this in the demo I was looking at, but the documentation suggests it should be available. The docs also suggest it has a fixed "dev, test, production" flow, and while that's probably fine for many, I have worked with some clients who would feel resrticted by that. But we'll have to wait and see how this works in Sitecore's version.
Pricing is a big question here too. Sitecore have suggested that there would be a usage-based pricing model, where you pay based on how many I/O operations your recipe flows perform. Loading records or writing data would be charged here, and I suspect that triggers which rely on external events might be part of that too. (If you have to ask SalesForce whether there's a new record, then that's I/O?) But there's not been any info on how much each operation would cost, or whether there might be some "free" tier for test and evaluation here.
This also raises some interesting ideas around how best to write recipies to manage costs. If fees are per-I/O-operation then a batch-processed model for flows seems like it would be cheaper. If you can read 50 records in one go, that's better than fifty individual reads from a costing perspective. So there will be some interesting architectural conversations to have around how to best build these integrations for cost efficiency.
Workato's demo includes a good chunk of training videos and exercises. Having gone through a few of these, they're a useful explanation of the technology, its terminology:
And training is another of the interesting unknowns - how Sitecore will handle this? Will they aquire and import this content to their learning platform? Write their own content? Or provide a way for us to get at the original? This will be an important point for agencies delivering projects, and clients looking to buy into this tech. Both will need to be able to get up to speed on the product efficiently.
Having spent some time playing with this, it seems like it's a good choice for the Sitecore space. If the pricing model supports it, I can think of a number of projects I've worked on where this could have been useful tooling for things like to mapping product data for enrichment in the CMS, or sending customer data to a CRM.
While it's probably going to be developers who do the initial work of integrating systems like this, it seems like its within the scope of more technical business users to maintain and extend too. The approach to the UI seems to be at a level that non-developers can understand, which will likely make adoption easier. This is orders of magnitude better than the UI for some workflow tooling I've looked at in the past...
So I'm enthusiastic to see how Sitecore take this technology on-board, and look forward to an oportunity to make use of it in a project...
↑ Back to top