Translating UI for Coveo Search [by Jeremy Davis]

This week I've spent a bit of time trying to knock up a quick demo of how Coveo can cope with searching in languages other than English. However, it's just my luck that the language I have to demo in is not on the list of languages supported out-of-the-box: Welsh. Hence I've been investigating how to customise the UI so that things look right for my demo. Turns out there are a few things to think about:

NB: I should start out with an apology to any Welsh speakers reading this – all the "translation" has been done with Google Translate and I have no real understanding of your language. I'm confident that there are failures of grammar and spelling in what follows. And I suppose it's not beyond the realms of possibility that I've accidentally ended up with something rude, insulting or just plain hilarious here. Sorry – it's lack of understanding rather than malice. Feel free to mock / correct me in the comments...

Configuring the test data url copied!

For want of any real copy, I created some test data items about animals. Each Item has Display Name, some descriptive text (stolen from Wikipedia) and a couple of metadata fields about the animal's categories of use and size. Each of these has been created in English and then translated into Welsh:

To get the indexing correct for my purposes I needed to make some configuration changes to the way that Coveo treated the data at index time. Note that while it looks like you can make some of these changes via the Coveo admin UI, due to the way the integration with Sitecore works you must configure these in your Sitecore search configuration data. Otherwise every time you run an index rebuild, the configuration you set via the Coveo UI is overwritten by that in the Sitecore config files. By default this configuration lives in the Coveo.SearchProvider.config file, but in a production solution you would probably create your own project-specific configuration patches.

The three changes I made were:

As discussed in a previous post, you must configure computed index fields that contain the appropriate display text for any metadata that is stored as GUIDs if you want to be able to create facets from them. Using the same tricks that I discussed previously, I created computed fields storing the display names of the referenced items using a variation on the out-of-the-box components:

<fields hint="raw:AddComputedIndexField">
  <!-- other computed fields -->
  <field fieldName="AnimalSizeFacet" sourceField="Size" referencedFieldName="DisplayName">CoveoDemo.ReferencedFieldComputedField, CoveoDemo</field>
  <field fieldName="AnimalCategoryFacet" sourceField="Category" referencedFieldName="DisplayName">CoveoDemo.ReferencedFieldComputedField, CoveoDemo</field>
</fields>

I wanted my descriptive text to be free-text searchable. To allow this, you have to set a specific indexing setting for that field. This is configured in the fieldMap section of the configuration:

<fieldMap type="Coveo.SearchProvider.CoveoFieldMap, Coveo.SearchProvider">
  <param desc="coveoReflectionFactory" type="Coveo.Framework.Utils.CoveoReflectionFactory, Coveo.Framework" />
  <fieldNames hint="raw:AddFieldByFieldName">
    <!-- other field mappings -->
    <fieldType fieldName="Description" includeForFreeTextSearch="true" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" />
  </fieldNames>
</fieldMap>

Finally, I wanted my "animal category" facet to be multi-valued as it used a UI control which allows multiple selections. Again this requires a setting in the Field Map data:

<fieldMap type="Coveo.SearchProvider.CoveoFieldMap, Coveo.SearchProvider">
  <param desc="coveoReflectionFactory" type="Coveo.Framework.Utils.CoveoReflectionFactory, Coveo.Framework" />
  <fieldNames hint="raw:AddFieldByFieldName">
    <!-- other field mappings -->
    <fieldType fieldName="AnimalCategoryFacet" isMultiValue="true" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" />
  </fieldNames>
</fieldMap>

Translating UI text url copied!

Usually when you're working in Sitecore, you translate labels and static text using Dictionary items stored as part of your website's content. The integration with Coveo does include a few dictionary entries, however, these appear to be for the administrative UI inside Sitecore rather than for the UI of the websites you publish. Since most of the UI behaviour is built using JavaScript, it's not surprising to find that the text resources for the UI live in JavaScript files. They can be found under your website root in the `Coveo\js\cultures` folder:

To add a new UI language you need to duplicate one or two of these files and translate / localise them in appropriately. First you need one of the "LanguageCode.js" files. These define the translatable strings and configure the appropriate locale information for dates and times for the core language. Then optionally you can also have one or more "globalize.culture.LangageCode-CultureCode.js" files which provide customisations based on the specific locales that the language is being used in – for example differences between UK and US English.

I created a Welsh language file as "cy.js" by copying the English language file. This allowed me to easily search the text for the English phrases I saw on screen, and translate them into Google's approximation of the correct Welsh text:

It's worth noting that for the purposes of my demo, I didn't need to translate all the strings here. Just the ones which were being displayed by the UI I was demonstrating.

Once you've adjusted all the settings you need, these files can be deployed simply by copying them to the appropriate location and forcing your web browser to re-load the page. The correct language files are chosen by the UI based on the language configured for the Sitecore Item of the current page. So as long as your language definition in Sitecore has the same code, the UI should use your new resource data automatically.

Allowing facet titles to work in another language url copied!

Once you've correctly configured your facets (again, see previous entry) they will mostly work in whatever language your page (and the related data) are published in. However one thing that is not immediately obvious is how to translate the titles of the individual facet controls:

The out-of-the-box UI controls are written to fetch their display labels from the properties of the facet:

And as I pointed out previously these are stored in the Layout XML in the Renderings field. Hence in Sitecore 7.2 (which I was using for this demo) there is no easy way to vary the by Item language. (This gets easier in Sitecore 8, with the adding of "final renderings" and the ability to vary the rendering data by Item Version – though this does make the editing experience a bit more prone to mistakes)

A bit of Googling found some posts on Coveo's forums which address this point. The suggestion is that you can create Items which contain the appropriate fields for a facet's configuration, and these can have language versions. You then need to modify the out-of-the-box code for the facet component so that it knows to fetch its data from the control's data source item rather than its rendering properties.

To adjust this, you need to create new item(s) based on the /sitecore/templates/CoveoModule/Search/Facet Parameters template. This contains all the same fields as the properties shown above, but note that the only fields of this item that will be used are those which you explicitly enable in the code changes to follow. Hence (for the purposes of my demo) I filled in only the title field in the appropriate languages. Then, for each of your facet controls, set its Data Source to point to the appropriate new Item.

The code for the facet UI lives in the website\Layouts\Coveo\CoveoFacet.ascx file. For the purposes of creating my demo site I opened this file and modified it directly. However, as pointed out by Jean-Francois in the comments, this isn't a good strategy if you ever plan to upgrade your version of Coveo. You should duplicate the file and create your own version in order to avoid having your changes wiped out by a future upgrade. Whichever way you go, you need to find the div element with the class "CoveoFacet". This will have an attribute called data-title. The value assigned to this attribute needs to be changed from the Model.Title it defaults to, to the appropriate field of the control's Data Source.

After a bit of fiddling about, I ended up testing this value to see if it was null/empty and falling back to the value in the Model.Title if so. This allows you to have facets with and without a data source on the same page. (I'm not sure if that's a real-world requirement or not – but for the purposes of my demo, I needed the UI to work in both scenarios)

So the code might end up looking something like:

<div class="CoveoFacet"
    data-title='<%= string.IsNullOrWhiteSpace(Model.BoundRendering.Item.GetFieldValue("FacetTitle")) ? Model.Title : Model.BoundRendering.Item.GetFieldValue("FacetTitle") %>'
    data-field='<%= Model.Field %>'
    data-number-of-values='<%= Model.NumberOfValues %>'
    data-id='<%= Model.UniqueId %>'
 -- other attributes removed for clarity --
    data-available-sorts='<%= String.Join(",", Model.AvailableSorts) %>'></div>

With this in place, you can now have translated facet titles:

I'm not sure about this "mix and match" approach to where the data is fetched from for configuring the facets. I think I will end up doing some rework here to try and come up with a way of storing this data that is less confusing to editors if I end up implementing this in a real project. Having the duplication of fields that occurs here is likely to lead to scenarios where editors are changing data in the "wrong" place and confused about why it's not being reflected in the UI. But that is a discussion for another day...

Getting suggested queries url copied!

Not really a language-specific issue here, but it did take me a bit of googling to work out how to get the system to display search suggestions as you type. There are some settings for controlling this on the properties of the "Coveo Search" component of the default search page:

But setting this didn't cause anything to be displayed as I typed. But reading documentation suggested that this can be enabled by adding the appropriate bit of mark-up to the CoveoSearch.ascx file. Inside the div with the ID "search" you can add something like:

<span class="CoveoTopFieldSuggestions"
    data-field="<%= Coveo.UI.SitecoreUtilities.ToCoveoFieldName("_displayname") %>"
    data-header-title="<%= Model.Labels["SuggestedQueries"] %>"
    data-query-override='<%= Coveo.UI.SitecoreUtilities.ToCoveoFieldName("templatename") %>="Animal"'></span>

The data-field attribute specifies which field from the search index should be displayed as the title for the selected items. A bit of trawling through the list of field names in my index found that _displayname is the field for the Item's Display Name property. Using this (rather than the item name) ensures they will be translated correctly.

The data-header-title property specifies the title text that should be applied to the set of suggested queries when it's displayed. Note that the example given in the documentation has a constant string here. Stupidly I spent ages trying to work out why that was the one string on the page which was not translating into Welsh. It was only writing up these notes that made me realise, that (obviously, in retrospect) you need to use the same code pattern as all the other localised fields to make sure translation happens. Hence fetching the value from the Model.Labels collection.

Finally, the data-query-override property allows you to add an extra clause to the search whose results are being displayed in the Omni-box. Here, I've added a query require only items based on my "animal" test data template are shown. This helps keep the results neat and focused. I'm wondering if I should also add a "return only results in the current language" query clause here as well – but for the moment I've decided to leave that out.

With those changes in place, you will get suggestions as you type: