Fun gotchas with Sitecore GraphQL and .http files

I was having a chat recently about alternatives to Postman if you needed to send HTTP requests to arbitrary web endpoints. I mentioned using Visual Studio's support for .http files for this during that discussion, and then found myself trying it out for some work too. But it seems there's a couple of tricky little bugs hiding in here, which tripped me up when I tried to set up a call to one of Sitecore's XM Cloud GraphQL endpoints.

Background

The theory here is pretty simple. If you create a text file with a .http extension in Visual Studio it gets some magic extra behaviour over a regular text file. You can use a predefined syntax to write out HTTP requests in the file, and when it sees this Visual Studio adds in an adorner above your URI for a "Send request" button so you can make this call and see the response. It's kind of like using something like Postman, but a bit more barebones and simple.

In the left pane you can specify the HTTP Verb, the URL and any headers or data. And when a URL is recognised the "Send request" adorner appears and you can click that to get results data in the right pane. You can do fancier stuff like have multiple requests in one file, click "debug" to attach the debugger to the current project in order to test what it does for the request, or use variables and environment settings.

There's a whole documentation page to explain the details of this.

Setting up my query

So I wanted to try a Sitecore GraphQL query via this route. In theory that's fairly simple:

• You need to know the URL of the GraphQL service to query
• You need to know the security key to send with the request
• And you need a GraphQL query to send

So I tried a first pass: There's a local development endpoint at https://xmcloudcm.localhost/sitecore/api/graph/edge, in a vanilla XM Cloud project. The key is sitting in the content tree as the Item ID of /sitecore/system/Settings/Services/API Keys/xmcloudpreview, and the docs have some example queries to try. So I stuck this into my .http file:

POST https://xmcloudcm.localhost/sitecore/api/graph/edge
sc_apikey: {09DA42A1-E647-4192-8D4D-1F2102DECEEB}

query { 
  layout(language: "en", routePath: "/", site: "site1") { 
    item { 
      rendered 
    } 
  } 
  site { 
    siteInfoCollection { 
      name 
      routes(first: 10, language: "en") { 
        results { 
          routePath 
          route { 
            id 
          } 
        } 
      } 
    } 
  } 
}

					

But alas it's not that simple. The response for that came back with this blob of json:

{
  "errors": [
    {
      "message": "Unable to bind request; unexpected Content-Type. Expected content types for a POST are either application/json or application/graphql. See http://graphql.org/learn/serving-over-http/   at Sitecore.Services.GraphQL.Hosting.Transports.Http.GraphQLHttpRequestParser.<ParseRequest>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Sitecore.Services.GraphQL.Hosting.Transports.HttpRequestHandler.<ProcessRequestAsync>d__4.MoveNext()"
    }
  ]
}

					

But like all good programmers should do, the devs here put the answer in the error message: My example is missing the right content type header! It wants us to specify if we're sending json or GraphQL so it knows how to parse the request. Now the answer here for my request is GraphQL, but it's worth noting that if you do send json as the content type with the style of query above you get a different error:

{
  "errors": [
    {
      "message": "GraphQL query parsing failed. Ensure the query is sent as a valid JSON object with 'query' and optionally 'operationName', 'variables', and 'extensions' elements. See http://graphql.org/learn/serving-over-http/   at Sitecore.Services.GraphQL.Hosting.Transports.Http.GraphQLHttpRequestParser.ProcessJsonRequest(String content, IHttpRequest context)\r\n   at Sitecore.Services.GraphQL.Hosting.Transports.Http.GraphQLHttpRequestParser.<ParseRequest>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Sitecore.Services.GraphQL.Hosting.Transports.HttpRequestHandler.<ProcessRequestAsync>d__4.MoveNext()"
    }
  ]
}

					

That's a bit messier - but it's basically saying "your query needs to be wrapped up in a proper json structure to use this approach". I'll not delve into it too much here, but the right structure for json looks more like:

{
  "query": "...",
  "operationName": "...",
  "variables": { "var": "val", ... }
}

					

But my original query can be modified to pass the GraphQL content type easily:

POST https://xmcloudcm.localhost/sitecore/api/graph/edge
sc_apikey: {09DA42A1-E647-4192-8D4D-1F2102DECEEB}
Content-Type: application/graphql

query { 
  layout(language: "en", routePath: "/", site: "site1") { 
    item { 
      rendered 
    } 
  } 
  site { 
    siteInfoCollection { 
      name 
      routes(first: 10, language: "en") { 
        results { 
          routePath 
          route { 
            id 
          } 
        } 
      } 
    } 
  } 
}

					

And that returns valid data:

{
  "data": {
    "layout": null,
    "site": {
      "siteInfoCollection": [
        {
          "name": "website",
          "routes": {
            "results": [
              {
                "routePath": "/",
                "route": {
                  "id": "110D559FDEA542EA9C1C8A5DF7E70EF9"
                }
              }
            ]
          }
        }
      ]
    }
  }
}

					

Another note here is that if you've done low-level HTTP stuff before you might be expecting to need a content length header into this request because you're sending a body. And you're right - one does need to be sent. But helpfully Visual Studio magically does that for us. You can see this in the request details available in the right column of the UI:

Note that you can manually add that header and Visual Studio will use your value, but if you get the number wrong you're going to see odd errors. If your length is too short you'll see something like:

Unable to write content to request stream; content would exceed Content-Length.

					

And if you give too big a length then:

Sent 339 request content bytes, but Content-Length promised 450.

					

So probably best to leave that to Visual Studio unless you have very specific needs.

It also magically handles adding an HTTP version to your request as well, defaulting to HTTP/1.1 - but you can override that too by adding your version after the URL if required.

The main problems I hit...

(NB: I'm using VS 17.10.6 - hopefully this will be resolved in newer versions)

So it's all grand that you can send requests like this for testing. But then I got my developer head on and started trying to neaten stuff up with variables for multiple queries in a single .http file. That seemed like a common scenario for an XM Cloud project - you'd likely have a set of GraphQL queries to test or prototype for your UI components - and having them all in one place could make life easier than having to find each rendering and use the GQL Playground separately. I figured the domain to call, the overall URL and the security key would be worth abstracting out if I was writing multiple queries. So I refactored my example above to:

@server = xmcloudcm.localhost
@previewapi = https://xmcloudcm.localhost/sitecore/api/graph/edge
@apikey = {09DA42A1-E647-4192-8D4D-1F2102DECEEB}

POST {{previewapi}}
sc_apikey: {{apikey}}
Content-Type: application/graphql

query { 
  layout(language: "en", routePath: "/", site: "site1") { 
    item { 
      rendered 
    } 
  } 
  site { 
    siteInfoCollection { 
      name 
      routes(first: 10, language: "en") { 
        results { 
          routePath 
          route { 
            id 
          } 
        } 
      } 
    } 
  } 
}

					

Variables in these files are defined using @name and assigned a string value. And then you can use those variables in your request with a Mustache template inspired {{name}} string. (And the UI colours variable declarations light blue, and variable references in red to make them obvious)

But this did not work. The "Send request" adorner disappeared from my request, and the @apikey variable didn't get coloured in like the others:

After some head-scratching I realised that the parsing in these files is not very clever. If you paste in GUIDs with curly braces, parsing gets confused. And removing the braces from above fixed the parsing again. Luckily server-side GUID parsing doesn't care if the braces are present or not:

But after fixing this, sending the request still did not work. It would run and return an error:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

					

And an interesting error at that - this looks more like Traefik or maybe something low-level in IIS having issues rather than the GraphQL API rejecting the query. What could that mean?

But we're looking at a tool for debugging HTTP requests - so lets use it! Changing to the "Request" tab of the right hand pane in Visual Studio gives an interesting clue:

Look at the URL it requested - the API Key header has ended up appended on to the end. That's clearly a bad request, but why?

Well looking back at the original query we can see that the POST request and that header are on adjacent lines with no text in between other than the newline's whitespace:

POST {{previewapi}}
sc_apikey: {{apikey}}
Content-Type: application/graphql

					

After a bit of messing about I realised that this is actually another issue caused by how .http files in Visual Studio parse their variables. If I add any valid URL character after the end of {{previewapi}} then the query works. So a trailing space, or a trailing / both fix this problem and get back the same results we saw for this query originally.

Clearly parsing the variable is messing up the CR/LF/whatever at the end of this line when it makes the HTTP request - causing the API Key header to get smooshed onto the URI. But if there's a character after the closing }} the parsing doesn't break.

Related to this, when I was testing adding a specific HTTP version to a request I found that POST {{previewapi}} HTTP/2 did not work either - despite having something coming after the URL. In this case I found you needed to add a trailing slash to make the request parse properly: POST {{previewapi}}/ HTTP/2.

So be warned people - this is a useful tool for making requests, but it can trip over the unwary at present...

Note: Since writing the draft of this post I've had a chance to test with the latest preview version of Visual Studio - that's v17.12.0 Preview 1.0 as I type. It appears that this release fixes the issues with "needing whitespace or `/` characters after the `{{previewapi}}` variable usage" issue above. But it has not fixed the issue with curly braces in variable declarations. So we're getting there, but it's not sorted yet...

↑ Back to top