🐛: JMAP references to previous method results are not working

[ralfbecker]

What happened?

I'm using the Staying-in-sync example from jmap.io together with push subscription to implemented instant updates to our mail UI (www.egroupware.org), as we already have that for Dovecot.

Unfortunately references in JMAP calls with multiple method-calls always return a reference error.

How can we reproduce the problem?

I receive the following change in the push subscription, when I change e.g. the seen flag of a mail:

{"@type":"StateChange","changed":{"s":{"Email":"sr9qk0l2jzhgkyay","Mailbox":"sr9qk0l2jzhgkyay"}}}

I know the previous states to be

{"Email":"srxqk071myhgkyay","Mailbox":"srxqk071myhgkyay"}

So I send the following JMAP call:

{
 "using":["urn:ietf:params:jmap:mail"],
 "methodCalls": [
  ["Mailbox/changes",{
    "accountId":"s",
    "sinceState":"srxqk071myhgkyay"
   },"0"],
  ["Mailbox/get",{
    "accountId":"s",
    "#ids":{"name":"Mailbox/changes","path":"/created","resultOf":"0"}
   },"1"], 
  ["Mailbox/get",{
    "accountId":"s",
    "#ids":{"name":"Mailbox/changes","path":"/updated","resultOf":"0"},
    "#properties":{"name":"Mailbox/changes","path":"/updatedProperties","resultOf":"0"}
   },"2"]
 ]
}

and receive this result:

{
  "methodResponses":[
    ["Mailbox/changes",{
     "accountId":"s",
     "oldState":"srxqk071myhgkyay",
     "newState":"sr9qk0l2jzhgkyay",
     "hasMoreChanges":false,
     "created":[],
     "updated":["a"],
     "destroyed":[],
     "updatedProperties":["totalEmails","unreadEmails","totalThreads","unreadThreads"]
    },"0"],
    ["error",{
      "type":"invalidResultReference",
      "description":"Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /created } result reference."
    },"1"],
    ["error",{
      "type":"invalidResultReference",
      "description":"Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /updated } result reference."
    },"2"]],
 "sessionState":"b898b3b6"
}

While the first response is ok: folder "a" (Inbox) has been updated due to seen flag change the counters changed.

The second result is already wrong, instead of an empty list for no newly created folder it returns a reference error. I could probably live with it, thought it's not according to the spec or the above linked example.

The third result is also wrong, thought the reference should succeed here, as folder "a" (Inbox) was updated.

As it follows so closely the example, I don't think somethings wrong with the JMAP commands I send.

Is that a know problem in Stalwart and is there a workaround or a fix to be expected soon?

Ralf

Version

v0.11.x

What database are you using?

RocksDB

What blob storage are you using?

RocksDB

Where is your directory located?

SQL

What operating system are you using?

Docker

Relevant log output

2025-05-15T17:42:47Z DEBUG JMAP method call (jmap.method-call) listenerId = "http", localPort = 8080, remoteIp = 172.18.0.4, remotePort = 51598, id = Mailbox/changes, accountId = 18, elapsed = 0ms
2025-05-15T17:42:47Z DEBUG Invalid JMAP result reference (jmap.invalid-result-reference) listenerId = "http", localPort = 8080, remoteIp = 172.18.0.4, remotePort = 51598, details = "Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /created } result reference."
2025-05-15T17:42:47Z DEBUG Invalid JMAP result reference (jmap.invalid-result-reference) listenerId = "http", localPort = 8080, remoteIp = 172.18.0.4, remotePort = 51598, details = "Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /updated } result reference."
2025-05-15T17:42:47Z TRACE HTTP response body (http.response-body) listenerId = "http", localPort = 8080, remoteIp = 172.18.0.4, remotePort = 51598, contents = "{"methodResponses":[["Mailbox/changes",{"accountId":"s","oldState":"srxqk071myhgkyay","newState":"sr9qk0l2jzhgkyay","hasMoreChanges":false,"created":[],"updated":["a"],"destroyed":[],"updatedProperties":["totalEmails","unreadEmails","totalThreads","unreadThreads"]},"0"],["error",{"type":"invalidResultReference","description":"Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /created } result reference."},"1"],["error",{"type":"invalidResultReference","description":"Failed to evaluate { resultOf: 0, name: Mailbox/changes, path: /updated } result reference."},"2"]],"sessionState":"b898b3b6"}", code = 200, size = 610

Code of Conduct

• I agree to follow this project's Code of Conduct

[ralfbecker]

To be able to continue with my implementation and not waiting for this to be fixed in Stalwart, I'm emulating multiple methodCalls now with a wrapper using single calls and resolving the references myself:

	/**
	 * Make a JMAP call - emulating multiple methodCalls with single calls and resolving references
	 *
	 * This fixes what seems to be a bug in Stalwart 0.11.x reference implementation
	 *
	 * @link https://github.com/stalwartlabs/mail-server/discussions/1508
	 * @param array $methodCalls [string $method, array $args][]
	 * @param string|array $using ='urn:ietf:params:jmap:mail'
	 * @return array response
	 */
	public function jmapCall(array $methodCalls, $using='urn:ietf:params:jmap:mail')
	{
		$responses = [];
		foreach($methodCalls as $methodCall)
		{
			foreach($methodCall[1] as $name => $value)
			{
				if ($name[0] === '#')
				{
					unset($methodCall[1][$name]);
					$name = substr($name, 1);
					if (count($reference = array_filter($responses, function($response) use ($value) {
						return $response[2] === $value['resultOf'];
					})) !== 1 || $reference[0][0] !== $value['name'] ||
						!isset($value['path']) || !is_array($reference[0][1] ?? null) ||
						($methodCall[1][$name] = self::jsonPath($reference[0][1], $value['path'])) === null)
					{
						$responses[] = ['error', [
							'type' => 'invalidResultReference',
							'description' => 'Failed to evaluate '.json_encode($value).' result reference.',
						], $methodCall[2]];
						continue 2;
					}
				}
			}
			$responses[] = ($response = $this->_jmapCall([$methodCall], $using))['methodResponses'][0];
		}
		return [
			'methodResponses' => $responses,
			'sessionState' => $response['sessionState'] ?? null,
		];
	}

Obviously performance and latency is not good, but I can continue with my implementation ...

Ralf

[ralfbecker]

@mdecimus I suggest to open that as bug report again ;)

Ralf

[mdecimus]

Thanks, I will test this once JMAP for Calendars is implemented. This is because the JSON pointer eval function will be reimplemented as part of a refactoring that is planned.

[mdecimus]

Hi @ralfbecker , this has been fixed. However, the version in main is under development and not yet ready for testing.

[ralfbecker]

Any rough idea when it will be released?

Thx :)

Ralf

[mdecimus]

Hopefully late next week, but if you want I can let you know once the refactoring is done so you can test if against eGroupware before the official release. I am doing the same with Mailtemi's developer.

[ralfbecker]

Sure, I'd like to :)

Ralf

[mdecimus]

Hi @ralfbecker - the version on the main branch is ready for testing if you like to give it a try before it is released in 2 days.