Register custom turbo-stream plugin for encoding/decoding additional types in single-fetch data

[robmonie]

Note: this has been moved from the Remix discussion

I'd like to propose an api for registering custom turbo-stream plugins for use in the encoding / decoding of loader-data. A specific use-case is to be able to send and receive Luxon DateTime or BigNumber instances which we've standardised on within our apps. Currently we do this via wrappers around the existing useDataLoader and friends but the move to turbo-stream makes this feel a bit less natural and continuing as we currently do would mean missing some of the router improvements.

An example of registering a plugin for encoding/decoding a Luxon DateTime.

registerTurboStreamPlugin({ 
  encoder: (value) => {
    if (value instanceof DateTime) {
      return ['DateTime"' value.toISODate()];
    }
  },
  decoder: (type, encodedValue) => {
    if (type === 'DateTime') {
      return { value: DateTime.fromIso(encodedValue); };
    }
  }
});

[glasody]

This is probably the only thing keeping us from migrating Remix to RR7, we also use Luxon DateTime and not being able to deserialize loader data to the proper types makes RR7 un-usable.

[robmonie]

In case it's helpful, we've been able to move forward using our previous approach by manually converting our models containing Luxon and BigNumber instances to JSON objects in loaders and then using hooks to deserialize back.

For example in a loader:

return toJson<LoaderData>({ someObjectWithDateTime });

the toJson implementation generically converts DateTime instances into ISO strings. This could be as simple as JSON.parse(JSON.stringify(object)); or a custom implementation that may perform better. I'm about to look at that now that i've proved the approach works more generally.

Next, in a route we use a hook that wraps useLoaderData and parses the json objects with our zod schemas. This approach could be adapted to different type-safe deserialization implementations.

export const useTypedLoaderData = <T extends z.ZodTypeAny>(schema: T): z.infer<T> => {
  const loaderData = useLoaderData();

  return useMemo(() => schema.parse(loaderData), [loaderData, schema]);
};

export default function Route() {
  const { someObjectWithDateTime } = useTypedLoaderData(loaderDataSchema);
}

This ends up looking very similar to what we had prior to RRV7. Unfortunately it means we miss out on full use of the route types, in particular the typed loaderData in route props which now appears as Json, but it allowed us to upgrade with the hope that we can get more flexibility with turbo-stream at some point.

Of course you could skip the useTypedLoaderData hook and do the following instead. We've just kept a pattern that already existed.

export default function Route({ loaderData }: Route.ComponentProps) {
  const { someObjectWithDateTime } = useMemo(() => schema.parse(loaderData), [loaderData, schema]);
}

We also have implementations such as useTypedLoaderDataForRoute and useTypedActionFetcher.

[glasody]

have you thought about using the superjson package just like how the remix-typedjson package was?

[robmonie]

We used superjson in one project and moved to zod schemas in another as we were already using them extensively elsewhere. I don't have a strong opinion one way or the other as to which is better. Superjson was probably a bit simpler to use in this context.

[lime517]

This would be massively helpful for Prisma users, as decimals as handled as Decimal.js objects and are not converted into anything immediately useful from loaders/actions. It's a huge headache.

[zwhitchcox]

@brookslybrand Would a PR be welcome on this? This has caused our team a lot of pain, and exposing the plugin functionality would be fairly easy to implement. Not sure if you guys had a reason for not exposing it in the first place though.

[jeyj0]

I'm quite certain turbo-stream will be replaced by RSC. Thus, I don't think any PR for this would be accepted, as it would soon be outdated again.

[brookslybrand]

@jeyj0 is correct, turbo-stream was always meant as a stepping stone to get us to the RSC transport layer.

Since there's a decent number of upvotes I added the "review" label to make sure I confirm this with others on the team

[zwhitchcox]

Thanks a lot! Just for context, we had problems with Prisma.Decimal and BigInt these things are easy to handle on the back end, but once we send it to the server, we have to have some type of custom helper like useDeserializedLoaderData and implement our own serializer/deserializer for us. I can't tell you all the specifics, but I have heard each member of our team spend multiple days on this for various reasons trying to come up with an optimal solution, but just exposing that plugin functionality would make it all go away, and we wouldn't have to think about it.

I understand not wanting to be dependent on turbo stream, but unless you want to serialize/deserialize every possible object, it might be good to have a plugin system for the final version too, and maybe you could prefix it with unstable_ to make sure people know it's not permanent, and you could gather feedback in the meantime.

Thanks for your response!

[jeyj0]

I'll also say that our project could also benefit from customizable serialization/serialization though.

RSC doesn't seem to want to offer this out of the box, but maybe there's something that RR could add on top? (I haven't looked into the implementation details of the RSC transport layer, so I don't know the feasibility of this)

For context, our use-case is a custom Result type, implemented as an Ok and Err class. The types mentioned above might also become relevant for us though.