Serializing Symfony Forms into JSON schema

TLDR; The Symfony Form Component is an exceptional piece of code, especially when used to generate HTML forms. However, when using it in an API, we lose many of its benefits. We have published LiformBundle/Liform to serialize forms into JSON schema, to get a representation of what the API expects, which can be used for documentation and validation. We have also published liform-react, a React library to generate React.js forms from a JSON schema representation in the client side, which is covered in a separate post.

Although one may think that it is only “building forms”, in fact it is doing a variety of tasks. This is a (probably non exhaustive) diagram of things that it does:

This is quite a lot of useful stuff. However, when using Forms in APIs, for instance to process requests to create or update resources, we don’t have as many benefits.

And since we are not using HTML but some rich frontent in JS or mobile, it is very likely that we want to take advantage of it with, for instance, the three kinds of form validation, like validation on blur so the user can be immediately alerted of invalid values when leaving the field, and other dynamic stuff, perhaps like suggesting alternatives in a purchase form.

We definitely lose the ability to give UI hints, as we are just presenting a POST endpoint, and we lose everything related to FormView, that is direclty related to HTML. We can, with FOSRestBundle, serialize errors into JSON, although we lose of course the integration with the view. Also, we can serialize our models, but in practice there are some edge cases where this doesn’t work.

This makes working with APIs more difficult than working with HTML forms, because, for instance, if we have a form with Choices (with value and a visual representation of the choice, as in 'gb' => 'United Kingdom'), we need to maintain a documentation in the API reflecting that these are the values that the form accepts.

Basically, the problem is that now we have to maintain a Symfony Form, a form in the client side, and a documentation that frontend and backend agrees on.

In the case of a single form, that may be ok. It is some extra overhead, but we can live with it. However, if you have to write, let’s say, a “wizard” form with 50 steps, or admin panels, the overhead adds up.

So, we were thinking, can we have something similar to $form->createView() but for APIs? It will involve serialization. But, to what format? I think that JSON schema is an excellent format for this.

We can inspect recursively the form and build an array that captures the information. It turns out that Matthias Noback had written a nifty library to transform Symfony Forms into Console forms. So why not doing the same?

So we wrote the library Liform and its companion LiformBundle that integrates the lib with Symfony.

It pretty much follows the same approach: there is a resolver that finds out the right transformer to apply based on the Form Type, and then transformers convert the types into slices of the JSON schema representation.

$task = new Task();
$form = $this->createForm(TaskType::Class, $task,
        array('csrf_protection' => false)
        );
$this->get('liform')->transform($form);

So if we have this Form Type:

public function buildForm(FormBuilderInterface $builder, array $options)
{
    $builder
        ->add('name', Type\TextType::class, [
            'label' => 'Name',
            'required' => true,
            'attr' => ['placeholder' => 'I\'m a placeholder’]])

        ->add('description', Type\TextType::class, [
            'label' => 'Description',
            'liform' => [
                'widget' => 'textarea',
                'description' => 'An explanation of the task',
            ]])

        ->add('dueTo', Type\DateTimeType::class, [
            'label' => 'Due to',
            'widget' => 'single_text']
        )
    ;
}

(Note that LiformBundle registers a form extension so forms accept a liform option that can be used to give more information.

It produces this JSON schema representation:

{"title":"task",
   "type":"object",
   "properties":{
      "name":{
         "type":"string",
         "title":"Name",
         "default":"I'm a placeholder",
         "propertyOrder":1
      },
      "description":{
         "type":"string",
         "widget":"textarea",
         "title":"Description",
         "description":"An explanation of the task",
         "propertyOrder":2
      },
      "dueTo":{
         "type":"string",
         "title":"Due to",
         "widget":"datetime",
         "format":"date-time",
         "propertyOrder":3
      }
   },
   “required":[ “name", "description","dueTo"]}

With this technique we can extract a lot of information, types, formasts, UI clues, and also validation rules (it is a work in progress to serialize as many validation rules as possible, and if you want to collaborate you are more than welcome). Note that we add information about which widgets to use. These options are not part of the JSON schema standard, but it is possible to add additional options to it, and it is common practice to do so.

It is also possible to add transformers for existing or new types, or even extensions to provide an extra pass to every transform. This way you can write your own specific FormType and serialize it into an accurate representation.

This is quite useful, since now we have a documentation of what our endpoint expects that is kept in sync with our code. Also, it allows us to use a variety of tools that work with JSON schema, such as client side validators, like the great ajv library.

It also provides two serializers, one form Forms with errors, to extract the errors (taken from FOSRestBundle, thanks and kudos), And a serializer for FormView that serializes initial values. There are code samples of this in the Symfony React Sandbox that we maintain.

So, with this, we have this:

Can we do more? Yes of course we can! Because apart from using JSON schema for validation we have all the information we need to generate forms in the client side. Now, this depends on the platform you use in your frontend, but you have, for instance, json-editor in vanilla Javascript. In our case, we love React. There are some form generators in React, with Mozilla’s React react-jsonschema-form being perhaps the most popular.

However, our needs were quite specific in terms of flexibility of the generator (we were building a quite big Wizard form with very custom widgets), and moreover we wanted to use the great redux-form to keep our state in Redux in a sane way. So we wrote another generator, liform-react that allows you to customize the widgets or write themes. In fact the default Bootstrap theme is not what we use, so this is important for us.

react-liform is covered in a different post, since it can be used by other React devs that I doubt would be so intersted in the details of serializing Symfony Forms, so if you are interested, keep reading. But, details apart, with a form generator in the mix, we have all we wanted:

And that is it. I would be happy to hear your thoughts, since this is something that we use in a project of ours, but it took some time to cover the features that we don’t use, such as a theme in react-liform or transformers for types that we don’t use, but are good to have in a public and more general library. This has saved us quite a lot of time when working with forms in the context of an API + a rich frontend.

There is an example of usage of all the components in the Symfony React Sandbox if you want to check them out.

Take care out there!