Implement Forms [Meta issue]

[pavish]

This is a meta-issue tracking the implementation of Forms: Product spec

This diverges from the product spec in several small areas based on design changes & discussions. Please make sure to follow the specification in this issue, and the refer to the product spec as a high level requirements document.

Please note that while the following form structure accounts for most cases, if the implementer hits any snags or sees a faster and/or better way, they should feel free to propose/make changes and/or discuss it with @pavish when necessary.

Form structure

Typescript types to describe the form structure

interface FormDefinition {
  id: number;
  // represents the url key & for obfuscation
  token: uuid; // uuid v4
  database_id: number;
  base_table_oid: Oid;
  // Could be obtained from base_table_oid, or persisted in DB
  schema_oid: Oid;
  version: number;

  // Form info
  // name should be unique per schema
  // if schema_oid is persisted, this will be a unique constraint on the db,
  // or it'll be a service layer validation
  name: string;
  description: string | null;

  // Ownership - YET TO BE CONFIRMED

  /* 
   * Each form is owned by a Mathesar user
   * Only the owner of the form or a Mathesar admin can edit the form definition
   * Other Mathesar users can view & submit data using the form but cannot edit the definitions
   * The owner information is not provided by the frontend during add/replace
   * The value for owner is identified by the backend i.e. using the current authenticated user
   * Important: However, when a Mathesar admin _replaces_ an existing form, the owner is not modified
   */
  owner: MathesarUser['id']

  // Display
  header: FormHeader;

  // Fields
  fields: FormField[];

  // Submission & sharing behaviour
  submission: SubmissionSettings;
  public_sharing: PublicShareSettings;

  // Audit
  created_at: string; // datetime string
  updated_at: string; // datetime string
}

Display

// Frontend needs full control over this
interface FormHeader {
  title: { text: string } & RichContentJson;
  subtitle?: { text: string } & RichContentJson;
}

Submission

interface SubmissionSettings {
  /*
   * When a form gets saved (added/replaced), the configured_role of the owner is
   * identified & persisted on the backend as the `submission_role`.
   * 
   * This can also be provided explicitly by the user/frontend.
   * When provided by the user, the backend should validate whether the Mathesar user has access to the submission_role.
   *   - The access is either explicitly provided via the Collaborator settings,
   *   - or the user's configured role is a member of the submission role,
   *   - or the user saving the form is a Mathesar admin (admins have access to all roles).
   */
  submission_role: ConfiguredRole['id'];

  message: { text: string } & RichContentJson;
  redirect_url: string | null;
  submit_label?: string;

  // List of callbacks - future consideration
  on_submit?: SubmissionAction[];
}

// Future consideration
interface SubmissionAction {
  action: string;                  // e.g. 'send_email'
  options?: Record<string, any>;   // action-specific
}

Public sharing

interface PublicShareSettings {
  is_enabled: boolean;

  // Future consideration
  slug: string;
  password?: string; // write-only on API
}

Fields

type FormField =
  | ScalarColumnField
  | ForeignKeyField
  | ReverseForeignKeyField;

interface BaseField {
  id: number;
  key: string;                      // API-friendly key
  kind: FieldKind;                 // discriminator
  /*
   * Column in the base_table.
   *  - For ScalarColumnField, it's the column that's attached to the field.
   *  - For ForeignKeyField, it's the column that's the foreign key.
   *  - For ReverseForeignKeyField, it's the column that the linked table references.
   */
  column_oid: Oid;
  label?: string;
  help?: string;
  placeholder?: string;
  readonly?: boolean;              // can only be true if there's a DB default
  styling?: Record<string, unknown>;
  order: integer; // inorder to order the fields in requests/responses & presenting visually
}

type FieldKind = 'scalar_column' | 'foreign_key' | 'reverse_foreign_key';

Scalar field

interface ScalarColumnField extends BaseField {
  kind: 'scalar_column';
  validation?: BasicValidation;
}

Foreign key field

interface ForeignKeyBase extends BaseField {
  kind: 'foreign_key';
  // The target table oid is retrieved during get. It is not sent from the frontend (it could be sent, if it'll make work easier on the backend). We do not have to persist this.
  target_table_oid: Oid;

  // TODO: Update this
  // The structure for lookup columns & record summary is yet to be confirmed as of 2025-05-27 (based on design iterations)
  lookup_columns?: LookupColumn[];
  record_summary_template?: RecordSummaryTemplate;

  validation?: BasicValidation;
}

// when allow_create === true, additional props are required
interface ForeignKeyCreateAllowed extends ForeignKeyBase {
  allow_create: true;
  create_label: string;
  // The immediate level of fields in `nested_fields` should only be columns of `target_table_oid`.
  nested_fields: FormField[];
}

// when allow_create === true, additional props should not be present
interface ForeignKeyNoCreate extends ForeignKeyBase {
  allow_create: false;
  create_label?: never;
  nested_fields?: never;
}

type ForeignKeyField = ForeignKeyCreateAllowed | ForeignKeyNoCreate;

Reverse Foreign key field

interface ReverseForeignKeyField extends BaseField {
  kind: 'reverse_foreign_key';
  linked_table_oid: Oid;

  // Always allow creating multiple child rows
  // The immediate level of fields in `nested_fields` should only be columns of `linked_table_oid`.
  nested_fields: FormField[];

  validation?: ReverseFkValidation;
}

Form level field validation (at the frontend & service layer, not the DB layer)

export interface BasicValidation {
  is_required?: boolean;
}

// Future consideration - more validations on how many entries user can add
export interface ReverseFkValidation extends BasicValidation {
  min_number_of_entries?: number;
  max_number_of_entries?: number;
}

Sample form structure

{
  "id": 1,
  "token": "837869fb-c815-47f2-8fa0-7c780672b228",
  "base_table_oid": 42,
  "schema_oid": 1, // Could be obtained from base_table_oid, or persisted
  "version": 1, // For frontend to keep track of changes across releases for objects it controls

  "name": "Movie addition form", // Should be unique per schema, if schema_oid is persisted, this will be a unique condition on the db, or it'll be a service layer validation
  "description": "string",

  // Ownership logic - Yet to be confirmed
  "owner": 2,

  "submission": {
    "submission_role": 112,
    "message": { // Message to show after submission, rich text, frontend needs ability to iterate on this rapidly
      "text": "Thank you!"
    },
    "redirect_url": "http://some-random-url", // To redirect UI after submission, default is None
    "submit_label": "Send",
    "on_submit": [ // Could be a separate table, Not part of the first feature release
      {
        "action": "send_email",
        "options": {
          "email_to": // some email
        }
      }
    ]
  },

  "public_sharing": {
    "is_enabled": false
  },

  "header": {
    "title": { // json with rich text, <required>, frontend needs ability to iterate on this rapidly
      "text": "Add a new movie to your collection!"
    },
    "subtitle": {
      "text": "Some long description" // Optional, Rich text
    }
  },

  "fields": [
    {
      "id": 1,
      "key": "fld_01", // id is text here & customizable, so that users can fill the form via API requests, we can also autogenerate docs for forms in the future
      "kind": "scalar_column",
      "column_oid": 7,

      "label": "Name", // Not mandatory
      "help": "", // Not mandatory
      "placeholder": "some movie name", // Placeholder for input, not mandatory
      "validation": { // Additional validation on the form apart from DB validation
        "is_required": true
      },
      "readonly": false, // Can only be true for columns that have default values
      "styling": {}, // frontend needs to be able to iterate on this rapidly
    },
    {
      "id": 2,
      "key": "fld_02",
      "kind": "foreign_key",
      "column_oid": 9,
      "target_table_oid": 57,
  
      // TODO: Update this
      // The structure for lookup columns & record summary is yet to be confirmed as of 2025-05-27 (based on design iterations)
      "lookup_columns": [], // User can configure lookup columns for each form
      "record_summary_template": [], // User can configure custom record summary for each form

      // For foreign key rows
      "allow_create": true, // “+ Add new” button
      // Shows when allow_create is true
      "create_label": "Add a new producer", //Optional
      // Only one row can be created newly for foreign_key field kind
      "nested_fields": [ // Present if allow_create is true
        // ...
       // The immediate level of fields should only be columns of `target_table_oid`.
      ],

      "label": "Director",
      "help": "",
      "placeholder": "",
      "validation": { "is_required": true },
      "readonly": false,
      "styling": {},
    },
    {
      "id": 3,
      "key": "fld_03",
      "kind": "reverse_foreign_key",
      "column_oid": 17,
      "linked_table_oid": 58,

      // Always allow creating records for reverse_foreign_key field types
      // Multiple rows can be created newly for reverse_foreign_key field
      "nested_fields": [
        {
          "id": 10,
          "key": "fld_03_01",
          "kind": "scalar_column",
          "column_oid": 3,
          "label": "Character"
        },
        // ...
      ],

      "label": "Cast Members",
      "description": "",
      "validation": {
        "is_required": false, // Form can be submitted with `0` rows.
        // More options in the future
      },
      "styling": {},
    }
  ],

  "created_at": "2015-10-03T00:00:00.0 AD",
  "updated_at": "2015-10-03T00:00:00.0 AD",
}

Sub-issues are created incrementally

• Implementers can pick issues that are marked ready.
• More sub-issues will be created based on progress on the designs and implementation work.

[mathemancer]

id: number;

I think this should be a UUID, that would allow us to defer the custom slug till a later iteration. For now, it'd just use the actual ID UUID for the link, and then the eventual logic would let a user navigate to either the custom-slug-link, or the UUID-link.

[mathemancer]

version: number;

I think this should be a string, so the user can make it something semantically meaningful if they want.

[pavish]

@mathemancer

id: number;
I think this should be a UUID, that would allow us to defer the custom slug till a later iteration. For now, it'd just use the actual ID UUID for the link, and then the eventual logic would let a user navigate to either the custom-slug-link, or the UUID-link.

Sounds good to me. I've updated the structure above.

version: number;
I think this should be a string, so the user can make it something semantically meaningful if they want.

This version is not related to the user (as in it's not for tracking their form versions). Versioning is not planned for forms for the first iteration.

This version is for future-proofing the frontend for reconciling any item stored as a json on the database, that only the frontend uses and has control over. This might of-course not be necessary if we don't store anything as json on the backend.