Implement Django db models for forms, and form management endpoints

[pavish]

Description

• This issue tracks implementing:
  • The Django DB models following the form structure from the Forms parent issue.
  • Form management RPC methods needed for creating, listing, retrieving, and deleting a form.
• Implementers can feel free to split this issue into multiple sub-issues.

Out of scope of this issue

• Public sharing and related RPC methods/models/logic.
• Form submission RPC method.
• Lookup columns & record summary related logic.
• Ownership.

Details

1. Forms Django DB models and necessary migrations.

• The decision to create separate models for Form and FormField is left to the implementer.

2. Form management RPC methods:

forms.add

• Adds a form.
• All logged-in users can perform this request.
• Request should contain the FormDefinition.
• The following fields in FormDefinition should be filled in at the backend service layer:
  • created_at
  • updated_at
  • target_table_oid within ForeignKeyField (it could be sent from the frontend, if it'll make work easier on the backend).
  • submission.submission_role - Use if provided by the frontend. If not provided, infer from owner current logged-in user.
  • owner - Use if provided by the frontend. If not provided, infer from current logged-in user.
• Performs the following validation before saving:
  • When submission.submission_role is provided by the user, the backend should validate whether the logged in user has access to the submission_role.
    • The access is either explicitly provided via the Collaborator settings,
    • or the user's configured role is a member of the submission role,
    • or the current logged-in user is a Mathesar admin.
  • When owner is provided by the user, the backend should ensure that:
    • the owner id is the same as that of the current logged-in user, or
    • the current logged-in user is a Mathesar admin (who could set the owner to anyone).
  • All columns specified in nested_fields should belong to the table with the oid:
    • target_table_oid (in case of foreign key parent fields),
    • or,linked_table_oid (in case of reverse foreign key parent fields).

forms.get

• Retrieves a saved form.
• All logged-in users can perform this request.
• Request should contain the form id.
• Provides an additional field_source_analysis_map. This is a map of all field ids with their respective column definitions. This might extend to provide additional information in the future based on frontend needs.
• Response structure:

  interface FormDefinitionGetResponse {
    form_defintion: FormDefinition,
    field_source_analysis_map: Record<FormField['id'], FieldSource>
  }
  
  type FieldSource = ValidFieldSource | ErrorWhileFetchingFieldSource;
  
  interface ValidFieldSource {
    // The column to fetch should be identified by FormField['column_oid']
    // The value is same as column information present in columns.list_with_metadata
    column: ColumnInfoWithMetadata,
    error: never
  }
  
  interface ErrorWhileFetchingFieldSource {
    column: null,
    error: ErrorCodeAndMessage
  }

• Sample JSON:

  {
    form_definition: {
      id: 1,
      base_table_oid: 12,
      name: 'New form'
      // ...,
    },
    field_source_analysis_map: {
      field_01: {
        column: {
          id: 1,
          name: 'Some column name',
          metadata: {
            // ...
          },
          // ...
        }
      },
      field_02: {
        column: null,
        error: {
          code: -12345,
          message: 'Column is not present'
        }
      }
    }
  }

• If a column is missing or cannot be fetched, the RPC method should not throw an error. It should:
  • return the column as null in field_source_analysis_map[FormField['column_oid']].column, and
  • provide an error in field_source_analysis_map[FormField['column_oid']].error.
• Note for implementers:
  • The implementer can split this method into two RPC methods, if they see fit (i.e., one for form information, one for the field_source_analysis_map).
  • However, it should be noted that the frontend should be able to call them both in a single batched request. One request should not depend on the other.
  • The request for field_source_analysis_map should only require the form_id.

forms.get_public

• Fetches the form definition that should be accessible publicly.
• The RPC method must be unauthenticated. Anyone can perform this request - whether logged-in or not.
• Request should contain the form id.
• Same response as forms.get minus the following fields in the form definition:
  • owner
  • submission['submission_role] // other data in submission should be sent in response
  • submission['on_submit']
  • public_sharing
• Note: We will be updating this method to only provide a response if the form is shared publicly. This is out of scope in this issue.
• If forms.get is split into two methods for the form definition & the field_source_analysis_map, same should be done for this method. In that scenario, the method for field_source_analysis_map should be unauthenticated.

forms.replace

• Replaces a form with the new content.
• Only the owner of a form or a Mathesar admin can perform this request to replace the form.
• (using the term replace instead of patch, because each user save updates the entire form definition).
• The input would be the same one provided to forms.add with an additional form id, id: FormDefinition['id'].
• All validations done as part of forms.add should be done in forms.replace.

forms.list

• Retrieves a list of saved forms - no pagination required as of now.
• All logged-in users can perform this request.

forms.delete

• Deletes a form
• Only the owner or a Mathesar admin can perform this request to delete the form.

[mathemancer]

@pavish I thought we were skipping the owner thing for now. Am I misremembering?

[mathemancer]

@Anish9901 Please submit a PR with just the models, i.e., don't try to get all the functions before getting the models merged.

[pavish]

I thought we were skipping the owner thing for now. Am I misremembering?

@mathemancer Yes, that was the plan before our call. Since we came to an agreement on how to handle permissions, I decided to update the issue to account for it.

[mathemancer]

I thought we were skipping the owner thing for now. Am I misremembering?

@mathemancer Yes, that was the plan before our call. Since we came to an agreement on how to handle permissions, I decided to update the issue to account for it.

Now I'm really confused. I thought that since we don't yet have an "owner" concept for any Mathesar-level object, we were going to wait to do it for forms. So, anyone with a login could try to update the form, but they wouldn't be able to save it without configuring the form to run with a role to which they have access. I'd like to avoid adding an ad-hoc ownership concept into Mathesar just for one object type.

[pavish]

@mathemancer

I thought that since we don't yet have an "owner" concept for any Mathesar-level object, we were going to wait to do it for forms.

Perhaps we arrived at different conclusions on our call.

I'll remove the owner concept from this issue. Let's have another sync call, and then create an issue for it if we need one.