Seeking Guidance on Training a MIONET Model with PINA

[reactingflow]

Hello everyone,

I wanted to start by saying that I find PINA to be an exceptionally good software, offering a plethora of useful tools. Currently, I am learning how to use PINA to train a MIONET model. In my endeavor, I have searched for tutorial cases but haven't been able to find any that match my specific needs.

All of my inputs (three in total) are one-dimensional arrays, and the output is also one-dimensional. Could anyone here provide some guidance on how to quickly set up a MIONET model for training from scratch? Your help would be greatly appreciated as I'm really eager to move forward with my project.

Thank you very much in advance for your support and insights!

Best regards,

[dario-coscia]

Hi @reactingflow! Have you already checked the doc on MIONET (here)?
There is a starting example, maybe it is useful :) Once the model is defined, you can train as you normally do with PINA.

Also, look at this minimal example below, adapted from the read me. I am reproducing the architecture of Fig.1 in the original paper with $n=2$.

import torch
from pina import Trainer, LabelTensor
from pina.model import MIONet, FeedForward
from pina.solver import SupervisedSolver
from pina.problem.zoo import SupervisedProblem

input_tensor = LabelTensor(torch.rand((10, 3)), ["v1", "v2", "y"])
output_tensor = LabelTensor(torch.rand((10, 1)), ["u"])

# Step 1. Define problem
problem = SupervisedProblem(
    input_tensor,
    output_tensor,
    input_variables=input_tensor.labels,
    output_variables=output_tensor.labels,
)
# Step 2. Design MIONET
branch_net1 = FeedForward(input_dimensions=1, output_dimensions=10)
branch_net2 = FeedForward(input_dimensions=1, output_dimensions=10)
trunk_net = FeedForward(input_dimensions=1, output_dimensions=10)
networks = {branch_net1: ["v1"], branch_net2: ["v2"], trunk_net: ["y"]}
model = MIONet(networks=networks, reduction="+", aggregator="*")
# Step 3. Define Solver
solver = SupervisedSolver(problem, model)
# Step 4. Train
trainer = Trainer(solver, max_epochs=1000, accelerator="cpu")
trainer.train()

Hope this helps! If you enjoy using PINA, consider leaving us a ⭐ — it really helps us grow the community. Thanks for your support!

[dario-coscia]

By the way @reactingflow, if you're interested in a tutorial on DeepONet or MIONet, feel free to open an issue! We don’t have one for those architectures yet, and it’d be great to add one. We’re always open to new contributions 😊

[reactingflow]

Hi Dario,

Thank you very much for your previous help and suggestions.

Inspired by your example and the first case study in the MIONet paper, I’ve started developing a test case implementation of MIONet that I’d be happy to share with you and potentially contribute to the community as a tutorial or example. The idea is to make it accessible and educational for others who are getting started with operator learning tasks.

However, while working on this, I encountered a recurring issue during training that I wanted to ask your opinion on. It seems to be related to network connectivity, and it often prevents the code from running properly. Specifically, when trying to download external data files (like those used in the PINA tutorials), I frequently receive a ConnectionRefusedError. Below is an excerpt of the error trace I get:

ConnectionError: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /mathLab/PINA/raw/refs/heads/master/tutorials/tutorial7/data/pts_0.5_0.5
(Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x72ff95d44950>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also, once the model runs successfully, I am struggling with visualizing the results effectively — for instance, plotting predicted vs. true solutions, loss curves. If you have any sample scripts or tools you use regularly for visualizations in MIONet or similar models, I’d love to take a look!

And here is the current testing code I am working on:

import numpy as np
import torch
from pina import Trainer, LabelTensor
from pina.model import MIONet, FeedForward
from pina.solver import SupervisedSolver
from pina.problem.zoo import SupervisedProblem

# === Alternative GRF Implementation ===
from sklearn.gaussian_process.kernels import RBF
from sklearn.gaussian_process import GaussianProcessRegressor

class GRF:
    def __init__(self, dim=1, length_scale=0.2, N=1000, interp="cubic"):
        self.kernel = RBF(length_scale=length_scale)  # Define RBF kernel
        self.N = N  # Number of discrete points
        self.dim = dim  # Dimensionality
        
    def random(self, n_samples):
        return [self._generate_sample() for _ in range(n_samples)]  # Generate n_samples
        
    def _generate_sample(self):
        X = np.linspace(0, 1, self.N)[:, None]  # Create input grid
        gp = GaussianProcessRegressor(kernel=self.kernel)
        return gp.sample_y(X).ravel()  # Sample from GP and flatten output
        
    def eval_u(self, features, points):
        from scipy.interpolate import interp1d
        x_ref = np.linspace(0, 1, len(features[0]))  # Reference x-axis
        # Interpolate each function to given points
        return np.array([interp1d(x_ref, f, kind='cubic')(points.ravel()) for f in features])
# ==================


def solve_ADVD(xmin, xmax, tmin, tmax, D, V, Nx, Nt):
    """Solve
    u_t + u_x - D * u_xx = 0
    u(x, 0) = V(x), periodic and twice continuously differentiable
    D(x), periodic and continuous
    """
    # Crank-Nicholson method implementation
    x = np.linspace(xmin, xmax, Nx)
    t = np.linspace(tmin, tmax, Nt)
    h = x[1] - x[0]
    dt = t[1] - t[0]
    lam = dt / h
    mu = dt / h**2
    u = np.zeros([Nx, Nt])
    u[:, 0] = V  # Initial condition
    d = D[1:]  # Exclude first element for compatibility with matrix dimensions

    I = np.eye(Nx - 1)
    I1 = np.roll(I, 1, axis=0)
    I2 = np.roll(I, -1, axis=0)

    A = (1 + d * mu) * I - (lam / 4 + d * mu / 2) * I1 + (lam / 4 - d * mu / 2) * I2
    B = 2 * I - A
    C = np.linalg.solve(A, B)

    for n in range(Nt - 1):
        u[1:, n + 1] = C @ u[1:, n]  # Time stepping
    u[0, :] = u[-1, :]  # Periodic boundary condition

    return x, t, u

# Functions for data generation
def eval_s(m, T, Nt, sensor_values1, sensor_values2):
    return solve_ADVD(0, 1, 0, T, sensor_values1, sensor_values2, m, Nt)[2]

def generate_data(space, m, T, Nt, num_samples):
    """Generate training or testing data"""
    # Generate sensor values
    features1 = space.random(num_samples)
    sensors1 = np.linspace(0, T, num=m)[:, None]
    sensor_values1 = (
        np.abs(space.eval_u(features1, np.sin(np.pi * sensors1)**2) * 0.01 + 0.1)
    )
    
    features2 = space.random(num_samples)
    sensors2 = np.linspace(0, T, num=m)[:, None]
    sensor_values2 = space.eval_u(features2, np.sin(np.pi * sensors2)**2)
    
    # Generate solutions
    s = np.stack([
        eval_s(m, T, Nt, sv1, sv2) 
        for sv1, sv2 in zip(sensor_values1, sensor_values2)
    ])
    
    # Generate fixed coordinate grid
    x = np.linspace(0, 1, m)
    t = np.linspace(0, T, Nt)
    xt = np.array([[xi, ti] for xi in x for ti in t])
    
    return sensor_values1, sensor_values2, xt, s


# Parameter settings
space = GRF(1, length_scale=0.2, N=1000, interp="cubic")
m = 20       # Number of spatial discretization points
T = 1        # Time range
Nt = 20      # Number of time steps
num_train = 500
num_test = 100

# Generate training data
sv1_train, sv2_train, xt_train, s_train = generate_data(space, m, T, Nt, num_train)

# Data preprocessing
# Repeat sensor data to match number of coordinate points
num_points = m * Nt
sv1_repeat = np.repeat(sv1_train, num_points, axis=0)
sv2_repeat = np.repeat(sv2_train, num_points, axis=0)

# Repeat coordinate data to match number of samples
xt_repeat = np.tile(xt_train, (num_train, 1))

# Combine input data
input_data = np.hstack([sv1_repeat, sv2_repeat, xt_repeat])
output_data = s_train.reshape(-1, 1)

# Create LabelTensors
input_labels = [f'D_{i}' for i in range(m)] + [f'V_{i}' for i in range(m)] + ['x', 't']
input_tensor = LabelTensor(torch.tensor(input_data, dtype=torch.float32), input_labels)
output_tensor = LabelTensor(torch.tensor(output_data, dtype=torch.float32), ['u'])

# Define supervised learning problem
problem = SupervisedProblem(
	input_tensor,
	output_tensor,
	input_variables=input_labels,
	output_variables=['u']
)

# Build MIONet
branch_net_D = FeedForward(input_dimensions=m, output_dimensions=64)
branch_net_V = FeedForward(input_dimensions=m, output_dimensions=64)
trunk_net = FeedForward(input_dimensions=2, output_dimensions=64)

model = MIONet(
	networks={
		branch_net_D: [f'D_{i}' for i in range(m)],
		branch_net_V: [f'V_{i}' for i in range(m)],
		trunk_net: ['x', 't']
	},
	reduction='+',
	aggregator='*',
)

# Configure solver and trainer
solver = SupervisedSolver(problem=problem, model=model)
trainer = Trainer(solver, 
				 max_epochs=1000,
				 accelerator='cpu',
				 batch_size=1024,
				 enable_model_summary=True)

trainer.train()

Looking forward to hearing your thoughts when you have time.

Best regards,

[dario-coscia]

Hi! We are aware of the ConnectionError issue (see #572), hopefully we will solve it shortly (we are working on it).
The tutorial looks pretty nice, I think that we could add it to the PINA tutorials if you want to collaborate. In that case, consider reading the Tutorial Guidelines first.

For model evaluation, it should be very simple:

test_output = solver(test_input)

where test_input is your input for testing. Once you have test_output, you can choose any viz. tool. If you care just about the error in testing, just run:

trainer.test()

which will print the table results.

For plotting the loss, in PINA there are two ways.
Easy way (recommended for fast development): Use the MetricTracker callback and plot the metrics after training. In this tutorial we show how to do it :)
Standard way: Use the Loggers from PyTorch Lightning, you can learn more about them in this tutorial in the Loggers section!