Generate spy mock

[JulianBissekkou]

Spy was removed when we migrated to null safety. Spy also used the mirrors API which introduces problems on some platforms.

Are we able to implement the Spy feature using code gen? A potential spy implementation could be another layer ontop of a mock that proxies the calls to an instance that can be provided when creating the spy object.

Here is some pseudo code to show you how this feature can look like.

@GenerateMocks([Cat])
// This will generate the "SpyCat" class.
// We can also add `final List<Type> spyClasses` to the `GenerateMocks` annotation.
@SpyProxyMock([Cat])
void main() {
  test(
    "Example test",
    () {
      final spy = SpyCat(realInstance: Cat("liam"));
      when(spy.eat(any)).thenAnswer((_) => print("..."));
    },
  );
}

In noSuchMethod we can then decide if we have to call the realInstance or if we execute a mocked invocation.

I would love to contribute to this feature.

[srawlins]

I think we probably could.

[JulianBissekkou]

I can help with this feature, but it would be good to agree on how the API should look like.
(See my comment on SpyProxyMock)

[srawlins]

I'm not sure that MockCat and SpyCat would be related at all. So maybe we would only need @GenerateSpy. I'm not sure.

[JulianBissekkou]

I have to gain some more knowledge on the source code and how the lib generates the actual Mock classes, but I think that SpyCat will extend MockCat and override noSuchMethod to call the realInstance.
Therefore we need to generate MockCat to make SpyCat possible.

Maybe it's easier to add the spy types to GenerateMocks because we can reuse the logic of the builders that generate code for the given annotation.
Do you think extending MockCat and generating SpyCat on top of that is a good idea or should we separate those things and only share code between both builders? 🤔

[JulianBissekkou]

I would love to get your opinion as well @kevmoo

[kevmoo]

In the 100% abstract, sounds good to me, but I don't use mockito...

[yanok]

@JulianBissekkou Hey, sorry, it's been a while. Would you still be interested to implement it?

Regarding the API, I would prefer to not follow Java's Mockito here and keep the Mock/Spy things separate (I think in Java's Mockito one can add expectations to Spies), unless we have a very good reason to keep it.

[JulianBissekkou]

A few month ago I tried implementing this feature but I was stuck when calling the actual method of the spy if the user didn't register a handler for a method.
I would still love to contribute. Keeping things seperated sounds like a good idea :)

[yanok]

A few month ago I tried implementing this feature but I was stuck when calling the actual method of the spy if the user didn't register a handler for a method.

What was the problem? How could I help? I didn't think about it a lot, but I think you have to generate an implementation of the class that overrides all the public methods/accessors, where you record the call and then dispatch it to the real instance. A slightly tricky part is supporting verify, since you don't want to call the real instance there.

Very schematically:

// Somewhere in the common code:
// I think you would need a base Spy class, it does many things that Mock does, but it has to do things slightly differently.
class Spy {
  Object? doStuff(
     Invocation i,
     Object inRealCallToken,
     {Object? returnValue}) {
    // pretty much nSM implementation from Mock class, but
    // 1. throws if _whenInProgress (Spies don't support when)
    // 2. for real call (last else block in Mock.nSM), just add the current call to realCalls and return inRealCallToken.
  }
}

// In the generated code:
class _InRealCallToken {}
class SpyFoo extends Spy {
  final realCallToken = _InRealCallToken();
  Foo real;
  @override
  int method(String s) {
    final retValue = /* generated fake return value, it will be discarded at runtime anyway */;
    final r = doStuff(Invocation.method('method', [s]), realCallToken, returnValue: retValue);
    if (r != realCallToken) {
       return r;
    }
    return real.method(s);
  }
}

[rafaelsetragni]

The only improvement needed is to create a new mock method similar to thenAnswer, thenReturn and thenThrow, called for example as thenExecuteRealMethod.

Here is a simple example:

import 'package:flutter_test/flutter_test.dart';
import 'package:mockito/annotations.dart';
import 'package:mockito/mockito.dart';

import 'test.mocks.dart';

enum MathOperation {
  add,
  subtract,
  multiply,
  divide,
}

class Calculator {

  int add(int a, int b) => a + b;
  int subtract(int a, int b) => a - b;
  int multiply(int a, int b) => a * b;
  int divide(int a, int b) => a % b;

  int execute(int a, int b, {required MathOperation operation}){
    late int result;
    switch(operation){

      case MathOperation.add:
        result = add(a, b);
        break;

      case MathOperation.subtract:
        result = subtract(a, b);
        break;

      case MathOperation.multiply:
        result = multiply(a, b);
        break;

      case MathOperation.divide:
        result = divide(a, b);
        break;
    }
    return result;
  }
}

@GenerateNiceMocks([MockSpec<Calculator>()])
void main(){
  final MockCalculator mockedCalculator = MockCalculator();
  setUpAll(() {
    when(mockedCalculator
        .execute(any, any, operation: anyNamed('operation'))
    ).thenExecuteRealMethod();

    when(mockedCalculator.add(any, any)).thenReturn(0);
    when(mockedCalculator.subtract(any, any)).thenReturn(0);
    when(mockedCalculator.multiply(any, any)).thenReturn(0);
    when(mockedCalculator.divide(any, any)).thenReturn(0);
  });

  test('calculation methods tests', () {
    final calculator = Calculator();
    expect(calculator.execute(1, 2, operation: MathOperation.add), 3);
    expect(calculator.execute(1, 2, operation: MathOperation.subtract), -1);
    expect(calculator.execute(1, 2, operation: MathOperation.multiply), 2);
    expect(calculator.execute(1, 2, operation: MathOperation.divide), 1);
  });

  test('calculation exception tests', () {
    final calculator = Calculator();
    expect(() => calculator.execute(0, 0, operation: MathOperation.add), returnsNormally);
    expect(() => calculator.execute(0, 0, operation: MathOperation.subtract), returnsNormally);
    expect(() => calculator.execute(0, 0, operation: MathOperation.multiply), returnsNormally);
    expect(() => calculator.execute(0, 0, operation: MathOperation.divide), throwsA(isA<UnsupportedError>()));
  });

  test('addition call tests', () {
    mockedCalculator.execute(0, 0, operation: MathOperation.add);

    verify(mockedCalculator.add(any, any)).called(1);
    verify(mockedCalculator.subtract(any, any)).called(0);
    verify(mockedCalculator.multiply(any, any)).called(0);
    verify(mockedCalculator.divide(any, any)).called(0);
  });

  test('subtraction call tests', () {
    mockedCalculator.execute(0, 0, operation: MathOperation.add);

    verify(mockedCalculator.add(any, any)).called(0);
    verify(mockedCalculator.subtract(any, any)).called(1);
    verify(mockedCalculator.multiply(any, any)).called(0);
    verify(mockedCalculator.divide(any, any)).called(0);
  });

  test('multiplication call tests', () {
    mockedCalculator.execute(0, 0, operation: MathOperation.add);

    verify(mockedCalculator.add(any, any)).called(0);
    verify(mockedCalculator.subtract(any, any)).called(0);
    verify(mockedCalculator.multiply(any, any)).called(1);
    verify(mockedCalculator.divide(any, any)).called(0);
  });

  test('divide call tests', () {
    mockedCalculator.execute(0, 0, operation: MathOperation.add);

    verify(mockedCalculator.add(any, any)).called(0);
    verify(mockedCalculator.subtract(any, any)).called(0);
    verify(mockedCalculator.multiply(any, any)).called(0);
    verify(mockedCalculator.divide(any, any)).called(1);
  });
}

In the provided example, I need to test the Calculator class by mocking its add, subtract, multiply, and divide methods individually while ensuring the execute method uses the real implementation. This is important because each method should be tested separately, without being affected by the others.

Its a simple improvement. If the method is flagged to use the real method, call the real method, and the others keep as the same.