Skip to content

fix: add extra_plugin_dir support to FullStackTest #1726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ fixes:
- fix: type hints (#1698)
- fix: update plugin config message (#1727)
- docs: add example on how to use threaded replies (#1728)
- fix: add extra_plugin_dir support to FullStackTest (#1726)


v6.2.0 (2024-01-01)
Expand Down
1 change: 1 addition & 0 deletions docs/user_guide/plugin_development/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ with sets of recipes on a range of topics describing how to handle more advanced
scheduling
webhooks
testing
testing_plugins_fullstack
logging
exceptions
plugin_compatibility_settings
Expand Down
31 changes: 0 additions & 31 deletions docs/user_guide/plugin_development/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -251,37 +251,6 @@ You can now have a look at coverage statistics through :command:`coverage report

It's also possible to generate an HTML report with :command:`coverage html` and opening the resulting `htmlcov/index.html`.

Travis and Coveralls
--------------------

Last but not least, you can run your tests on Travis-CI_ so when you update code or others submit pull requests the tests will automatically run confirming everything still works.

In order to do that you'll need a `.travis.yml` similar to this:

.. code-block:: yaml

language: python
python:
- 3.6
- 3.7
install:
- pip install -q errbot pytest pytest-pep8 --use-wheel
- pip install -q coverage coveralls --use-wheel
script:
- coverage run --source myplugin -m py.test --pep8
after_success:
- coveralls
notifications:
email: false

Most of it is self-explanatory, except for perhaps the `after_success`. The author of this plugin uses Coveralls.io_ to keep track of code coverage so after a successful build we call out to coveralls and upload the statistics. It's for this reason that we `pip install [..] coveralls [..]` in the `.travis.yml`.

The `-q` flag causes pip to be a lot more quiet and `--use-wheel` will cause pip to use wheels_ if available, speeding up your builds if you happen to depend on something that builds a C-extension.

Both Travis-CI and Coveralls easily integrate with Github hosted code.

.. _py.test: http://pytest.org
.. _conftest.py: http://doc.pytest.org/en/latest/writing_plugins.html#conftest-py-local-per-directory-plugins
.. _Coveralls.io: https://coveralls.io
.. _Travis-CI: https://travis-ci.org
.. _wheels: http://www.python.org/dev/peps/pep-0427/
167 changes: 167 additions & 0 deletions docs/user_guide/plugin_development/testing_plugins_fullstack.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
Testing your plugins with unittest
==================================

This guide explains how to test your Errbot plugins using the built-in testing framework. Errbot provides a powerful testing backend called ``FullStackTest`` that allows you to write unit tests for your plugins in a familiar unittest style.

Basic Test Setup
--------------

To test your plugin, create a test file (e.g., `test_myplugin.py`) in your plugin's directory. Here's a basic example:

.. code-block:: python

import unittest
from pathlib import Path

from errbot.backends.test import FullStackTest

path = str(Path(__file__).resolve().parent)
extra_plugin_dir = path


class TestMyPlugin(FullStackTest):
def setUp(self):
super().setUp(extra_plugin_dir=extra_plugin_dir)

def test_my_command(self):
# Simulate a user sending a command
self.push_message('!hello')
self.assertIn('Hello!', self.pop_message())

Running Tests
------------

You can run your tests using Python's unittest framework:

.. code-block:: bash

python -m unittest test_myplugin.py

Test Methods
-----------

FullStackTest provides several methods to help test your plugin's behavior:

1. **Message Handling**:
- ``push_message(command)``: Simulate a user sending a command
- ``pop_message(timeout=5, block=True)``: Get the bot's response
- ``assertInCommand(command, response, timeout=5)``: Assert a command returns expected output
- ``assertCommandFound(command, timeout=5)``: Assert a command exists

2. **Room Operations**:
- ``push_presence(presence)``: Simulate presence changes
- Test room joining/leaving
- Test room topic changes

3. **Plugin Management**:
- ``inject_mocks(plugin_name, mock_dict)``: Inject mock objects into a plugin
- Test plugin configuration
- Test plugin dependencies

Example Test Cases
----------------

Here are some example test cases showing different testing scenarios:

1. **Basic Command Testing**:

.. code-block:: python

def test_basic_command(self):
self.push_message('!echo test')
self.assertIn('test', self.pop_message())

2. **Command with Arguments**:

.. code-block:: python

def test_command_with_args(self):
self.push_message('!repeat test 3')
response = self.pop_message()
self.assertIn('testtesttest', response)

3. **Error Handling**:

.. code-block:: python

def test_error_handling(self):
self.push_message('!nonexistent')
response = self.pop_message()
self.assertIn('Command not found', response)

4. **Mocking Dependencies**:

.. code-block:: python

def test_with_mocks(self):
# Create mock objects
mock_dict = {
'external_api': MockExternalAPI()
}
self.inject_mocks('MyPlugin', mock_dict)

# Test plugin behavior with mocks
self.push_message('!api_test')
self.assertIn('Mock response', self.pop_message())

Best Practices
-------------

1. **Test Isolation**: Each test should be independent and not rely on the state from other tests.

2. **Setup and Teardown**: Use ``setUp()`` to initialize your test environment and ``tearDown()`` to clean up.

3. **Timeout Handling**: Always specify appropriate timeouts for message operations to avoid hanging tests.

4. **Error Cases**: Include tests for error conditions and edge cases.

5. **Documentation**: Document your test cases to explain what they're testing and why.

Complete Example
--------------

Here's a complete example of a test suite for a plugin:

.. code-block:: python

import unittest
from pathlib import Path

from errbot.backends.test import FullStackTest

path = str(Path(__file__).resolve().parent)
extra_plugin_dir = path

class TestGreetingPlugin(FullStackTest):
def setUp(self):
super().setUp(extra_plugin_dir=extra_plugin_dir)

def test_basic_greeting(self):
"""Test the basic greeting command."""
self.push_message('!greet Alice')
self.assertIn('Hello, Alice!', self.pop_message())

def test_greeting_with_options(self):
"""Test greeting with different options."""
# Test with count
self.push_message('!greet Bob --count 2')
response = self.pop_message()
self.assertIn('Hello, Bob!Hello, Bob!', response)

# Test with shout
self.push_message('!greet Charlie --shout')
self.assertIn('HELLO, CHARLIE!', self.pop_message())

def test_error_handling(self):
"""Test how the plugin handles errors."""
# Test missing name
self.push_message('!greet')
self.assertIn('Please provide a name', self.pop_message())

# Test invalid count
self.push_message('!greet Eve --count abc')
self.assertIn('must be an integer', self.pop_message())


if __name__ == '__main__':
unittest.main()
21 changes: 14 additions & 7 deletions errbot/backends/test.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import importlib
import logging
import pathlib
import sys
import textwrap
import unittest
from os.path import abspath, sep
from os.path import sep
from queue import Empty, Queue
from tempfile import mkdtemp
from threading import Thread
Expand Down Expand Up @@ -604,25 +605,31 @@ def test_about(self):
self.assertIn('Err version', self.pop_message())
"""

def __init__(
self,
methodName,
extra_plugin_dir=None,
loglevel=logging.DEBUG,
extra_config=None,
):
self.bot_thread = None
super().__init__(methodName)

def setUp(
self,
extra_plugin_dir=None,
extra_test_file=None,
loglevel=logging.DEBUG,
extra_config=None,
) -> None:
"""
:param extra_plugin_dir: Path to a directory from which additional
plugins should be loaded.
:param extra_test_file: [Deprecated but kept for backward-compatibility,
use extra_plugin_dir instead]
Path to an additional plugin which should be loaded.
:param loglevel: Logging verbosity. Expects one of the constants
defined by the logging module.
:param extra_config: Piece of extra bot config in a dict.
"""
if extra_plugin_dir is None and extra_test_file is not None:
extra_plugin_dir = sep.join(abspath(extra_test_file).split(sep)[:-2])
if extra_plugin_dir is None:
extra_plugin_dir = str(pathlib.Path(".").resolve().parent.parent.absolute())

self.setup(
extra_plugin_dir=extra_plugin_dir,
Expand Down