Skip to content

Update Testcase and TestScript in parameters.rst #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
7c251d4
update the changelog
omehrabi Jul 22, 2022
ffb845f
release_22.7
omehrabi Jul 25, 2022
57e9330
Merge pull request #175 from CiscoTestAutomation/release_22.7
omehrabi Jul 27, 2022
e409b73
Update docs for keys() markup
dwapstra Jul 29, 2022
370c942
Update pyats validate docs for jinja2_config subcommand
dwapstra Aug 2, 2022
e712ca7
Update manifest doc for boolean and meta argument handling
dwapstra Aug 23, 2022
adeab86
Merge pull request #177 from CiscoTestAutomation/pyats_validate_jinja…
dwapstra Aug 24, 2022
1278a3b
Merge pull request #178 from CiscoTestAutomation/manifest_boolean_arg…
dwapstra Aug 24, 2022
928e82f
Releasing v22.8
GerriorL Aug 26, 2022
1aa1a8f
Merge pull request #179 from CiscoTestAutomation/release_22.8
GerriorL Aug 30, 2022
ae2cca4
Add docs for gTask class
dwapstra Sep 7, 2022
f67d946
Merge pull request #180 from CiscoTestAutomation/gtask
dwapstra Sep 20, 2022
addb5cb
Releasing v22.9
Sep 23, 2022
79f5567
Merge pull request #181 from CiscoTestAutomation/release_22.9
bastell Sep 27, 2022
87b11df
Update YAML markup reference
dwapstra Oct 14, 2022
4326a94
Merge pull request #176 from CiscoTestAutomation/markup_keys
dwapstra Oct 18, 2022
f6920a5
Releasing v22.10
Oct 21, 2022
7092d4d
Merge pull request #186 from CiscoTestAutomation/release_22.10
GerriorL Oct 31, 2022
a678199
update changelog
omehrabi Nov 29, 2022
a7fe7fd
Merge pull request #194 from CiscoTestAutomation/release_22.11
omehrabi Nov 30, 2022
145164a
Releasing v23.1
GerriorL Jan 31, 2023
76eef28
Merge pull request #200 from CiscoTestAutomation/release_23.1
domachad Jan 31, 2023
ef6383a
update documentation
omehrabi Jan 31, 2023
2d33a05
Merge pull request #201 from CiscoTestAutomation/update_markup_adding…
omehrabi Feb 6, 2023
c267a12
Update docs for CLI markup
dwapstra Feb 17, 2023
35eb4c0
new migrate command
bastell Feb 23, 2023
6944260
Merge pull request #203 from CiscoTestAutomation/cli_markup_default
Taarini Feb 23, 2023
f82bf28
Releasing v23.2
lsheikal Feb 24, 2023
51fd117
updated changelogs
lsheikal Mar 3, 2023
62ac81e
Merge pull request #206 from CiscoTestAutomation/release_23.2
lsheikal Mar 7, 2023
f5d6ea3
changelogs 23.3
bastell Mar 30, 2023
8a61214
Merge pull request #207 from CiscoTestAutomation/release_23.3
bastell Mar 30, 2023
d9d6d1f
23.4 changelogs
bastell Apr 25, 2023
10ea694
Update docs for incoming changes
GerriorL May 24, 2023
f1e77c6
Update table
GerriorL May 25, 2023
986477a
Update docs/aetest/debugging.rst
GerriorL May 25, 2023
4f3669d
Update docs/aetest/debugging.rst
GerriorL May 25, 2023
72f9fd5
Merge pull request #211 from CiscoTestAutomation/GerriorL-update_debu…
GerriorL May 25, 2023
e8f59f0
Update index.rst
GerriorL May 26, 2023
1d7a2e6
Merge pull request #212 from CiscoTestAutomation/release_23.5
GerriorL May 29, 2023
b2ada19
Update Changelogs
May 30, 2023
6e62262
Merge remote-tracking branch 'origin/release_23.4' into release_23.5
May 26, 2023
0c7741e
Releasing v23.6
Jun 22, 2023
e5ca3f6
Merge pull request #214 from CiscoTestAutomation/release_23.6
ThomasJRyan Jun 27, 2023
52bc8a5
Update changelogs
lsheikal Jul 25, 2023
1740a82
Releasing v23.7
lsheikal Jul 25, 2023
a147a63
Docs for CALLABLE markup with kwarg
dwapstra Jul 25, 2023
c42d2b9
Releasign v23.7
lsheikal Jul 26, 2023
e5fb8e9
Merge pull request #218 from CiscoTestAutomation/release_23.7
bastell Jul 27, 2023
a79148d
Add section stats docs
dwapstra Jul 18, 2023
2edc39e
Update section stats doc
dwapstra Jul 25, 2023
dc71a1d
Update email docs
dwapstra Aug 3, 2023
11be925
Merge pull request #220 from CiscoTestAutomation/section_stats
dwapstra Aug 9, 2023
d8660b4
Releasing v23.8
omehrabi Aug 29, 2023
221b8e6
Merge pull request #221 from CiscoTestAutomation/release_23.8
bastell Aug 29, 2023
3eff740
Releasing v23.9
Sep 22, 2023
8673598
Merge pull request #222 from CiscoTestAutomation/release_23.9
Taarini Sep 27, 2023
b542376
update schema
omehrabi Oct 16, 2023
6ec71b8
update the documentation
omehrabi Oct 16, 2023
4888f04
address comments
omehrabi Oct 18, 2023
de98af4
Merge pull request #223 from CiscoTestAutomation/update_schema_for_fa…
bastell Oct 24, 2023
bca90ee
Unify changelogs
bastell Nov 1, 2023
a0565bd
Merge pull request #224 from CiscoTestAutomation/release_23.10
bastell Nov 3, 2023
33478d2
Releasing 23.11
domachad Nov 28, 2023
b5c339a
Merge pull request #225 from CiscoTestAutomation/release_23.11
domachad Nov 29, 2023
27e2dea
Releasing v24.1
Jan 25, 2024
e342e50
Merge pull request #228 from CiscoTestAutomation/release_24.1
Taarini Jan 31, 2024
01233bf
update changelogs
lsheikal Feb 26, 2024
bafd869
Releasing v24.2
lsheikal Feb 26, 2024
6a1aa69
Merge pull request #231 from CiscoTestAutomation/release_24.2
lsheikal Feb 27, 2024
45bee4c
Releasing v24.3
omehrabi Mar 21, 2024
aca0f13
Releasing v24.3
omehrabi Mar 26, 2024
cd7c756
Merge pull request #232 from CiscoTestAutomation/release_24.3
omehrabi Mar 26, 2024
946f206
Added service wrapper docs
Apr 15, 2024
232ad0c
Addressed comments
Apr 16, 2024
b073a6e
Merge pull request #234 from CiscoTestAutomation/thomryan/service_wra…
ThomasJRyan Apr 22, 2024
648f9b9
Releasing v24.4
Apr 30, 2024
0a31680
Merge pull request #235 from CiscoTestAutomation/release_24.4
Taarini May 7, 2024
6b54be8
Merge pull request #219 from CiscoTestAutomation/markup_callable
dwapstra May 9, 2024
0d584c9
Updated docs
Apr 30, 2024
7112826
Releasing v24.5
May 27, 2024
7c67433
Merge pull request #239 from CiscoTestAutomation/release_24.5
ThomasJRyan May 31, 2024
fc20c40
Releasing v24.6
omehrabi Jun 24, 2024
efba1b6
Merge pull request #241 from CiscoTestAutomation/release_24.6
omehrabi Jun 26, 2024
666dab6
fixed make docs warnings
lsheikal Jul 11, 2024
98e108c
Merge remote-tracking branch 'origin/master' into pyats_luke_docs_cle…
lsheikal Jul 11, 2024
d25ff39
cleanup docs
lsheikal Jul 12, 2024
434c581
Merge pull request #244 from CiscoTestAutomation/pyats_luke_docs_cleanup
lsheikal Jul 12, 2024
d987b37
Release_24.7
lsheikal Jul 30, 2024
2e5d8ae
Releasing v24.7
lsheikal Jul 30, 2024
a2bc70c
Releasing v24.7
lsheikal Jul 31, 2024
1d81b7b
Merge pull request #245 from CiscoTestAutomation/release_24.7
lsheikal Aug 1, 2024
5f39515
Releasing v24.9
Sep 25, 2024
3a79fd3
Merge pull request #249 from CiscoTestAutomation/release_24.9
Taarini Oct 1, 2024
26b7c1c
Releasing v24.10
Oct 22, 2024
af30a6a
Merge pull request #251 from CiscoTestAutomation/release_24.10
ThomasJRyan Oct 25, 2024
0ad726a
Improve readibility and wording
aitestino Jul 22, 2022
9ae07a0
Update Testcase and TestScript object names
mjbear Nov 5, 2024
3e225f5
Remove trailing spaces at end of parameters.rst
mjbear Nov 5, 2024
55cfad7
Merge branch 'aetest-parameters-doc-improv' into aetest-parameters-do…
mjbear Nov 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 46 additions & 2 deletions docs/aetest/debugging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -227,8 +227,9 @@ want to pause the testscript execution somewhere & check the current state
and/or configuration of your environment, logs, and testbed devices?

The **pause on phrase** feature allows you to pause on **any** log messages
generated by the current script run, without requiring any modifications to the
scripts and/or its libraries. When enabled, it actively filters all log messages
generated by the current script run, including CLI output from devices,
without requiring any modifications to the scripts and/or its libraries.
When enabled, it actively filters all log messages
propagated to the ``logging.root`` logger, and pauses when a match is found.

Three distinct types of pause actions are supported:
Expand Down Expand Up @@ -486,3 +487,46 @@ Below are some examples of the pause on phrase feature in action:
# -> /path/to/my/example/testscript.py[20] test()
# (press Ctrl-D to resume execution)
# >>>

The pause will also dump the connection information, including which devices are currently connected.

.. code-block:: python

# --------------------------------------------------------------------------------
# Pause On Phrase: Connection Information
#
# +-------------------+--------------------------------+
# | Device Property | Value |
# +-------------------+--------------------------------+
# | Name | device.name |
# | Alias | device.alias |
# | Active Connection | alias of active connection |
# | Status | Connected|Disconnected |
# | Spawn Command | command for active connection |
# | -------- | -------- |
# | Connection Alias | connection alias e.g. cli |
# | Class | Unicon, Gnmi, Netconf, etc |
# | IP | IPV4|IPV6 address |
# | Protocol | ssh, telnet, etc |
# | -------- | -------- |
# | URL | <protocol>://<ip>:<port> |
# +-------------------+--------------------------------+


.. note::

If it is required to connect to the device directly while the test is paused, the device connection must be disconnected
from before it can be accessed directly.

.. code-block:: python

>>> import pyats.easypy as ep
>>> dev = ep.runtime.testbed.devices['<device>']
>>> dev.disconnect()

# the device can now be connected to in a separate terminal. Once ready to resume the test,
# break the connection and reattach with

>>> dev.connect()


44 changes: 22 additions & 22 deletions docs/aetest/parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,16 @@ Test Parameters
``aetest`` is a data-driven test infrastructure. Its scripts and test cases are
intended to be driven dynamically by:

- Data in the form of input arguments to the test script
- Data in the form of input arguments to the TestScript
- Dynamic data generated during runtime

The collection of dynamic data that artificially affects the behavior of
test scripts and test cases in ``aetest`` is called **parameters**. It adheres to
TestScripts and Testcases in ``aetest`` is called **parameters**. It adheres to
a pre-defined set of parent/child propagation relationships and may be used as
`Function Arguments`_ to each test section.

This feature is a supplement to static test case data (attribute values stored
within each test case).
This feature is a supplement to static Testcase data (attribute values stored
within each Testcase).

.. tip::

Expand Down Expand Up @@ -74,8 +74,8 @@ methods as the "doer," then parameters are the "what to do with." E.g., an
add_to_c(1, 2)
# 103

In data-driven testing, test scripts are the *doers*, performing the act
of testing a facet of some product. Test script arguments and parameters are thus the
In data-driven testing, TestScripts are the *doers*, performing the act
of testing a facet of some product. TestScript arguments and parameters are thus the
input data that influences the specific actions of testing. Here
are some possible use cases:

Expand All @@ -86,13 +86,13 @@ are some possible use cases:
- The ``vlan`` argument to the ``layer2_traffic`` script can dynamically control the
VLAN to be configured for traffic testing.

- Other toggle arguments that dynamically turn on/off certain test cases,
- Other toggle arguments that dynamically turn on/off certain Testcases,
and a combination of features to be configured & tested

- Etc.

Of course, the parameters feature in ``aetest`` is much more than just script
arguments. It enables users to write test cases and test scripts that are capable
arguments. It enables users to write Testcases and TestScripts that are capable
of being driven by inputs, varying the degree of testing, etc.

Relationship Model
Expand Down Expand Up @@ -162,7 +162,7 @@ execution, this happens behind the scenes automatically.
new_testcase_parameters.update(testcase.parameters)
testcase.parameters = new_testcase_parameters

# so that the new parameters seen in the test case
# so that the new parameters seen in the Testcase
# level, is:
testcase.parameters
# {'param_A': 100, 'param_B': 2, 'param_C': 3}
Expand Down Expand Up @@ -203,7 +203,7 @@ within the test script.
# note that this also applies to CommonSetup & CommonCleanup
class Testcase(aetest.Testcase):

# all default parameters specific to this test case are declared
# all default parameters specific to this Testcase are declared
# in its own parameters dictionary.
parameters = {
'generic_param_A': 200
Expand Down Expand Up @@ -247,7 +247,7 @@ dynamically access & update parameters.
#
# continuing from the above

# re-defining the test case for the sake of code-continuity
# re-defining the Testcase for the sake of code-continuity
class Testcase(aetest.Testcase):

# local parameters defaults, same as above
Expand All @@ -272,7 +272,7 @@ dynamically access & update parameters.

@aetest.test
def test(self):
# access & print parent test script parameters
# access & print parent TestScript parameters
# (following the parent model)
print(self.parent.parameters)
# {'generic_param_A': 100,
Expand All @@ -291,7 +291,7 @@ dynamically access & update parameters.
Consider the above example: parameters can be set and accessed as the script
runs, opening the opportunity for scripts to dynamically discover the runtime
environment and modify test behavior (parameters) as required. E.g., the ``setup``
section of modifying test case parameters based on the current testbed state, and
section of modifying Testcase parameters based on the current testbed state, and
altering the behavior of ensuing ``test`` sections, etc.

.. tip::
Expand All @@ -306,7 +306,7 @@ altering the behavior of ensuing ``test`` sections, etc.
Script Arguments
----------------

In short, any arguments passed to the test script before startup becomes part of
In short, any arguments passed to the TestScript before startup becomes part of
the ``TestScript`` parameter. This includes all the arguments passed through the
:ref:`easypy_jobfile` during :ref:`aetest_jobfile_execution`, and/or any command
line arguments parsed and passed to ``aetest.main()`` during
Expand All @@ -323,7 +323,7 @@ line arguments parsed and passed to ``aetest.main()`` during
# without going into details about how script parameters/arguments are
# passed in (covered under Running Scripts section)

# assuming that the test script was called with the following inputs
# assuming that the TestScript was called with the following inputs
script_arguments = {
'arg_a': 100,
'arg_c': 3,
Expand Down Expand Up @@ -353,7 +353,7 @@ script's base parameters and overwrite existing ones.
.. tip::

Define your default parameters in the script, and change the behavior of
the test script by overwriting specific ones using script arguments.
the TestScript by overwriting specific ones using script arguments.

.. _parameters_as_funcargs:

Expand Down Expand Up @@ -404,7 +404,7 @@ their support.

# this setup section definition identifies "param_B"
# as an input requirement. As this parameter is available at this
# test case level (aggregated from the parent test script), it
# Testcase level (aggregated from the parent TestScript), it
# is passed in as input
@aetest.setup
def setup(self, param_B):
Expand Down Expand Up @@ -533,7 +533,7 @@ support arguments, can identify the current execution context, and
can act accordingly.

A parametrized function is declared when the ``@parameters.parametrize``
decorator is used on a function within a test script. This also adds the newly
decorator is used on a function within a TestScript. This also adds the newly
created parametrized function automatically as part of ``TestScript``
parameters, using the function name as the parameter name.

Expand Down Expand Up @@ -583,7 +583,7 @@ identical to its callable parameter sibling, with the following additions:
# As previously stated, there's no need to add parametrized functions
# to the parameters dict(). They are automatically discovered and added.

# Defining two tests under this test case
# Defining two tests under this Testcase
# ----------------------------------------------
# Similar to callable parameters, the above parameters
# are evaluated when used as function arguments to section
Expand Down Expand Up @@ -686,8 +686,8 @@ property, and is not useable as a function argument.
# names as keyword arguments to methods
@aetest.subsection
def subsection_one(self, testscript, section, steps):
# Testscript object has an attribute called module
# which is this test script's module
# TestScript object has an attribute called module
# which is this TestScript's module
print (testscript.module)
# <module 'example_script' from '/path/to/example.py'>

Expand Down Expand Up @@ -718,7 +718,7 @@ property, and is not useable as a function argument.

Reserved parameters provide ``aetest`` a mechanism to offer optional features
without polluting the :ref:`object_model` with additional attributes. It also
allows users to write test scripts that delve deeper and interact with the
allows users to write TestScripts that delve deeper and interact with the
internals of ``aetest`` using a supported method instead of hacking around.

*With great power comes great responsibilities* - use them wisely.
Expand Down
10 changes: 0 additions & 10 deletions docs/apidoc/easypy/pyats.easypy.runinfo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,4 @@ Submodules
:show-inheritance:


.. automodule:: pyats.easypy.runinfo.kafka
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.easypy.runinfo.s3
:members:
:undoc-members:
:show-inheritance:

1 change: 0 additions & 1 deletion docs/apidoc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ API Documentation
robot/index
async/index
datastructures/index
tcl/index
log/index
reporter/index
results/index
Expand Down
19 changes: 10 additions & 9 deletions docs/apidoc/kleenex/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,38 +11,39 @@ Subpackages

.. toctree::

pyats.kleenex.loader
pyats.clean.loader
pyats.kleenex.reporter
pyats.bringup

Submodules
----------


.. automodule:: pyats.kleenex.bases
.. automodule:: pyats.bringup.bases
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.bringup_base_cli_parser
.. automodule:: pyats.bringup.base_cli_parser
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.bringup_manager
.. automodule:: pyats.bringup.manager
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.bringup_manager_cli_parser
.. automodule:: pyats.bringup.manager_cli_parser
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.bringup_signals
.. automodule:: pyats.bringup.signals
:members:
:undoc-members:
:show-inheritance:
Expand All @@ -66,19 +67,19 @@ Submodules
:show-inheritance:


.. automodule:: pyats.kleenex.kleenex_traceback
.. automodule:: pyats.clean.traceback
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.schema
.. automodule:: pyats.clean.schema
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.utils
.. automodule:: pyats.clean.utils
:members:
:undoc-members:
:show-inheritance:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyats.kleenex.loader package
pyats.clean.loader package
============================

.. automodule:: pyats.kleenex.loader
.. automodule:: pyats.clean.loader
:members:
:undoc-members:
:show-inheritance:
Expand All @@ -10,7 +10,7 @@ Submodules
----------


.. automodule:: pyats.kleenex.loader.markup
.. automodule:: pyats.clean.loader.markup
:members:
:undoc-members:
:show-inheritance:
Expand Down
14 changes: 1 addition & 13 deletions docs/apidoc/kleenex/pyats.kleenex.reporter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,7 @@ Submodules
----------


.. automodule:: pyats.kleenex.reporter.base
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.reporter.clean
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.kleenex.reporter.context
.. automodule:: pyats.kleenex.reporter.reporter
:members:
:undoc-members:
:show-inheritance:
Expand Down
1 change: 1 addition & 0 deletions docs/apidoc/log/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Subpackages
.. toctree::

pyats.log.commands
pyats.atslog

Submodules
----------
Expand Down
6 changes: 3 additions & 3 deletions docs/apidoc/log/pyats.log.commands.parser.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ Submodules
----------


.. automodule:: pyats.log.commands.parser.base
.. automodule:: pyats.log.commands.parser.base_parser
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.log.commands.parser.xml
.. automodule:: pyats.log.commands.parser.xml_parser
:members:
:undoc-members:
:show-inheritance:


.. automodule:: pyats.log.commands.parser.yaml
.. automodule:: pyats.log.commands.parser.yaml_parser
:members:
:undoc-members:
:show-inheritance:
Expand Down
Loading