v1.3.2 (#208)

* adding another quick example

* cutting 200 pep8 errors

* working on py3 merge

* updating to python3

* reverting back changes, remember Switch flag next time

* adding some magic methods, just cause

* adding documentation pdf

* Fixes issue 49

* Adds True and False constants for older python versions.

Update issue 50

* adding tests for a few fixes

* Fixes issue 51

* fixing tests broken by interface change

* Updates issue 52

* Fixes issue 53

* Fix to add Python < 2.5 compatibility

This fix simply removes all the ternaries that were added
in Python 2.5.

Fixes issue 50

* adding a mostly complete diagnostic register implementation

* Adding more documentation and helpful factory methods.

The documentation added was to address some questions
with the synchronous and asynchrounous server implementations
as well as the functionality of the data contexts.

The factory methods were added by request to simplify the
creation of a fully populated DataBlock address space.

* adding modbus plus registers

* adding endian ability to payload builder

* Finishing the remaining modbus protocol

* Adding the remaining portions of the protocol (request/response)
* Tieing these into the factory decoder
* Adding tests to cover the new code (need more)
* Fixing a few bugs found along the way

* finished file read/write record requests

* documentation, more tests, fixing extra commands

* adding more tests

* Fixing the client/server async/sync implementations

- asynchronous client/server for tcp working and tested
- synchronous client/server for tcp/udp working and tested
- adding tests to exercise the synchronous client/server
- more documentation

* adding test stubs to get full coverage count

* working on jamod, need to set up more complete project

* adding GetDeviceInformationRequest

* adding read device information to decoder factory

* fixing some example errors

* working on the serial implementations

* fixing the synchronous server implementation

* updating functional tests and documentation for sync-serial

* updating tac files

* Updating the documentation for the serial client/servers

* adding readme to root

* Fixes issue 56

* Fixes issue 1

* adding a requirements file for virtualenv folk

* adding more cohesive code to the payload utilities

* forgot to add the new example to the doc index

* Fixing the rtu size calculation isse #55 on google code

* using the endian flag in register factory

* fixing bad documentation

* fixing issue 58 on google code

* Fix binary framer and add start and end tokens

Fixes a TypeError occuring while escaping token bytes in the message and
adds the start and end tokens to the packet.

* Adding checking in the client for an unconnected socket.

* Fixing issue 60 on google code (including ez_setup)

* fixing github issue #7

* adding tests and two utilities

* adding message parser utility

* Fixing extended message decoding

* Now correctly decoding sub function messages (diagnostic)
* Message parser now finished and documented
* Fixed broken logger unit test

* ModbusSparseDataBlock handled dictionaries incorrectly (they have an __iter__ attribute). Changed 'if' to 'elif'

* Updating the Readme.rst to explain how to install in zero dependency mode.

* Fixing issue #10

* updating the client documentation

* updating the server context documentation

* fixes #9 on github

* fixing the factory decoding and adding examples

* pushing to version 1.0

* Fixing a bug in the asynchronous client that slipped through.

* complete tests for the sync client

* cleaning up some pep8 errors

* finished unit tests for async client

* adding another test to the test-install script

* finished covering sync client

* updating documentation

* adding another example, adding debug checks

* Fixing the serial implementation of everything

* updating lots of reference documentation
* fixing the fifo semantics of serial clients
* using transaction manager in async clients
* fixing references

* fixing incorrect rtu lenght calculation

* Add optional server_address to Sync and Async TCP/UDP Server API

* Improve erver_address defaults to passing None

* adding a few messages

* Adding a message encoding generator for testing.

- fixed messages not passing **kwargs to base
- fixed binary framer off by 1
- fixed mei_message rtu size tests
- added a message generator to use with message parser
- fixed message parser with ascii
- tested message parser with all formats (added to messages)

* adding documentation for message formats

* updating documentation and being pedantic

* Allowing overloading of message encoding

* allow codes like payload builder to encode
* added IPayloadBuilder interface (future)
* renamed builder methods to reflect vision
* added error code decoding to name
* fixed affected tests

* Cleaning up the build tools

- moving custom datastores to examples
- bumping required versions
- making the debug server console optional
- updating documentation

* adding support for pydev

* fixing broken nosetest (/dev/pts) and pydev issues

* adding bcd payload builder

* reworking contrib packages

* adding bcd payload contrib

* moving complex examples to contrib

* syncing version on pypi

* adding server rest api

* moving web -> bottle

* fixing documentation

* adding gui base

* adding initial manager page

* adding data view

* fixing error in example

* adding the ability to override sync client framer

* updating the performance test

* fixing documentation

* fixing memory leak in sync client

* adding modicon encoder/decoder

* Fix bogus calculation in ModbusSocketFramer.checkFrame
(one of the tests also seemed wrong, I had to fix it)

* Adding sunspec client example

* Changing @staticmethod to @classmethod to fix inheritance

* adding method to retrieve all the device blocks

* adding the initial mapper interface

* adding ideas in progress

* adding a modbus datastore saver

* making lrc/crc read stdin

* adding the callback server example

* adding a periodic updating example

* fixing documentation

* adding fix for stuck RTU streams

* fixing documentation error

* Adding an example for changing framers

- new example changing-framers.py
- added documentation link

* Reworking the transaction managers to be explicit

- Serial framers use the FIFO manager (results in order)
- Socket framers use the Dict manager (tid -> result)
- Fixed tests and removed bad global managers
- Managers no longer use global state (now instance)

* Bumping the version and adding a changelog

* Fixing a logic error in client code

- Now correctly choosing transaction manager

* Fixing 2to3 common warnings

* Fixed example typo in readme

* Adding slave context delete ability

- Fixes #20

* Fixing google code issue 69

- custom-message example had bugs
- http://code.google.com/p/pymodbus/issues/detail?id=69

* Fixed docstring in synchronous serial client connect() method

I think there was a copy/paste error in the docstring. It said tcp but I think it is supposed to be serial.

* Fix for issue #21

* Allowing options to be passed through

- This fixes issue #21
- Options now go to the transaction manager

* Adding source_address to TCP client inputs

* Typo

* Typo

* Typo

* Typo

* Adding some new examples

* modbus scraper (and documentation)
* thread safe datastore (and documentation)

* Adding a concurrent client example

* Allow specification of client connection handler in Modbus...Server
o ModbusTcpServer and ModbusUdpServer allows handler class other than
  default Modbus{Connected,Disconnected}RequestHandler
o Add debug logging on request failure, to help diagnose root causes

* Fixes #46

* Fixes #48

* Adds the option to drop into zero mode for slaves

* Fixes #42

* Bumping the version to 1.3.0

* add missing self operator

The parameter values of the constructor within  class WriteMultipleRegistersRequest can specified as single value or list of values. But if a single value is given the missing self operator on line 137 leads to crash in line 138 with error: TypeError("object of type 'int' has no len()",)

* typo: fix spelling of Parameter

Signed-off-by: Karl Palsson <[email protected]>

* async: missing slaves: add missing imports

1e0bcde is missing imports into the
async server, the import was only added to the sync server.

Further fix for #42

Signed-off-by: Karl Palsson <[email protected]>

* Fix zero_mode argument for ModbusSlaveContext

Using "-" in the argument name causes SyntaxError
'non-keyword arg after keyword arg'

* Fix typo in constants

* Adding an example and adding tcp timeout

* fixes #70

* Endianness now used in packing bytes

* Adding a libmodbus wrapper

* adding new contrib example of libmodbus client
* updating example documentation so users can find it

* closes #73

* Execute methods of Requests accept context

There were errors whenever a request that got handles in
other_message.py got it's execute method called, as it was being called
with a context parameter. The functions now accept this parameter,
although they don't do anything with it.

The tests have also been updated.

* Add Modbus RTU syncronous server example

* Signal handlers to stop in shutdown().

* WriteMultipleRegistersRequest: Handle values=None case

Currently, if None is passed in explicitly, or if values is not given,
the 'values' object is correctly identified as *not* having an __iter__
attribute, but is incorrectly identified as being a valid register value.
This breaks testInvalidWriteMultipleRegistersRequest.

Solution: if we see None, replace this with [] and skip the check for
__iter__.

* Fix payload tests.

Looking at the "encoded" string, it looks identical to that of the big
endian encoding string, and my understanding is that the data shown in
"encoded" *IS* big-endian, not little-endian.

I have no idea how this passed before, but it passes now.

* sync.ModbusUdpClient: Correct reference to settimeout.

I could not find a 'settimeout' (case-insensitive) anywhere in the
codebase other than on this line, but I *did* see it was a method of
socket.socket.

So I'll assume that this was *supposed* to be calling
socket.socket.settimeout.

* test-client-sync: Fix UDP connection test.

Rather than passing in a vanilla object, we should pass in something
that implements the settimeout method.

* Lazily initialize InstallManagementConsole

Simply importing the management console takes ~0.3 sec on an 1GHz Atom.
Since it is (I suspect) often not used, lazily importing it should help
startup for many programs.

* Fixes the UDP client socket timeout bug

fixes #51

* Implement mask_write_register method in client mixin

* Workaround for bug 101 #101

* Workaround for bug 101 #101

* Riptide 1.3.0 pymodbus internal pre-release (riptide1) with dhoomakethu's fix for bashwork#101.

* Riptide 1.3.0 pymodbus internal pre-release (riptide1) with dhoomakethu's fix for bashwork#101.

* Use a PEP 404 compliant version.  Start at rc93101 to usurp any other real release candidates.

* Use a PEP 404 compliant version.  Start at rc93101 to usurp any other real release candidates.

* Document Riptide specific instructions.

* Include Workaround for bug 101 bashwork#101

* Include Workaround for bug 101 bashwork#101

* Better tag names.

* pymodbus fix for modes RTU error cases.

* version bumped to c93103

* version bumped to c93103

* fixed issue with pymodbus timing over serial

* fixed issue with pymodbus timing over serial

* Add RIPTIDE.md  to .gitignore

* bump version to rc93105

*  fix merge conflicts

* fix version file

*  fix unit test failures

* cleanup

* #121 changelog updated

* Misc update Add __maintainer__ field in setup.py

* 1. #162, creating universal distribution for py2 and py3
2. Merge PR #152 , create compatible versions

* fix test failures

* fix ReadDeviceInformationRequest  encoding probelm

*  #166 fix failing tests on python 3.5

* mis - Updated documentation

* #167 updated documentation for CI

* #167 disable pep8/flake checks for the time being

* #167, fix install dependencies osx, remove pypy from supported python versions

* #167, fix install dependencies osx, remove pypy from supported python versions

* #167 , fix minior error in travis.yml

* Bump version to 1.3.0rc2

* Fix #170 and other misc updates

* Updated Documents

* Add coverage.rc to show missing lines with coverage >4.0

* new documentation

* #171 Minor bug fix in BinaryPayloadBuilder.add_string method for python3

* 1. #179 fix error calculating response pdu length for read coil/discrete input requests
2. #179 add tests

* Enable debug logging for send packets

* #184 prepare for v1.3.1

* Make examples compatible withg python3

* fix #165, #174, #169, #175, #147, #146

* #184 updated changelog

* #142 move
MaskWriteRegisterRequest/MaskWriteRegisterResponse  to register_write_message.py from file_message.py

* fix #186 +misc updates

* Fixed obvious indentation error in InvalidResponseRecievedException class definition (#193)

Signed-off-by: Tomas Hozza <[email protected]>

* #190 fix examples - callback-server, modbus-payload

* #191 handle interframe timeout  for baudrate >19200 on Modbus RTU

* #201 Recover modbus server and client from previous errors

* Update changelog

* #149 SerialServer could be stopped when running on a thread

* #149 update changelogs

* unit tests for #191, #149 , fix obvious server_start errors

* Add new skip_bytes method to BinaryPayloadDecoder to skip over bytes (#205)

* Add new skip_bytes method to BinaryPayloadDecoder to skip over bytes
in the payload we are not interested in.

* Add an example of skip_bytes usage to the modbus-payload.py example.

* update readme

Author: dhoomakethu

.gitignore

@@ -12,7 +12,7 @@ pymodbus.egg-info/
 .tox/
 doc/api/epydoc/html/
 .vscode/
-
+.venv
 __pycache__/
 pymodbus/__pycache__/
 pymodbus/client/__pycache__/

CHANGELOG.rst

@@ -1,4 +1,13 @@
 
+Version 1.3.2
+------------------------------------------------------------
+* ModbusSerialServer could now be stopped when running on a seperate thread.
+* Fix issue with server and client where in the frame buffer had values from previous unsuccesful transaction
+* Fix response length calculation for ModbusASCII protocol
+* Fix response length calculation ReportSlaveIdResponse, DiagnosticStatusResponse
+* Fix never ending transaction case when response is recieved without header and CRC
+* Fix tests
+
 Version 1.3.1
 ------------------------------------------------------------
 * Recall socket recv until get a complete response

README.rst

@@ -125,7 +125,6 @@ like your device tested, I accept devices via mail or by IP address.
 That said, the current work mainly involves polishing the library as
 I get time doing such tasks as:
 
-  * Add CI support
   * Make PEP-8 compatible and flake8 ready
   * Fixing bugs/feature requests
   * Architecture documentation
@@ -152,6 +151,10 @@ Use make to perform a range of activities
      make tox        run the tests on all Python versions
      make clean      cleanup all temporary files 
 
+------------------------------------------------------------
+Contributing
+------------------------------------------------------------
+Just fork the repo and raise your PR against `dev` branch.
 
 ------------------------------------------------------------
 License Information

examples/common/callback-server.py

@@ -16,6 +16,7 @@
 from pymodbus.datastore import ModbusSlaveContext, ModbusServerContext
 from pymodbus.transaction import ModbusRtuFramer, ModbusAsciiFramer
 
+
 #---------------------------------------------------------------------------# 
 # import the python libraries we need
 #---------------------------------------------------------------------------# 
@@ -31,7 +32,8 @@
 
 #---------------------------------------------------------------------------# 
 # create your custom data block with callbacks
-#---------------------------------------------------------------------------# 
+#---------------------------------------------------------------------------#
+
 class CallbackDataBlock(ModbusSparseDataBlock):
     ''' A datablock that stores the new value in memory
     and passes the operation to a message queue for further
@@ -44,7 +46,7 @@ def __init__(self, devices, queue):
         self.devices = devices
         self.queue = queue
 
-        values = {k:0 for k in devices.iterkeys()}
+        values = {k:0 for k in devices.keys()}
         values[0xbeef] = len(values) # the number of devices
         super(CallbackDataBlock, self).__init__(values)
 

examples/common/device-mapping

@@ -0,0 +1,2 @@
+0x0001,/dev/ptyp0
+0x0002,/dev/ptyp1

examples/common/modbus-payload-server.py

@@ -35,10 +35,10 @@
 # build your payload
 #---------------------------------------------------------------------------# 
 builder = BinaryPayloadBuilder(endian=Endian.Little)
-builder.add_string('abcdefgh')
-builder.add_32bit_float(22.34)
-builder.add_16bit_uint(0x1234)
-builder.add_8bit_int(0x12)
+# builder.add_string('abcdefgh')
+# builder.add_32bit_float(22.34)
+# builder.add_16bit_uint(4660)
+# builder.add_8bit_int(18)
 builder.add_bits([0,1,0,1,1,0,1,0])
 
 #---------------------------------------------------------------------------# 

examples/common/modbus-payload.py

@@ -9,6 +9,7 @@
 from pymodbus.payload import BinaryPayloadDecoder
 from pymodbus.payload import BinaryPayloadBuilder
 from pymodbus.client.sync import ModbusTcpClient as ModbusClient
+from pymodbus.compat import iteritems
 
 #---------------------------------------------------------------------------# 
 # configure the client logging
@@ -34,13 +35,15 @@
 # - a 8 byte string 'abcdefgh'
 # - a 32 bit float 22.34
 # - a 16 bit unsigned int 0x1234
+# - another 16 bit unsigned int 0x5678
 # - an 8 bit int 0x12
 # - an 8 bit bitstring [0,1,0,1,1,0,1,0]
 #---------------------------------------------------------------------------# 
 builder = BinaryPayloadBuilder(endian=Endian.Big)
 builder.add_string('abcdefgh')
 builder.add_32bit_float(22.34)
 builder.add_16bit_uint(0x1234)
+builder.add_16bit_uint(0x5678)
 builder.add_8bit_int(0x12)
 builder.add_bits([0,1,0,1,1,0,1,0])
 payload = builder.build()
@@ -57,6 +60,7 @@
 # - a 8 byte string 'abcdefgh'
 # - a 32 bit float 22.34
 # - a 16 bit unsigned int 0x1234
+# - another 16 bit unsigned int which we will ignore
 # - an 8 bit int 0x12
 # - an 8 bit bitstring [0,1,0,1,1,0,1,0]
 #---------------------------------------------------------------------------# 
@@ -68,15 +72,16 @@
     'string': decoder.decode_string(8),
     'float': decoder.decode_32bit_float(),
     '16uint': decoder.decode_16bit_uint(),
+    'ignored': decoder.skip_bytes(2),
     '8int': decoder.decode_8bit_int(),
     'bits': decoder.decode_bits(),
 }
 
-print "-" * 60
-print "Decoded Data"
-print "-" * 60
-for name, value in decoded.iteritems():
-    print ("%s\t" % name), value
+print("-" * 60)
+print("Decoded Data")
+print("-" * 60)
+for name, value in iteritems(decoded):
+    print ("%s\t" % name, value)
 
 #---------------------------------------------------------------------------# 
 # close the client

examples/common/performance.py

@@ -13,7 +13,7 @@
 import logging, os
 from time import time
 from multiprocessing import log_to_stderr
-from pymodbus.client.sync import ModbusTcpClient
+# from pymodbus.client.sync import ModbusTcpClient
 from pymodbus.client.sync import ModbusSerialClient
 
 #---------------------------------------------------------------------------# 
@@ -61,9 +61,10 @@ def single_client_test(host, cycles):
         client = ModbusSerialClient(method="rtu", port="/dev/ttyp0", baudrate=9600)
         while count < cycles:
             with _thread_lock:
-                result = client.read_holding_registers(10, 1, unit=1).getRegister(0)
+                client.read_holding_registers(10, 1, unit=1).registers[0]
                 count += 1
-    except: logger.exception("failed to run test successfully")
+    except:
+        logger.exception("failed to run test successfully")
     logger.debug("finished worker: %d" % os.getpid())
 
 #---------------------------------------------------------------------------# 
@@ -73,6 +74,10 @@ def single_client_test(host, cycles):
 # threads that was specified. We then start all the threads and block on
 # them to finish. This may need to switch to another mechanism to signal
 # finished as the process/thread start up/shut down may skew the test a bit.
+
+# RTU 32 requests/second @9600
+# TCP 31430 requests/second
+
 #---------------------------------------------------------------------------# 
 args  = (host, int(cycles * 1.0 / workers))
 procs = [Worker(target=single_client_test, args=args) for _ in range(workers)]

pymodbus/client/sync.py

@@ -308,7 +308,10 @@ def __init__(self, method='ascii', **kwargs):
         self.timeout  = kwargs.get('timeout',  Defaults.Timeout)
         if self.method == "rtu":
             self._last_frame_end = 0.0
-            self._silent_interval = 3.5 * (1 + 8 + 2) / self.baudrate
+            if self.baudrate > 19200:
+                self._silent_interval = 1.75/1000  # ms
+            else:
+                self._silent_interval = 3.5 * (1 + 8 + 2) / self.baudrate
 
     @staticmethod
     def __implementation(method):

pymodbus/diag_message.py

@@ -74,7 +74,6 @@ def get_response_pdu_size(self):
         return 1 + 2 + 2 * len(self.message)
 
 
-
 class DiagnosticStatusResponse(ModbusResponse):
     '''
     This is a base class for all of the diagnostic response functions
@@ -118,7 +117,11 @@ def decode(self, data):
 
         :param data: The data to decode into the function code
         '''
-        self.sub_function_code, self.message = struct.unpack('>HH', data)
+        word_len = len(data)//2
+        if len(data) % 2:
+            word_len += 1
+        data = struct.unpack('>' + 'H'*word_len, data)
+        self.sub_function_code, self.message = data[0], data[1:]
 
 
 class DiagnosticStatusSimpleRequest(DiagnosticStatusRequest):
@@ -708,6 +711,23 @@ class GetClearModbusPlusRequest(DiagnosticStatusSimpleRequest):
     '''
     sub_function_code = 0x0015
 
+    def __init__(self, **kwargs):
+        super(GetClearModbusPlusRequest, self).__init__(**kwargs)
+
+    def get_response_pdu_size(self):
+        """
+        Returns a series of 54 16-bit words (108 bytes) in the data field of the response
+        (this function differs from the usual two-byte length of the data field). The data
+        contains the statistics for the Modbus Plus peer processor in the slave device.
+        Func_code (1 byte) + Sub function code (2 byte) + Operation (2 byte) + Data (108 bytes)
+        :return:
+        """
+        if self.message == ModbusPlusOperation.GetStatistics:
+            data = 2 + 108 # byte count(2) + data (54*2)
+        else:
+            data = 0
+        return 1 + 2 + 2 + 2+ data
+
     def execute(self, *args):
         ''' Execute the diagnostic request on the given device
 
@@ -716,9 +736,23 @@ def execute(self, *args):
         message = None # the clear operation does not return info
         if self.message == ModbusPlusOperation.ClearStatistics:
             _MCB.Plus.reset()
-        else: message = _MCB.Plus.encode()
+            message = self.message
+        else:
+            message = [self.message]
+            message += _MCB.Plus.encode()
         return GetClearModbusPlusResponse(message)
 
+    def encode(self):
+        '''
+        Base encoder for a diagnostic response
+        we encode the data set in self.message
+
+        :returns: The encoded packet
+        '''
+        packet = struct.pack('>H', self.sub_function_code)
+        packet += struct.pack('>H', self.message)
+        return packet
+
 
 class GetClearModbusPlusResponse(DiagnosticStatusSimpleResponse):
     '''

pymodbus/exceptions.py

@@ -78,19 +78,19 @@ def __init__(self, string=""):
         ModbusException.__init__(self, message)
 
 
-class InvalidResponseRecievedException(ModbusException):
+class InvalidMessageRecievedException(ModbusException):
     """
     Error resulting from invalid response received or decoded
     """
 
+    def __init__(self, string=""):
+        ''' Initialize the exception
 
-def __init__(self, string=""):
-    ''' Initialize the exception
+        :param string: The message to append to the error
+        '''
+        message = "[Invalid Message] %s" % string
+        ModbusException.__init__(self, message)
 
-    :param string: The message to append to the error
-    '''
-    message = "[Invalid Response] %s" % string
-    ModbusException.__init__(self, message)
 
 #---------------------------------------------------------------------------#
 # Exported symbols

pymodbus/other_message.py

@@ -403,7 +403,7 @@ def encode(self):
             status = ModbusStatus.SlaveOn
         else:
             status = ModbusStatus.SlaveOff
-        length  = len(self.identifier) + 2
+        length  = len(self.identifier) + 1
         packet  = int2byte(length)
         packet += self.identifier  # we assume it is already encoded
         packet += int2byte(status)

pymodbus/payload.py

@@ -349,6 +349,14 @@ def decode_string(self, size=1):
         self._pointer += size
         return self._payload[self._pointer - size:self._pointer]
 
+    def skip_bytes(self, nbytes):
+        ''' Skip n bytes in the buffer
+
+        :param nbytes: The number of bytes to skip
+        '''
+        self._pointer += nbytes
+        return None
+
 #---------------------------------------------------------------------------#
 # Exported Identifiers
 #---------------------------------------------------------------------------#