feature: testsuite configurable. Docu: simulation and testing

This commit is contained in:
uhi22 2023-05-16 19:13:21 +02:00
parent 537387ff80
commit e7725c1a8d
3 changed files with 79 additions and 2 deletions

View file

@ -0,0 +1,50 @@
# Testing and Simulation
The purpose of the pyPlc project is, to talk to a real charger or to a real car. But these scenarios
have a certain effort to be tested.
To have better testing possibilities "on the desk", the following methods can be used.
## Simulation without any modem or other hardware
This works on Windows10, and should also work on Raspberry (not tested).
Open two console windows. In one window, start the pyPlc in EvseMode and simulation enabled:
```
python pyPlc.py E S
```
In the other console window, start pyPlc in PevMode and also simulation enabled:
```
python pyPlc.py P S
```
## Use two different machines
Use two machines (can be a Windows10 and a Raspberry for instance) and connect them with appropriate configured modems.
On one machine, run the charger side:
```
python pyPlc.py E
```
On the other machine start pyPlc in PevMode:
```
python pyPlc.py P
```
Wireshark can be used on any side, to make a trace of the communication for later inspection.
## Automatic Test Case execution
The above simulation environment is nice to test the standard charging process, but also the error cases should be
tested, to ensure a convinient and safe operation. For such tests, the pyPlc project contains the possiblity to
inject faults into the normal procedure, and so to test the reaction in case of errors.
### How does this work?
The main test functionality is contained in mytestsuite.py. Here the test cases are configured.
The Evse state machine triggers the selection of the next test case by calling testsuite_choose_testcase().
In the interesting points in the state machines, where we want to inject faults, we call the testsuite_faultinjection_is_triggered() inside an if condition. This function returns True, if the fault injection is triggered.
The state machine which shall be tested (can be on an other machine) reports certain execution pathes via UDP syslog messages. Based on this feedback, the testsuite decides whether the expected behavior is fulfilled or not.
### Turning the Testsuite on and off
The testsuite feature can be enabled by setting testsuite_enable = Yes in the pyPlc.ini.
### Inspecting the test results
The progress of the tests and their result can be best seen in Wireshark, by applying a display filter for "syslog".

View file

@ -8,8 +8,12 @@
# In normal software run, this condition is never fulfilled and does not disturb. If the related test case is activated, # In normal software run, this condition is never fulfilled and does not disturb. If the related test case is activated,
# by setting testsuite_testcase_number = TC_MY_TESTCASE_FOR_SOMETHING below, the condition will fire and the fault is injected. # by setting testsuite_testcase_number = TC_MY_TESTCASE_FOR_SOMETHING below, the condition will fire and the fault is injected.
# A number of delay cycles can be configured with testsuite_delayCycles below. # A number of delay cycles can be configured with testsuite_delayCycles below.
#
# Detailled docu see doc/testing_and_simulation.md
from udplog import udplog_log from udplog import udplog_log
from configmodule import getConfigValue, getConfigValueBool
# The list of test cases. Each must have a unique test case ID. # The list of test cases. Each must have a unique test case ID.
@ -40,6 +44,9 @@ testsuite_delayCycles = 0
testsuite_counter = 0 testsuite_counter = 0
def testsuite_getTcNumber(): def testsuite_getTcNumber():
if (testsuite_testcase_number==0):
return "(no tests)"
else:
return testsuite_testcase_number return testsuite_testcase_number
def testsuite_faultinjection_is_triggered(context): def testsuite_faultinjection_is_triggered(context):
@ -60,6 +67,10 @@ def testsuite_choose_testcase():
global testsuite_observedResult global testsuite_observedResult
global testsuite_expectedResult global testsuite_expectedResult
if (not getConfigValueBool("testsuite_enable")):
testsuite_testcase_number = TC_NOTHING_TO_TEST
return
try: try:
if (testsuite_expectedResult is None): if (testsuite_expectedResult is None):
testsuite_expectedResult = "" testsuite_expectedResult = ""

View file

@ -346,6 +346,10 @@ is active (current limitation, voltage limitation, power limitation).
See [charger_test_results.md](doc/charger_test_results.md) See [charger_test_results.md](doc/charger_test_results.md)
## Testing And Simulation On Desk
See [testing_and_simulation.md](doc/testing_and_simulation.md)
## Biggest Challenges ## Biggest Challenges
- [*ListenMode*] Find a way to enable the sniffer mode or monitor mode in the AR7420. Seems to be not included in the public qca/open-plc-utils. - [*ListenMode*] Find a way to enable the sniffer mode or monitor mode in the AR7420. Seems to be not included in the public qca/open-plc-utils.
Without this mode, we see only the broadcast messages, not the TCP / UDP traffic between the EVSE and the PEV. Without this mode, we see only the broadcast messages, not the TCP / UDP traffic between the EVSE and the PEV.
@ -379,10 +383,22 @@ measured current flow (discharging). The car may stop the complete process, if t
However, the software will help to explore and understand the behavior of the car. However, the software will help to explore and understand the behavior of the car.
### Q3: Is it possible to use this software in a car without CCS, to make it ready for CCS charging? ### Q3: Is it possible to use this software in a car without CCS, to make it ready for CCS charging?
That's is definitely a goal, at it looks reachable. Of course, two aspects need to be considered: That's is definitely a goal, and it was reached in the meanwhile. Of course, two aspects need to be considered:
- This project is not a final product. Further development will be necessary to ensure compatibility with chargers, and make it flexible for practical use. - This project is not a final product. Further development will be necessary to ensure compatibility with chargers, and make it flexible for practical use.
- Some parts are not covered by this project at all, e.g. communication with the BMS, connector lock, safety considerations. - Some parts are not covered by this project at all, e.g. communication with the BMS, connector lock, safety considerations.
### Q4: Do I need to go to a public charger to test my Pev integration?
For testing the communication at home, the pyPlc supports both sides of the CCS, the car and the charger. Have a
look to [testing_and_simulation.md](doc/testing_and_simulation.md)
## Credits
Thanks to catphish to start the investigations regarding the homeplug modems and publishing them on OpenInverter forum.
Thanks to johu for the OpenInverter forum, and for the first video of the early experiments, and the beaglebone integration.
Thanks to CCSknowitall for clarifying tricky details.
Thanks to celeron55 for the large number of test drives to various public chargers, and for improving the software. Thanks for helpful discussions to Pete9008, peternooy, tom91, asavage and all which I forgot to mention.
Thanks to all patreons.
## References ## References
* [i] https://www.goingelectric.de/wiki/CCS-Technische-Details/ knowledge collection * [i] https://www.goingelectric.de/wiki/CCS-Technische-Details/ knowledge collection
* [ii] https://openinverter.org/forum/viewtopic.php?p=37085#p37085 discussion of this project * [ii] https://openinverter.org/forum/viewtopic.php?p=37085#p37085 discussion of this project